1
$\begingroup$

In many machine learning algorithm, it is often assumed that outputs of unknown function and their corresponding inputs are given to estimate the unknown function. However, I wonder whether there exist some algorithms which can estimate unknown function using derivative data of unknown function and their corresponding inputs. For example,

$$ Given\;D = {(x_1, y_1'),\ldots,(x_n, y_n')} \\ where\;y_{i}' = \frac{df(x_{i})}{dx} \; for \; 1 \leq i \leq n\\ Find \;\; f(x) $$

I found one curve fitting method which is called Hermite interpolation in wiki and some papers about Hermite learning. However, in these methods, outputs of unknown function and derivative outputs of it and corresponding inputs are given. Especailly, I want to know there exist some methods that utilize only derivative data. Also is it possible to find unknown function from derivative data, in particular, the error bound between unknown function and our estimator can decrease as the number of training data increases. How can we prove it?

$\endgroup$
3
$\begingroup$

If your function is $f:\mathbb{R}\to\mathbb{R}$, you can "learn" $f'$ as a standard regression problem (linear, polynomial, etc.) and then recover $f$ up to an additive constant by integrating $f'$. Obviously, you will only able to recover $f$ up to an additive constant from the derivative.

Update: As for error estimates, here's a simple one. Suppose you've managed to recover $f'$ up to $\epsilon$-accuracy in $L_\infty$. Suppose further that you know that $f(0)=0$ and you only care about $f$ on $[0,T]$. Then you estimate of $f$ (obtained by integrating $f'$) is accurate up to $T\epsilon$ in $L_\infty$.

$\endgroup$
  • $\begingroup$ Thank you for your kind example. However, as you said, when $f$ is estimated from $f'$ with error $\epsilon$, the error of estimation linearly grows (i.e. $T\epsilon$). In particular, if the interval of integration goes to infinity such as $[-\infty,\infty] = \mathbb{R} $, then the cumulative error $\int\epsilon dt$ also goes to infinity. So the error of estimation through the whole input domain always goes to infinity.In this case, is it possible to discuss the same thing as your answer? Minor question aoubt notation, does $L_{\infty}$ mean the set of function whose infinite norm is bounded? $\endgroup$ – Kyoungjae Lee Dec 5 '16 at 3:13
  • $\begingroup$ If you assume something about the noise, you might be able to argue that the errors in the derivative cancel out on average (plausible, since you're integrating) and the deviation is bounded over all of $R$. $\endgroup$ – Aryeh Dec 5 '16 at 8:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.