Consider a function F (think of neural networks) with two sets of parameters: (1) model parameters $\mathbf{w}$, and (2) input data ${\bf x} \in {\mathbb R}^d$. Fix $i \in [d]$, consider the following path integral:

$$C_i({\bf x}, {\bf w}) = {\bf x}_i \int_0^1\frac{\partial F}{\partial {\bf x}_i}(\alpha {\bf x})d\alpha $$

Basically, one can think of $C_i({\bf x}, {\bf w})$ as the "contribution" of the $i$-th dimension to the final output -- by integrating along a line from $\bf 0$ to $\bf x$.

My question is whether there are known methods (references, etc.) that give "backpropagation-like" algorithms for computing:

$$\frac{\partial C_i({\bf x}, {\bf w})}{\partial {\bf w}_i}$$

That is, basically I want to understand the first order information of the contribution of the $i$-th dimension w.r.t. ${\bf w}_i$.

Thanks in advance for any insight.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Browse other questions tagged or ask your own question.