Consider a function F (think of neural networks) with two sets of parameters: (1) model parameters $\mathbf{w}$, and (2) input data ${\bf x} \in {\mathbb R}^d$. Fix $i \in [d]$, consider the following path integral:

$$C_i({\bf x}, {\bf w}) = {\bf x}_i \int_0^1\frac{\partial F}{\partial {\bf x}_i}(\alpha {\bf x})d\alpha $$

Basically, one can think of $C_i({\bf x}, {\bf w})$ as the "contribution" of the $i$-th dimension to the final output -- by integrating along a line from $\bf 0$ to $\bf x$.

My question is whether there are known methods (references, etc.) that give "backpropagation-like" algorithms for computing:

$$\frac{\partial C_i({\bf x}, {\bf w})}{\partial {\bf w}_i}$$

That is, basically I want to understand the first order information of the contribution of the $i$-th dimension w.r.t. ${\bf w}_i$.

Thanks in advance for any insight.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.