1
$\begingroup$

Consider a function F (think of neural networks) with two sets of parameters: (1) model parameters $\mathbf{w}$, and (2) input data ${\bf x} \in {\mathbb R}^d$. Fix $i \in [d]$, consider the following path integral:

$$C_i({\bf x}, {\bf w}) = {\bf x}_i \int_0^1\frac{\partial F}{\partial {\bf x}_i}(\alpha {\bf x})d\alpha $$

Basically, one can think of $C_i({\bf x}, {\bf w})$ as the "contribution" of the $i$-th dimension to the final output -- by integrating along a line from $\bf 0$ to $\bf x$.

My question is whether there are known methods (references, etc.) that give "backpropagation-like" algorithms for computing:

$$\frac{\partial C_i({\bf x}, {\bf w})}{\partial {\bf w}_i}$$

That is, basically I want to understand the first order information of the contribution of the $i$-th dimension w.r.t. ${\bf w}_i$.

Thanks in advance for any insight.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.