I know that in general, a function $g$ can be a good auto-encoder (i.e., $g(x) \approx x$ for $x \sim D$) and on the same time $I(g(x);x)$ is small. This is the case when $g$ forms a good correlation but no compression of the data.

I'm curious if there is any work that suggests that the training process or some regularization tends to implicitly select an auto-encoder that maximizes the mutual information $I(x;g(x))$.

Please let me know if there are some missing mathematical technicalities and if the question needs further clarifications.


  • 3
    $\begingroup$ Please define your terms and notations. $\endgroup$ – Jan Johannsen Sep 11 '18 at 7:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.