Is there a theoretical guarantee that an autoencoder $g$ has $I(x;g(x)) \approx H(x)$?

I know that in general, a function $g$ can be a good auto-encoder (i.e., $g(x) \approx x$ for $x \sim D$) and on the same time $I(g(x);x)$ is small. This is the case when $g$ forms a good correlation but no compression of the data.

I'm curious if there is any work that suggests that the training process or some regularization tends to implicitly select an auto-encoder that maximizes the mutual information $I(x;g(x))$.

Please let me know if there are some missing mathematical technicalities and if the question needs further clarifications.

Thanks!!

• Please define your terms and notations. – Jan Johannsen Sep 11 '18 at 7:31