Bounds on $E[f(x)]$ in terms of $f(E[x])$ other than Jensen's inequality?

Question

If $f$ is a convex function then Jensen's inequality states that $f(\textbf{E}[x]) \le \textbf{E}[f(x)]$, and mutatis mutandis when $f$ is concave. Clearly in the worst case you cannot upper bound $\textbf{E}[f(x)]$ in terms of $f(\textbf{E}[x])$ for a convex $f$, but is there a bound that goes in this direction if $f$ is convex but "not too convex"? Is there some standard bound that gives conditions on a convex function $f$ (and possibly the distribution as well, if necessary) that would allow you to conclude that $\textbf{E}[f(x)] \le \varphi(f)f(\textbf{E}[x])$, where $\varphi(f)$ is some function of the curvature/degree of convexity of $f$? Something akin to a Lipschitz condition, perhaps?

Answer 1

EDIT: original version missed an absolute value. sorry!!

Hi Ian. I will briefly outline two sample inequalities, one using a Lipschitz bound, the other using a bound on the second derivative, and then discuss some difficulties in this problem. Although I'm being redundant, since an approach using one derivative explains what happens with more derivatives (via Taylor), it turns out that the second derivative version is quite nice.

First, with a Lipschitz bound: simply re-work the standard Jensen inequality. The same trick applies: compute the Taylor expansion at the expected value.

Specifically, Let $X$ have corresponding measure $\mu$, and set $m := \textrm E(x)$. If $f$ has Lipschitz constant $L$, then by Taylor's theorem

$$ f(x) = f(m) + f'(z)(x-m) \leq f(m) + L|x-m|, $$

where $z \in [m, x]$ (note that $x\leq m$ and $x> m$ are possible). Using this and re-working the Jensen proof (I am paranoid and checked that the standard one is indeed on wikipedia),

\begin{align} \operatorname{E}(f(X)) & = \int f(x) \, d\mu(x) \leq f(m) \int d\mu(x) + L\int |x-m| \, d\mu(x) \\[6pt] & = f(\operatorname{E}(X)) + L \operatorname{E} (|X-\operatorname{E}(X)|). \end{align}

Now, suppose $|f''(x)| \leq \lambda$. In this case,

\begin{align} f(x) & = f(m) + f'(m)(x-m) + f''(z) \frac{(x-m)^2} 2 \\[6pt] & \leq f(m) + f'(m)(x-m) + \lambda \frac{(x-m)^2} 2, \end{align}

and so

\begin{align} \operatorname{E}(f(X)) & \leq f(m) + f'(m)(\operatorname{E}(X) - m) + \frac {\lambda \operatorname{E}((X-m)^2)}{2} \\[6pt] & = f(\operatorname{E}(X)) + \frac {\lambda \operatorname{Var}(X)}2. \end{align}

I'd like to briefly mention a few things. Sorry if they are obvious.

One is that, you can't merely say "wlog $\operatorname{E}(X) = 0$" by shifting the distribution, because you are changing the relationship between $f$ and $\mu$.

Next is that the bound must depend on the distribution in some way. To see this, imagine that $X \sim \textrm{Gaussian}(0, \sigma^2)$ and $f(x) = x^2$. Whatever the value of $\sigma$, you still get $f(\operatorname{E}(X)) = f(0) = 0$. On the other hand, $\operatorname{E}(f(X)) = \operatorname{E}(X^2) = \sigma^2$. Thus, by changing $\sigma$, you can make the gap between the two quantities arbitrary! Intuitively, more mass is pushed away from the mean, and thus, for any strictly convex function, $\operatorname{E} (f(X))$ will increase.

Lastly, I don't see how to get a multiplicative bound like you suggest. Everything I have used in this post is standard: Taylor's theorem and derivative bounds are bread&butter in statistics bounds, and they automatically give additive, not multiplicative errors.

I will think about it though, and post something. Vague intuition is it will need very strenous conditions on both the function and the distribution, and that the additive bound is actually at the heart of it.

Answer 2

For insight, consider a distribution concentrated on two values; say, with equal probabilities of 1/2 that it equal 1 or 3, whence $\textbf{E}[x] = 2$. Take $N >> 0$ and $\epsilon > 0$. Consider functions $f$ for which $f(1) = f(3)= N\epsilon$ and $f(\textbf{E}[x]) = f(2) = \epsilon$. By making $\epsilon$ sufficiently small and connecting $f$ continuously among these three points we can make the curvature of $f$ as small as desired. Then

$\textbf{E}[f(x)] = N\epsilon$, yet

$N = N\epsilon / \epsilon = \textbf{E}[f(x)] / f(\textbf{E}[x]) \le \varphi(f)$.

This shows $\varphi(f)$ must be arbitrarily large.