On p.87 in this online Understanding Machine Learning book, the authors wrote:
Unlike the ERM paradigm discussed in previous chapters, we no longer just care about the empirical risk, $L_S(h)$, but we are willing to trade some of our bias toward low empirical risk with a bias toward classes for which $\epsilon_{n(h)}(m, w(n(h))\cdot\epsilon)$ is smaller, for the sake of a smaller estimation error.
With the output $h$ of SRM defined to be:
Output: $h\in\underset{h\in\mathcal{H}}{argmin}\ L_S(h)+\epsilon_{n(h)}(m, w(n(h))\cdot\delta)$.
Where $\epsilon_n(m, \delta) := min\ \lbrace\epsilon\in (0, 1): m_\mathcal{H_n}^{UC}(\epsilon, \delta)\le m \rbrace$, $w(n)$ is a function from $n\in\mathbb{N}\to[0, 1]$ such that $\sum_{n=1}^{\infty}w(n)\le1$ and $n(h):=min\lbrace n: h\in\mathcal{H}_n\rbrace$
That is the output learner $h$ will minimize $L_S(h)+\epsilon_{n(h)}(m, w(n(h))\cdot\delta)$.
However, $\epsilon_n$ as defined above is the lowest possible upper bound on the gap between $L_S(h)-L_\mathcal{D}(h)$ achieved by using a sample of $m$ examples from the hypothesis class $\mathcal{H}_n$ , while the estimation error is defined to be:
$$\epsilon_{est}:=L_\mathcal{D}(h)-\underset{h\in\mathcal{H}}{min} L_\mathcal{D}(h).$$
Besides, the Theorem 7.4. in the book shows that:
$$ \mathbb{P}\lbrace S\sim\mathcal{D}^m: (\forall h\in\mathcal{H}) (L_\mathcal{D}(h)\le L_S(h) + \epsilon_{n(h)}(m, w(n(h))\cdot\delta)) \rbrace \ge 1-\delta, $$
so minimizing $L_S(h)+\epsilon_{n(h)}(m, w(n(h))\cdot\delta)$ will, with at least $1-\delta$ probability, minimize $L_\mathcal{D}(h)$ but it's not the same as to minimize $\epsilon_{est}$?
And so I couldn't find the connection between the minimization of $L_S(h)+\epsilon_{n(h)}(m, w(n(h))\cdot\delta)$ and the smaller estimation error.
