# Latest word on cross validation?

It's a standard result leave-one-out cross-validation is an unbiased estimator of the risk (see, e.g., Lemma 4.1 in Mohri, Rostamizadeh, Talwalkar). Are there any "better" results? Such as, say, with high probability rather than just in expectation?

• From my phone at the moment, but is this type of results (see discussion in Section 3) what you are looking for? math.arizona.edu/~hzhang/math574m/Read/LOOtheory.pdf – Clement C. Jan 6 at 0:05
• Yes, very much in this spirit! But is that the last word as of 2020? :) – Aryeh Jan 6 at 5:39

It is not hard to see that without additional stability assumptions one won't be able to get high probability bounds. For example consider predicting unbiased coin using majority label in the sample. With probability $$~1/\sqrt{n}$$ we get that majority of leave-one-out is exactly the opposite of the excluded point so LOO will give error of 1. Note that the true error is always 1/2.