Biased estimators are useful in statistics because they can optimize mean-squared error more than what an unbiased estimator can manage. I was wondering if in theoryCS if there are any very notable examples of the effective use of biased estimators. I realize this list could get long, and if it does I can modify this question to a big-list CW question, but for now I'm just curious.
In machine learning, the sample error of an ERM learner is a biased estimator of the true generalization error. Here the bias is not by choice but by necessity: we simply don't know of a way to construct an unbiased estimator of the generalization error, and I'm pretty sure it's impossible from general no-free-lunch principles.