1
$\begingroup$

An additive model constructed using the exponential loss function

$$L(y, f (x)) = \exp(−yf (x))$$

gives Adaboost. How can we derive the corresponding additive model (known as logitboost) using the logistic loss function

$$L(y, f (x)) = \log(1 + \exp(−yf (x)))$$

What steps I should take to do the above proof?

| cite | improve this question | |
$\endgroup$
  • 2
    $\begingroup$ I suggest modifying this to ask a concrete theoretical question -- e.g. "How does one minimize logistic loss...". $\endgroup$ – Lev Reyzin♦ Nov 1 '12 at 22:22
2
$\begingroup$

Originally, Logitboost was derived by Friedman, Hastie, and Tibshirani (paper) -- their algorithm internally used a numerical procedure, via Newtons method to solve a regression problem.

Later, Collins, Schapire, and Singer (paper) found an equivalent formulation, with a single-line modification from AdaBoost, setting $$D(i) \propto \frac{1}{1+e^{y_i f_{t-1}(x_i)}}.$$

Schapire has a nice summary of boosting here, which also discusses Logitboost.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ looks like I didn't ask the question clearly, I have cleaned up. I understand the adaboost. $\endgroup$ – add-semi-colons Nov 2 '12 at 0:57
2
$\begingroup$

To get you answer,you may wanna look at this paper http://dept.stat.lsa.umich.edu/~gmichail/ada_final.pdf. Algorithm 2 summarizes the step you have to take to derive a boosting algorithm from any given convex loss including logitboost.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.