-2
$\begingroup$

I have recently come across a Matlab implementation that appears to be using only the first term (i.e. in itself a product) of the typical logistic regression cost function. enter image description here

f = - full(mean(sum(Y.*log(Yr))));
f = f + 0.5*lambda*sum(W(:).^2);

When I create a two class simple dataset and adjust the cost function to its traditional form I still get separation between classes albeit the vector of parameters are slightly different (for a fixed lambda regularizer in both cases).

enter image description here

f = - full(mean(sum(Y.*log(Yr) + (1-Y).*log(1-Yr))));
f = f + 0.5*lambda*sum(W(:).^2);

For better visualization see the following plot Orange due to simpler, blue due to full cost

Once the cost function is set, the parameters are learned using limited-memory BFGS. Gradient functions remain the same (i.e. derived from the correct classical cost function). enter image description here

Naturally, my question is: Has this choice been deliberate ? Is the first objective function sufficient ? Or is it just an obvious inconsistency.

$\endgroup$
-1
$\begingroup$

Actually I have realized where the misunderstanding came from. In fact the implementation aforementioned specifies a cost for the general case, namely the multinational logistic regression. Therefore the implementation is deliberate and correct.

See the screenshot below.

enter image description here

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.