Octave : logistic regression : difference between fmincg and fminunc

Question

I often use fminunc for a logistic regression problem. I have read on web that Andrew Ng uses fmincg instead of fminunc, with same arguments. The results are different, and often fmincg is more exact, but not too much. (I am comparing the results of fmincg function fminunc against the same data)

So, my question is : what is the difference between these two functions? What algorithm does each function have implemented? (Now, I just use these functions without knowing exactly how they work).

Thanks :)

Answer 1

The fmincg function is used not to get a more accurate result, your cost function should be the same in either case, and your hypothesis will become less simple or more complex, but because it is more efficient at doing gradient descent for especially complex hypotheses. You should use fminunc where the hypothesis has few features, but fmincg where it has hundreds.

Visit here if you wish to know more about What is Logistic Regression.

Hope this answer helps.