2 views

I am using scipy.optimize.fmin_l_bfgs_b to solve a gaussian mixture problem. The means of mixture distributions are modeled by regressions whose weights have to be optimized using EM algorithm.

sigma_sp_new, func_val, info_dict = fmin_l_bfgs_b(func_to_minimize, self.sigma_vector[si][pj],

args=(self.w_vectors[si][pj], Y, X, E_step_results[si][pj]),

approx_grad=True, bounds=[(1e-8, 0.5)], factr=1e02, pgtol=1e-05, epsilon=1e-08)

But sometimes I got a warning 'ABNORMAL_TERMINATION_IN_LNSRCH' in the information dictionary:

func_to_minimize value = 1.14462324063e-07

6.75015599e-06,  -4.97379915e-06,  -1.06581410e-06]), 'nit': 0, 'warnflag': 2}

RUNNING THE L-BFGS-B CODE

Machine precision = 2.220D-16

N = 6     M = 10

This problem is unconstrained.

At X0         0 variables are exactly at the bounds

At iterate    0    f=  1.14462D-07    |proj g|=  3.51719D-05

Tit   = total number of iterations

Tnf   = total number of function evaluations

Tnint = total number of segments explored during Cauchy searches

Skip  = number of BFGS updates skipped

Nact  = number of active bounds at final generalized Cauchy point

Projg = norm of the final projected gradient

F     = final function value

N    Tit     Tnf  Tnint  Skip  Nact     Projg        F

6      1     21      1     0     0   3.517D-05   1.145D-07

F =  1.144619474757747E-007

ABNORMAL_TERMINATION_IN_LNSRCH

Line search cannot locate an adequate point after 20 function

and gradient evaluations.  Previous x, f and g restored.

Possible causes: 1 error in function or gradient evaluation;

2 rounding error dominate computation.

Cauchy                time 0.000E+00 seconds.

Subspace minimization time 0.000E+00 seconds.

Line search           time 0.000E+00 seconds.

Total User time 0.000E+00 seconds.

I do not get this warning every time, but sometimes.

by (33.1k points)

Scipy calls the original L-BFGS-B implementation. In your problem, the descent direction is actually going up.

For example:

gd = ddot(n,g,1,d,1)

if (ifun .eq. 0) then

gdold=gd

if (gd .ge. zero) then

c                               the directional derivative >=0.

c                               Line search is impossible.

if (iprint .ge. 0) then

write(0,*)' ascent direction in projection gd = ', gd

endif

info = -4

return

endif

endif

In general, you are telling it to go down the hill by going up the hill. The code tries something called line search a total of 20 times in the descent direction that you provide and realizes that you are NOT telling it to go downhill, but uphill, every 20 times.