scipy.optimize.fmin_l_bfgs_b returns 'ABNORMAL_TERMINATION_IN_LNSRCH'

Question

I am using scipy.optimize.fmin_l_bfgs_b to solve a gaussian mixture problem. The means of mixture distributions are modeled by regressions whose weights have to be optimized using EM algorithm.

sigma_sp_new, func_val, info_dict = fmin_l_bfgs_b(func_to_minimize, self.sigma_vector[si][pj], 

                       args=(self.w_vectors[si][pj], Y, X, E_step_results[si][pj]),

                       approx_grad=True, bounds=[(1e-8, 0.5)], factr=1e02, pgtol=1e-05, epsilon=1e-08)

But sometimes I got a warning 'ABNORMAL_TERMINATION_IN_LNSRCH' in the information dictionary:

func_to_minimize value = 1.14462324063e-07

information dictionary: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'funcalls': 147, 'grad': array([  1.77635684e-05,   2.87769808e-05,   3.51718654e-05,

         6.75015599e-06,  -4.97379915e-06,  -1.06581410e-06]), 'nit': 0, 'warnflag': 2}

RUNNING THE L-BFGS-B CODE

Machine precision = 2.220D-16

 N = 6     M = 10

 This problem is unconstrained.

At X0         0 variables are exactly at the bounds

At iterate    0    f=  1.14462D-07    |proj g|=  3.51719D-05

Tit   = total number of iterations

Tnf   = total number of function evaluations

Tnint = total number of segments explored during Cauchy searches

Skip  = number of BFGS updates skipped

Nact  = number of active bounds at final generalized Cauchy point

Projg = norm of the final projected gradient

F     = final function value

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F

    6      1     21      1     0     0   3.517D-05   1.145D-07

  F =  1.144619474757747E-007

ABNORMAL_TERMINATION_IN_LNSRCH                              

 Line search cannot locate an adequate point after 20 function

  and gradient evaluations.  Previous x, f and g restored.

 Possible causes: 1 error in function or gradient evaluation;

                  2 rounding error dominate computation.

 Cauchy                time 0.000E+00 seconds.

 Subspace minimization time 0.000E+00 seconds.

 Line search           time 0.000E+00 seconds.

 Total User time 0.000E+00 seconds.

I do not get this warning every time, but sometimes.

Answer 1

Scipy calls the original L-BFGS-B implementation. In your problem, the descent direction is actually going up. 

For example:

gd = ddot(n,g,1,d,1)

  if (ifun .eq. 0) then

     gdold=gd

     if (gd .ge. zero) then

c                               the directional derivative >=0.

c                               Line search is impossible.

        if (iprint .ge. 0) then

            write(0,*)' ascent direction in projection gd = ', gd

        endif

        info = -4

        return

     endif

  endif

In general, you are telling it to go down the hill by going up the hill. The code tries something called line search a total of 20 times in the descent direction that you provide and realizes that you are NOT telling it to go downhill, but uphill, every 20 times.

Hope this answer helps you!