Explore Courses Blog Tutorials Interview Questions
0 votes
in Machine Learning by (19k points)

I am using scipy.optimize.fmin_l_bfgs_b to solve a gaussian mixture problem. The means of mixture distributions are modeled by regressions whose weights have to be optimized using EM algorithm.

sigma_sp_new, func_val, info_dict = fmin_l_bfgs_b(func_to_minimize, self.sigma_vector[si][pj], 

                       args=(self.w_vectors[si][pj], Y, X, E_step_results[si][pj]),

                       approx_grad=True, bounds=[(1e-8, 0.5)], factr=1e02, pgtol=1e-05, epsilon=1e-08)

But sometimes I got a warning 'ABNORMAL_TERMINATION_IN_LNSRCH' in the information dictionary:

func_to_minimize value = 1.14462324063e-07

information dictionary: {'task': b'ABNORMAL_TERMINATION_IN_LNSRCH', 'funcalls': 147, 'grad': array([  1.77635684e-05,   2.87769808e-05,   3.51718654e-05,

         6.75015599e-06,  -4.97379915e-06,  -1.06581410e-06]), 'nit': 0, 'warnflag': 2}


Machine precision = 2.220D-16

 N = 6     M = 10

 This problem is unconstrained.

At X0         0 variables are exactly at the bounds

At iterate    0    f=  1.14462D-07    |proj g|=  3.51719D-05

Tit   = total number of iterations

Tnf   = total number of function evaluations

Tnint = total number of segments explored during Cauchy searches

Skip  = number of BFGS updates skipped

Nact  = number of active bounds at final generalized Cauchy point

Projg = norm of the final projected gradient

F     = final function value

   N    Tit     Tnf  Tnint  Skip  Nact     Projg        F

    6      1     21      1     0     0   3.517D-05   1.145D-07

  F =  1.144619474757747E-007


 Line search cannot locate an adequate point after 20 function

  and gradient evaluations.  Previous x, f and g restored.

 Possible causes: 1 error in function or gradient evaluation;

                  2 rounding error dominate computation.

 Cauchy                time 0.000E+00 seconds.

 Subspace minimization time 0.000E+00 seconds.

 Line search           time 0.000E+00 seconds.

 Total User time 0.000E+00 seconds.

I do not get this warning every time, but sometimes.

1 Answer

0 votes
by (33.1k points)

Scipy calls the original L-BFGS-B implementation. In your problem, the descent direction is actually going up. 

For example:

gd = ddot(n,g,1,d,1)

  if (ifun .eq. 0) then


     if (gd .ge. zero) then

c                               the directional derivative >=0.

c                               Line search is impossible.

        if (iprint .ge. 0) then

            write(0,*)' ascent direction in projection gd = ', gd


        info = -4




In general, you are telling it to go down the hill by going up the hill. The code tries something called line search a total of 20 times in the descent direction that you provide and realizes that you are NOT telling it to go downhill, but uphill, every 20 times.

Hope this answer helps you!

Browse Categories