0 votes
1 view
in Machine Learning by (17.1k points)

I just try to find out how I can use Caffe. To do so, I just took a look at the different .prototxt files in the examples folder. There is one option I don't understand:

# The learning rate policy

lr_policy: "inv"

Possible values seem to be:

  • "fixed"

  • "inv"

  • "step"

  • "multistep"

  • "stepearly"

  • "poly"

Could somebody please explain those options?

1 Answer

0 votes
by (33.2k points)

You can check following description about lr_policy (Learning rate decay):

// The learning rate decay policy. The currently implemented learning rate

// policies are as follows:

//    - fixed: always return base_lr.

//    - step: return base_lr * gamma ^ (floor(iter / step))

//    - exp: return base_lr * gamma ^ iter

//    - inv: return base_lr * (1 + gamma * iter) ^ (- power)

//    - multistep: similar to step but it allows non uniform steps defined by

//      stepvalue

//    - poly: the effective learning rate follows a polynomial decay, to be

//      zero by the max_iter. return base_lr (1 - iter/max_iter) ^ (power)

//    - sigmoid: the effective learning rate follows a sigmod decay

//      return base_lr ( 1/(1 + exp(-gamma * (iter - stepsize))))

//

// where base_lr, max_iter, gamma, step, stepvalue and power are defined

// in the solver parameter protocol buffer, and iter is the current iteration.

Hope this answer helps.

Welcome to Intellipaat Community. Get your technical queries answered by top developers !


Categories

...