Explore Courses Blog Tutorials Interview Questions
0 votes
in Machine Learning by (19k points)

I just try to find out how I can use Caffe. To do so, I just took a look at the different .prototxt files in the examples folder. There is one option I don't understand:

# The learning rate policy

lr_policy: "inv"

Possible values seem to be:

  • "fixed"

  • "inv"

  • "step"

  • "multistep"

  • "stepearly"

  • "poly"

Could somebody please explain those options?

1 Answer

0 votes
by (33.1k points)

You can check following description about lr_policy (Learning rate decay):

// The learning rate decay policy. The currently implemented learning rate

// policies are as follows:

//    - fixed: always return base_lr.

//    - step: return base_lr * gamma ^ (floor(iter / step))

//    - exp: return base_lr * gamma ^ iter

//    - inv: return base_lr * (1 + gamma * iter) ^ (- power)

//    - multistep: similar to step but it allows non uniform steps defined by

//      stepvalue

//    - poly: the effective learning rate follows a polynomial decay, to be

//      zero by the max_iter. return base_lr (1 - iter/max_iter) ^ (power)

//    - sigmoid: the effective learning rate follows a sigmod decay

//      return base_lr ( 1/(1 + exp(-gamma * (iter - stepsize))))


// where base_lr, max_iter, gamma, step, stepvalue and power are defined

// in the solver parameter protocol buffer, and iter is the current iteration.

Hope this answer helps.

Welcome to Intellipaat Community. Get your technical queries answered by top developers!

29.3k questions

30.6k answers


104k users

Browse Categories