Neural network training is about finding weights that minimize prediction error. In general, we start our training with a set of randomly generated weights. Then, backpropagation is used to update the weights in an attempt to correctly map arbitrary inputs to outputs.

Backpropagation, short for “backward propagation of errors”, is a mechanism used to update the weights using __gradient descent__. It calculates the gradient of the error function with respect to the neural network’s weights. The calculation proceeds backward through the network.