Meaning of an Epoch in Neural Networks Training

Question

while I'm reading in how to build ANN in pybrain, they say:

Train the network for some epochs. Usually, you would set something like 5 here,

trainer.trainEpochs( 1 )

I looked for what is that mean, then I conclude that we use an epoch of data to update weights, If I choose to train the data with 5 epochs as pybrain advice, the dataset will be divided into 5 subsets, and the wights will update 5 times as maximum.

I'm familiar with online training where the wights are updated after each sample data or feature vector, My question is how to be sure that 5 epochs will be enough to build a model and setting the weights probably? what is the advantage of this way on online training? Also, the term "epoch" is used on online training, does it mean one feature vector?

Answer 1

One epoch in neural networks consists of one full training cycle iteration on the training set. One epoch tries to iterate over every sample once in the set, then it starts again, marking the beginning of the 2nd epoch.

Epoch has no relation with batch or online training. Batch training means weights will get updated at the end of the epoch. Online training means that weights will be updated after each sample.

You can never be sure about the number of epochs should be used. You can stop training when the error converges or gets lower than a certain threshold. You can learn about early stopping and cross-validation regarding that.

Hope this answer helps.

Learn Neural Network with the help of this Neural Network Tutorial.