The on-line and batch modes are slightly different, although both will perform
well for parabolic performance surfaces. One major difference is that the
batch algorithm keeps the system weights constant while computing the error
associated with each sample in the input. Since the on-line version is constantly
updating its weights, its error calculation (and thus gradient estimation) uses
different weights for each input sample. This means that the two algorithms visit
different sets of points during adaptation. However, they both converge to the
same minimum.
Note that the number of weight updates of the two methods for the same number
of data presentations is very different. The on-line method (LMS) does an
update each sample, while batch does an update each epoch, that is,
LMS updates = (batch updates) x (# of samples in training set).
The batch algorithm is also slightly more efficient in terms of number of
computations.
Use your browser's back button to
return to text.