Activation
Error Criterion
Gradient Search
Input
Probe

Adaptive Step DLL

The adaptive step size algorithm has individual step sizes, one for each weight. Each step size is increased whenever the gradient component relative to that weight has the same sign in two consecutive epochs. Otherwise, the step size is decreased. The error control continues to monitor the average cost. If this cost goes up, three actions are taken: 1) backtrack to the set of weights where the lowest cost was found, 2) sgnificantly decrease all step sizes, and 3) set all momentum memories to zero. A theoretical description can be found in: Silva F. and Almeida L., "Speeding up backpropagation," Advanced Neural Computers, (ed. Eckmiller), pp 151-160.

To see the Adaptive Step Size DLL in action, please run the NeuroSolutions Demos (within the Help menu), select "Dynamic Link Library Demos", click "Run", then select "Adaptive Step Size" and click "Run".

Download zip file (0.1M)