Derivation of the Largest Step Size

The best way to find the upper bound for NEURAL AND ADAPTIVE SYSTEMS00090002.gif is to write the equation that produces the weight values. Let us rewrite the ideal performance surface (Eq.1B.10)as

NEURAL AND ADAPTIVE SYSTEMS00000137.gif (1B.11)

where

NEURAL AND ADAPTIVE SYSTEMS00000138.gif (1B.12)

By computing the gradient of J Eq.1 B.11 , we get

NEURAL AND ADAPTIVE SYSTEMS00000139.gif (1B.13)

so the iteration that produces the weight updates Eq.1.11 can be written as

NEURAL AND ADAPTIVE SYSTEMS00000140.gif (1B.14)

This is a first-order linear constant-coefficient difference equation, which can be solved by induction. First, let us subtract w* from both sides to yield

NEURAL AND ADAPTIVE SYSTEMS00000141.gif

Start with a solution w(0).

NEURAL AND ADAPTIVE SYSTEMS00000142.gif

which provides by induction the equation

NEURAL AND ADAPTIVE SYSTEMS00000143.gif

Use your browser's back button to return to text.