Derivation of the Largest Step Size
The best way to find the upper bound for is to write the equation that produces the weight values. Let us rewrite the
ideal performance surface (Eq.1B.10)as
(1B.11)
where
(1B.12)
By computing the gradient of J Eq.1 B.11 , we get
(1B.13)
so the iteration that produces the weight updates Eq.1.11 can be written as
(1B.14)
This is a first-order linear constant-coefficient difference equation, which
can be solved by induction. First, let us subtract w* from both sides to yield
Start with a solution w(0).
which provides by induction the equation