Derivation of the Time Constant of Adaptation

Writing exp(-1/NEURAL AND ADAPTIVE SYSTEMS00090012.gif)= NEURAL AND ADAPTIVE SYSTEMS00090004.gif and expanding the exponential in Taylor series,

NEURAL AND ADAPTIVE SYSTEMS00000144.gif

we get approximately NEURAL AND ADAPTIVE SYSTEMS00090004.gif~1-1/NEURAL AND ADAPTIVE SYSTEMS00090012.gif. We saw that geometric ratio of the gradient descent is Eq.1.17 so we get

NEURAL AND ADAPTIVE SYSTEMS00000145.gif

Use your browser's back button to return to text.