Derivation of Normal Equations

When the derivative of J with respect to the unknown quantities (the weights) is taken, we end up with a set of D+1 equations in D+1 unknowns:

NEURAL AND ADAPTIVE SYSTEMS00000146.gif (1B.15)

The solution is the famous normal matrix equation

NEURAL AND ADAPTIVE SYSTEMS00000147.gif (1B.16)

or expanding

NEURAL AND ADAPTIVE SYSTEMS00000148.gifNEURAL AND ADAPTIVE SYSTEMS00000149.gif (1B.17)

Notice that these equations are linear in the unknowns (the wj), so they can be easily solved.

Return to text

Use your browser's back button to return to text.