Greetings from
NeuroDimension!
Makers of NeuroSolutions,
the Neural Network Simulation Environment.
This brief issue of the
newsletter highlights our upcoming neural network courses in Orlando while we
prepare for the release of several exciting new products in the second quarter
of 2000, to be announced soon.
In this issue you’ll find:
What’s News?
* Deadline Approaching for Neural Network Course Discount
NeuroSolutions Tip
Box
* Using Weight-Decay to Improve Generalization
Note: You are receiving this newsletter because you requested to stay informed concerning new developments at NeuroDimension. If you would like to stop receiving these newsletters, please see the bottom of this newsletter for removal instructions.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
What’s News?
The deadline for an early registration discount is approaching soon for our May neural network courses in Orlando. The courses will take place May 8-12, 2000 in Orlando, Florida at the Doubletree Guest Suites Hotel in the Walt Disney World Resort. The deadline to qualify for a 10% discount is April 1.
Our course format allows both novice and advanced users to find a suitable course. Offered courses include: "Introduction to NeuroSolutions", "Fundamentals of Neural Networks and NeuroSolutions", and "Advanced NeuroSolutions".
The courses include a copy of our new textbook, Neural and Adaptive Systems: Fundamentals Through Simulations. We are also happy to work with attendees who would like to use their own data in the sample projects.
Our previous courses continue to receive rave reviews from our attendees. In fact, everyone has said that they would recommend our courses to friends. We average a score of 4.6 of 5 when attendees are asked to rate the overall quality of the course.
For details on the May courses, or to sign-up from the internet, see http://www.nd.com/course/may.htm
For general ND course information, see http://www.nd.com/course
For more information and samples of the interactive book, see http://www.nd.com/products/nsbook.htm
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
NeuroSolutions Tip Box
One of the points emphasized during our neural network course is the importance of generalization. A neural network is not going to be very useful if it “memorizes” the training set and does a poor job of modeling an out-of-sample test set. One cause of this poor generalization is overtraining. Overtraining can be prevented by stopping the training (or at least saving the network weights) when the error of the cross validation set begins to rise.
Another cause of poor generalization is using a neural
network that has too many free parameters (weights). A good rule of thumb is to
have a system that has only 1 weight for every 10 exemplars (input/output
pairs) of training data. One way to reduce the number of weights in the neural
network is using a technique called weight decay. Weight
decay subtracts a small amount from each weight each time it is updated. If the
weight is not relevant, it eventually moves to zero.
A weight decay algorithm is
available to all users of NeuroSolutions in the form of a public DLL. To use
this DLL you simply need to perform the following steps:
1. Download the DLL from http://www.nd.com/dll/WeightDecay.htm
and unzip the file.
2. Replace all GradientSearch components
(e.g., Momentum) with Step components.
3. From the Engine page of the
inspector for each Step component, load the DLL.
The decay rate of the weight decay
algorithm can be adjusted from the DLL page of the inspector. Higher decay
rates result in the weights decaying faster, causing a smoother (more general) output.
Lower decay rates decrease this effect, with a decay rate of zero being
equivalent to the normal Step component.
This is just an example of the
type of functionality that can be implemented using DLL’s. Many components,
like the GradientSearch components, can be replaced with DLL’s to implement new
functionality. The neural network course also covers this as an advanced topic.
For more information on weight decay
and similar algorithms, see the interactive book, Section 4.7, Network
Size and Generalization.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
We appreciate your feedback! Please send us your comments or
suggestions concerning this newsletter, our web site, or part of the
NeuroDimension product line. Write to us at: feedback@nd.com
Have questions about NeuroDimension products or training services? Send your questions to: info@nd.com
This issue and previous issues of this newsletter are available on the NeuroDimension web site at: http://www.nd.com/mailinglist.htm
If you would prefer not to receive these newsletters or subsequent product updates from NeuroDimension, please reply to this letter with the subject heading changed to the word REMOVE.
Thank you again for your support of NeuroDimension products!