Greetings
from NeuroDimension!
Makers
of NeuroSolutions, the Neural Network Simulation Environment.
This issue of the
newsletter provides insights on several techniques for using neural networks and
announces a new neural network course offering.
In
this issue you’ll find:
What’s News?
* New Neural Network Course Offering
Designing Neural
Networks
* Generalization and Data Sets
Did You Know?
* Time-Series Prediction using NeuroSolutions for Excel: Part 2
Customer Spotlight
* Anticipate and Localize Faults within Telecommunication Networks
Note: You are receiving this newsletter because you requested to stay informed concerning new developments at NeuroDimension. If you would like to stop receiving these newsletters, please see the bottom of this newsletter for removal instructions.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
What’s News?
NeuroDimension has scheduled a new course offering for May 8-12, 2000 in Orlando, Florida at the Doubletree Guest Suites Hotel in the Walt Disney World Resort.
Our course format allows both novice and advanced users to find a suitable course. Offered courses include: "Introduction to NeuroSolutions", "Fundamentals of Neural Networks and NeuroSolutions", and "Advanced NeuroSolutions".
The courses include a copy of our new textbook, Neural and Adaptive Systems: Fundamentals Through Simulations. We are also happy to work with attendees who would like to use their own data in the sample projects.
Our previous courses continue to receive rave reviews from our attendees. In fact, everyone has said that they would recommend our courses to friends. We average a score of 4.6 of 5 when attendees are asked to rate the overall quality of the course.
Remember to sign-up early and receive a 10% early registration discount.
For details on this new offering, or to sign-up from the internet, see http://www.nd.com/course/may.htm
For general ND course information, see http://www.nd.com/course
For more information and samples of the interactive book,
see http://www.nd.com/products/nsbook.htm
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
Designing Neural Networks
One of the primary goals in training neural networks is to ensure that the network will perform well on data that it has not been trained on (called “generalization”). The standard method of ensuring good generalization is to divide your training data into multiple data sets. The most common data sets are the training, cross validation, and testing data sets. While the training data set is the data that is actually used to train the network the usage of the other two may need some explanation.
Like the training data set, the cross validation data set is also used by the network during training. Periodically, while training on the training data set, the network is tested for performance on the cross validation set. During this testing, the weights are not trained, but the performance of the network on the cross validation set is saved and compared to past values. If the network is starting to overtrain on the training data, the cross validation performance will begin to degrade. Thus, the cross validation data set is used to determine when the network has been trained as well as possible without overtraining (e.g. maximum generalization).
Although the network is not trained with the cross validation set, it uses the cross validation set to choose a “best” set of weights. Therefore, it is not truly an out-of-sample test of the network. For a true test of the performance of the network the testing data set is used. This data set is used to provide a true indication of how the network will perform on new data.
A common question is how much data should be set aside for cross validation and testing sets. The answer really depends on the amount of data you have. If you have a small amount of data (in neural network terms, a small data set is less than 1000 exemplars), you need to use as much data as possible in your training set, thus your cross validation and testing sets should be small (say 10% each). In this case, however, you may not get a true indication of the ability of your network to generalize. If you have a great deal of data, then you may want to dedicate 20% to cross validation and 50% to testing.
One important pointer: make sure that the data in your training set, cross validation set, and test sets have similar characteristics. For instance, if your classification data is ordered by class and you select the last 20% of the file to be the test set, you may only get one class of information in your test set which is a very poor indicator of the performance of your network.
The Neural Wizard and NeuroSolutions for Excel both provide easy mechanisms for creating training, cross validation and testing data sets.
For more information on this topic, see the interactive book, Section 4.7, Network Size and Generalization.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
Did
You Know?
The first article in this two part series discussed one way in which to use NeuroSolutions for Excel to perform time series prediction (see Volume 1, Issue 9). This method involved pre-processing (shifting) the time series data in Excel to produce multiple lags of the inputs. A standard multilayer perceptron (MLP) was then used to perform the modeling.
The second method uses a focused time delay neural network (TDNN). In this method, the inputs are lagged by the neural network (the first layer, a TDNNAxon, is just a tap delay line which automatically lags the inputs). The details of this alternative method are described below:
Method 2: Using a TDNN
1. Starting with a single labeled column of data (the time series you want to predict), copy this column of data (select the data only, not the whole column or the label) and paste it into the adjacent column to the right. Paste the column one cell higher than the original column. This will serve as your desired column (set up to predict one period in advance).
2. Add a label to the new column overwriting the data value that exists in the first row.
3. Tag the original column as input and the new column as desired.
4. Delete the last row of data since the input column will have one extra value.
5. Tag the rows of data as training, cross validation, and/or testing.
6. Create a TDNN using the NeuralWizard. Use the defaults for all options except the following: Neural Model = Time-Lag Recurrent Network, Focused = checked, Memory = TDNNAxon, Depth in samples = 3.
7. Train and test the neural network as usual.
Here is an example of what the prepared data will look like when starting with a single column of data consisting of the numbers 1 through 7 (the time series).
x(0) x(1)
-----------------------------
1 2
2 3
3 4
4 5
5 6
6 7
7 8
This article has shown you how to prepare data for time series prediction for use with a TDNN neural network. In this method, the input data is lagged by the TDNNAxon in the first layer of the neural network, rather than being pre-lagged within the Excel spreadsheet.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
S. Rigg, J. Tindle, S. Brewis
University of Sunderland
BT London
The application of IT technologies that make use of the telephone network is dramatically increasing. There is a growing demand to rapidly detect and repair faulty telecommunication lines. This paper identifies a method to anticipate and localize possible fault areas within the Local Access Copper Networks (LACNs). The system employs a proactive approach to identify potential faults within the network. Therefore it is possible to replace plant before an actual fault state occurs. The paper outlines a typical network topology, overview of artificial neural networks, current anticipation and localization procedures and suggests an artificial intelligent technique for a new anticipation and localization method. Ultimately these methods may lead to tools being developed to aid an expert and provide better guidance for field maintenance engineers to replace faulty plant.
NOTE: This is just an abstract of the application summary. The entire summary is available at:
http://www.nd.com/application%20summaries/appsum-lacn.htm
Want to have your solutions spotlighted? We strongly encourage our customers to send their 1-2 page application summaries to submissions@nd.com so that we may post them on our web site at: http://www.nd.com/applicationsum.htm. We frequently spotlight solutions in our newsletters and include a link for people to get more information.
=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=~=
We appreciate your feedback! Please send us your comments
or suggestions concerning this newsletter, our web site, or part of the
NeuroDimension product line. Write to us at: feedback@nd.com
Have questions about NeuroDimension products or training services? Send your questions to: info@nd.com
This issue and previous issues of this newsletter are available on the NeuroDimension web site at: http://www.nd.com/mailinglist.htm
If you would prefer not to receive these newsletters or subsequent product updates from NeuroDimension, please reply to this letter with the subject heading changed to the word REMOVE.
Thank you again for your support of NeuroDimension products!