Q: Why is normalization needed?
In order for a neural network to function properly, the values of the input data and the desired output data must fall within the range of the processing element transfer functions. For TanhAxons this range is between -1 and +1 and for SigmoidAxons the range is between 0 and 1.
Q: Do I need to handle the normalization when building a neural network?
The wizards automatically configure the File components to normalize (i.e., transform) the data between the bounds of the transfer functions. They also configure the output probes to denormalize (i.e., reverse the transform of) the output data back to the range of the desired output file. There is usually no need to make additional changes, but these can be made after the network is built.
Q: Why does my network output fall outside the range of my desired output?
The NeuralBuilder configures the File components to normalize the data between the range of -0.9 and 0.9 (for TanhAxons). The networks tend to learn a little better when they don’t have to work from the “flat” part of the transfer function. The side effect of this is that the raw network outputs can be greater than 0.9 or less then -0.9, which will produce values outside of the range of the desired output when denormalized. The solution to this is to change the normalization range of the desired output to -1 and +1 (right-click on the desired File component, select “Properties”, then select the “Stream” tab, then change the “Upper” and “Lower” settings).
Q: How is the normalization computed?
There are two coefficients computed for each column of data – amplitude and offset. These normalization coefficients are computed based on the minimum and maximum values found across all of the data sets selected from the Data Set Inspector, using the following formula:
Amp(i) = (UpperBound - LowerBound) / (Max(i) - Min(i))
Off(i) = UpperBound - Amp(i) * Max(i)
where Max(i) and Min(i) are the maximum and minimum values found within column i, and UpperBound and LowerBound are the values entered within the Stream tab of the File inspector.
Q: Where are these coefficients stored?
They are stored within a normalization file (*.nsn). This file can be selected and viewed from the “Data Sets” tab of the File inspector. Each row of the file contains the two normalization coefficients for the respective column of data.
These coefficients are also stored within the weights file under the sections for the File components. This copy of the coefficients is used by the generated C++ code and the Custom Solution Wizard DLLs.
Q: Can I use my own coefficients?
Yes. Once NeuroSolutions has generated a default normalization file, simply edit the file and make the desired changes to the coefficients. You then need to check the “Read Only” switch to keep the normalization file from being overwritten with the default coefficients.
