Steps for Maximizing/Minimizing Output(s) in NeuroSolutions

This list of steps is useful for those applications where a system is needed to find a set of inputs that will maximize or minimize a single output or a set of outputs. It involves two phases of work: 1) Develop a neural network to model the real-world “plant” using the historical inputs and outputs, and 2) Fix the neural network weights and use genetic optimization to do an intelligent search of the input space to maximize or minimize the outputs. Note that these steps assume you are working with a static network (i.e., a problem type of classification or function approximation) that uses the Tanh function for the hidden layer(s). It is also worth noting that these steps can be automated by recording a Macro while performing them for the first time.

Steps

1. Create a neural network model for your historical data using either NeuroSolutions for Excel, the NeuralBuilder or the NeuralExpert and save the breadboard.
2. Train the network.
3. Test the network using the data you have set aside as your Testing set, to make sure the neural network model is a good representation of the real-world “plant”.
4. Save the breadboard.
5. Make a copy of the two normalization files (*.nsn) that NeuroSolutions created. These should be in the same directory as the breadboard. NeuroSolutions will automatically delete the original copies in a later step.
6. Right-click on the BackStaticControl and select “Properties”. Click the “Remove” button on the “Backpropagation” tab of the inspector. This will fix the network weights to those obtained from the training process.
7. If the Axon Components toolbar or the Control Components toolbar is not displayed, then select “Customize” from the “Tools” menu and check the “Axon Components” box and/or the “Control Components” box.
8. Left-click on the BiasAxon toolbar button and move your cursor on top of the input Axon (the Axon furthest to the left). Left-click to replace the input Axon with a BiasAxon.
9. Left-click on the GeneticControl toolbar button and move your cursor on top of the BackStaticControl. Left-click to stamp down a GeneticControl component.
10. Create a new input data file that has the exact same column headings as your training input file.
11. Add one row of data – all 0’s for each input. If your desired data is contained in the same file as your input data, then enter a -1 for each output you want to minimize and a +1 for each output you want to maximize. Save the file.
12. Right-click on the input File component and select “Properties”. Click the “Add” button and select the new input file you created. Leave the default data set as “Training” and click “OK”.
13. Select each of the files from the learning breadboard that are listed and click the “Remove” button. This should leave only your new file listed in the File List.
14. If your desired data is contained in a separate file from your input data, then create a new desired file that has the same column headings as your training desired file. Add one row of data: enter a -1 for each output you want to minimize and a +1 for each output you want to maximize. Save the file.
15. Repeat steps 12 & 13 for the desired File component.
16. Switch to the “Stream” tab of the desired File inspector and uncheck the “Normalize” checkbox. Repeat for the input File component.
17. Stamp a DataGraph probe on top of the GeneticControl component, open the inspector for the probe and change the “Access” to “Best Fitness”. Double-click the probe to open the window. This will show the cost during the optimization. Note that the cost is based on the difference between the optimal output (the maximum or minimum value given in the desired output file) and the output produced by the system for the given set of inputs. This “Best Fitness” should decrease during the optimization process.
18. Open the inspector for the BiasAxon and switch to the “Genetic” tab. Check the “weightValue” box and change the “Lower Bound” to -1.
19. Open the inspector for the StaticControl and change the “Active Data Set” to “Training” and change the “Cross Validation Data Set” to “None”.
20. Change the “Epochs/Run” to 1.
21. Open the inspector for the GeneticControl component and check the “Save Best Weights” box.
22. Click the “Zero Count” toolbar button, followed by the “Start” button.
23. After the optimization has completed, open the GeneticControl inspector and click the “Load Best Parameters” button.
24. Stamp a MatrixViewer on the right side of the BiasAxon. Open the inspector and switch to the “Probe” tab. Click the “Denormalize” checkbox and select the copy of the input normalization file that you saved in step #5.
25. Double-click on the MatrixViewer icon to open the probe window.
26. Open the GeneticControl inspector and uncheck the “Enable Optimization” button.
27. Repeat step #22 to feed the data into the network.
28. Observe the optimal inputs shown in the MatrixViewer window.
29. To view the output value(s) produced by those inputs, add a probe to the network output (“Activity” access point of the Axon to the left of the L2Criterion), denormalize it using the desired normalization file saved in step #5 and run the data through the network again.
30. Save the breadboard under a different name (i.e., don’t overwrite the original breadboard used to train the weights in case you need to modify the model).