Authors: Moody, John; Utans, Joachim
Source: Neural Networks in the Capital Markets, Refenes A.N. (ed.), John Wiley Sons, 1994
Click to receive a postscript copy of the paper via FTP
We propose strategies for selecting a good neural network architecture for modeling any specific data set. Our approach involves efficiently searching the space of possible architectures and selecting a ``best'' architecture based on estimates of generalization performance. Since an exhaustive search over the space of architectures is computationally infeasible, we propose heuristic strategies which dramatically reduce the search complexity. These employ directed search algorithms, including selecting the number of nodes via sequential network construction (SNC), sensitivity based pruning (SBP)} of inputs, and optimal brain damage (OBD) pruning for weights. A selection criterion, the estimated generalization performance or prediction risk, is used to guide the heuristic search and to choose the final network. Both predicted squared error (PSE) and nonlinear cross--validation (NCV) are used for estimating the prediction risk from the available data. We apply these heuristic search and prediction risk estimation techniques to the problem of predicting corporate bond ratings. This problem is very attractive as a case study, since it is characterized by a limited set of data and by the lack of a complete a priori model which could be used to impose a structure to the network architecture.