General References
Bankruptcy Prediction
Bond Rating
Chemical Engineering
Communications
Financial Forecasting
Industrial Applications
Marketing/Scheduling
Medical Diagnosis
OCR
Robotics
Speech Recognition

Application Summaries
Neural Net Reference
Company History
Customer Quotes
Customer Forum
Mailing List
Reviews
Search
FAQ

Books

Core Research Papers

Hints
Author: Abu-Mostafa, Yaser S.
Source: Neural Computation 7, pp. 639-671, 1995
Abstract: The systematic use of hints in the learning-from-examples paradigm is the subject of this review. Hints are the properties of the target function that are known to us independently of the training examples. The use of hints is tantamount to combining rules and data in learning, and is compatible with different learning models, optimization techniques, and regularization techniques. The hints are represented to the learning process by virtual examples, and the training examples of the target function are treated on equal footing with the rest of the hints. A balance is achieved between the information provided by the different hints through the choice of objective functions and learning schedules. The Adaptive Minimization algorithm achieves this balance by relating the performance on each hint to the overall performance. The application of hints in forecasting the very noisy foreign-exchange markets is illustrated. On the theoretical side, the information value of hints is contrasted to the complexity value and related to the VC dimension.

Prediction Risk and Architecture Selection for Neural Networks
Author: Moody, John
Source: From Statistics to Neural Networks: Theory and Pattern Recognition Applications, V. Cherkassky, J.H. Friedman and H. Wechsler (eds.), NATO ASI Series F, Springer-Verlag, 1994
Abstract: We describe two important sets of tools for neural network modeling: prediction risk estimation and network architecture selection. Prediction risk is defined as the expected performance of an estimator in predicting new observations. Estimated prediction risk can be used both for estimating the quality of model predictions and for model selection. Prediction risk estimation and model selection are especially important for problems with limited data. Techniques for estimating prediction risk include data resampling algorithms such as nonlinear cross--validation (NCV) and algebraic formulae such as the predicted squared error (PSE) and generalized prediction error (GPE). We show that exhaustive search over the space of network architectures is computationally infeasible even for networks of modest size. This motivates the use of heuristic strategies that dramatically reduce the search complexity. These strategies employ directed search algorithms, such as selecting the number of nodes via sequential network construction (SNC) and pruning inputs and weights via sensitivity based pruning (SBP) and optimal brain damage (OBD) respectively. Click to receive a postscript copy of the paper via FTP

Trading Committees: A Comparative Study
Authors: Rehfuss, Steve; Wu, Lizhong; Moody, John
Source: Proc.3rd Int'l. Conf. Neural Networks in the Capital Markets '95, pp. 11-13, London, England, 1995
Abstract: Combining experts by averaging their forecasts can be useful for prediction in environments where the individual experts are noisy. In decision-making situations such as trading systems, another possibility is to use voting committees, where committee members first decide individually, and then the individual decisions are used to produce the final decision. We compare combining forecasts with three types of voting using a trading system developed for the INFFC competition.Click to receive a postscript copy of the paper via FTP

A Smoothing Regularizer for Feedfoward and Recurrent Neural Networks
Authors: Wu, Lizhong; Moody, John
Source: Neural Computation, Vol. 7, 1995
Abstract: We derive a smoothing regularizer for dynamic network models by requiring robustness in prediction performance to perturbations of the training data. The regularizer can be viewed as a generalization of the first order Tikhonov stabilizer to dynamic models. We have successfully tested this regularizer in a number of case studies and found that it performs better than standard quadratic weight decay. Click to receive a postscript copy of the paper via FTP

Economic Forecasting: Challenges and Neural Network Solutions
Authors: Moody, John
Source: Keynote talk presented at the International Symposium on Artificial Neural Networks, Hsinchu, Taiwan, December 1995.
Abstract: Macroeconomic forecasting is a very difficult task due to the lack of an accurate, convincing model of the economy. The most accurate models for economic forecasting, ``black box'' time series models, assume little about the structure of the economy. Constructing reliable time series models is challenging due to short data series, high noise levels, nonstationarities, and nonlinear effects. This paper describes these challenges and surveys some neural network solutions to them. Important issues include balancing the bias/variance tradeoff and the noise/nonstationarity tradeoff. The methods surveyed include hyperparameter selection (regularization parameter and training window length), input variable selection and pruning, network architecture selection and pruning, new smoothing regularizers, and committee forecasts. Empirical results are presented for forecasting the U.S. Index of Industrial Production. These demonstrate that, relative to conventional linear time series and regression methods, superior performance can be obtained using state-of-the-art neural network models.Click to receive a postscript copy of the paper via FTP

Fast Pruning Using Principal Components
Authors: Levin, Asriel U.; Leen, Todd K.; Moody, John E.
Source: Advances in Neural Information Processing 6, J. Cowan, G. Tesauro, and J. Alspector (eds.), Morgan Kaufmann, San Mateo, CA, 1994
Abstract: We present a new algorithm for eliminating excess parameters and improving network generalization after supervised training. The method, "Principal Components Pruning (PCP)", is based on principal component analysis of the node activations of successive layers of the network. It is simple, cheap to implement, and effective. It requires no network retraining, and does not involve calculating the full Hessian of the cost function. Only the weight and the node activity correlation matrices for each layer of nodes are required. We demonstrate the efficacy of the method on a regression problem using polynomial basis functions, and on an economic time series prediction problem using a two-layer, feedforward network. Click to receive a postscript copy of the paper via FTP

Active Research Sites

Neural Network Publications at the Electronics Institute

Products | Support | Order | Download | Search | Contact

Product questions? Contact info@nd.com Website questions? Contact webmaster@nd.com
Web Site Design and Implementation Copyright © 2002 NeuroDimension, Inc.