THEORY GENERATION, USING NON-MONOTONIC GAUSSIAN PROCESSING UNITS.

THEORY GENERATION, USING NON-MONOTONIC GAUSSIAN PROCESSING UNITS.

Keywords: Problem solving, network analysis, data mining, inference rules.

Istvan S. N. Berkeley, Ph.D.

Philosophy

The University of Southwestern Louisiana

One well known problem which has faced workers in the field of Artificial Neural Networks (ANNs) is the complexity which can arise in a network after it has learned to solve a problem through supervised training. Indeed, some (e.g. McCloskey, 1991) have gone as far as to suggest that ANNs may not be able to play a theoretically significant role in Cognitive Science, or other research areas, until the problem of analyzing trained networks in detail is over come. The ramifications of this difficulty are extremely wide reaching as they influence not only theoretical concerns, but also have implications for the deployment of ANNs in many domains, especially those in which ANNs function as integral parts of critical systems.

Istvan Berkeley Ph.D. is a faculty member in the Philosophy program at the University of Southwestern Louisiana (USL). He also works closely with researchers from various fields as part of the USL Cognitive Science Group. Berkeley received his Ph.D. from the University of Alberta in Edmonton, Alberta, Canada where he worked with the Biological Computation Project. His Ph.D. dissertation focussed upon the theoretical consequences of network interpretation and described in detail a novel method of network interpretation. His work has been published in both theoretical and technical journals.

In the current research project, feedforward networks which employ Dawson and Schopflocher's (1992) value units are being deployed. Value units have a non-monotonic Gaussian activation function and exhibit a number of attractive computational properties, such as faster learning and better generalization than traditional processing units employing sigmoidal activation functions. In addition, the nature of the value unit activation function makes these units especially ameanable to analysis and interpretation after training using a variant of the backpropogation learning procedure. At the current time, this analytic method hasw been applied to a logic problem, originally investigated by Bechtel and Abrahamsen, as well as a number of other standard problems (e.g. parity, majority). Details of some of this research are published in Berkeley, Dawson, Schopflocher, Medler and Hornsby (1995). The thrust of the current work is to extend and refine the analytic methodology, so as to provide a general purpose means by which trained networks of value units can be subject to analysis. This research is proceding by the training and analysis of networks trained upon a wide variety of tasks. The hope is that once the analytic method is fully developed, it may be possible to deploy value unit networks in domains where there is no detailed understanding of input/output relationships and then use the subsequent results of analysis to develop a theoretical understanding of those domains. Alternatively, value unit networks could be employed in an analogous manner to search for alternative solutions to well-known problems.

The Neurosolutions software package was selected for this research project, as it provided both the power and flexibility to make the research progress quickly, without undue delays due to technical limitations. In particular, the ability to be able to reconfigure the Neurosolutions software to implement non-standard activation functions was a crucial feature which led to it's adoption. In addition, the fact that the data generated by the Neurosolutions software easily and simply interfaced with software such as Microsoft Excel also made the process of network analysis considerably easier and faster than had previously been possible.

REFERENCES

Bechtel, W. and Abrahamsen, A. (1991) *Connectionism and the Mind*

Blackwells (Cambridge, Mass.).

Berkeley, I., Dawson, M., Schopflocher, D., Medler, D. and Hornsby, L.

(1995), "Density Plots of Hidden Unit Activations Reveal Interpretable

Bands", in *Connection Science* 7/2, pp. 167-186.

Dawson, M. and Schopflocher, D. (1992), "Modifying the Generalized Delta

Rule to Train Networks of Non-Monotonic Processors for Pattern

Classification", in *Connection Science* 4/1, pp. 19-31.

McCloskey, M. (1991), "Networks and Theories: The Place of Connectionism

in Cognitive Science", in *Psychological Science* 2/6, pp. 387-395.

Istvan S. N. Berkeley Ph.D, E-mail: istvan@USL.edu,

Philosophy, The University of Southwestern Louisiana,

USL P. O. Box 43770, Lafayette, LA 70504-3770, USA.

Tel:(318) 482 6807, Fax: (318) 482 6195, http://www.ucs.usl.edu/~isb9112

Download this Application Summary in Word97 format.
If you don't have Word97....Download the Word Viewer.