University Neural Networks Course

Application Category: Educational.

Keywords: Neural Networks course, Practical exercises, Simulation.

University Name: Instituto Superior Técnico, Universidade Técnica de Lisboa, Lisbon, Portugal.

Department: Electrical and Computer Engineering.

Contact Emails:
mailto:lba@inesc.pt
mailto:ajmb@isr.ist.utl.pt

Summary:

The Department of Electrical and Computer Engineering, Instituto Superior Técnico, offers a course on Neural Networks for graduate and undergraduate students. During the course, there are 8 lab assignments, consisting of practical problems that students must solve using neural networks. Each problem must be solved during one lab session (3 hours). The problem description is available one week before the lab session, so that it can be adequately prepared. The problems are solved using NeuroSolutions version 2, in evaluation mode or with breadboards semi-prepared by the instructor. NeuroSolutions is also used for some practical demos presented during the lectures.

The course is lectured by Luís B. Almeida and Alexandre Bernardino. Luís B. Almeida is a full professor at the Instituto Superior Técnico, in the fields of Neural Networks and Signal Processing. He is also head of the Neural Networks and Signal Processing Group at INESC, Lisbon, Portugal. Alexandre Bernardino is a teaching assistant at the Instituto Superior Técnico and a research assistant at the ISR, Lisbon, Portugal. He is also a graduate student at the Instituto Superior Técnico. His main interests in the field of Neural Networks are their biological aspects and their applications to Vision and Control.

There are 8 lab sessions during the Neural Networks course:

1 – Emulation of Logic Functions. Students try to emulate the AND, OR and XOR functions using linear networks and the delta rule. They notice that the XOR can’t be emulated using single-unit networks and use a very simple nonlinear two-layer network trained with backpropagation. They observe the influence of step and momentum parameters on the training speed.

2 – Digit recognition. A multilayer perceptron trained with backpropagation is used to classify binary images of the digits 0-9. A set of images is given to the students and they have to prepare the training and test sets. Cross-validation and stopping criteria are important topics in this session. Two custom DLLs are used: ADPTSTEP.DLL implements the adaptive step size algorithm; PERCENT.DLL computes the percentage of right and wrong classifications.

3 – Image Restoration. Two sets of images (original and blurred) are given to the students. The purpose of this session is to develop a system to enhance the blurred images. First, a linear network is used and the results are compared to Wiener filtering. Then, nonlinear networks are used and the results are compared to the linear methods. The students have to define the topology and training methods for the network. The custom DLL DISCRIM.DLL is used to convert arrays of data into images.

4 – Recurrent Neural Networks. As a first experiment, a linear recurrent network is trained with input and output data obtained from a first order low-pass IIR filter. Students must find the adequate topology for the network and, after training, discover the filter coefficients. The training method is backpropagation through time. In the second experiment, the purpose is to emulate an RS flip-flop. Topology, training data and training methods are selected by the students.

5 – Statistical aspects of Neural Networks. MLPs are used to estimate the sample mean, variance, median and quantiles of data sets and functions. The interpretation of the outputs of an MLP, trained for classification purposes, as estimates of a posteriori probabilities of the classes is explicitly addressed. Three custom DLLs are used in this session: NORMAL.DLL generates a 1-D Gaussian function; MULT.DLL multiplies two values and L1ASYM.DLL implements an "absolute value" type of error criterion but with adjustable slopes for positive and negative errors.

6 – Radial Basis Functions. Supervised classification of bidimensional data is performed using an MLP and a radial basis function network. The two methods must be compared. Special emphasis is given to the selection of the number of radial basis functions and to the unsupervised training of their centers and variances.

7 - Competitive Networks. Two problems are addressed in this session: clustering and vector quantization of 2D data. Both standard competitive learning and "conscience" competitive learning are used. For vector quantization, Kohonen maps are also used. Students evaluate the importance of adaptation step, conscience parameters and neighborhood dimension.

8 – Hopfield Networks. Hopfield Networks are used both as associative memories and for solving optimization problems. In the first set of experiments, two 3x3 binary patterns are memorized in a discrete Hopfield network, and the trajectories of a 2D continuous network are visualized to illustrate the convergence to stable points from different initial conditions. In the second experiment, a continuous Hopfield network is used to solve the traveling salesman problem with four cities. The students must define the weights for the network. Two custom DLLs are used in this session: HOPFIELD.DLL implements a continuous type of update for the elements of the network; DISCHOPF.DLL implements an asynchronous update of discrete units.

Download this Application Summary in Word97 format.
If you don't have Word97....Download the Word Viewer.