Principal Component Analysis (PCA) networks are a mixture of unsupervised and supervised networks. Principal component analysis is a linear procedure to find the direction in input space where most of the energy of the input lies. In other words, PCA performs feature extraction. The projections of these components correspond to the eigenvalues of the input covariance matrix. The unsupervised segment of the network performs the feature extraction and the supervised segment of the network performs the (linear or nonlinear) classification of these features using a MLP.
The principal component analysis is performed first, and then the MLP is trained. The reason for this is that the PCA network trains faster when it does not have to share computing resources with the MLP, since there is no point in training the MLP until the eigenvalues are stable.
Principal component analysis is a well known method of orthogonalizing data. It converges very fast and the theoretical method is well understood. Since the features are orthogonal, the MLP is able to train easily. There are usually fewer features extracted than there are inputs, so the unsupervised segment provides a means of data reduction.
