Time-Lag recurrent networks (TLRNs) are MLPs extended with short term memory structures that have local recurrent connections. The TLRN is a very appropriate model for processing temporal (time-varying) information. Examples of temporal problems include time series prediction, system identification and temporal pattern recognition. The training algorithm used with TLRNs (Backpropagation Through Time) is more advanced than standard Backpropagation.
The main advantage of TLRNs is the smaller network size required to learn temporal problems when compared to MLPs that use extra inputs to represent the past samples (equivalent to time delay neural networks). An added advantage of TLRNs is their low sensitivity to noise. The recurrence of the TLRN provides the advantage of an adaptive memory depth (i.e., it finds the best duration to represent the input signal's past).
From a system identification point of view, TLRNs implement nonlinear moving average (NMA) models. With global feedback from the output to the hidden layer, they can be extended to nonlinear autoregressive moving average (ARMA) models. These nonlinear models can be used for optimal control applications, surpassing the performance of their linear counterparts.
