Jordan and Elman networks extend the multilayer perceptron with context units, which are PEs that remember past activity. Context units are required when learning patterns over time (i.e., when the past value of the network influences the present processing).
The context units can be locally recurrent (i.e., they feedback onto themselves). The local recurrence decreases the values by a multiplicative constant t (time constant) as they are fed back. This constant determines the memory depth (i.e., how long a given value fed to the context unit will be "remembered"). Context units (neurons that self excite) are very common in the brain.
