2.6 Dynamic Neural Networks
Techniques to estimate a system process from observed data fall under the general category of system identification. Figure 2.9 illustrates the concept of a system.
Figure 2.9. A system with input signal u, disturbance signal e, and output signal y.
Loosely speaking, a system is an object in which different kinds of signals interact and produce an observable output signal. A system may be a real physical entity, such as an engine, or entirely abstract, such as the stock market.
There are three types of signals that characterize a system, as indicated in Figure 2.9. The output signal y(t) of the system is an observable/measurable signal, which you want to understand and describe. The input signal u(t) is an external measurable signal, which influences the system. The disturbance signal e(t) also influences the system but, in contrast to the input signal, it is not measurable. All these signals are time dependent.
In a singleinput, singleoutput (SISO) system, these signals are timedependent scalars. In the multiinput, multioutput (MIMO) systems, they are represented by timedependent vectors. When the input signal is absent, the system corresponds to a timeseries prediction problem. This system is then said to be noise driven, since the output signal is only influenced by the disturbance e(t).
The Neural Networks package supports identification of systems with any number of input and output signals.
A system may be modeled by a dynamic neural network that consists of a combination of neural networks of FF or RBF types, and a specification of the input vector to the network. Both of these two parts have to be specified by the user. The input vector, or regressor vector, which it is often called in connection with dynamic systems, contains lagged input and output values of the system specified by three indices: , and . For a SISO model the input vector looks like this:
Index represents the number of lagged output values; it is often referred to as the order of the model. Index is the input delay relative to the output. Index represents the number of lagged input values. In a MIMO case each individual lagged signal value is a vector of appropriate length. For example, a problem with three outputs and two inputs ={1,2,1}, ={2,1}, and ={1,0} gives the following regressor:
For timeseries problems, only has to be chosen.
The neural network part of the dynamic neural network defines a mapping from the regressor space to the output space. Denote the neural network model by g(, ) where is the parameter vector to be estimated using observed data. Then the prediction (t) can be expressed as
Models with a regressor like Eq. (2.0) are called ARX models, which stands for AutoRegressive with eXtra input signal. When there is no input signal u(t), its lagged valued may be eliminated from Eq. (2.0), reducing it to an AR model. Since the mapping g(, ) is based on neural networks, the dynamic models are called neural ARX and neural AR models, or neural AR(X) as short form for both them. Figure 2.10 shows a neural ARX model, based on a onehiddenlayer FF network.
Figure 2.10. A neural ARX model.
The special case of an ARX model where no lagged outputs are present in the regressor (that is, when =0 in Eq. (2.0)), is often called a Finite Impulse Response (FIR) model.
Depending on the choice of the mapping g( , ) you obtain a linear or a nonlinear model using an FF network or an RBF network.
Although the disturbance signal e(t) is not measurable, it can be estimated once the model has been trained. This estimate is called the prediction error and is defined by
A good model that explains the data well should yield small prediction errors. Therefore, a measure of (t) may be used as a modelquality index.
System identification and timeseries prediction examples can be found in Section 8.2 and Section 12.2, Prediction of Currency Exchange Rate.
