Nonlinear system identification

Uses your system voice; voice and rate in settings.

Snapshot

≈ 11 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

System identification is a method of identifying or measuring the mathematical model of a system from measurements of the system inputs and outputs. The applications of system identification include any system where the inputs and outputs can be measured and include industrial processes, control systems, economic data, biology and the life sciences, medicine, social systems and many more. A nonlinear system is defined as any system that is not linear, that is any system that does not satisfy the superposition principle. This negative definition tends to obscure that there are very many different types of nonlinear systems. Historically, system identification for nonlinear systems has developed by focusing on specific classes of system and can be broadly categorized into five basic approaches, each defined by a model class: # Volterra series models, # Block-structured models, # Neural network models, # NARMAX models, and # State-space models. There are four steps to be followed for system identification: data gathering, model postulate, parameter identification, and model validation. Data gathering is considered as the first and essential part in identification terminology, used as the input for the model which is prepared later. It consists of selecting an appropriate data set, pre-processing and processing.…

System identification is a method of identifying or measuring the mathematical model of a system from measurements of the system inputs and outputs. The applications of system identification include any system where the inputs and outputs can be measured and include industrial processes, control systems, economic data, biology and the life sciences, medicine, social systems and many more. A nonlinear system is defined as any system that is not linear, that is any system that does not satisfy the superposition principle. This negative definition tends to obscure that there are very many different types of nonlinear systems. Historically, system identification for nonlinear systems has developed by focusing on specific classes of system and can be broadly categorized into five basic approaches, each defined by a model class: Volterra series models, Block-structured models, Neural network models, NARMAX models, and State-space models. There are four steps to be followed for system identification: data gathering, model postulate, parameter identification, and model validation. Data gathering is considered as the first and essential part in identification terminology, used as the input for the model which is prepared later. It consists of selecting an appropriate data set, pre-processing and processing. It involves the implementation of the known algorithms together with the transcription of flight tapes, data storage and data management, calibration, processing, analysis, and presentation. Moreover, model validation is necessary to gain confidence in, or reject, a particular model. In particular, the parameter estimation and the model validation are integral parts of the system identification. Validation refers to the process of confirming the conceptual model and demonstrating an adequate correspondence between the computational results of the model and the actual data. y(k)& = h_0+\sum\limits_{m_1=1}^M h_1(m_1)u(k-m_1) + \sum\limits_{m_1=1}^M \sum\limits_{m_2=1}^M h_2(m_1,m_2)u(k-m_1)u(k-m_2) \\ & {}\quad{}+\sum\limits_{m_1=1}^M \sum\limits_{m_2=1}^M \sum\limits_{m_3=1}^M h_3(m_1,m_2,m_3)u(k-m_1)u(k-m_2)u(k-m_3) + \cdots \end{align} where u(k), y(k); k = 1, 2, 3, ... are the measured input and output respectively and h_\ell(m_1,\ldots ,m_\ell) is the lth-order Volterra kernel, or lth-order nonlinear impulse response. The Volterra series is an extension of the linear convolution integral. Most of the earlier identification algorithms assumed that just the first two, linear and quadratic, Volterra kernels are present and used special inputs such as Gaussian white noise and correlation methods to identify the two Volterra kernels. In most of these methods the input has to be Gaussian and white which is a severe restriction for many real processes. These results were later extended to include the first three Volterra kernels, to allow different inputs, and other related developments including the Wiener series. A very important body of work was developed by Wiener, Lee, Bose and colleagues at MIT from the 1940s to the 1960s including the famous Lee and Schetzen method. While these methods are still actively studied today there are several basic restrictions. These include the necessity of knowing the number of Volterra series terms a priori, the use of special inputs, and the large number of estimates that have to be identified. For example, for a system where the first order Volterra kernel is described by say 30 samples, 30x30 points will be required for the second order kernel, 30x30x30 for the third order and so on and hence the amount of data required to provide good estimates becomes excessively large. These numbers can be reduced by exploiting certain symmetries but the requirements are still excessive irrespective of what algorithm is used for the identification. y(k) & =F[y(k-1),y(k-2),\ldots ,y(k-n_y),u(k-d),u(k-d-1),\ldots ,u(k-d-n_u), \\ & {}\quad e(k-1),e(k-2),\ldots ,e(k-n_e)]+e(k) \end{align} where y(k), u(k) and e(k) are the system output, input, and noise sequences respectively; n_y , n_u , and n_e are the maximum lags for the system output, input and noise; F[•] is some nonlinear function, d is a time delay typically set to d = 1.The model is essentially an expansion of past inputs, outputs and noise terms. Because the noise is modelled explicitly, unbiased estimates of the system model can be obtained in the presence of unobserved highly correlated and nonlinear noise. The Volterra, the block structured models and many neural network architectures can all be considered as subsets of the NARMAX model. Since NARMAX was introduced, by proving what class of nonlinear systems can be represented by this model, many results and algorithms have been derived based around this description. Most of the early work was based on polynomial expansions of the NARMAX model. These are still the most popular methods today but other more complex forms based on wavelets and other expansions have been introduced to represent severely nonlinear and highly complex nonlinear systems. A significant proportion of nonlinear systems can be represented by a NARMAX model including systems with exotic behaviours such as chaos, bifurcations, and subharmonics. While NARMAX started as the name of a model it has now developed into a philosophy of nonlinear system identification,. The NARMAX approach consists of several steps: Structure detection: which terms are in the model Parameter estimation: determine the model coefficients Model validation: is the model unbiased and correct Prediction: what is the output at some future time Analysis: what are the dynamical properties of the system Structure detection forms the most fundamental part of NARMAX. For example, a NARMAX model which consists of one lagged input and one lagged output term, three lagged noise terms, expanded as a cubic polynomial would consist of eighty two possible candidate terms. This number of candidate terms arises because the expansion by definition includes all possible combinations within the cubic expansion. Naively proceeding to estimate a model which includes all these terms and then pruning will cause numerical and computational problems and should always be avoided. However, only a few terms are often important in the model. Structure detection, which aims to select terms one at a time, is therefore critically important. These objectives can easily be achieved by using the Orthogonal Least Squares algorithm and its derivatives to select the NARMAX model terms one at a time. These ideas can also be adapted for pattern recognition and feature selection and provide an alternative to principal component analysis but with the advantage that the features are revealed as basis functions that are easily related back to the original problem. NARMAX methods are designed to do more than find the best approximating model. System identification can be divided into two aims. The first involves approximation where the key aim is to develop a model that approximates the data set such that good predictions can be made. There are many applications where this approach is appropriate, for example in time series prediction of the weather, stock prices, speech, target tracking, pattern classification etc. In such applications the form of the model is not that important. The objective is to find an approximation scheme which produces the minimum prediction errors. A second objective of system identification, which includes the first objective as a subset, involves much more than just finding a model to achieve the best mean squared errors. This second aim is why the NARMAX philosophy was developed and is linked to the idea of finding the simplest model structure. The aim here is to develop models that reproduce the dynamic characteristics of the underlying system, to find the simplest possible model, and if possible to relate this to components and behaviours of the system under study. The core aim of this second approach to identification is therefore to identify and reveal the rule that represents the system. These objectives are relevant to model simulation and control systems design, but increasingly to applications in medicine, neuro science, and the life sciences. Here the aim is to identify models, often nonlinear, that can be used to understand the basic mechanisms of how these systems operate and behave so that we can manipulate and utilise these. NARMAX methods have also been developed in the frequency and spatio-temporal domains. x_{t+1} &= f(x_t, u_t, w_t; \theta),\\ y_t &= g(x_t, u_t, v_t; \theta), \quad t =1,2, \dots \end{aligned} in which t is a positive integer referring to time. The functions f and g are general nonlinear functions. The first equation is known as the state equation and the second is known as the output equation. All the signals are modeled using stochastic processes. The process x_t is known as the state process, w_t and v_t are usually assumed independent and mutually independent such that w_t \sim p(w; \theta),\; v_t \sim p(v; \theta) . The parameter \theta is usually a finite-dimensional (real) parameter to be estimated (using experimental data). Observe that the state process does not have to be a physical signal, and it is normally unobserved (not measured). The data set is given as a set of input-output pairs (y_t, u_t) for t = 1, \dots, N for some finite positive integer value N . Unfortunately, due to the nonlinear transformation of unobserved random variables, the likelihood function of the outputs is analytically intractable; it is given in terms of a multidimensional marginalization integral. Consequently, commonly used parameter estimation methods such as the Maximum Likelihood Method or the Prediction Error Method based on the optimal one-step ahead predictor are analytically intractable. Recently, algorithms based on sequential Monte Carlo methods have been used to approximate the conditional mean of the outputs or, in conjunction with the Expectation-Maximization algorithm, to approximate the maximum likelihood estimator. These methods, albeit asymptotically optimal, are computationally demanding and their use is limited to specific cases where the fundamental limitations of the employed particle filters can be avoided. An alternative solution is to apply the prediction error method using a sub-optimal predictor. The resulting estimator can be shown to be strongly consistent and asymptotically normal and can be evaluated using relatively simple algorithms.

References

Further reading

Lennart Ljung: System Identification — Theory For the User, 2nd ed, PTR Prentice Hall, Upper Saddle River, N. J., 1999. R. Pintelon, J. Schoukens, System Identification: A Frequency Domain Approach, IEEE Press, New York, 2001. T. Söderström, P. Stoica, System Identification, Prentice Hall, Upper Saddle River, N.J., 1989. R. K. Pearson: Discrete-Time Dynamic Models. Oxford University Press, 1999. P. Marmarelis, V. Marmarelis, V. Analysis of Physiological Systems, Plenum, 1978. K. Worden, G. R. Tomlinson, Nonlinearity in Structural Dynamics, Institute of Physics Publishing, 2001.

Text derived from https://en.wikipedia.org/w/index.php?oldid=1313903988 under CC BY-SA 4.0; modifications have been made.

Provenance

snapshot enwiki-001313903988 · captured 1759078050.0 · version history

Cite this page as: encyclopegia.org, “Nonlinear system identification”, snapshot enwiki-001313903988. https://encyclopegia.org/a/nonlinear-system-identification

Sources cited in this revision

8 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.