Vito Volterra (1860-1940) was one of the founding fathers of functional analysis. At the turn of the twentieth century, he introduced the notion of functions of lines, which are defined over a functional space. He studied their derivative and obtained results on integral equations. Practically, a Volterra series is a polynomial functional expansion similar to a Taylor series that provides an approximation of weakly nonlinear systems. One of the first application to nonlinear system analysis is due to Wiener in the 1940s, who developed a method for determining the nonlinear response to a white noise input. Nowadays, this approach is widely used for system identification in many domains such as electrical engineering or biological sciences.
Given an input space of signals , an output space of signals , and a nonlinear input-output operator , we look for a polynomial approximation of . The Volterra series provide such an approximation (under certain conditions) as follows :
where is the input, is the output, and each is called the n-th order Volterra kernel. This is equivalent to the operator formulation
where each is called the n-th order Volterra operator
The first order operator coincides with the standard convolution
whereas the second order operator can be considered as a two-fold convolution
and so on for higher order operators.
The formulation of the Volterra kernels in the frequency domain is obtained by applying the Fourier transform, namely
The response to a multi-sinusoidal input is
We deduce the formulation in the frequency domain
The mathematical foundations of the Volterra series can be established from the Stone-Weierstrass theorem. We give the general principle below following the approach of the Rugh’s book.
Basically, the approximation of continuous functions by polynomials is established by the Weierstrass theorem.
Every continuous function can be uniformly approximated by polynomials such that
The uniform topology ensures that the limit of continuous functions is a continuous function. However, uniform approximation of continuous functions by polynomials is not always possible. For example, the function cannot be uniformly approximated by polynomials over , essentially because is bounded and is not compact, whereas non constant polynomials are not bounded over . In contrast, on a compact domain, polynomials are bounded and often useful to approximate continuous functions.
In fact, the set of polynomials is a subalgebra of the Banach algebra of real valued continuous functions over . Thus, the Weierstrass theorem is equivalent to say that is dense in . More generally, we have the Stone-Weierstrass theorem.
Let be a compact Hausdorff space, and a subalgebra of which contains the constant function and separates the points. Then is dense in .
The key aspect of this theorem is that must separate the points. This means that if are distinct then there exists such that . Otherwise, no function can set different values to and . For example, the algebra of constant functions does not separate the points when there are at least two points, it is not dense in . A polynomial algebra, however, does separate the points.
In order to be able to use the Stone-Weierstrass theorem, the operator must be defined as a real valued continuous function on a compact space. Thus, we choose as a compact space. Let be a space of square integrable functions satisfying the two following properties :
(1) There exists a constant such that for all
(2) For all , there exists such that for all and
The resulting space is compact, the proof is given in Liusternik and Sobolev.
Although is not a real valued function, it can be seen as such in the following way. Let be the space of real valued continuous functions over with norm
Let and be two continuous, stationary and causal operators such that for all
There exists such that
Hence, when posing on and on
Therefore, and can be seen as real valued functions , it suffices to consider and at . The majoration is valid for all .
The last step consists of defining the algebra of continuous, stationary and causal operators . We take as generators the operator
and all operators
The algebra is obtained by repeated addition, scalar multiplication and multiplication of the generators, which leads to operators
The stationarity and causality of operators is trivial. The continuity is obtained if we assume that is square integrable over , moreover the algebra does separate the points (see Rugh). The Stone-Weierstrass theorem concludes that a continuous, stationary, causal system can be approximated by a continuous, stationary, causal polynomial system . However, the compactness of is quite restrictive in practice.
Consider the nonlinear system
for an input built as superposition of pure frequencies 0.1, 0.4 , 1 , 1.7 , 2.9 , 5.2 , 6.7 , 8.9 , 13.2 , 16.4. The input is shown below
The following figure shows the first order approximation (in green) versus the second order approximation (in red) of the output (in blue). Higher order contributions have been neglected. Clearly, the accuracy increases with the number of Volterra kernels.
- The Volterra and Wiener Theories of Nonlinear Systems, Martin Schetzen, Krieger Publishing Company, 2006
- Theory of Functionals and of Integral and Integro-Differential Equations, Vito Volterra, Dover Publications, 2005
- Nonlinear System Theory, Wilson J.Rugh, The Johns Hopkins University Press, 1981, Web version, 2002
- Elements of Functional Analysis, Liusternik and Sobolev, 1961