A Dynamical Systems Approach to Modeling Input-Output Systems Martin Casdagli Santa Fe Institute, 1120 Canyon Road Santa Fe, New Mexico 87501
Total Page:16
File Type:pdf, Size:1020Kb
A Dynamical Systems Approach to Modeling Input-Output Systems Martin Casdagli Santa Fe Institute, 1120 Canyon Road Santa Fe, New Mexico 87501 Abstract Motivated by practical applications, we generalize theoretical results on the nonlinear modeling of autonomous dynamical systems to input-output systems. The inputs driving the system are assumed to be observed, as well as the out puts. The underlying dynamics coupling inputs to outputs is assumed to be deterministic. We give a definition of chaos for input-output systems, and develop a theoretical framework for state space reconstruction and modeling. Most of the results for autonomous deterministic systems are found to gener alize to input-output systems with some modifications, even if the inputs are stochastic. 1 Introduction There has been much recent interest in the nonlinear modeling and prediction of time series data, as evidenced by this conference proceedings volume. For approaches mo tivated by deterministic chaos, see [2, 5, 7, 8] and references therein. For approaches motivated by nonlinear stochastic models, see [15] and references therein. In both these approaches, it is assumed that a time series x(t) is obtained from observations of an autonomous dynamical system, possibly perturbed by unobserved forces or noise. By contrast, when modeling input-output systems, in addition to an observed output time series x(t), there is also available an observed input time series u(t); see Figure 1. Modeling input-output systems is appropriate in many applications. For example, in vibration testing, a mechanical device may be subjeeted to a controlled random forcing to test its robustness in a simulated environment. Also, in scientific experi ments, one may be interested in the response of a system to various forms of stimulus. There are many other disciplines, for example meteorology and economics, in which pairs of input-output time series from input-output systems are available for analysis. In this paper we will develop a theory for the nonlinear modeling of input-output systems based on deterministic dynamics. This deterministic approach assumes that the input-output time series arises from a finite dimensional dynamical system ds dt = f(s{t),u(t)) (1) x(t) = h(s(t)) (2) 1 Input u(t) Unknown Output x(t) System Figure 1: Conceptual model of a single input-single output system where s(t) E ~d denotes a d-dimensional state, and for simplicity we take u(t) E ~ to be a scalar input, f : ~d X ~ --> ~d to be a smooth flow, and h : ~d --> ~ to be a scalar measurement function. It is then natural to attempt to model and forecast the behavior of the input-output system with a nonlinear deterministic model of the form x(t) = P(x(t - r), x(t - 2r), .. , x(t - mr), u(t), u(t - r), .. ,u(t - (1- l)r)), (3) where P is a nonlinear function fitted to the input-output time series data. This deterministic approach has been applied by Hunter to a variety of practical examples [11]. Of course, the deterministic system (1,2) is only an approximation to reality. Firstly, it is assumed that only d independent modes of the system are excited to a significant amplitude, where in practice d is reasonably small. Secondly, it is assumed that effects due to unobserved sources of noise and measurement errors are small enol.\gh to be ignored. If these assumptions are violated, a stochastic approach to nonlinear modeling may be more appropriate, and it is natural to include noise terms in the above equations. For a stochastic approach to the nonlinear modeling of input output systems, see Billings [1]. Under the above deterministic assumptions, we are interested in the following theoretical questions. Firstly, how should chaos be defined and quantified for the system (I)? If the input time series u(t) is periodic, this question trivially reduces to that of autonomous systems by considering the appropriate time-T Poincare map. However, we are mostly interested in the case where u(t) is a random time series. If u(t) is random, then x(t) is random, so neither time series by itselfis chaotic. Hence this must be a question about the structure of the pair of time series u(t), x(t). We will address this question is Section 2. Secondly, if we wish to construct a nonlinear deterministic model of the form (3), how many lags m and I should be chosen for a system of dimension d? In the case of autonomous systems, this question has been addressed by Takens [14]; see also Sauer et al. [13]. We are also interested in how accurate such a model is likely to be as a function of the length of the time series available to construct it, and the dimension d of the underlying dynamical system. In the case of autonomous systems, this question has been addressed by Farmer and Sidorowich [7]; see also Casdagli [2]. We will investigate these questions for input output systems in Section 3. Finally, we summarize the conclusions in Section 4. 2 2 Chaos in Input-Output Systems In this section we give a definition of chaos for input-output systems. We also in vestigate the usefulness of this definition in quantifying predictability for a numerical example. For simplicity we will assume that time is discrete, so that Equation (1) is replaced by Sn+l = j(Sn' un) (4) Xn+l = h( Sn+l) (5) All of the results in this section generalize naturally to continuous time input-output systems by using results about Liapunov exponents in continuous time autonomous systems; see [6, 16J. 2.1 Definition of the largest Liapunov exponent Suppose the system (4) is initialized at two slightly different states, and in both cases is subjected to the same sequence of inputs Un' Then if the system states diverge exponentially in time, we say the input-output system is chaotic, and the rate of divergence is given by the largest Liapunov exponent. The largest Liapunov exponent quantifies the degree to which the system is predictable in the long term, assuming that the input time series is always observed. This observability assumption is satisfied in many of the applications mentioned in the introduction. For example, in vibration testing, one may only be able to observe the state of the system at rare intervals in the past due to measurement problems, and desire to predict the present state of the system given a sequence of inputs to the system. The assumption of having the input sequence available is also relevant for problems of reducing noise on output sequences observed in the past. Note that if the input time series is random and unobserved, the system is unpredictable even in the short term. We now make the above notions more precise. We will only be concerned with the largest Liapunov exponent, which is defined as follows. Let D j(s, u) denote the derivative of j at S with u held constant. Let D jT denote the matrix product T-1 DjT(so,uO,u,UT_1) = II Dj(Si,Ui) (6) i;:;O where the Si are generated from (4) starting from a given initial state so. Then the largest Liapunov exponent A, is defined by (7), where ds is an arbitrary initial vector, and II . II denotes the Euclidean norm. A, = lim .!:..log(11 DFds II / II ds II) (7) T~oo T If Al > 0 we say the system is chaotic. 3 The above definition of the Liapunov exponent Al at first sight depends on the initial state So, the sequence of inputs uo, Ul, .. and the tangent vector ds. However, suppose that the sequence of inputs is drawn from a realization of a stationary ran dom or deterministic process. Then in the case of random inputs, by multiplicative ergodic theorems (see [6]), the limit (7) exists with probability one, and depends only on the ergodic invariant measure to which the initial state So is attracted. This in variant measure is often unique, for example in the case of Gaussian inputs. In the case of deterministic inputs, although there are no general theorems, it is observed numerically that the limit (7) exists, and depends only on the basin of attraction in which So lies. The above definition coincides with the definition of Liapunov exponents for ran domly driven dynamical systems with unobserved inputs [6]. In this case, the input is assumed to be small, and models noise perturbations. However, if the inputs are observed, the largest Liapunov exponent may be used to describe the divergence of trajectories even for large amplitude inputs as follows. Let So and s~ denote two close initial conditions, subjected to the same sequence Uo, .. , UT_l of inputs. Then the divergence of trajectories will be described for moderate T by (8) Hence using (7) we obtain (9) We now illustrate the above ideas with the randomly driven Ikeda map where f is taken to be f(x,y,u) = 1 + a(xcost -ysint) +u,a(xsint +ycost) (10) 2 where t = 0.4 - 6.0/(1 + x + y2), a = 0.7 and the inputs Un are independently identically distributed (IID) Gaussians with variance 1]2. Figure 2 illustrates the de pendence of the Liapunov exponent Al on the noise level 1]. The Liapunov exponent was computed numerically using 10 5 iterates for each value of 1], with the QR al gorithm described in [6] to avoid overflow. Observe that a smooth transition from chaotic to non-chaotic behavior occurs at 1] "'" 0.95. Unlike autonomous systems, it is impossible to locate this transition by inspection of the invariant measure, which in randomly driven systems is always smooth.