NATIONAL TECHNICAL UNIVERSITY OF ATHENS SCHOOL OF NAVAL ARCHITECTURE & MARINE ENGINEERING Section of Naval & Marine Hydrodynamics Stochastic Analysis with Applications to Dynamical Systems by Themistoklis P. Sapsis Diploma Thesis Supervisor: G. A. Athanassoulis, Professor NTUA Athens September 2005 ii iii Preface This thesis deals with the analysis of multidimensional nonlinear dynamical systems subject to general external stochastic excitation. The method used to address this problem is based on the Characteristic Functional. As it is known, the characteristic functional includes the full probability structure of a stochastic process. Hence the knowledge of the characteristic functional induces the knowledge of every statistical quantity related with the stochastic process. For Stochastic Dynamical Systems, the joint Characteristic Functional of the response and the excitation is governed by linear Functional Differential Equations, i.e. differential equations with Volterra derivatives. This kind of equations appeared for the first time at the paper “Statistical Hydromechanics and Functional Calculus” by E. Hopf (1952) in the context of the analysis of Turbulence. Even thought such an equation contains the full probabilistic structure of the system response cannot be a straight forward method of solution of the problem. Apparently no practical methods have been found for the efficiently solution of this type of equations. The present work deals with two very important issues, concerning Functional Differential Equations that describe Stochastic Dynamical Systems. First, we show that apparently all known equations partially describing the probability structure of the response under specific assumptions (such as Fokker-Planck Equation, Liouville Equation, Moment Equations, etc) can be derived directly through the Functional Differential Equation. Additionally, we derive some new (to the best our knowledge) Partial Differential Equations governing the characteristic functions of the response for systems under general stochastic excitation. In a second level, a new method is proposed for the solution of Functional Differential Equations. This method is related with the concepts of Kernel Probability Measures and Kernel Characteristic Functionals in infinite dimensional spaces, which are defined and discussed extensively. In this way we propose a method for treating the Functional Differential Equations at the infinite dimensional level, without any reduction to PDE’s. A simplification of the proposed method is implemented and a numerical example is presented. This thesis is divided into five chapters. In Chapter 1 we briefly present the basic background needed for our study. We recall some basic elements of set theory as well as the notions of probability measure and probability space. In the last section of the chapter we review the concept of random variable and give a summary of some results concerning their characteristic properties. The first part of Chapter 2 presents the concept of stochastic process as well as an extensive discussion concerning the Kolmogorov Theorem. This discussion will be very useful for Chapter 3 where we will define the stochastic process using the measure theoretic approach. The second part of chapter 2 deals with some classifications of the stochastic processes. Stochastic processes are categorized with respect to their memory, their regularity and their ergodic properties. Chapter 3 deals with the stochastic processes through the measure theoretic approach. In the first two sections we extensively discuss the properties of a probability measure defined on a Banach space as well as its finite dimensional produced measures. In Sections 3.3-3.5 the characteristic functional is defined and studied. Especially we study the connection of characteristic functional with more conventional statistical quantities describing the stochastic processes, such as moments, iv characteristic functions etc. Finally, in the three last sections of the chapter, we study specific cases of probability measures/characteristic functionals, such as the Gaussian and the Poisson. Special attention has been given to the characteristic functional introduced by V. Tatarskii, which induces as special (or asymptotic) cases almost every known characteristic functional. Before we proceed to the general study of Stochastic Dynamical Systems, we present the topic of mean-square calculus. Although classical, it’s very important for a complete description of Stochastic Differential Equations. Among the topics we examine, of special interest is the presentation for the Integral of stochastic processes where we distinguish the case of integration of a stochastic process with respect to another independent stochastic process and the case of integration over Martingales with non-anticipative dependence. At the end of the chapter we present some results and criteria for the sample path properties of stochastic process. Finally in the beginning of Chapter 5 the notion of a stochastic differential equation is defined and some general theorems concerning the existence and uniqness of its solutions are proved. Moreover, necessary conditions for bounded of all moments of the probability measure describing the system response are given. Section 5.2 deals with the formulation of functional differential equations for the joint characteristic functional describing (jointly) the probability structure of the excitation and the response. The formulation concerns stochastic systems described by nonlinear ODE’s with polynomial nonlinearities. We distinguish between the case of a m.s.- continuous stochastic excitation and an orthogonal-increment excitation. Before the general results we present two specific systems, the Duffing oscillator and the van der Poll oscillator. In the next section 5.3 we prove how the general functional differential equation describing the probability structure of a system can produce a family of partial differential equations for the characteristic functions of various orders. Special cases are the Fokker-Planck equation, as well as the Liouville equation. Of great importance are the partial differential equations proved at Section 5.3.4 that describe the probability structure of the system for the case of m.s.-continuous excitation. In the last Section 5.5 we introduce the notion of the kernel characteristic functional that generalizes the corresponding idea of kernel density functions in infinite dimensional spaces. We prove that the Gaussian characteristic functional has the kernel property, thus being an efficient tool for the study of FDEs. The proof is based on existent theorems for extreme values of stochastic processes. In Section 5.4.4 we discuss how the kernel characteristic functionals can be efficiently used for the solution of functional differential equations. The basic idea is the localization in the physical phase space domain (probability measure) expressed in terms of the characteristic functional. It should be noted that the Gaussian measures are of great importance in this direction since most (if not all) of the existing analytical results concerning infinite dimensional integrals pertain Gaussian measures. In this way we derive a set of partial differential equations that governs the functional parameters of the kernel characteristic functionals. Finally, in Section 5.4.5, we a simplified set of equations is derived in the basis of specific assumptions concerning the probability interchange between probability kernels. The latter equations are applied for the numerical study of a simple nonlinear ODE. v Acknowledgments Acknowledgments are owed to everyone who helped and assisted me, each one on his/her own way, during the elaboration of this thesis. However, let me take this opportunity to thank some people individually. First, I would like to express my deep gratitude to the supervisor of this work, Prof. Gerassimos A. Athanassoulis, for his guidance, encouragement and support throughout the whole duration of my studies at the school of Naval Architecture and Marine Engineering of NTUA. I would like to thank him for bringing me in touch, through his inspiring teaching, with various mathematical problems and especially the examined one. This thesis could never have been written or even conceived without all the stimulating discussions that we had the last four years. The valuable contribution of Mr. Panagiotis Gavriliadis, PhD candidate, through the critical discussions as well as of Mr. Agissilaos Athanassoulis, PhD candidate, through his stimulating comments that led to a number of improvements, are greatly appreciated. Finally I have to thank my family Panagiotis, Euaggelia and Pantelis Sapsis as well as Kiriaki Kofiani, student of NTUA. This thesis could never have been taken the current form without their continuous and unconditional love and support. This thesis is dedicated to them with immense gratitude. Th. Sapsis September 2005 vi vii Contents Preface iii Acknowledgments v Contents vii List of Symbols xi Chapter 1: BACKGROUND CONCEPTS FROM PROBABILITY AND ANALYSIS 1.1. Elements of set theory 2 1.1.1. Set operations 2 1.1.2. Borel Fields 4 1.2. Axioms of Probability – Definition of Probability Measure 5 1.3. Random Variables 7 1.3.1. Distribution Functions and Density Functions 7 1.3.2. Moments 9 1.3.3. Characteristic Functions 10 1.3.4. Gaussian Random Variables and the Central Limit Theory 13 1.3.5. Convergence of a Sequence of Random Variables 16 1.3.5.a. Mean-Square Convergence 16 1.3.5.b.