Faculty of Technology and Science Department of Physics and Electrical Engineering Paluri Suraj and Patluri Sandeep A Study of Impulse Response System Identification Degree Project of 10 credit points Master Program in Electrical Engineering Date/Term: 07-06-14 Supervisor: Magnus Mossberg Examiner: Magnus Mossberg Karlstads universitet 651 88 Karlstad Tfn 054-700 10 00 Fax 054-700 14 60 [email protected] www.kau.se A Study of Impulse Response System Identification Abstract: In system identification, different methods are often classified as parametric or non- parametric methods. For parametric methods, a parametric model of a system is considered and the model parameters are estimated. For non-parametric methods, no parametric model is used and the result of the identification is given as a curve or a function. One of the non-parametric methods is the impulse response analysis. This approach is dynamic simulation. This thesis introduces a new paradigm for dynamic simulation, called impulse-based simulation. This approach is based on choosing a Dirac function d (t) as input, and as a result, the output will be equal to the impulse response. However, a Dirac function cannot be realized in practice, and an approximation has to be used. As a consequence, the output will deviate from the impulse response. Once the impulse response is estimated, a parametric model can be fitted to the estimation. This thesis aims to determine the parameters in a parametric model from an estimated impulse response. The process of investigating the models is a critical aspect of the project. Correlation analysis is used to obtain the weighting function from the estimates of covariance functions. Later, a relation formed between the parameters and the estimates (obtained by correlation analysis) in the form of a linear system of equations. Furthermore, simulations are carried out using Monte Carlo for investigating the properties of the two step approach, which involves in correlation analysis to find h-parameters and least squares and total least squares methods to solve for the parameters of the model. In order to evaluate the complete capability of the approach to the noise variation a study of signal to noise ratio and mean, mean square error and variances of the estimated parameters is carried out. The results of the Monte Carlo study indicate that two-step approach can give rather accurate parameter estimates. In addition, the least squares and total least squares methods give similar results. 1 ACKNOWLEDGEMENT This thesis project is a compulsory part of the International Master’s programs that the authors attend, and it leads to the degree of Master of Science in Electrical Engineering. First of all, we would like to take this opportunity to express our sincere gratitude to our supervisor Dr.Magnus Mossberg for giving the opportunity of doing our Master Thesis Project and have given us invaluable expertise throughout the project. In addition, we would also like to thank Dr.Andreas Jakobsson for his invaluable support and kind consideration. Finally, we would like to thank our parents, friends and well wishers for their moral and unconditional support. 2 Contents 1 Introduction 5 1.1 System Identification 5 1.2 System 5 1.3 Model Structure 5 1.4 Modelling a dynamic system 6 1.5 Types of models 6 1.6 System Identification Procedure 7 2 Types of System Identification Methods 8 2.1 Parametric Methods 8 2.2 Least Squares 8 2.3 Prediction Error Method (PEM) 8 2.4 Instrumental Variable Method 9 2.5 Total Least Squares 9 3 Non-Parametric Methods 10 3.1 Transient Analysis 10 3.1.1 Step Response 10 3.1.2 Impulse Response 10 3.2 Freqency Analysis 12 3.3 Correlation Analysis 13 3.4 Spectral Analysis 15 4 Estimation of Parameters 16 4.1 Task1 16 4.2 Task 2 17 4.3 Task 3 19 4.4 Task 4 20 4.5 Two-Step Approach 21 5 Monte Carlo 23 5.1 Simulation and Results 26 Conclusions 35 References 36 Appendix 37 3 Overview: The purpose of this thesis is to develop a model which has a weighting function that coincides with an estimated sequence. A parametric model is determined from the impulse response for this study. Methods such as parametric and non parametric way of estimation are used. To conduct this study we use correlation technique among the non- parametric methods. This leads to estimating the weighting function. Initially, we introduce system identification and its importance in science and engineering. Also, basic model structures involved are discussed. Later, different types of methods in system identification such as parametric and non-parametric methods are explained in detail. Our interest is on impulse response system identification, which is a non-parametric method. Later, a series of tasks involving polynomial formulation and impulse response are undergone to effectively estimate the parameters. A Monte Carlo study is done to investigate the properties of the two-step procedure, which involves correlation analysis for finding the h-parameters. These h-parameters are plugged into a linear system of equation of the form AX=B. Later least squares and total least squares are performed to solve for X (will be studied in chapter 4 and 5). Further investigations of the parameters include impulse based simulations modeled with increased noise and the plots for signal to noise (SNR) ratio versus mean, mean square error (MSE) and variances of the estimated parameters in order to achieve a model that is best suited to the system. 4 1. Introduction: In control and systems engineering, system identification methods are used to get appropriate models for design of prediction algorithm, or simulation. In signal processing applications, models obtained by system identification are used for spectral analysis, fault detection, pattern recognition, adaptive filtering, linear prediction and other purposes. In recent times, system identification techniques have delivered powerful methods and tools for data-based system modeling. While the theory for identification of linear systems has become very mature, the challenges for the field are in modeling and developing of more complex dynamical systems (nonlinear, distributed, hybrid, large scale), and in task-oriented issues (identification for control, diagnosis etc).They also have wide applications in non-technical fields such as biology, environmental sciences and econometrics to develop models for increasing scientific knowledge on the identified object, or for prediction and control. 1.1 System Identification: System identification deals with the problem of constructing the mathematical model of a dynamic system based on a given set of experimental data. Several fundamental concepts about system identification are introduced in this section. The procedures of building the estimated model by means of system identification technique are also given. 1.2 System: To denote a mathematical description of a process we use the word system. In reality that provides the experimental data will generally be referred to as a process. In order to perform a theoretical analysis of the identification methods it is necessary to introduce assumptions on the data. As it is not necessary to know the system, we will use the system concept only for investigating of how different identification methods behave under different circumstances. 1.3 Model Structure: In many cases while dealing with problems related to system identification we deal with parametric models, such models are characterized by a parameter vector, which we denote by q .The corresponding model will be denoted M(q ).When q is varied over a set of feasible values we obtain a model structure. Sometimes, we use non-parametric models such a model is described by a curve, function or table. An impulse response is an example. It is curve or function which carries some information about the characteristics of the system, which we later on study in detail. 5 1.4 Modeling a Dynamic System: A system is an object in which variables of different kinds interact and produce observable signals. The observable signals that are of interest to us are usually called outputs. The system is also affected by external stimuli. External signals that can be manipulated by the observer are called inputs and others are called disturbances, which are further divided into those that are directly measured and those that are only observed through their influence on the output. The distinction between input and measured disturbance is of less importance for the modeling process. System identification is a field of modeling dynamic systems from experimental data. A Dynamic system is described in the following figure. v (t) w (t) y (t) u (t) Figure 1.1 A Dynamic Model This system has an input u(t), output y(t), measured disturbance w(t), unmeasured disturbance v(t). For a dynamic system the control at time t will influence the time instants s>t. 1.5 Types of Models: The purpose of a model is to describe how the various variables of the system relate to each other. The relationship among these variables is called a model of the system. Modeling the system of interest is considerably useful in many areas of science as an aid to properly describe the system’s behavior. A good model must reflect all properties of 6 such an unknown system. There are several kinds of models which can be classified as follows: · Mental models which do not involve a mathematical formula. For instance, when driving a car, one is required the knowledge of turning the wheel, pushing the brake, etc. · Graphical models which are also suitable for some certain systems using numerical tables and/or plots. For example, any linear systems can be uniquely described by their impulse, step, or frequency response which can be obviously represented in graphical forms.