Wavelet Methods and System Identification
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Happy 80Th, Ulf!
From SIAM News, Volume 37, Number 7, September 2004 Happy 80th, Ulf! My Grenander Number is two, a distinction that goes all the way back to 1958. In 1951, I wrote a paper on an aspect of the Bergman kernel function with Henry O. Pollak (who later became head of mathematics at the Bell Telephone Laboratories). In 1958, Ulf Grenander, Henry Pollak, and David Slepian wrote on the distribution of quadratic forms in normal varieties. Through my friendship with Henry, I learned of Ulf’s existence, and when Ulf came to Brown in 1966, I felt that this friendship would be passed on to me. And so it turned out. I had already read and appreciated the second of Ulf’s many books: On Toeplitz Forms and Their Applications (1958), written jointly with Gabor Szegö. Over the years, my wife and I have lived a block away from Ulf and “Pi” Grenander. (Pi = π = 3.14159...) We have known their children and their grandchildren. And I shouldn’t forget their many dogs. We have enjoyed their hospitality on numerous occasions, including graduations, weddings, and Lucia Day (December 13). We have visited with them in their summer home in Vastervik, Sweden. We have sailed with them around the nearby islands in the Baltic. And yet, despite the fact that our offices are separated by only a few feet, despite the fact that I have often pumped him for mathematical information and received mathematical wisdom in return, my Grenander Number has never been reduced to one. On May 7, 2004, about fifty people gathered in Warwick, Rhode Island, for an all-day celebration, Ulf Grenander from breakfast through dinner, in honor of Ulf’s 80th birthday. -
Prediction Error Methods and Pseudo-Linear Regressions
System Identification Lecture 10: Prediction error methods and pseudo-linear regressions Roy Smith 2016-11-22 10.1 Prediction e k p q H z p q v k y k p q u k p q G z p q ` p q Typical assumptions G z and H z are stable, p q p q H z is stably invertible (no zeros outside the unit disk) p q e k has known statistics: known pdf or known moments. p q One-step ahead prediction Given ZK u 0 , y 0 , . , u K 1 , y K 1 , “ t p q p q p ´ q p ´ qu what is the best estimate of y K ? p q 2016-11-22 10.2 Prediction v k e k p q H z p q p q Noise model invertibility Given, v k , k 0,...,K 1, can we determine e k , k 0,...,K 1? p q “ ´ p q “ ´ 8 Inverse filter: Hinv z : e k hinv i v k i p q p q “ i 0 p q p ´ q ÿ“ We also want the inverse filter to be causal and stable: 8 hinv k 0, k 0, and hinv k . p q “ ă k 0 | p q| ă 8 ÿ“ If H z has no zeros for z 1, then, p q | | ě 1 Hinv z . p q “ H z p q 2016-11-22 10.3 Prediction v k e k p q H z p q p q One step ahead prediction Given measurements of v k , k 0,...,K 1, can we predict v K ? p q “ ´ p q Assume that we know H z , how much can we say about v K ? p q p q Assume also that H z is monic (h 0 1). -
10. Prediction Error Methods (PEM)
EE531 - System Identification Jitkomut Songsiri 6. Prediction Error Methods (PEM) • description • optimal prediction • examples • statistical results • computational aspects 6-1 Description idea: determine the model parameter θ such that "(t; θ) = y(t) − y^(tjt − 1; θ) is small y^(tjt − 1; θ) is a prediction of y(t) given the data up to time t − 1 and based on θ general linear predictor: y^(tjt − 1; θ) = N(L; θ)y(t) + M(L; θ)u(t) where M and N must contain one pure delay, i.e., N(0; θ) = 0;M(0; θ) = 0 example: y^(tjt − 1; θ) = 0:5y(t − 1) + 0:1y(t − 2) + 2u(t − 1) Prediction Error Methods (PEM) 6-2 Elements of PEM one has to make the following choices, in order to define the method • model structure: the parametrization of G(L; θ);H(L; θ) and Λ(θ) as a function of θ • predictor: the choice of filters N; M once the model is specified • criterion: define a scalar-valued function of "(t; θ) that will assess the performance of the predictor we commonly consider the optimal mean square predictor the filters N and M are chosen such that the prediction error has small variance Prediction Error Methods (PEM) 6-3 Loss function let N be the number of data points sample covariance matrix: 1 XN R(θ) = "(t; θ)"T (t; θ) N t=1 R(θ) is a positive semidefinite matrix (and typically pdf when N is large) loss function: scalar-valued function defined on positive matrices R f(R(θ)) f must be monotonically increasing, i.e., let X ≻ 0 and for any ∆X ⪰ 0 f(X + ∆X) ≥ f(X) Prediction Error Methods (PEM) 6-4 Example 1 f(X) = tr(WX) where W ≻ 0 is a weighting matrix f(X + ∆X) = tr(WX) + tr(W ∆X) ≥ f(X) (tr(W ∆X) ≥ 0 because if A ⪰ 0;B ⪰ 0, then tr(AB) ≥ 0) Example 2 f(X) = det X f(X + ∆X) − f(X) = det(X1/2(I + X−1/2∆XX−1/2)X1/2) − det X = det X[det(I + X−1/2∆XX−1/2) − 1] " # Yn −1/2 −1/2 = det X (1 + λk(X ∆XX )) − 1 ≥ 0 k=1 −1/2 −1/2 the last inequalty follows from X ∆XX ⪰ 0, so λk ≥ 0 for all k both examples satisfy f(X + ∆X) = f(X) () ∆X = 0 Prediction Error Methods (PEM) 6-5 Procedures in PEM 1. -
5 Identification Using Prediction Error Methods
5 Identification using Prediction Error Methods The previous chapters have shown how the dynamic behaviour of a civil engineering structure can be modelled by a discrete-time ARMAV model or an equivalent stochastic state space realization. The purpose of this chapter is to investigate how an estimate of a p-variate discrete-time model can be obtained from measurements of the response y(tkk). It is assumed that y(t ) is a realization of a Gaussian stochastic process and thus assumed stationary. The stationarity assumption implies that the behaviour of the underlying system can be represented by a time-invariant model. In the following, the general ARMAV(na,nc), for na nc, model will be addressed. In chapter 2, it was shown that this model can be equivalently represented by an m- dimensional state space realization of the innovation form, with m na#p. No matter how the discrete-time model is represented, all adjustable parameters of the model are assembled in a parameter vector , implying that the transfer function description of the discrete-time model becomes y(tk ) H(q, )e(tk ), k 1, 2, ... , N (5.1) T E e(tk )e (tks ) ( ) (s) The dimension of the vectors y(tkk) and e(t ) are p × 1. Since there is no unique way of computing an estimate of (5.1), it is necessary to select an appropriate approach that balances between the required accuracy and computational effort. The approach used in this thesis is known as the Prediction Error Method (PEM), see Ljung [71]. The reason for this choice is that the PEM for Gaussian distributed prediction errors is asymptotic unbiased and efficient. -
Wavelet Analysis for System Identification∗
Wavelet analysis for system identification∗ 大阪教育大学・数理科学 芦野 隆一 (Ryuichi Ashino) Mathematical Sciences, Osaka Kyoiku University 大阪電気通信大学・数理科学研究センター 萬代 武史 (Takeshi Mandai) Research Center for Physics and Mathematics, Osaka Electro-Communication University 大阪教育大学・情報科学 守本 晃 (Akira Morimoto) Information Science, Osaka Kyoiku University Abstract By the Schwartz kernel theorem, to every continuous linear sys- tem there corresponds a unique distribution, called kernel distribution. Formulae using wavelet transform to access time-frequency informa- tion of kernel distributions are deduced. A new wavelet based system identification method for health monitoring systems is proposed as an application of a discretized formula. 1 System Identification A system L illustrated in Figure 1 is an object in which variables of different kinds interact and produce observable signals. The observable signals g that are of interest to us are called outputs. The system is also affected by external stimuli. External signals f that can be manipulated by the observer are called inputs. Others are called disturbances and can be divided into those, denoted by w, that are directly measured, and those, denoted by v, that are only observed through their influence on the output. Mathematically, a system can be regarded as a mapping which relates inputs to outputs. Hereafter, we will use the terminology “system” rather ∗This research was partially supported by the Japanese Ministry of Education, Culture, Sports, Science and Technology, Grant-in-Aid for Scientific Research (C), 15540170 (2003– 2004), and by the Japan Society for the Promotion of Science, Japan-U.S. Cooperative Science Program (2003–2004). 1 Unmeasured Measured disturbance v disturbance w System L Output g Input f Figure 1: System L. -
Localization and System Identification of a Quadcopter AU V
Western Michigan University ScholarWorks at WMU Master's Theses Graduate College 6-2014 Localization and System Identification of a Quadcopter AU V Kenneth Befus Follow this and additional works at: https://scholarworks.wmich.edu/masters_theses Part of the Aeronautical Vehicles Commons, Navigation, Guidance, Control and Dynamics Commons, and the Propulsion and Power Commons Recommended Citation Befus, Kenneth, "Localization and System Identification of a Quadcopter AU V" (2014). Master's Theses. 499. https://scholarworks.wmich.edu/masters_theses/499 This Masters Thesis-Open Access is brought to you for free and open access by the Graduate College at ScholarWorks at WMU. It has been accepted for inclusion in Master's Theses by an authorized administrator of ScholarWorks at WMU. For more information, please contact [email protected]. LOCALIZATION AND SYSTEM IDENTIFICATION OF A QUADCOPTER UAV by Kenneth Befus A thesis submitted to the Graduate College in partial fulfillment of the requirements for the degree of Master of Science in Engineering Department of Mechanical and Aeronautical Engineering Western Michigan University June 2014 Doctoral Committee: Kapseong Ro, Ph.D., Chair Koorosh Naghshineh, Ph.D. James W. Kamman, Ph.D. LOCALIZATION AND SYSTEM IDENTIFICATION OF A QUADCOPTER UAV Kenneth Befus, M.S.E. Western Michigan University, 2014 The research conducted explores the comparison of several trilateration algorithms as they apply to the localization of a quadcopter micro air vehicle (MAV). A localization system is developed employing a network of combined ultrasonic/radio frequency sensors used to wirelessly provide range (distance) measurements defining the location of the quadcopter in 3-dimensional space. A Monte Carlo simulation is conducted using the extrinsic parameters of the localization system to evaluate the adequacy of each trilateration method as it applies to this specific quadcopter application. -
A Pattern Theoretic Approach
The Modelling of Biological Growth: a Pattern Theoretic Approach Nataliya Portman, Postdoctoral fellow McConnell Brain Imaging Centre Montreal Neurological Institute McGill University June 21, 2011 The Modelling of Biological Growth: a Pattern Theoretic Approach 2/ 47 Dedication This research work is dedicated to my PhD co-supervisor Dr. Ulf Grenander, the founder of Pattern Theory Division of Applied Mathematics, Brown University Providence, Rhode Island, USA. http://www.dam.brown.edu/ptg/ The Modelling of Biological Growth: a Pattern Theoretic Approach 3/ 47 Outline 1 Mathematical foundations of computational anatomy 2 Growth models in computational anatomy 3 A link between anatomical models and the GRID model 4 GRID view of growth on a fine time scale 5 GRID equation of growth on a coarse time scale (macroscopic growth law) 6 Image inference of growth properties of the Drosophila wing disc 7 Summary, concluding remarks and future perspectives 8 Current work The Modelling of Biological Growth: a Pattern Theoretic Approach 4/ 47 Mathematical foundations of computational anatomy Mathematical foundations of computational anatomy Computational anatomy1 focuses on the precise study of the biological variability of brain anatomy. D'Arcy Thompson laid out the vision of this discipline in his treatise \On Growth and Form" 2. In 1917 he wrote \In a very large part of morphology, our essential task lies in the comparison of related forms rather than in the precise definition of each; and the deformation of a complicated figure may be a phenomenon easy of comprehension, though the figure itself may be left unanalyzed and undefined." D'Arcy Thompson introduced the Method of Coordinates to accomplish the process of comparison. -
Sequential Monte Carlo Methods for System Identification (A Talk About
Sequential Monte Carlo methods for system identification (a talk about strategies) \The particle filter provides a systematic way of exploring the state space" Thomas Sch¨on Division of Systems and Control Department of Information Technology Uppsala University. Email: [email protected], www: user.it.uu.se/~thosc112 Joint work with: Fredrik Lindsten (University of Cambridge), Johan Dahlin (Link¨opingUniversity), Johan W˚agberg (Uppsala University), Christian A. Naesseth (Link¨opingUniversity), Andreas Svensson (Uppsala University), and Liang Dai (Uppsala University). [email protected] IFAC Symposium on System Identification (SYSID), Beijing, China, October 20, 2015. Introduction A state space model (SSM) consists of a Markov process x ≥ f tgt 1 that is indirectly observed via a measurement process y ≥ , f tgt 1 x x f (x x ; u ); x = a (x ; u ) + v ; t+1 j t ∼ θ t+1 j t t t+1 θ t t θ;t y x g (y x ; u ); y = c (x ; u ) + e ; t j t ∼ θ t j t t t θ t t θ;t x µ (x ); x µ (x ); 1 ∼ θ 1 1 ∼ θ 1 (θ π(θ)): (θ π(θ)): ∼ ∼ Identifying the nonlinear SSM: Find θ based on y y ; y ; : : : ; y (and u ). Hence, the off-line problem. 1:T , f 1 2 T g 1:T One of the key challenges: The states x1:T are unknown. Aim of the talk: Reveal the structure of the system identification problem arising in nonlinear SSMs and highlight where SMC is used. 1 / 14 [email protected] IFAC Symposium on System Identification (SYSID), Beijing, China, October 20, 2015. -
2018 Ulf Grenander Prize in Stochastic Theory and Modeling
AMS Prize Announcements FROM THE AMS SECRETARY 2018 Ulf Grenander Prize in Stochastic Theory and Modeling Judea Pearl was awarded the inaugural Ulf Grenander Prize in Stochastic Theory and Modeling at the 124th Annual Meeting of the AMS in San Diego, California, in January 2018. Citation belief propagation algorithm in Bayesian networks, which The 2018 Grenander Prize in recast the problem of computing posterior distributions Stochastic Theory and Modeling given evidence as a scheme for passing local messages is awarded to Judea Pearl for between network variables. Although exact computations the invention of a model-based through dynamic programming are possible, Pearl recog- approach to probabilistic and nized that in most problems of interest this would not be causal reasoning, for the discov- feasible, and in fact belief propagation turned out to be ery of innovative tools for infer- remarkably effective in many applications. ring these models from observa- Pearl’s primary goal in adopting Bayesian networks for tions, and for the development formulating structured models of complex systems was of novel computational methods his conviction that Bayesian networks would prove to be Judea Pearl for the practical applications of the right platform for addressing one of the most funda- these models. mental challenges to statistical modeling: the identifica- Grenander sought to develop general tools for con- tion of the conditional independencies among correlated structing realistic models of patterns in natural and variables that are induced by truly causal relationships. man-made systems. He believed in the power of rigorous In a series of papers in the 1990s, Pearl clearly showed mathematics and abstraction for the analysis of complex that statistical and causal notions are distinct and how models, statistical theory for efficient model inference, graphical causal models can provide a formal link between and the importance of computation for bridging theory causal quantities of interest and observed data. -
What Is System Identification
What is System identification Using experimental data obtained from input/output relations to model dynamic systems. disturbances input output System Different approaches to system identification depending on model class Linear/Non-linear Parametric/Non-parametric ¾Non-parametric methods try to estimate a generic model. (step responses, impulse responses, frequency responses) ¾Parametric methods estimate parameters in a user-specified model. (transfer functions, state-space matrices) 1 System identification System identification includes the following steps: Experiment design: its purpose is to obtain good experimental data, and it includes the choice of the measured variables and of the character of the input Signals. Selection of model structure: A suitable model structure is chosen using prior knowledge and trial and error. Choice of the criterion to fit: A suitable cost function is chosen, which reflects how well the. model fits the experimental data. Parameter estimation: An optimisation problem is solved to obtain the numerical values of the model parameters. Model validation: The model is tested in order to reveal any inadequacies. 2 Procedure of system identification Experimental design Data collection Data pre-filtering Model structure selection Parameter estimation Model validation Model ok? No Yes 3 Different mathematical models Model description: Transfer functions State-space models Block diagrams (e.g. Simulink) Notation for continuous time models Complex Laplace variable ‘s’ and differential operator ‘p’ ∂x(t) x&()t -
Norges Teknisk
ÆÇÊGEË ÌEÃÆÁËùÆAÌÍÊÎÁÌEÆËÃAÈEÄÁGE ÍÆÁÎEÊËÁÌEÌ EרiÑaØiÒg BÐÓÓd Îe××eÐ AÖea× iÒ ÍÐØÖa×ÓÙÒd ÁÑage× Í×iÒg a DefÓÖÑabÐe ÌeÑÔÐaØe ÅÓ deÐ bÝ ÇddÚaÖ ÀÙ×bÝ aÒd ÀaÚaÖd ÊÙe ÈÊEÈÊÁÆÌ ËÌAÌÁËÌÁCË ÆÇº ½7»¾¼¼½ ÁËËÆ: ¼8¼4¹9½7¿ ÆÇÊÏEGÁAÆ ÍÆÁÎEÊËÁÌY ÇF ËCÁEÆCE AÆD ÌECÀÆÇÄÇGY ÌÊÇÆDÀEÁŸ ÆÇÊÏAY Ìhi× ÖeÔ ÓÖØ ha× ÍÊÄ hØØÔ:»»ÛÛÛºÑaØhºÒØÒÙºÒÓ»ÔÖeÔÖiÒØ»×ØaØiרic×»¾¼¼½»Ë½7¹¾¼¼½ºÔ× ÀaÚaÖd ÊÙe ha× hÓÑeÔage: hØØÔ:»»ÛÛÛºÑaØhºÒØÒÙºÒÓ»hÖÙe E¹ÑaiÐ: hÖÙe@ÑaØhºÒØÒÙºÒÓ AddÖe××: DeÔaÖØÑeÒØ Óf ÅaØheÑaØicaÐ ËcieÒce׸ ÆÓÖÛegiaÒ ÍÒiÚeÖ×iØÝ Óf ËcieÒce aÒd ÌechÒÓÐÓgݸ ƹ749½ ÌÖÓÒdheiѸ ÆÓÖÛaݺ Estimating blood vessel areas in ultrasound images using a deformable template model Oddvar HUSBY and Havard˚ RUE Department of Mathematical Sciences Norwegian University of Science and Technology Norway October 9, 2001 Abstract We consider the problem of obtaining interval estimates of vessel areas from ul- trasound images of cross sections through the carotid artery. Robust and automatic estimates of the cross sectional area is of medical interest and of help in diagnosing atherosclerosis, which is caused by plaque deposits in the carotid artery. We approach this problem by using a deformable template to model the blood vessel outline, and use recent developments in ultrasound science to model the likelihood. We demonstrate that by using an explicit model for the outline, we can easily adjust for an important feature in the data: strong edge reflections called specular reflection. The posterior is challenging to explore, and naive standard MCMC algorithms simply converge to slowly. To obtain an efficient MCMC algorithm we make extensive use of computational efficient Gaus- sian Markov Random Fields, and use various block-sampling constructions that jointly update large parts of the model. -
The Pennsylvania State University the Graduate School Eberly College of Science
The Pennsylvania State University The Graduate School Eberly College of Science STUDIES ON THE LOCAL TIMES OF DISCRETE-TIME STOCHASTIC PROCESSES A Dissertation in Mathematics by Xiaofei Zheng c 2017 Xiaofei Zheng Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy August 2017 The dissertation of Xiaofei Zheng was reviewed and approved∗ by the following: Manfred Denker Professor of Mathematics Dissertation Adviser Chair of Committee Alexei Novikov Professor of Mathematics Anna Mazzucato Professor of Mathematics Zhibiao Zhao Associate Professor of Statistics Svetlana Katok Professor of Mathematics Director of Graduate Studies ∗Signatures are on file in the Graduate School. Abstract This dissertation investigates the limit behaviors of the local times `(n; x) of the n Pn 1 partial sum fSng of stationary processes fφ ◦ T g: `(n; x) = i=1 fSi=xg. Under the conditional local limit theorem assumption: n kn BnP (Sn = knjT (·) = !) ! g(κ) if ! κ, P − a:s:; Bn we show that the limiting distribution of the local time is the Mittag-Leffler distri- bution when the state space of the stationary process is Z. The method is from the infinite ergodic theory of dynamic systems. We also prove that the discrete-time fractional Brownian motion (dfBm) admits a conditional local limit theorem and the local time of dfBm is closely related to but different from the Mittag-Leffler dis- tribution. We also prove that the local time of certain stationary processes satisfies an almost sure central limit theorem (ASCLT) under the additional assumption that the characteristic operator has a spectral gap. iii Table of Contents Acknowledgments vi Chapter 1 Introduction and Overview 1 1.0.1 Brownian local time .