DOCSLIB.ORG

Non-Autonomous Random Dynamical Systems: Stochastic Approximation and Rate-Induced Tipping

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Non-Autonomous Random Dynamical Systems: Stochastic Approximation and Rate-Induced Tipping Imperial College London Department of Mathematics Non-autonomous Random Dynamical Systems: Stochastic Approximation and Rate-Induced Tipping Michael Hartl Supervised by Prof Sebastian van Strien and Dr Martin Rasmussen A thesis presented for the degree of Doctor of Philosophy at Imperial College London. Declaration I certify that the research documented in this thesis is entirely my own. All ideas, theories and results that originate from the work of others are marked as such and fully referenced, and ideas originating from discussions with others are also acknowledged as such. 1 Copyright The copyright of this thesis rests with the author and is made available under a Creative Commons Attribution Non-Commercial No Derivatives license. Researchers are free to copy, distribute or transmit the thesis on the condition that they attribute it, that they do not use it for commercial purposes and that they do not alter, transform or build upon it. For any reuse or redistribution, researchers must make clear to others the license terms of this work. 2 Abstract In this thesis we extend the foundational theory behind and areas of application of non- autonomous random dynamical systems beyond the current state of the art. We generalize results from autonomous random dynamical systems theory to a non-autonomous realm. We use this framework to study stochastic approximations from a different point of view. In particular we apply it to study noise induced transitions between equilibrium points and prove a bifurcation result. Then we turn our attention to parameter shift systems with bounded additive noise. We extend the framework of rate induced tipping in deterministic parameter shifts for this case and introduce tipping probabilities. Finally we perform a case study by developing and applying a numerical method for calculating tipping probabilities and examining the results thereof. 3 Acknowledgments I consider myself very lucky that I was included in the Innovative Training Network CRITICS1, funded entirely by the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 643073. Special thanks to: Hassan Alkhayuon, Peter Ashwin, Michel Benaïm, Jens Bendel, Daniele Castellana, Andrew Clarke, Michael Collins, Peter Deiml, Gabriela Depetri, Maximilian En- gel, Gabriel Fuhrmann, Tobias Jäger, Gerhard Keller, Jeroen Lamb, Johannes Lohmann, Iacopo Longo, Usman Mirza, Karl Nyman, Christian Oertel, Guillermo Olicon, Greg Pavli- otis, Courtney Quinn, Martin Rasmussen, Flavia Remo, Chris Richley, Paul Ritchie, Pablo Rodríguez-Sánchez, Edmilson Roque, Anderson Santos, Cristina Sargent, Tobias Schwedes, Jakob Seifert, Jan Sieber, Leif Stolberg, Sebastian van Strien, Damian Smug, Kalle Timperi, Dmitry Turaev, and my family. Dedicated to Rudi Wutz. 1Critical Transitions in Complex Systems 4 Contents 1 Introduction6 1.1 Non-autonomous random dynamical systems...................6 1.2 Stochastic approximations..............................8 1.3 Rate-induced tipping................................. 10 1.4 Structure of the thesis................................ 13 1.5 Notation........................................ 15 2 Random dynamical systems 17 2.1 Background on autonomous RDS.......................... 17 2.1.1 Skew product systems............................ 17 2.1.2 Random invariant sets and measures.................... 18 2.1.3 Attractors and repellers........................... 22 2.2 Non-autonomous random dynamical systems................... 24 2.2.1 Non-autonomous sets and measures.................... 24 2.2.2 Attractors and repellers........................... 32 3 Stochastic Approximations 35 3.1 Setup and notation.................................. 35 3.2 Examples....................................... 36 3.2.1 Urn models and market competition.................... 36 3.2.2 Learning in games.............................. 37 3.2.3 Stochastic gradient descent......................... 40 3.3 The Limit Set Theorem............................... 41 3.4 Noise induced tipping in one dimension...................... 43 3.4.1 Preliminaries................................. 43 3.4.2 The invertible case.............................. 46 3.4.3 The non-invertible case........................... 57 3.4.4 Further remarks............................... 60 3.5 A bifurcation arising from touchpoints....................... 62 3.6 Conclusions and outlook............................... 66 4 Rate-induced tipping in random systems 68 4.1 Deterministic R-tipping............................... 68 4.2 Asymptotically autonomous NRDS......................... 69 4.3 Random parameter shift systems and rate induced tipping............ 75 4.4 An example...................................... 83 4.5 Further remarks and outlook............................ 94 A Hausdorff distance 104 B Conditional expectation, martingales and stopping times 105 C Chain recurrence and asymptotic pseudotrajectories 109 5 1 Introduction 1.1 Non-autonomous random dynamical systems The theory of dynamical systems dates back to at least the 17th century, when Isaac Newton and contemporaries began exploring the motion of celestial bodies, which would later develop into the field of classical mechanics. Very loosely speaking, a dynamical system is a space with a prescribed set of motion laws. The classical theory of dynamical systems assumes that those laws are known and fixed for all times. In many areas of applications however this setting is far too restrictive. This gives rise to the notion of non-autonomous dynamical systems. There have been efforts towards developing a unified theory of non-autonomous dynamical systems. A foundational account of this is the book [49] by Kloeden and Rasmussen, which also contains a comprehensive historical overview of the developments in this field. The basic idea is to describe non-autonomy by a base flow on some abstract parameter space which influences the observed dynamics on the phase space. Generally there are two main categories of such base flows. Firstly, one may think of situations where a system is subject to some driving force, which does not follow any known or predescribed motion laws. This can be simply due to the fact that a system is way too complex in for every detail of it to be included in the model, as it is the case for example in climate studies. Another possibility is that a system is subject to a truly random influence, like many economic or financial markets who are subject to human decisions. Models in those cases are often based on stochastic differential equations driven by a random process, often a Brownian motion; examples include [23, 34, 58]. In systems with multiple time scales, a fast variable can seem random from the viewpoint of a slow timescale, such that a stochastic differential equation describes the motion of slow variables fairly well, see e.g. [57] for a theoretical approach and [43] for practical examples. Discrete time random systems are also of interest, where points in a space are iterated according to a map depending on some random parameter, such as random circle diffeomorphisms [78] or iterated function systems [10, 42]. The abstract parameter space in this case is usually a measure space, equipped with a measure preserving transformation; a setup popularized by Arnold in his seminal book [3]. On the other hand, motion laws can explicitly depend on time, for example through a varying parameter. Examples include the FitzHugh–Naguro model for the firing of neurons in the brain (cf. [39]), where the parameter is the level of stimulation. An early theoretical account is [72, 71]. In this setting, the abstract parameter space is the real “time” line, and the dynamics are given by a right shift. In the case of a periodic time dependency one can also use the unit circle as the abstract parameter space. Despite the two types of systems being included within the umbrella term non-autonomous dynamical system, a big part of the existing literature treats them separately. Random dynamical systems with their measure preserving base transformations give rise to a lot of measure theoretic tools that are not available for the time shift case. We will present a few highlights in Chapter 2.1. On the other hand this limits the way a non-autonomous system can depend on the driving force; the right shift on the real line does not possess an invariant probability measure. Only very recently, mathematicians have turned their attention to simultaneously time and noise dependent systems. To the best of our knowledge, the non-autonomous random 6 dynamical system formalism as presented in this thesis appeared for the first time in [30] under the name partial-random dynamical system, as the solution operator of a stochastic partial differential equation with a time varying domain. The term “partial-random” refers to the fact that the non-autonomous aspect of the dynamical system is only partially due to a random influence. The authors introduce a concept of non-autonomous random attractors and prove the existence of a global attractor in their sense for an example class of systems. Their ideas were followed on by Wang [73, 74]. Other work, such as [1, 24, 74], is particularly concerned with existence of pullback attractors under periodic deterministic forcing. Cui and Langa [32] discuss various concepts of non-autonomous random attractors with compact deterministic component of the base flow. Remarkably this paper does not necessarily assume that their NRDS is induced by a non-autonomous SDE. A NRDS on
Load more

Recommended publications
  • Data Assimilation and Parameter Estimation for a Multiscale Stochastic System with Α-Stable Lévy Noise
    Data assimilation and parameter estimation for a multiscale stochastic system with α-stable L´evy noise Yanjie Zhang1, Zhuan Cheng2, Xinyong Zhang3, Xiaoli Chen1, Jinqiao Duan2 and Xiaofan Li2 1Center for Mathematical Sciences & School of Mathematics and Statistics, Huazhong University of Science and Technology, Wuhan 430074, China 2Department of Applied Mathematics, Illinois Institute of Technology, Chicago, IL 60616, USA 3Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China E-mail: [email protected], [email protected], [email protected], [email protected], [email protected] and [email protected] . ∗This work was partly supported by the NSF grant 1620449, and NSFC grants 11531006, 11371367, and 11271290. Abstract. This work is about low dimensional reduction for a slow-fast data assimilation system with non-Gaussian α stable L´evy noise via stochastic averaging. − When the observations are only available for slow components, we show that the averaged, low dimensional filter approximates the original filter, by examining the corresponding Zakai stochastic partial differential equations. Furthermore, we demonstrate that the low dimensional slow system approximates the slow dynamics of the original system, by examining parameter estimation and most probable paths. Keywords: Multiscale systems, non-Gaussian L´evy noise, averaging principle, Zakai equation, parameter estimation, most probable paths arXiv:1801.02846v1 [math.DS] 9 Jan 2018 1. Introduction Data assimilation is a procedure to extract system state information with the help of observations [18]. The state evolution and the observations are usually under random fluctuations. The general idea is to gain the best estimate for the true system state, in terms of the probability distribution for the system state, given only some noisy observations of the system.
    [Show full text]
  • Trajectory Averaging for Stochastic Approximation MCMC Algorithms
    The Annals of Statistics 2010, Vol. 38, No. 5, 2823–2856 DOI: 10.1214/10-AOS807 c Institute of Mathematical Statistics, 2010 TRAJECTORY AVERAGING FOR STOCHASTIC APPROXIMATION MCMC ALGORITHMS1 By Faming Liang Texas A&M University The subject of stochastic approximation was founded by Rob- bins and Monro [Ann. Math. Statist. 22 (1951) 400–407]. After five decades of continual development, it has developed into an important area in systems control and optimization, and it has also served as a prototype for the development of adaptive algorithms for on-line esti- mation and control of stochastic systems. Recently, it has been used in statistics with Markov chain Monte Carlo for solving maximum likelihood estimation problems and for general simulation and opti- mizations. In this paper, we first show that the trajectory averaging estimator is asymptotically efficient for the stochastic approxima- tion MCMC (SAMCMC) algorithm under mild conditions, and then apply this result to the stochastic approximation Monte Carlo algo- rithm [Liang, Liu and Carroll J. Amer. Statist. Assoc. 102 (2007) 305–320]. The application of the trajectory averaging estimator to other stochastic approximation MCMC algorithms, for example, a stochastic approximation MLE algorithm for missing data problems, is also considered in the paper. 1. Introduction. Robbins and Monro (1951) introduced the stochastic approximation algorithm to solve the integration equation (1) h(θ)= H(θ,x)fθ(x) dx = 0, ZX where θ Θ Rdθ is a parameter vector and f (x), x Rdx , is a density ∈ ⊂ θ ∈X ⊂ function depending on θ. The dθ and dx denote the dimensions of θ and x, arXiv:1011.2587v1 [math.ST] 11 Nov 2010 respectively.
    [Show full text]
  • Limit Theorems for Non-Stationary and Random
    LIMIT THEOREMS FOR NON-STATIONARY AND RANDOM DYNAMICAL SYSTEMS by Yaofeng Su A dissertation submitted to the Department of Mathematics, College of Natural Sciences and Mathematics in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Mathematics Chair of Committee: Andrew T¨or¨ok Committee Member: Vaughn Climenhaga Committee Member: Gemunu Gunaratne Committee Member: Matthew Nicol University of Houston May 2020 Copyright 2020, Yaofeng Su ACKNOWLEDGMENTS This dissertation would not have been possible without the guidance and the help of several in- dividuals who in one way or another contributed and extended their valuable assistance in the preparation and completion of this study. First, my utmost gratitude to my advisor Dr. Andrew T¨or¨okfor the support and help he provided me throughout my graduate education; my commit- tee members, Dr. Vaughn Climenhaga, Dr. Gemunu Gunaratne, and Dr. Matthew Nicol for their participation and their thoughtful comments; all professors in the dynamical system research group of University of Houston, Dr. Vaughn Climenhaga, Dr. Alan Haynes, Dr. Matthew Nicol, Dr. William Ott, and Dr. Andrew T¨or¨okfor organizing the wonderful \Houston Summer School on Dynamical Systems", where I learned a lot about dynamical systems, ergodic theory, number theory, and quantum mechanics. iii ABSTRACT We study the limit behavior of the non-stationary/random chaotic dynamical systems and prove a strong statistical limit theorem: (vector-valued) almost sure invariance principle for the non- stationary dynamical systems and quenched (vector-valued) almost sure invariance principle for the random dynamical systems. It is a matching of the trajectories of the dynamical system with a Brownian motion in such a way that the error is negligible in comparison with the Birkhoff sum.
    [Show full text]
  • Quasi-Invariant Gaussian Measures for the Cubic Fourth Order Nonlinear Schrödinger Equation
    Probab. Theory Relat. Fields DOI 10.1007/s00440-016-0748-7 Quasi-invariant Gaussian measures for the cubic fourth order nonlinear Schrödinger equation Tadahiro Oh1,2 · Nikolay Tzvetkov3 Received: 4 August 2015 / Revised: 25 November 2016 © The Author(s) 2016. This article is published with open access at Springerlink.com Abstract We consider the cubic fourth order nonlinear Schrödinger equation on the circle. In particular, we prove that the mean-zero Gaussian measures on Sobolev spaces s(T) > 3 H , s 4 , are quasi-invariant under the flow. Keywords Fourth order nonlinear Schrödinger equation · Biharmonic nonlinear Schrödinger equation · Gaussian measure · Quasi-invariance Mathematics Subject Classification 35Q55 Contents 1 Introduction ............................................... 1.1 Background ............................................. 1.2 Cubic fourth order nonlinear Schrödinger equation ........................ 1.3 Main result ............................................. 1.4 Organization of the paper ...................................... 2 Notations ................................................. 3 Reformulation of the cubic fourth order NLS .............................. B Tadahiro Oh [email protected] Nikolay Tzvetkov [email protected] 1 School of Mathematics, The University of Edinburgh, James Clerk Maxwell Building, The King’s Buildings, Peter Guthrie Tait Road, Edinburgh EH9 3FD, UK 2 The Maxwell Institute for the Mathematical Sciences, James Clerk Maxwell Building, The King’s Buildings, Peter Guthrie Tait Road,
    [Show full text]
  • Math.DS] 5 Aug 2014 Etns E PK O Opeesv Uvyo H Ujc.I Subject
    SYNCHRONIZATION PROPERTIES OF RANDOM PIECEWISE ISOMETRIES VICTOR KLEPTSYN AND ANTON GORODETSKI Abstract. We study the synchronization properties of the random double rotations on tori. We give a criterion that show when synchronization is present in the case of random double rotations on the circle and prove that it is always absent in dimensions two and higher. 1. Introduction The observation of a synchronization effect goes back at least to 17th century, when Huygens [Hu] discovered the synchronization of two linked pendulums. Since then, synchronization phenomena have been observed in numerous systems and settings, see [PRK] for a comprehensive survey of the subject. In the theory of dynamical systems, synchronization usually refers to random dynamical system trajectories of different initial points converging to each other under the applica- tion of a sequence of random transformations. A first such result is the famous Furstenberg’s Theorem [Fur1], stating that under some very mild assumptions, the angle (mod π) between the images of any two vectors under a long product of random matrices (exponentially) tends to zero. Projectivizing the dynamics, it is easy to see that this theorem in fact states that random trajectories of the quotient system on the projective space (exponentially) approach each other. For random dynamical systems on the circle, several results are known. Surely, in random projective dynamics there is a synchronization due to the simplest pos- sible case of Furstenberg’s Theorem. In 1984, this result was generalized to the setting of homeomorphisms (with some very mild and natural assumptions of min- imality of the action and the presence of a North-South map) in the work of a physicist V.
    [Show full text]
  • A Model of Gene Expression Based on Random Dynamical Systems Reveals Modularity Properties of Gene Regulatory Networks†
    A Model of Gene Expression Based on Random Dynamical Systems Reveals Modularity Properties of Gene Regulatory Networks† Fernando Antoneli1,4,*, Renata C. Ferreira3, Marcelo R. S. Briones2,4 1 Departmento de Informática em Saúde, Escola Paulista de Medicina (EPM), Universidade Federal de São Paulo (UNIFESP), SP, Brasil 2 Departmento de Microbiologia, Imunologia e Parasitologia, Escola Paulista de Medicina (EPM), Universidade Federal de São Paulo (UNIFESP), SP, Brasil 3 College of Medicine, Pennsylvania State University (Hershey), PA, USA 4 Laboratório de Genômica Evolutiva e Biocomplexidade, EPM, UNIFESP, Ed. Pesquisas II, Rua Pedro de Toledo 669, CEP 04039-032, São Paulo, Brasil Abstract. Here we propose a new approach to modeling gene expression based on the theory of random dynamical systems (RDS) that provides a general coupling prescription between the nodes of any given regulatory network given the dynamics of each node is modeled by a RDS. The main virtues of this approach are the following: (i) it provides a natural way to obtain arbitrarily large networks by coupling together simple basic pieces, thus revealing the modularity of regulatory networks; (ii) the assumptions about the stochastic processes used in the modeling are fairly general, in the sense that the only requirement is stationarity; (iii) there is a well developed mathematical theory, which is a blend of smooth dynamical systems theory, ergodic theory and stochastic analysis that allows one to extract relevant dynamical and statistical information without solving
    [Show full text]
  • Stochastic Approximation for the Multivariate and the Functional Median 3 It Can Not Be Updated Simply If the Data Arrive Online
    STOCHASTIC APPROXIMATION TO THE MULTIVARIATE AND THE FUNCTIONAL MEDIAN Submitted to COMPSTAT 2010 Herv´eCardot1, Peggy C´enac1, and Mohamed Chaouch2 1 Institut de Math´ematiques de Bourgogne, UMR 5584 CNRS, Universit´ede Bourgogne, 9, Av. A. Savary - B.P. 47 870, 21078 Dijon, France [email protected], [email protected] 2 EDF - Recherche et D´eveloppement, ICAME-SOAD 1 Av. G´en´eralde Gaulle, 92141 Clamart, France [email protected] Abstract. We propose a very simple algorithm in order to estimate the geometric median, also called spatial median, of multivariate (Small, 1990) or functional data (Gervini, 2008) when the sample size is large. A simple and fast iterative approach based on the Robbins-Monro algorithm (Duflo, 1997) as well as its averaged version (Polyak and Juditsky, 1992) are shown to be effective for large samples of high dimension data. They are very fast and only require O(Nd) elementary operations, where N is the sample size and d is the dimension of data. The averaged approach is shown to be more effective and less sensitive to the tuning parameter. The ability of this new estimator to estimate accurately and rapidly (about thirty times faster than the classical estimator) the geometric median is illustrated on a large sample of 18902 electricity consumption curves measured every half an hour during one week. Keywords: Geometric quantiles, High dimension data, Online estimation al- gorithm, Robustness, Robbins-Monro, Spatial median, Stochastic gradient averaging 1 Introduction Estimation of the median of univariate and multivariate data has given rise to many publications in robust statistics, data mining, signal processing and information theory.
    [Show full text]
  • Simulation Methods for Robust Risk Assessment and the Distorted Mix
    Simulation Methods for Robust Risk Assessment and the Distorted Mix Approach Sojung Kim Stefan Weber Leibniz Universität Hannover September 9, 2020∗ Abstract Uncertainty requires suitable techniques for risk assessment. Combining stochastic ap- proximation and stochastic average approximation, we propose an efficient algorithm to compute the worst case average value at risk in the face of tail uncertainty. Dependence is modelled by the distorted mix method that flexibly assigns different copulas to different regions of multivariate distributions. We illustrate the application of our approach in the context of financial markets and cyber risk. Keywords: Uncertainty; average value at risk; distorted mix method; stochastic approximation; stochastic average approximation; financial markets; cyber risk. 1 Introduction Capital requirements are an instrument to limit the downside risk of financial companies. They constitute an important part of banking and insurance regulation, for example, in the context of Basel III, Solvency II, and the Swiss Solvency test. Their purpose is to provide a buffer to protect policy holders, customers, and creditors. Within complex financial networks, capital requirements also mitigate systemic risks. The quantitative assessment of the downside risk of financial portfolios is a fundamental, but arduous task. The difficulty of estimating downside risk stems from the fact that extreme events are rare; in addition, portfolio downside risk is largely governed by the tail dependence of positions which can hardly be estimated from data and is typically unknown. Tail dependence is a major source of model uncertainty when assessing the downside risk. arXiv:2009.03653v1 [q-fin.RM] 8 Sep 2020 In practice, when extracting information from data, various statistical tools are applied for fitting both the marginals and the copulas – either (semi-)parametrically or empirically.
    [Show full text]
  • An Empirical Bayes Procedure for the Selection of Gaussian Graphical Models
    An empirical Bayes procedure for the selection of Gaussian graphical models Sophie Donnet CEREMADE Universite´ Paris Dauphine, France Jean-Michel Marin∗ Institut de Mathematiques´ et Modelisation´ de Montpellier Universite´ Montpellier 2, France Abstract A new methodology for model determination in decomposable graphical Gaussian models (Dawid and Lauritzen, 1993) is developed. The Bayesian paradigm is used and, for each given graph, a hyper inverse Wishart prior distribution on the covariance matrix is considered. This prior distribution depends on hyper-parameters. It is well-known that the models’s posterior distribution is sensitive to the specification of these hyper-parameters and no completely satisfactory method is registered. In order to avoid this problem, we suggest adopting an empirical Bayes strategy, that is a strategy for which the values of the hyper-parameters are determined using the data. Typically, the hyper-parameters are fixed to their maximum likelihood estimations. In order to calculate these maximum likelihood estimations, we suggest a Markov chain Monte Carlo version of the Stochastic Approximation EM algorithm. Moreover, we introduce a new sampling scheme in the space of graphs that improves the add and delete proposal of Armstrong et al. (2009). We illustrate the efficiency of this new scheme on simulated and real datasets. Keywords: Gaussian graphical models, decomposable models, empirical Bayes, Stochas- arXiv:1003.5851v2 [stat.CO] 23 May 2011 tic Approximation EM, Markov Chain Monte Carlo ∗Corresponding author: [email protected] 1 1 Gaussian graphical models in a Bayesian Context Statistical applications in genetics, sociology, biology , etc often lead to complicated interaction patterns between variables.
    [Show full text]
  • The Mathematics of Critical Shifts in Ecological Networks with Alternative Stable State Theory, a Potential Framework for Early Warning Indicators
    The Mathematics of Critical Shifts in Ecological Networks with Alternative Stable State Theory, A Potential Framework for Early Warning Indicators by Michael Kupperman A THESIS submitted to Oregon State University Honors College in partial fulfillment of the requirements for the degree of Honors Bachelors of Science in Mathematics (Honors Scholar) Honors Bachelors of Science in Biochemistry & Molecular Biology (Honors Scholar) Presented June 14, 2019 Commencement June 2020 AN ABSTRACT OF THE THESIS OF Michael Kupperman for the degree of Honors Bachelors of Science in Mathematics and Honors Bachelors of Science in Biochemistry & Molecular Biology presented on June 14, 2019. Title: The Mathematics of Critical Shifts in Ecological Networks with Alternative Stable State Theory, A Potential Framework for Early Warning Indicators Abstract approved: David Koslicki A long running problem in mathematical biology is the prediction of extinction events, a specialized case of the larger global stability problem found in differential equations and dynamical systems theory. A central technical question is how to introduce the randomness observed in real ecological systems not accounted for in deterministic models. This work introduces the SP-system as a new mathematical object in which ecological parameters are treated as sequences of random variables that attain values over intervals of random lengths of time. The SP-system characterization of ecological networks leads to two differ- ent novel approaches for the simulation and extinction prediction. The first approach uses a construct new to modeling literature to describe the probability of a parameter transition event into an extinction event. The second approach utilizes Markov chains for both sim- ulation and extinction prediction.
    [Show full text]
  • Stochastic Approximation of Score Functions for Gaussian Processes
    The Annals of Applied Statistics 2013, Vol. 7, No. 2, 1162–1191 DOI: 10.1214/13-AOAS627 c Institute of Mathematical Statistics, 2013 STOCHASTIC APPROXIMATION OF SCORE FUNCTIONS FOR GAUSSIAN PROCESSES1 By Michael L. Stein2, Jie Chen3 and Mihai Anitescu3 University of Chicago, Argonne National Laboratory and Argonne National Laboratory We discuss the statistical properties of a recently introduced un- biased stochastic approximation to the score equations for maximum likelihood calculation for Gaussian processes. Under certain condi- tions, including bounded condition number of the covariance matrix, the approach achieves O(n) storage and nearly O(n) computational effort per optimization step, where n is the number of data sites. Here, we prove that if the condition number of the covariance matrix is bounded, then the approximate score equations are nearly optimal in a well-defined sense. Therefore, not only is the approximation ef- ficient to compute, but it also has comparable statistical properties to the exact maximum likelihood estimates. We discuss a modifi- cation of the stochastic approximation in which design elements of the stochastic terms mimic patterns from a 2n factorial design. We prove these designs are always at least as good as the unstructured design, and we demonstrate through simulation that they can pro- duce a substantial improvement over random designs. Our findings are validated by numerical experiments on simulated data sets of up to 1 million observations. We apply the approach to fit a space–time model to over 80,000 observations of total column ozone contained in the latitude band 40◦–50◦N during April 2012.
    [Show full text]
  • Stochastic Approximation Methods and Applications in Financial Optimization Problems
    Stochastic Approximation Methods And Applications in Financial Optimization Problems by Chao Zhuang (Under the direction of Qing Zhang) Abstract Optimizations play an increasingly indispensable role in financial decisions and financial models. Many problems in mathematical finance, such as asset allocation, trading strategy, and derivative pricing, are now routinely and efficiently approached using optimization. Not until recently have stochastic approximation methods been applied to solve optimization problems in finance. This dissertation is concerned with stochastic approximation algorithms and their appli- cations in financial optimization problems. The first part of this dissertation concerns trading a mean-reverting asset. The strategy is to determine a low price to buy and a high price to sell so that the expected return is maximized. Slippage cost is imposed on each transaction. Our effort is devoted to developing a recursive stochastic approximation type algorithm to esti- mate the desired selling and buying prices. In the second part of this dissertation we consider the trailing stop strategy. Trailing stops are often used in stock trading to limit the maximum of a possible loss and to lock in a profit. We develop stochastic approximation algorithms to estimate the optimal trailing stop percentage. A modification using projection is devel- oped to ensure that the approximation sequence constructed stays in a reasonable range. In both parts, we also study the convergence and the rate of convergence. Simulations and real market data are used to demonstrate the performance of the proposed algorithms. The advantage of using stochastic approximation in stock trading is that the underlying asset is model free. Only observed stock prices are required, so it can be performed on line to provide guidelines for stock trading.
    [Show full text]