DOCSLIB.ORG

An Introduction to Large Deviations

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Introduction to Large Deviations An Introduction to Large Deviations Vitalii Konarovskyi [email protected] February 16, 2021 Abstract The large deviations theory is one of the key techniques of modern probability. It concerns with the study of probabilities of rare events and its estimates is the crucial tool required to handle many questions in statistical mechanics, engineering, applied probability, statistics etc. The course is build as the first look at the theory and is oriented on master and PhD students. 1 Introduction and some examples 1.1 Introduction We start from the considering of a coin-tossing experiment. Let us assume that we toss a fair coin. 1 The law of large numbers says us that the frequency of occurrence of \heads" becomes close to 2 as the number of trials increases to infinity. In other words, if X1;X2;::: are independent random variables 1 1 taking values 0 and 1 with probabilities 2 , i.e. P fXk = 0g = P fXk = 1g = 2 , then we know that for 1 X1+···+Xn the empirical mean n Sn = n 1 1 1 P Sn − > " ! 0 ; n ! 1; n 2 or more strongly 1 1 S ! a.s.2; n ! 1: n n 2 1 1 We are going stop more precisely on the probabilities P n Sn − 2 > " . We see that this events becomes more unlikely for large n and their probabilities decay to 0. During the course, we will work with such kind of unlike events and will try to understand the rate of their decay to zero. The knowledge of decay of probabilities of such unlike events has many applications in insurance, information theory, statistical mechanics etc. The aim of the course is to give an introduction to one of the key technique of the modern probability which is called the large deviation theory. 1 1 Before to investigate the rate of decay of the probabilities P n Sn − 2 > " , we consider an example of other random variable where computations are much more simpler. Let ξ1; ξ2;::: be independent identically distribyted random variables. We also assume that E ξ1 = µ 2 R; 2 Var ξ1 = σ > 0; 1according to the weak law of large numbers 2according to the strong law of large numbers 1 Universit¨atHamburg { WS20/21 An introduction to large deviations / Vitalii Konarovskyi and denote Sn = ξ1 + ··· + ξn. Then the weak law of large numbers says that 1 S ! µ in probability, as n ! +1; n n 1 that is, for all " > 0 one has P n Sn − µ ≥ " ! 0 as n ! +1. This convergence simply follows from Chebyshev's inequality. Indeed, 2 2 1 1 1 1 1 σ P Sn − µ ≥ " ≤ E Sn − µ = Var Sn = ! 0; n ! +1: (1) n "2 n "2 n n"2 1 Estimate (1) shows that the rate of convergence must be at least n , but this estimate is too rough. Later we will see that those probabilities decay exponentially fast. Let us demonstrate it on a particular example. Example 1.1. Let ξ1; ξ2;::: be independent normal distributed random variables with mean µ = 0 and variance σ = 1 (shortly ξk ∼ N(0; 1)). Then the random variable Sn has the normal distribution p1 with mean 0 and variance n. This implies n Sn ∼ N(0; 1). Now we can consider for x > 0 +1 1 1 p 1 Z y2 1 nx2 − 2 − 2 P Sn ≥ x = P p Sn ≥ x n = p p e dy ∼ p p e ; n ! +1; n n 2π x n 2πx n by Exercise 1.1 below. Thus, we have for x > 0 2 1 1 1 1 − nx lim ln P Sn ≥ x = lim ln p p e 2 n!1 n n n!1 n 2πx n 1 p p x2 x2 = − lim ln 2πx n − lim = − n!1 n n!1 2 2 due to Exercise 1.2 3). Remark 1.1. By symmetry, one can show that 1 1 x2 lim ln P Sn ≤ x = − n!1 n n 2 for all x < 0. Indeed, 1 1 1 1 1 1 (−x)2 lim ln P Sn ≤ x = lim ln P − Sn ≥ −x = lim ln P Sn ≥ −x = − n!1 n n n!1 n n n!1 n n 2 because −ξk ∼ N(0; 1) for k ≥ 1. Exercise 1.1. Show that Z +1 2 2 − y 1 − x e 2 dy ∼ e 2 ; x ! +1: x x Exercise 1.2. Let (an)n≥1 and (bn)n≥1 be two sequences of positive real numbers. We say that they are logarithmically equivalent and write an ' bn if 1 lim (ln an − ln bn) = 0: n!1 n 2 Universit¨atHamburg { WS20/21 An introduction to large deviations / Vitalii Konarovskyi o(n) 1. Show that an ' bn iff bn = ane . 2. Show that an ∼ bn implies an ' bn and that the inverse implication is not correct. 3. Show that an + bn ' maxfan; bng. Exercise 1.3. Let ξ1; ξ2;::: be independent normal distributed random variables with mean µ and 2 1 1 variance σ . Let also Sn = ξ1 + ··· + ξn. Compute limn!1 n ln P n Sn ≥ x for x > µ. 1.2 Coin-tossing In this section, we come back to the coin-tossing experiment and compute the decay of the probability 1 P n Sn ≥ x . Let, us before, X1;X2;::: be independent random variables taking values 0 and 1 with 1 probabilities 2 and Sn = X1 + ··· + Xn denote their partial sum. We will show that 1 1 lim ln P Sn ≥ x = −I(x) (2) n!1 n n 1 for all x ≥ 2 , where I is some function of x. We first note that for x > 1 1 1 3 lim ln P Sn ≥ x = −∞: n!1 n n 1 Next, for x 2 2 ; 1 we observe that 1 X 1 X S ≥ x = fS ≥ xng = fS = kg = Ck; P n n P n P n 2n n k≥xn k≥xn k n! where Cn = k!(n−k)! . Then we can estimate 1 k n + 1 k max Cn ≤ P fSn ≥ xng ≤ max Cn: (3) 2n k≥xn 2n k≥xn 1 Note that the maximum is attained at k = bxnc, the smallest integer ≥ xn, because x ≥ 2 . We denote l := bxnc. Using Stirling's formula p 1 n! = nne−n 2πn 1 + O ; n we have 1 k 1 l 1 lim ln max Cn = lim ln Cn = lim (ln n! − ln l! − ln(n − l)!) n!1 n k≥xn n!1 n n!1 n l l n − l n − l = lim ln n − 1 − ln l + − ln (n − l) + n!1 n n n n l n − l l n − l = lim ln n + ln n − ln l − ln(n − l) n!1 n n n n l l n − l n − l = lim − ln − ln = −x ln x − (1 − x) ln(1 − x); n!1 n n n n 3we always assume that ln 0 = −∞ 3 Universit¨atHamburg { WS20/21 An introduction to large deviations / Vitalii Konarovskyi l bxnc because n = n ! x as n ! +1. This together with estimate (3) implies 1 1 lim ln P Sn ≥ x = − ln 2 − x ln x − (1 − x) ln(1 − x) n!1 n n 1 for all x 2 2 ; 1 . So, we can take ( ln 2 + x ln x + (1 − x) ln(1 − x) if x 2 [0; 1]; I(x) = (4) +1 otherwise: 4 Remark 1.2. Using the symmetry, we have that 1 1 lim ln P Sn ≤ x = −I(x) (5) n!1 n n 1 for all x ≤ 2 . Indeed, 1 1 1 n 1 lim ln P Sn ≤ x = lim ln P − Sn ≥ 1 − x n!1 n n n!1 n n n 1 (1 − X1) + ··· + (1 − Xn) = lim ln P ≥ 1 − x = −I(1 − x) = −I(x) n!1 n n because Xk and 1 − Xk have the same distribution. Theorem 1.1. Let ξ1; ξ2;::: be independent Bernoulli distributed random variables with parameter p for some p 2 (0; 1), that is, P fξk = 1g = p and P fξk = 0g = 1 − p for all k ≥ 1. Let also Sn = ξ1 + ··· + ξn. Then for all x ≥ p 1 1 lim ln P Sn ≥ x = −I(x); n!1 n n where ( x ln x + (1 − x) ln 1−x if x 2 [0; 1]; I(x) = p 1−p +1 otherwise. 4The picture was taken from [dH00] 4 Universit¨atHamburg { WS20/21 An introduction to large deviations / Vitalii Konarovskyi Exercise 1.4. Prove Theorem 1.1. Exercise 1.5. Using (2) and (5) show that 1 X Sn 1 P − ≥ " < 1; n 2 n=1 Sn 1 for all " > 0. Conclude that n ! 2 a.s. as n ! 1 (strong low of large numbers). (Hint: Use the Borel-Cantelly lemma to show the convergence with probability 1) 2 Cramer's theorem 2.1 Comulant generating function The aim of this section is to obtain an analog of Theorem 1.1 for any sequeness of independent identically distributed random variables. In order to understand the form of the rate function I, we 1 will make the following computations, trying to obtain the upper bound for P n Sn ≥ x . Let ξ1; ξ2;::: be independent identically distributed random variables with mean µ 2 R. Let also Sn = ξ1 + ··· + ξn. We fix x ≥ µ and λ > 0 and use Chebyshev's inequality in order to estimate the following probability n 1 n o 1 1 Y 1 n S ≥ x = fS ≥ xng = eλSn ≥ eλxn ≤ eλSn = eλξk = eλξ1 : P n n P n P eλxn E eλxn E eλxn E k=1 Thus, we have 1 1 1 n −λxn λξ1 lim ln P fSn ≥ xng ≤ lim ln e + lim ln E e = −λx + '(λ); (6) n!1 n n!1 n n!1 n λξ where '(λ) := ln E e 1 .
Load more

Recommended publications
  • Large Deviations and Exit Time Asymptotics for Diffusions and Stochastic Resonance
    Large deviations and exit time asymptotics for diffusions and stochastic resonance DISSERTATION zur Erlangung des akademischen Grades doctor rerum naturalium (Dr. rer. nat.) im Fach Mathematik eingereicht an der Mathematisch-Naturwissenschaftlichen Fakultät II Humboldt-Universität zu Berlin von Diplom-Mathematiker Dierk Peithmann geboren am 29.06.1972 in Rahden Präsident der Humboldt-Universität zu Berlin: Prof. Dr. Christoph Markschies Dekan der Mathematisch-Naturwissenschaftlichen Fakultät II: Prof. Dr. Wolfgang Coy Gutachter: 1. Prof. Dr. Peter Imkeller 2. Prof. Dr. Ludwig Arnold 3. Prof. Dr. Michael Scheutzow Tag der mündlichen Prüfung: 13. August 2007 Abstract In this thesis, we study the asymptotic behavior of exit and transition times of certain weakly time inhomogeneous diffusion processes. Based on these asymptotics, a prob- abilistic notion of stochastic resonance is investigated. Large deviations techniques play the key role throughout this work. In the first part we recall the large deviations theory for time homogeneous diffusions. Chapter1 gives a brief overview of the classical results due to Freidlin und Wentzell. In Chapter2 we present an extension of their theory to stochastic differential equa- tions with locally Lipschitz coefficients that depend on the noise amplitude. Kramers’ exit time law, which makes up the foundation stone for the results obtained in this thesis, is recalled in Chapter3. The second part deals with the phenomenon of stochastic resonance. That is, we study periodicity properties of diffusion processes. First, in Chapter4 we explain the paradigm of stochastic resonance. Afterwards, physical notions of measuring pe- riodicity properties of diffusion trajectories are discussed, and their drawbacks are pointed out. The latter suggest to follow an alternative probabilistic approach, which is introduced in Section 4.3 and discussed in subsequent chapters.
    [Show full text]
  • Large-Deviation Principles, Stochastic Effective Actions, Path Entropies
    Large-deviation principles, stochastic effective actions, path entropies, and the structure and meaning of thermodynamic descriptions Eric Smith Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501, USA (Dated: February 18, 2011) The meaning of thermodynamic descriptions is found in large-deviations scaling [1, 2] of the prob- abilities for fluctuations of averaged quantities. The central function expressing large-deviations scaling is the entropy, which is the basis for both fluctuation theorems and for characterizing the thermodynamic interactions of systems. Freidlin-Wentzell theory [3] provides a quite general for- mulation of large-deviations scaling for non-equilibrium stochastic processes, through a remarkable representation in terms of a Hamiltonian dynamical system. A number of related methods now exist to construct the Freidlin-Wentzell Hamiltonian for many kinds of stochastic processes; one method due to Doi [4, 5] and Peliti [6, 7], appropriate to integer counting statistics, is widely used in reaction-diffusion theory. Using these tools together with a path-entropy method due to Jaynes [8], this review shows how to construct entropy functions that both express large-deviations scaling of fluctuations, and describe system-environment interactions, for discrete stochastic processes either at or away from equilibrium. A collection of variational methods familiar within quantum field theory, but less commonly applied to the Doi-Peliti construction, is used to define a “stochastic effective action”, which is the large-deviations rate function for arbitrary non-equilibrium paths. We show how common principles of entropy maximization, applied to different ensembles of states or of histories, lead to different entropy functions and different sets of thermodynamic state variables.
    [Show full text]
  • Sample Path Large Deviations for Lévy Processes and Random
    Sample Path Large Deviations for L´evyProcesses and Random Walks with Regularly Varying Increments Chang-Han Rhee1;3 Jose Blanchet2;4 Bert Zwart1;3 December 10, 2017 Abstract Let X be a L´evyprocess with regularly varying L´evymeasure ν. We ¯ obtain sample-path large deviations for scaled processes Xn(t) , X(nt)=n and obtain a similar result for random walks. Our results yield detailed asymptotic estimates in scenarios where multiple big jumps in the incre- ment are required to make a rare event happen; we illustrate this through detailed conditional limit theorems. In addition, we investigate connec- tions with the classical large deviations framework. In that setting, we show that a weak large deviation principle (with logarithmic speed) holds, but a full large deviation principle does not hold. Keywords Sample Path Large Deviations · Regular Variation · M- convergence · L´evyProcesses Mathematics Subject Classification 60F10 · 60G17 · 60B10 1 Introduction In this paper, we develop sample-path large deviations for one-dimensional L´evy processes and random walks, assuming the jump sizes are heavy-tailed. Specif- ically, let X(t); t ≥ 0; be a centered L´evyprocess with regularly varying L´evy measure ν. Assume that P(X(1) > x) is regularly varying of index −α, and that P(X(1) < −x) is regularly varying of index −β; i.e. there exist slowly varying functions L+ and L− such that −α −β P(X(1) > x) = L+(x)x ; P(X(1) < −x) = L−(x)x : (1.1) Throughout the paper, we assume α; β > 1. We also consider spectrally one- sided processes; in that case only α plays a role.
    [Show full text]
  • Large Deviations for Markov Sequences
    Chapter 32 Large Deviations for Markov Sequences This chapter establishes large deviations principles for Markov sequences as natural consequences of the large deviations principles for IID sequences in Chapter 31. (LDPs for continuous-time Markov processes will be treated in the chapter on Freidlin-Wentzell theory.) Section 32.1 uses the exponential-family representation of Markov sequences to establish an LDP for the two-dimensional empirical dis- tribution (“pair measure”). The rate function is a relative entropy. Section 32.2 extends the results of Section 32.1 to other observ- ables for Markov sequences, such as the empirical process and time averages of functions of the state. For the whole of this chapter, let X1, X2, . be a homogeneous Markov se- quence, taking values in a Polish space Ξ, with transition probability kernel µ, and initial distribution ν and invariant distribution ρ. If Ξ is not discrete, we will assume that ν and ρ have densities n and r with respect to some reference measure, and that µ(x, dy) has density m(x, y) with respect to that same ref- erence measure, for all x. (LDPs can be proved for Markov sequences without such density assumptions — see, e.g., Ellis (1988) — but the argument is more complicated.) 32.1 Large Deviations for Pair Measure of Markov Sequences It is perhaps not sufficiently appreciated that Markov sequences form expo- nential families (Billingsley, 1961; Ku¨chler and Sørensen, 1997). Suppose Ξ is 226 CHAPTER 32. LARGE DEVIATIONS FOR MARKOV SEQUENCES 227 discrete. Then t 1 n t − P X1 = x1 = ν(x1) µ(xi, xi+1) (32.1) i=1 ! " #t 1 − log µ(xi,xi+1) = ν(x1)e i=1 (32.2) P t x,y Ξ2 Tx,y (x1) log µ(x,y) = ν(x1)e ∈ (32.3) P t where Tx,y(x1) counts the number of times the state y follows the state x in the t sequence x1, i.e., it gives the transition counts.
    [Show full text]
  • Large Deviations of Poisson Cluster Processes 1
    Stochastic Models, 23:593–625, 2007 Copyright © Taylor & Francis Group, LLC ISSN: 1532-6349 print/1532-4214 online DOI: 10.1080/15326340701645959 LARGE DEVIATIONS OF POISSON CLUSTER PROCESSES Charles Bordenave INRIA/ENS, Départment d’Informatique, Paris, France Giovanni Luca Torrisi Istituto per le Applicazioni del Calcolo “Mauro Picone” (IAC), Consiglio Nazionale delle Ricerche (CNR), Roma, Italia In this paper we prove scalar and sample path large deviation principles for a large class of Poisson cluster processes. As a consequence, we provide a large deviation principle for ergodic Hawkes point processes. Keywords Hawkes processes; Large deviations; Poisson cluster processes; Poisson processes. Mathematics Subject Classification 60F10; 60G55. 1. INTRODUCTION Poisson cluster processes are one of the most important classes of point process models (see Daley and Vere-Jones[7] and Møller; Waagepetersen[24]). They are natural models for the location of objects in the space, and are widely used in point process studies whether theoretical or applied. Very popular and versatile Poisson cluster processes are the so-called self-exciting or Hawkes processes (Hawkes[12,13]; Hawkes and Oakes[14]). From a theoretical point of view, Hawkes processes combine both a Poisson cluster process representation and a simple stochastic intensity representation. Poisson cluster processes found applications in cosmology, ecology and epidemiology; see, respectively, Neyman and Scott[25], Brix and Chadoeuf[5], and Møller[19,20]. Hawkes processes are particularly appealing for seismological applications. Indeed, they are widely used as statistical models for the standard activity of earthquake series; see the papers by Received October 2006; Accepted August 2007 Address correspondence to Charles Bordenave, INRIA/ENS, Départment d’Informatique, 45 rue d’Ulm, F-75230 Paris Cedex 05, France; E-mail: [email protected] 594 Bordenave and Torrisi Ogata and Akaike[28], Vere-Jones and Ozaki[32], and Ogata[26,27].
    [Show full text]
  • Large Deviations Theory and Emergent Landscapes in Biological Dynamics
    The mathematical theory of large deviations provides a nonequilibrium thermodynamic description of complex biological systems that consist of heterogeneous individuals. In terms of the notions of stochastic elementary reactions and pure kinetic species, the continuous-time, integer- valued Markov process dictates a thermodynamic structure that generalizes (i) Gibbs’ macroscopic chemical thermodynamics of equilibrium matters to nonequilibrium small systems such as living cells and tissues; and (ii) Gibbs’ potential function to the landscapes for biological dynamics, such as that of C. H. Waddington’s and S. Wright’s. Large Deviations Theory and Emergent Landscapes in Biological Dynamics Hong Qian Department of Applied Mathematics University of Washington The biological world is stochastic The maximum entropy principle (MEP) is a consequence of the theory of large deviations (Cover and van Campenhout, 1981). It is a twin of the Gibbs’ statistical mechanics, which applies to only thermodynamic equilibrium. How to represent (e.g., describe) a biochemical or a biological system that consists of populations of individuals? (i) classifying biochemical (or biological) individuals into populations of kinetic species; (ii) counting the number of individuals in each and every “pure kinetic species”; (iii) representing changes in terms of “stochastic elementary processes”. A stochastic elementary process + + + + + = , , … , 2 3 ( + ) = + | = + ∆, = = 1 + , = 0, ∆ . � − ∆ 표 instantaneous rate function stoichiometric coefficients
    [Show full text]
  • On the Form of the Large Deviation Rate Function for the Empirical Measures
    Bernoulli 20(4), 2014, 1765–1801 DOI: 10.3150/13-BEJ540 On the form of the large deviation rate function for the empirical measures of weakly interacting systems MARKUS FISCHER Department of Mathematics, University of Padua, via Trieste 63, 35121 Padova, Italy. E-mail: fi[email protected] A basic result of large deviations theory is Sanov’s theorem, which states that the sequence of em- pirical measures of independent and identically distributed samples satisfies the large deviation principle with rate function given by relative entropy with respect to the common distribution. Large deviation principles for the empirical measures are also known to hold for broad classes of weakly interacting systems. When the interaction through the empirical measure corresponds to an absolutely continuous change of measure, the rate function can be expressed as relative entropy of a distribution with respect to the law of the McKean–Vlasov limit with measure- variable frozen at that distribution. We discuss situations, beyond that of tilted distributions, in which a large deviation principle holds with rate function in relative entropy form. Keywords: empirical measure; Laplace principle; large deviations; mean field interaction; particle system; relative entropy; Wiener measure 1. Introduction Weakly interacting systems are families of particle systems whose components, for each fixed number N of particles, are statistically indistinguishable and interact only through the empirical measure of the N-particle system. The study of weakly interacting systems originates in statistical mechanics and kinetic theory; in this context, they are often referred to as mean field systems. arXiv:1208.0472v4 [math.PR] 16 Oct 2014 The joint law of the random variables describing the states of the N-particle system of a weakly interacting system is invariant under permutations of components, hence determined by the distribution of the associated empirical measure.
    [Show full text]
  • Arxiv:1604.06828V2 [Math.PR]
    AN INTRODUCTION TO LARGE DEVIATIONS FOR RANDOM GRAPHS SOURAV CHATTERJEE Abstract. This article gives an overview of the emerging literature on large deviations for random graphs. Written for the general mathematical audience, the article begins with a short introduction to the theory of large deviations. This is followed by a description of some large deviation questions about ran- dom graphs, and an outline of the recent progress on this topic. A more elaborate discussion follows, with a brief account of graph limit theory and its application in constructing a large deviation theory for dense random graphs. The role of Szemer´edi’s regularity lemma is explained, together with a sketch of the proof of the main large deviation result and some examples. Appli- cations to exponential random graph models are briefly touched upon. The remainder of the paper is devoted to large deviations for sparse graphs. Since the regularity lemma is not applicable in the sparse regime, new tools are needed. Fortunately, there have been several new breakthroughs that man- aged to achieve the goal by an indirect method. These are discussed, together with an exposition of the underlying theory. The last section contains a list of open problems. Contents 1. Large deviations 1 2. The problem with nonlinearity 3 3. Recent developments 3 4. Graph limit theory 4 5. Largedeviationsfordenserandomgraphs 6 6. Exponential random graphs 10 7. Largedeviationsforsparserandomgraphs 12 8. The low complexity gradient condition 13 9. Nonlinear large deviations 17 arXiv:1604.06828v2 [math.PR] 27 Apr 2016 10. The naive mean field approximation 20 11. Open problems 23 Acknowledgments 24 References 24 1.
    [Show full text]
  • Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains
    Article Large Deviations Properties of Maximum Entropy Markov Chains from Spike Trains Rodrigo Cofré 1,* , Cesar Maldonado 2 and Fernando Rosas 3,4 1 Centro de Investigación y Modelamiento de Fenómenos Aleatorios, Facultad de Ingeniería, Universidad de Valparaíso, Valparaíso 2340000, Chile 2 IPICYT/División de Matemáticas Aplicadas, Instituto Potosino de Investigación Científica y Tecnológica, San Luis Potosí 78216, Mexico; [email protected] 3 Centre of Complexity Science and Department of Mathematics, Imperial College London, London SW7 2AZ, UK; [email protected] 4 Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK * Correspondence: [email protected]; Tel.: +56-32-2603625 Received: 6 June 2018; Accepted: 11 July 2018; Published: 3 August 2018 Abstract: We consider the maximum entropy Markov chain inference approach to characterize the collective statistics of neuronal spike trains, focusing on the statistical properties of the inferred model. To find the maximum entropy Markov chain, we use the thermodynamic formalism, which provides insightful connections with statistical physics and thermodynamics from which large deviations properties arise naturally. We provide an accessible introduction to the maximum entropy Markov chain inference problem and large deviations theory to the community of computational neuroscience, avoiding some technicalities while preserving the core ideas and intuitions. We review large deviations techniques useful in spike train statistics to describe properties of accuracy and convergence in terms of sampling size. We use these results to study the statistical fluctuation of correlations, distinguishability, and irreversibility of maximum entropy Markov chains. We illustrate these applications using simple examples where the large deviation rate function is explicitly obtained for maximum entropy models of relevance in this field.
    [Show full text]
  • Large Deviations for IID Sequences: the Return of Relative Entropy
    Chapter 31 Large Deviations for IID Sequences: The Return of Relative Entropy Section 31.1 introduces the exponential version of the Markov in- equality, which will be our major calculating device, and shows how it naturally leads to both the cumulant generating function and the Legendre transform, which we should suspect (correctly) of being the large deviations rate function. We also see the reappearance of rela- tive entropy, as the Legendre transform of the cumulant generating functional of distributions. Section 31.2 proves the large deviations principle for the empir- ical mean of IID sequences in finite-dimensional Euclidean spaces (Cram´er’s Theorem). Section 31.3 proves the large deviations principle for the empiri- cal distribution of IID sequences in Polish spaces (Sanov’s Theorem), using Cram´er’s Theorem for a well-chosen collection of bounded con- tinuous functions on the Polish space, and the tools of Section 30.2. Here the rate function is the relative entropy. Section 31.4 proves that even the infinite-dimensional empirical process distribution of an IID sequence in a Polish space obeys the LDP, with the rate function given by the relative entropy rate. The usual approach in large deviations theory is to establish an LDP for some comparatively tractable basic case through explicit calculations, and then use the machinery of Section 30.2 to extend it to LDPs for more complicated cases. This chapter applies this strategy to IID sequences. 216 CHAPTER 31. IID LARGE DEVIATIONS 217 31.1 Cumulant Generating Functions and Rela- tive Entropy Suppose the only inequality we knew in probability theory was Markov’s inequal- ity, P (X a) E [X] /a when X 0.
    [Show full text]
  • Large Deviations in Mathematical Finance
    Large deviations in mathematical finance ∗ Huy^enPHAM Laboratoire de Probabilit´eset Mod`elesAl´eatoires CNRS, UMR 7599 Universit´eParis 7 e-mail: [email protected] and Institut Universitaire de France January 5, 2010 Abstract The area of large deviations is a set of asymptotic results on rare events probabilities and a set of methods to derive such results. Large deviations theory is a very active field in applied probability, and finds important applications in finance, where ques- tions related to extremal events play an increasingly major role. Financial applications are various, and range from Monte-Carlo methods and importance sampling in option pricing to estimates of large portfolio losses subject to credit risk, or long term portfo- lio investment The purpose of these lectures is to explain some essential techniques in large deviations theory, and to illustrate how they are applied recently for example in stochastic volatility models to compute implied volatilities near maturities. Key words: large deviations, importance sampling, rare event simulation, exit probability, small time asymptotics, implied volatilities, credit risk, portfolio performance. MSC Classification (2000): 60F10, 62P05, 65C05, 91B28, 91B30. ∗Based on Natixis lectures, Fall 2008, and Master course at University Paris Diderot and ENSAE. 1 Contents 1 Introduction 3 2 An overview of large deviations theory 5 2.1 Laplace transform and change of probability measures . 5 2.2 Cramer's theorem . 7 2.3 Large deviations and Laplace principles . 10 2.4 Relative entropy and Donsker-Varadhan formula . 14 2.5 Sanov's theorem . 15 2.6 Sample path large deviation results . 18 3 Importance sampling and large deviations approximation in option pri- cing 22 3.1 Importance sampling for diffusions via Girsanov's theorem .
    [Show full text]
  • Some Applications and Methods of Large Deviations in Finance and Insurance Huyen Pham
    Some applications and methods of large deviations in finance and insurance Huyen Pham To cite this version: Huyen Pham. Some applications and methods of large deviations in finance and insurance. 2007. hal-00131578v1 HAL Id: hal-00131578 https://hal.archives-ouvertes.fr/hal-00131578v1 Preprint submitted on 16 Feb 2007 (v1), last revised 19 Feb 2007 (v2) HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Some applications and methods of large deviations in finance and insurance ∗ Huyˆen PHAM Laboratoire de Probabilit´es et Mod`eles Al´eatoires CNRS, UMR 7599 Universit´eParis 7 e-mail: [email protected] and Institut Universitaire de France February 16, 2007 Abstract In these notes, we present some methods and applications of large deviations to finance and insurance. We begin with the classical ruin problem related to the Cramer’s theorem and give en extension to an insurance model with investment in stock market. We then describe how large deviation approximation and importance sampling are used in rare event simulation for option pricing. We finally focus on large deviations methods in risk management for the estimation of large portfolio losses in credit risk and portfolio performance in market investment.
    [Show full text]