CRMs Sinead Williamson Background Completely random measures and related L´evyprocesses Completely models random measures Applications Normalized Sinead Williamson random measures Neutral-to-the- right processes Computational and Biological Learning Laboratory Exchangeable University of Cambridge matrices January 20, 2011 Outline CRMs Sinead Williamson 1 Background Background L´evyprocesses Completely 2 L´evyprocesses random measures Applications 3 Completely random measures Normalized random measures Neutral-to-the- right processes 4 Applications Exchangeable matrices Normalized random measures Neutral-to-the-right processes Exchangeable matrices A little measure theory CRMs Sinead Williamson Set: e.g. Integers, real numbers, people called James. Background May be finite, countably infinite, or uncountably infinite. L´evyprocesses Completely Algebra: Class T of subsets of a set T s.t. random measures 1 T 2 T . 2 If A 2 T , then Ac 2 T . Applications K Normalized 3 If A1;:::; AK 2 T , then [ Ak = A1 [ A2 [ ::: AK 2 T random k=1 measures (closed under finite unions). Neutral-to-the- right K processes 4 If A1;:::; AK 2 T , then \k=1Ak = A1 \ A2 \ ::: AK 2 T Exchangeable matrices (closed under finite intersections). σ-Algebra: Algebra that is closed under countably infinite unions and intersections. A little measure theory CRMs Sinead Williamson Background L´evyprocesses Measurable space: Combination (T ; T ) of a set and a Completely σ-algebra on that set. random measures Measure: Function µ between a σ-field and the positive Applications reals (+ 1) s.t. Normalized random measures 1 µ(;) = 0. Neutral-to-the- right 2 For all countable collections of disjoint sets processes P Exchangeable A1; A2; · · · 2 T , µ([k Ak ) = µ(Ak ). matrices k Probability measures CRMs Sinead Williamson Background Probability distribution: Measure P on some measurable L´evyprocesses space (Ω; F) s.t. P(Ω) = 1. Completely Intuition: Subsets = events; measures of subsets = random measures probability of that event. Applications Normalized Discrete probability distribution: assigns measure 1 to a random measures countable subset of Ω. Neutral-to-the- right processes Continuous probability distribution: assigns measure 0 to Exchangeable matrices singletons x 2 Ω. Atoms: singletons with positive measure. Representing the real world CRMs Sinead Williamson Background Kolmogorov: Two types of object - experimental observations, L´evyprocesses and the random phenomena underlying them. Completely random measures Real world Mathematical world Applications Normalized Random phenomena Probability space (Ω; F; P) random measures Neutral-to-the- right processes Experiment Algebra Exchangeable matrices Experimental observations Collection of random variables Representing the real world CRMs Random variables X : (Ω; F) ! (SX ; SX ) are mappings Sinead Williamson from the underlying probability space to our observation space. Background L´evyprocesses This mapping, combined with the probability distribution Completely on (Ω; F), induces a probability distribution random −1 measures µX := P ◦ X on the observation space. Applications We call µX the distribution of our observations. Normalized random measures Neutral-to-the- right processes X Exchangeable matrices SX Ω Characteristic functions CRMs Sinead Williamson Often, it is useful to represent random variables and Background probability distributions in terms of their characteristic L´evyprocesses function. Completely d random For a random variable X taking values in with measures R distribution µ , Applications X Normalized Z random ihuyi ihuyi measures ΦX (u) = e µX (dy) = E[e ] Neutral-to-the- d right R processes Exchangeable matrices If µX admits a density (i.e. µX (dy) = p(y)ν(dy)), then the characteristic function is the Fourier transform of that density. Infinitely divisible distributions CRMs Sinead Williamson Background We say a probability measure µ is infinitely divisible if, for each L´evyprocesses n 2 N: Completely We can write µ as the n-fold self-convolution random (n) (n) (n) measures µ ∗ · · · ∗ µ of some distribution µ . Applications (n) Normalized (Equivalently) The nth root Φ of the characteristic random measures function of µ is the characteristic function of some Neutral-to-the- right processes probability measure. Exchangeable matrices (Equivalently) For any X ∼ µ, we can write Pn (i) (i) (n) X = i=1 X , where X ∼ µ . (The celebrated) L´evy-Khintchineformula CRMs Theorem: L´evy-Khintchine Sinead Williamson d A distribution µ on R is infinitely divisible iff its characteristic Background function Φµ can be represented in the form: L´evyprocesses Completely 1 random Φµ(u) = exp ihb; ui − hu; Aui measures 2 Applications Z Normalized ihu;zi random + (e − 1 − ihu; ziI (jz ≤ 1))ν(dz; ds) ; measures d ( −{0g)×SX Neutral-to-the- R right processes Exchangeable d matrices for some uniquely defined vector b 2 R , positive-definite d symmetric matrix A, and measure ν on R satisfying: Z (jzj2 ^ 1)ν(dz; ds) < 1 : d R −{0}×SX Notation CRMs Sinead Williamson Background L´evyprocesses We call: Completely random b the drift; measures Applications A the Gaussian covariance matrix; Normalized random measures ν the L´evymeasure; Neutral-to-the- right processes the triplet (A; ν; b) the generating triplet. Exchangeable matrices L´evyprocesses CRMs Sinead Williamson AL´evyprocess is a stochastic process X = (Xt )t≥0 s.t. Background 1 X0 = 0. L´evyprocesses 2 X has independent increments, i.e. for each n 2 N and Completely random each t1 ≤ · · · ≤ tn+1, the random variables measures (Xti+1 − Xti ; 1 ≤ i ≤ n) are independent. Applications Normalized 3 X is stochastically continuous, i.e. for every > 0 and random measures s ≥ 0, Neutral-to-the- right processes lim P(jXt − Xs j > ) = 0 : (1) Exchangeable s!t matrices 4 Sample paths of X are right-continuous with left limits. A L´evyprocess is homogeneous if its increments are stationary { i.e. if the distribution of Xt+s − Xt does not depend on t. L´evyprocesses and infinite divisibility CRMs Sinead Williamson Background L´evyprocesses Theorem: Infinite divisibility Completely X is infinitely divisible for all t ≥ 0. random t measures Applications Proof Normalized random measures (Homogeneous case) Since X has independent increments, we Neutral-to-the- right processes can write Xt as the sum of n independent random variables for Exchangeable matrices any n 2 N. Therefore, Xt is infinitely divisible. L´evyprocesses and infinite divisibility CRMs Sinead Williamson Infinite divisibility means the L´evy-Khintchineformula Background holds. L´evyprocesses So, we can describe a L´evyprocess in terms of a drift Completely random vector, a Gaussian covariance matrix and a L´evy measure. measures Applications A related result - the L´evy-It^odecomposition, tells us that Normalized any L´evyprocess can be decomposed into the random measures superposition of three L´evy processes: Neutral-to-the- right processes A continuous, deterministic process, governed by the drift. Exchangeable matrices A continuous, random process (Brownian motion), governed by the Gaussian covariance matrix. A pure-jump, random process, governed by the L´evy measure. Subordinators CRMs Sinead Williamson Background A subordinator is a L´evyprocess with strictly increasing L´evyprocesses sample paths. Completely random A L´evyprocess on R+ has increasing sample paths iff: measures A = 0 no Gaussian component. Applications Normalized b ≥ 0 deterministic component is strictly nondecreasing. random R measures (−∞;0) ν(dz × R+) = 0 no negative jumps. Neutral-to-the- R right zν(dz × ) < 1 ensures conditions of L´evy processes (0;1] R+ Exchangeable matrices process. If 0 < ν < 1, then X has countably infinite jumps. Completely random measures CRMs Sinead Williamson Random measure: Mapping M : (Ω; F) ! (SM; SM), where (SM; SM) is a set of measures. Background L´evyprocesses Completely random measure (CRM): Random measures Completely where (SM; SM) is a set of measures such that µ(A1) and random measures µ(A2) are independent whenever A1 and A2 are disjoint. Applications CRMs can be decomposed into three parts: Normalized random measures 1 An atomic measure with random atom locations and Neutral-to-the- right random atom masses. processes Exchangeable 2 An atomic measure with (at most countable) fixed atom matrices locations and random atom masses. 3 A non-random measure. Parts 2 and 3 can be easily dealt with, so we only consider part 1. Completely random measures and L´evyprocesses CRMs CRM: Distribution over measures that assign independent Sinead Williamson masses to disjoint subsets. This distribution is infinitely divisible, so L´evy-Khintchine Background applies. L´evyprocesses CRMs are closely related to L´evyprocesses: Completely random If X is a subordinator, then the measure M defined so measures M(t; s] = Xt − Xs is a CRM. Applications If M is a completely random measure on , then it's Normalized R+ random measures cumulative function is a subordinator. Neutral-to-the- right Just as a subordinator (with ν > 0) has a countably processes Exchangeable matrices infinite number of jumps, a CRM assigns positive mass to a countably infinite number of locations: 1 X M = πi δti ; i=1 where πi > 0 for all i. Completely random measures and Poisson processes CRMs Sinead Williamson Background L´evyprocesses Can catgorize atoms as (size, location) pairs in some space Completely R+ × SX . random measures Define a Poisson point process on this space with L´evy Applications measure ν(dz; ds). Normalized random measures Events of Poisson