Random fields, Fall 2014 1

fMRI brain scan Ocean waves Nobel prize 2996 to The oceans cover 72% of John C. Mather the earth’s surface. George F. Smoot Essential for life on earth,

“for discovery of the and huge economic blackbody form and importance through anisotropy of the cosmic fishing, transportation, microwave background oil and gas extraction radiation" PET brain scan

TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A The course

• Kolmogorov existence theorem, separable processes, measurable processes • Stationarity and isotropy • Orthogonal and spectral representations • Geometry • Exceedance sets • Rice formula • Slepian models

Literature

An unfinished manuscript “Applications of RANDOM FIELDS AND GEOMETRY: Foundations and Case Studies” by Robert Adler, Jonathan Taylor, and Keith Worsley.

Complementary literature:

“Level sets and extrema of random processes and fields” by Jean- Marc Azais and Mario Wschebor, Wiley, 2009 “Asymptotic Methods in the Theory of Gaussian Processes” by Vladimir Piterbarg, American Mathematical Society, ser. Translations of Mathematical Monographs, Vol. 148, 1995 “Random fields and Geometry” by Robert Adler and Jonathan Taylor, Springer 2007

Slides for ATW ch 2, p 24-39

Exercises: 2.8.1, 2.8.2, 2.8.3, 2.8.4, 2.8.5, 2.8.6 + excercises in slides Stochastic convergence (assumed known) 푎.푠. Almost sure convergence: 푋푛 X 퐿2 Mean square convergence: 푋푛 X 푃 Convergence in probability: 푋푛 X 푑 푑 푑 Convergence in distribution: 푋푛 X, 퐹푛 F, 푃푛 P, …

푎.푠. 푃 • 푋푛 X ⇒ 푋푛 푋 퐿2 푃 • 푋푛 X ⇒ 푋푛 푋 푃 퐿2 • 푋푛 푋 plus uniform integrability ⇒ 푋푛 X 푃 푎.푠. • 푋푛 푋 ⇒ there is a subsequence {푛푘} with 푋푛푘 X 푑 • The random variables don’t really mean anything for In particular, the 푋푛 and 푋 don’t need to be defined on the same probability space, and don’t need to have a simultaneous distribution Random field

푇 parameter space. In this course 푇 is 푹푁 some 푁 ≥ 1 or a subset (e.g. box or sphere or surface of sphere) of 푹푁 푹푑 value space

An (푁, 푑) random field is a collection (or family) of random variables 푓푡; 푡 ∈ 푇 where 푇 is a set of dimension 푁 and the 푓푡 (or 푓(푡)) take values in 푹푑

푇 Or, a random function with values in 푹푑

푇 Or, a probablility measure on 푹푑 A realisation (or sample function, or sample path, or sample field, or observation, or trajectory, or …) is the function

푑 푓푡 휔 : 푇 푹 푡 ft(휔) for 휔 fixed. Two examples below:

Microscopy image of Thresholded Gaussian tablet coating field Terminology

random variable stochastic variable

random element random vector stochastic element stochastic vector

random field random process stochastic field stochastic process Finite dimensional distributions

The finite-dimensional distribution functions of an (푁, 푑) random field {푓푡} are defined as

퐹푡1,…,푡푛 풙1, … , 풙푛 = 푃(푓푡1 ≤ 풙1, … , 푓푡푛 ≤ 풙n) and the family of finite-dimensional distribution functions is the set 푑 {퐹푡1,…,푡푛 풙1, … , 풙푛 ; 푡1, … , 푡푛 ∈ 푇, 풙1, … , 풙푛 ∈ 푅 , 푛 ≥ 1}

This family has the following obvious properties: Symmetry: it is not changed under a simultaneous permutation of 푡1, … , 푡푛 and 풙1, … , 풙푛 Consistency:

퐹푡1,…,푡푛 풙1, … , 풙푛−1, ∞ = 퐹푡1,…,푡푛−1 풙1, … , 풙푛−1 Example of symmetry: 퐹푡1,푡2 풙1, 풙2 = 퐹푡2,푡1 풙2, 풙1

Example of consistency: Marginal distributions may be obtained from bivariate distributions,

퐹푡 풙 = 퐹푡,푠 풙, ∞

Three sample paths of a 1,1 random field. 퐹2,5,8 푥1, 푥2, 푥3 is the probability to obtain a sample path which passes through all three vertical lines

(in a more general theory one instead of finite-dimensional distributions uses probabilities of cylindersets,

푃(푓푡1 ∈ 푩1, … , 푓푡푛 ∈ 푩n) ) (Daniell-)Kolmogorov extension theorem To any symmetric and consistent family of finite-dimensional distributions 푑 {퐹푡1,…,푡푛 풙1, … , 풙푛 ; 푡1, … , 푡푛 ∈ 푇, 풙1, … , 풙푛 ∈ 푅 , 푛 ≥ 1} there exists a probability triple (Ω, , 푃) and an 푁, 푑 B random field {푓푡; 푡 ∈ 푇 } which has these finite-dimensional distributions In the proof one takes 푇 푇 Ω = 푹푑 , = (푹푑) , 푇 and 푃 as the measure on (푹푑) which is uniquely determined by the finite-dimensional distribution. Thus an element of Ω is a function 푓: 푇 푅푑 which maps a point 푡 ∈ 푇 to the value 푓(푡). The field is defined as

{푓푡 휔 = 푓 푡 ; 푡 ∈ 푇} Limitations of Kolmogorov’s theorem Many interesting sets, such as the set

퐶 = {휔; 푓푡 휔 is a continous function of 푡} 푇 do not belong to = (푹푑) , and hence, in Kolmogorov’s construction , the probabability of such events is not defined.

One important way around this problem is to make a direct construction of the field on some other probability space (Ω, , 푃) where the interesting sets belong to , say 퐶 ∈ , so thatB their probabilities, say 푃(퐶), is well defined. And then, more fields are obtained as functions of the already constructed field! Modifications

A field 푔푡; 푡 ∈ 푇 is a modification of the field 푓푡; 푡 ∈ 푇 if

푃 푔푡 = 푓푡 = 1, ∀푡 ∈ 푇

It is obvious (!) that 푔푡 has the same finite dimensional distributions as 푓푡. The other common way to circumvent the limitation is to construct, on Kolmogorov’s (Ω, , 푃) a modification of 푓 B 푡 which has the desired properties, say continuity. Whether this is possible or not (of course) dependes on which finite-dimensional distributions one is interested in. E.g. if they correspond to a Browninan motion it is possible, if they correspond to a Poisson process, it isn’t. Doob’s separability

A field 푓푡; 푡 ∈ 푇 is separable if there is a countable subset 푇 푆 ∈ 푇 and a null set Λ ∈ (푹푑) such that for every B closed set 퐵 ∈ 푹푑 and open set 퐼 ∈ 푇 it holds that

푓푡 휔 ∈ 퐵, ∀푡 ∈ 푆 ∩ 퐼 ⇒ 휔 ∈ Λ or 푓푡 휔 ∈ 퐵, ∀푡 ∈ 퐼 A separable modification of a field always exists (at least for 푁 = 푑 = 1? ), and it can be seen that e.g. if a continuous modification of a field exists, then the separable modification is continuous. Example of modification: Ω = 0,1 , is the Borel sets on [0, 1], 푃 is Lebesgue measure, 푓푡 휔 = 0, ∀푡, 휔 and 0 if 푡 ≠ 휔 푔 휔 = 푡 1 if 푡 = 휔 Measurable fields

A field 푓푡; 푡 ∈ 푇 is measurable if for almost all 휔 the sample path (function) 푑 푓 . 휔 : 푇 푅

푡 푓푡(휔) is (푅푑)-measurable (holds e.g. if the field is a.s. continuous).B It then follows that the function of two variables 푓푡(휔) is measurable with respect to the product sigma-algebra × (푅푑), and one can then define integrals like 푇 ℎ 푓푡 푑푡 and use Fubini’s theorem for calculations like

퐸 ℎ 푓푡 푑푡 = 퐸(ℎ 푓푡 )푑푡 푇 푇 (above we have assumed that and (푅푑) are complete) ATW basically say that it is nice if one has seen the concepts of Kolmogorov extension, modification, and Doob separability, but that this has been taken care of once and for all by Kolmogorov, Doob and others, and that we shouldn’t worry about it any more in this course. And this is right (I hope). However, things are different for the theory of ”Empirical Processes”, the so far most efficient and high-tech tool to find asymptotic distributions of statistical estimators. In this theory, such ”measureability problems” pose important techical problems, and has formed much of the entire theory. Empirical process theory is closely related to the metods used to prove continuity and differentiability in this course. Gaussian fields (ATW p. 25-28)

A random vector 퐗 = 푋1, … , 푋푑 has a multivariate Gaussian distribution iff one of the following conditions hold: 푑 • 훼, 푥 ≜ 푖 훼푖푋푖 has a univariate normal distribution for all 훼 ∈ 푅푑. • There exist a vector 풎 ∈ 푹푑 and a non-negative definite matrix 퐶 such that for all 휽 ∈ 푹푑 1 휙 휽 = 퐸 푒푖휽푋 = e푖휃풎−2휽퐶휽´

If 퐶 is positive definite and 푋 has the probability density 1 1 푒−2 풙−풎 퐶 풙−풎 ´ 2휋 푑 퐶 1/2 then 푋 is Gaussian. 풅 Here 푚 = 퐸 푿 and 퐶 = 퐶표푣 푿 . Similarly if the 푿풊 ∈ 푹 We write 퐗~푁푑 풎, 퐶 if 퐗 has a d-variate Gaussian distribution with mean 푚 and covariance matrix 퐶.

Excercises (the first is (2.2.5), the second Exercise 2.8.2):

(i) if 퐗~푁푑 풎, 퐶 and 퐴 is a 푑 × 푑 matrix, then 퐗퐴~푁푑 풎퐴, 퐴´퐶퐴 1 2 1 2 (ii) If 푿 = 푿 , 푿 with 푿 = 푋1, … , 푋푛 , 푿 = 푋푛+1, … , 푋푑 , with mean vectors 푚1 and 푚2 and covariance matrix 퐶 = 퐶1,1 퐶1,2 , then the conditional distribution of 푿1 given 퐶2,1 퐶2,2 푿2 is n-variate normal with mean 1 2 2 −1 풎1|2 = 풎 + (푿 −풎 )퐶2,2 퐶2,1 and covariance matrix −1 퐶1|2 = 퐶1,1 − 퐶1,2퐶2,2 퐶2,1 A Gaussian random field is hence, by the Kolmogorov theorem, determined by its means and covariances

Conversely, it also follows from the Kolmogorov theorem that given a function 푚: 푇 푹 and a non-negative definite function 퐶: 푇 × 푇 푹 there exist an 푁, 1 Gaussian random field which has 풎 as mean function and 퐶 as covariance function.

푁, 푑 Gaussian random fields for 푑 > 1 are the same, one just has to use more general notation. Gaussian related fields (ATW p. 28-30)

An (푁, 푑) Gaussian related field 푓(푡); 푡 ∈ 푇 is defined from a (푁, 푘) Gaussian field 푔(푡); 푡 ∈ 푇 using a function 퐹: 푅푘 푅푑 by the formula

푓 푡 = 퐹 푔 푡 = 퐹 g1 t , … , 푔푘 푡 . Examples: • Instantaneous function of Gaussian field: 푘 = 푑 and 퐹 is invertible 2 푘 2 • 휒 -field: 푑 = 1 and 퐹 풙 = 푖=1 푥푖

푥1 푘−1 • 푡-field: 푑 = 1 and 퐹 풙 = 푘 2 1/2 ( 푖=2 푥푖 ) 푛 2 푚 푖=1 푥푖 • 퐹-field: 푑 = 1, 푘 = 푚 + 푛 and 퐹 풙 = 푛+푚 2 푛 푖=푛+1 푥푖 Stationarity and isotropy (ATW p. 30-31) Weak stationarity: A random field is weakly stationary if • 풎 푡 ≜ 퐸 푓(푡) is constant • 퐶 푠, 푡 ≜ 퐸{(푓 푠 − 풎 푠 )´(푓 푡 − 풎 푡 } only depends on 푡 − 푠 Weak isotropy: A random field is weakly isotropic if 퐶 푠, 푡 only depends on |푡 − 푠| A random field is strictly stationary if the joint distribution of {푓 푡1 + 휏 , … , 푓 푡푛 + 휏 ) doesn’t depend on 휏, for all 푁 푛 ≥ 1, 푡1, … , 푡푛 ∈ 푹 . A random field is strictly isotropic if it is stationary and the joint distribution of {푓 푡1 , … , 푓 푡푛 } is invariant under 푁 rotations, for all 푛 ≥ 1, 푡1, … , 푡푛 ∈ 푹 . Weak is the same as strict for real Gaussian fields

”Weak” is sometimes instead called ”second order”

Abuse of notation: For weakly stationary fields one writes 퐶 푠, 푡 = 퐶 푡 − 푠 For istropic fields one writes 퐶 푠, 푡 = 퐶 |푡 − 푠| Cosine processes and fields (ATW p. 32-36)

Cosine process (a (1,1) field): 푓 푡 ≜ 휉 cos 휆푡 + 휉′ sin 휆푡 = 푅푐표푠(휆푡 − 휃) where 휉 and 휉′ are uncorrelated and have the same distribution, and (for convenience?) mean 0, and 푅2 = 휉2 + 휉′ 휉′ 2, and 휃 = arctan( ). R is ”amplitude”, 휃 is ”phase”, 휉 and 휆 is ”angular frequency” . Then 퐸 푓 푡 = 0 and 퐶 푠, 푡 = 퐸{푓 푠 푓 푡 } = 퐸{(휉 cos 휆푠 + 휉′ sin 휆푠)(휉 cos 휆푡 + 휉′ sin 휆푡)} = 퐸 휉2 (cos 휆푠 cos 휆푡 + sin 휆푠 sin 휆푡) = 퐸(휉2) cos 휆 푡 − 푠 휆 in the cosine process is “angular frequency”. Sometimes one instead writes 푓 푡 = Rcos (2휋휔푡 + 휃) 휔 then is “ frequency”

If 휉, 휉′ are Gaussian, then 푅2 is exponential with parameter 2휎2 (why?), so that 푃(푅 ≥ 푢) = exp (−푢2/휎2), and 휃 is independent of 푅 and uniformly distributed on 0, 2휋 (do the calculation!). The following are of central interest in the course: 푑푓 푡 • 푁 = 푁 (푓, 푇) ≜ #{푡 ∈ 푇; 푓 푡 = 푢 and > 0} 푢 푢 푑푡 • 푃( sup 푓 푡 ≥ 푢) 0≤푡≤푇 For a Gaussian cosine process, and Ψ 푢 ≜ 푃 푁 0,1 > 푢 ,

푃( sup 푓 푡 ≥ 푢) = 푃 푓 0 ≥ 푢 + 푃(푓 0 < 푢, 푁푢 ≥ 1) 0≤푡≤푇 = Ψ 푢/휎 + 푃(푓 0 < 푢, 푁푢 ≥ 1) If 푇 ≤ 휋/휆 then

푃 푓 0 < 푢, 푁푢 ≥ 1 = 푃 푁푢 ≥ 1 = 푃 푁푢 = 1 , and 푁푢 = 1 iff both 푅 ≥ 푢 and 휃 falls in an interval of lenght 휆푇 (requires some thinking: draw a picture). Since these two events are independent, 푢 휆푇 2 2 푃( sup 푓 푡 ≥ 푢) = Ψ + × 푒−푢 /2휎 0≤푡≤푇 휎 2휋 If 푇 > 2휋/휆, then sup 푓 푡 ≥ 푢 iff 푅 > 푢, so that 0≤푡≤푇 2 2 푃( sup 푓 푡 ≥ 푢) = 푃(푅 ≥ 푢) = 푒−푢 /2휎 0≤푡≤푇 Without assuming Gaussianity, for any differentiable stochastic process (i.e. (1,1)-field) we get the important general bound

푃( sup 푓 푡 ≥ 푢) = 푃 푓 0 ≥ 푢 + 푃 푓 0 < 푢, 푁푢 ≥ 1 0≤푡≤푇 ≤ 푃 푓 0 ≥ 푢 + 푃 푁푢 ≥ 1 ≤ 푃 푓 0 ≥ 푢 + 퐸(푁푢)

Cosine field (a (푁, 1) field): 푁 1 푓 푡 = 푓 푡1, … , 푡푁 ≜ 푓푘 휆푘푡푘 , 푁 푘=1 where ′ 푓푘 푡 = 휉푘 cos 푡 + 휉푘 sin 푡 ′ and the 휉푘 and 휉푘 are uncorrelated and have the same distribution, with mean 0. 푁 If 푇 = 푘=1 0, 푇푘 , then taking the supremum first over 푡1, then over 푡2, then … we get that 푁 1 sup 푓 푡 = sup 푓푘 휆푘푡 . 0≤푡≤푇 푁 0≤푡≤푇 푘=1 ′ If the 휉푘 and 휉푘 are Gaussian, and 푇푘 ≤ 휋/휆푘, k = 1, … , 푁 this gives an explicit (but complicated) formula. Orthogonal expansions (ATW p. 36-39)

An orthogonal exansion of an (푁, 1) field is an expression ∞ 푓 푡 = 휉푛휑푛(푡) , 푛=1 with 휉푛 uncorrelated centered (i.e. 퐸 휉푛 = 0, just for 2 2 convenience) random variables with 퐸 휉푛 = 휎푛 , and 휑푛 non-random orthogonal functions 푇 푹. (For 푁, 푑 fields the 휉푛 are matrices and the 휑푛 are vector valued.) The moment functions then are 퐸 푓 푡 = 0 and ∞ 2 퐶 푠, 푡 = 퐸 푓 푠 푓 푡 = 휎푛 휑푛 푠 휑푛(푡) 푛=1 ∞ 2 2 2 푉 푡 ≜ 퐸 푓 푡 = 휎푛 휑푛 푡 푛=1 Important for theory, application , and computation.

A Gaussian field with continuous covariance always has a Reproducing Kernel Hilbert Space (RKHS) ortogonal expansion. Loosley it is obtained as follows: set 푛

푆 = {푢: 푇 푹; 푢 ∙ = 푎푖퐶 푠푖,∙ , 푎푖real, 푠푖 ∈ 푇, 푛 ≥ 1}, 푖=1 and define an inner product on 푆 by 푛 푚 푛 푚

푢, 푣 = ( 푎푖퐶 푠푖,∙ , 푏푖퐶 푡푗,∙ ) = 푎푖푏푗퐶(푠푖, 푡푗) 푖=1 푗=1 푖_1 푗=1 ”Reproducing kernel ” comes from

푛 푛

푢, 퐶(푡,∙ ) = 푎푖퐶 푠푖,∙ , 퐶 푡,∙ = 푎푖퐶 푠푖, 푡 = 푢(푡) 푖=1 푖=1 If 퐶 푠, 푡 is positive definite, then 푢 = 푢, 푢 1/2 is a norm and one can define the RKHS 퐻(푆) as the closure of 푆 in this norm. If {휑푛} is a complete orhtonormal system in 퐻(푆), and the 휉푖 are 푁(0,1) then the field {푓 푡 } has an orthogonal expansion ∞ 푓 푡 =푑 휉푛휑푛(푡) 푛=1

This is important, but not always easy to handle. We will next briefly describe a somewhat more concrete expansion, the Karhunen-Loeve expansion, and then, in much more detail, the by far most important orthogonal expansions, the spectral representations, which expresses the field as a sum of cosine processes. The Karhunen-Loeve expansion applies to the case when 푇 is a compact set in 푹푁. Let the operator 퐶: 퐿2 푇 퐿2 푇 be defined by

퐶휓 푡 = 퐶 푠, 푡 휓 푠 푑푠 푇 and let 휆1 ≥ 휆2 ≥ ⋯ its eigenvalues and 휓1 ≥ 휓2 ≥ ⋯ the corresponding orthonormal eigenfunctions. It can be shown that 휆푛휓푛 is an orthonormal system in the RKHS 퐻 퐶 , and hence ∞ 푓 푡 =푑 푛=1 휉푛 휆푛 휓(푡). In general convergence is in mean square. For continuous fields, the sum also converges 푎. 푠. Again it may be difficult to find the eigenvalues and eigenfunctions, but discretization may often work