Stochastic Pdes and Markov Random Fields with Ecological Applications

Intro SPDE GMRF Examples Boundaries Excursions References

Stochastic PDEs and Markov random ﬁelds with ecological applications

Finn Lindgren

Spatially-varying Stochastic Differential Equations with Applications to the Biological Sciences OSU, Columbus, Ohio, 2015

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References “Big” data

Z(Dtrn)

Illustration: Synthetic data mimicking satellite based CO2 measurements. Iregular data locations, uneven coverage, features on different scales.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Sparse spatial coverage of temperature measurements

raw data (y) 200409 kriged (eta+zed) etazed field

20 15 15

10 10 10

5 5

lat lat lat 0

0 0 −10

−5 −5 44 46 48 50 52

44 46 48 50 52 44 46 48 50 52 −20

2 4 6 8 10 14 2 4 6 8 10 14 2 4 6 8 10 14

lon lon lon

residual (y − (eta+zed)) climate (eta) eta field

20 2 15 18

10 16 1 14 5 lat 0 lat lat 12

0 10 −1 8 −5

44 46 48 50 52 6 −2 44 46 48 50 52 44 46 48 50 52

2 4 6 8 10 14 2 4 6 8 10 14 2 4 6 8 10 14

lon lon lon Regional observations: ≈ 20,000,000 from daily timeseries over 160 years

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Spatio-temporal modelling framework

Spatial statistics framework ◮ Spatial domain D, or space-time domain D × T, T ⊂ R. ◮ Random ﬁeld u(s), s ∈ D, or u(s, t), (s, t) ∈ D × T. ◮ Observations yi . In the simplest setting, yi = u(si )+ ǫi , but more generally yi ∼ GLMM, with u(·) as a structured random effect. ◮ Needed: models capturing stochastic dependence on multiple scales ◮ Partial solution: Basis function expansions, with large scale functions and covariates to capture static and slow structures, and small scale functions for more local variability

Two basic model and method components ◮ Stochastic models for u(·). ◮ Computationally efﬁcient (i.e. avoid MCMC whenever possible) inference methods for the posterior distribution of u(·) given data y.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Covariance functions and stochastic PDEs

The Matérn covariance family on Rd

21−ν Cov(u(0), u(s)) = σ2 (κksk)ν K (κksk) Γ(ν) ν

Scale κ> 0, smoothness ν > 0, variance σ2 > 0

Whittle (1954, 1963): Matérn as SPDE solution Matérn ﬁelds are the stationary solutions to the SPDE

α/2 κ2 −∇·∇ u(s)= W(s), α = ν + d/2

2 d ∂ 2 Γ(ν) W(·) white noise, ∇·∇ = i 2 , σ = 2ν d/2 =1 ∂si Γ(α)κ (4π) P Questions: What is “white noise”? How can we deﬁne fractional operators? How can we deal with SPDEs in hierarchical models? Why do we want to?

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Generalised Spectrum Markov White noise on continuous domains, and Brownian motion

∗ Let W (f ) be a Gaussian random measure for functions f ∈ L2(D), such that ∗ ∗ ∗ E(W (1A)) = 0, Cov(W (1A), W (1B )) = |A ∩ B|Leb(D). for all Lebesgue-measurable A, B ⊆ D. Gaussian white noise W(s) is then formally deﬁned by deﬁning ∗ hf , WiD ≡W (f ), so that

E(hf , WiD ) = 0, s s s Cov(hf , WiD , hg, WiD )= hf , giD ≡ f ( )g( )d , ZD hf , WiD ∼N (0, hf , f iD )

The Brownian motion Wt on R and the associated stochastic differential ∗ dWt from Radu’s lectures are related to W and W via ∗ Wt = W (1[0,t])

dWt = W(t)dt

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Generalised Spectrum Markov

A Gaussian random ﬁeld u : D 7→ R is deﬁned via

E(u(s)) = m(s), Cov(u(s), u(s′)) = K (s, s′),

u(si ), i = 1,..., n ∼N (m = m(si ), i = 1,..., n , Σ s s = K ( i , j ), i, j = 1,..., n ) for all finite location sets {s1,..., sn }, and K (·, ·) symmetric positive definite. A generalised Gaussian random field u : D 7→ R is defined via a random ∗ R measure, hf , uiD = u (f ): HR(D) 7→ ,

s s s E(hf , uiD )= hf , miD = f ( )m( )d , ZD s s s′ s′ s s′ Cov(hf , uiD , hg, uiD )= hf , RgiD ≡ f ( )K ( , )g( )d d , ZZD×D hf , uiD ∼N (hf , miD , hf , Rf iD ) R for all f , g ∈ HR(D) ≡ {f : D 7→ ; hf , Rf iD < ∞}. This allows for singular covariance kernels K (·, ·).

Gaussian white noise on continuous domains Standard Gaussian white noise W(·) is a generalised random ﬁeld, with

s s s′ s′ m( ) = 0, K ( , )= δs( ), hf , WiD ∼N (0, hf , f iD ), s for all f ∈ L2(D). Since hδs,δsiD = ∞ for all ∈ D, W(·) does not have pointwise meaning. We can only do calculus!

Independent Gaussian noise Spatially independent Gaussian noise w(·) is a random ﬁeld, with

′ m(s) = 0, K (s, s )= 1{s=s′}, w(s) ∼N (0, 1),

′ for all s, s ∈ D. However, for every set A ⊂ D with |A|Leb(D) > 0,

P(sup w(s)= ∞)= P(inf w(s)= −∞) = 1, s s∈A ∈A

and the generalised calculus is not applicable.

Bochner’s theorem on Rd A symmetric kernel K (s, s′), s, s′ ∈ Rd , is a positive (semi-)deﬁnite stationary covariance kernel if and only if there exists a non-negative spectral measure S ∗(ω) such that

K (s, s′)= exp(i(s′ − s) · ω)dS ∗(ω) Rd Z If the measure has a density S(ω),

K (s, s′)= exp(i(s′ − s) · ω)S(ω)dω Rd Z 1 s s s S(ω)= d exp(−i · ω)K (0, )d (2π) Rd Z d d White noise on R has spectral density SW (ω) = 1/(2π) .

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Generalised Spectrum Markov

Rd s s s Let D = , and f (ω)=(Ff )(ω)= Rd exp(i · ω)f ( )d . Very informally, S (ω)= E(|u(ω)|2). u R Differential operatorsb can also be interpreted spectrally: Lf b f ∇f −∇·∇f Lα/2f Lf ≡ F(Lf ) f iωf kωk2f |L|α/2f The rightmost column is a deﬁnition of a fractional operator! For the Whittle-Matérnbb SPDE, informally,b b b b b

(κ2 −∇·∇)α/2u(s)= W(s) (κ2 + kωk2)α/2u(ω)= W(ω) E(|(κ2 + kωk2)α/2u(ω)|2)= E(|W(ω)|2) c 2 2 α b (κ + kωk ) Su (ω)= SW (ω) c b 1 S (ω)= u (2π)d (κ2 + kωk2)α

′ Whittle (1954, 1963) showed that (FSu (·))(s − s) is equal to the Matérn covariance (up to a known scaling constant), with smoothness ν = α − d/2.

d For space-time ﬁelds, we write u(s, t), (s, t) ∈ R × R, and Su (k,ω), (k,ω) ∈ Rd × R. We drive a heat equation with a noise process E that is white noise in time and Matérn noise in space, with parameters matching the heat operator: ∂ γ + κ2 −∇ ·∇ u(s)= E(s, t), ∂t s s 2 α/2 (κ −∇s ·∇s ) E(s, t)= W(s, t). The Fourier domain version is iγω + κ2 + kkk2 u(k,ω)= E(k,ω), (κ2 + kkk2)α/ 2E(k,ω)= W(k,ω), b b and b c 1 S (k,ω)= u (2π)d+1(γ2ω2 +(κ2 + kkk2)2)(κ2 + kkk2)α

How differentiable are the realisations?

Using that, in the standardised Whittle-Matérn SPDE, the variance is

Γ(ν) σ2 = , ν = α − d/2, Γ(α)κ2ν (4π)d/2

the marginal spatial spectrum for the heat model is

1 1 Su (k)= Su (k,ω)dω = d , R 4πγ (2π) (κ2 + kkk2)α+1 Z which is a scaled Whittle spectrum for a Matérn covariance with smoothness ν = α + 1 − d/2.

If α = 0, d = 2, then ν = 0, which is just outside of the allowed range of the Matérn family. However, for every t, u(·, t) is a generalised random ﬁeld with s s′ 1 1 s′ s singular kernel K ( , )= 4πγ 2π K0(κk − k).

To help understand the temporal properties, take the Fourier transform in only the spatial directions:

∂ W(k, t) γ + κ2 + kkk2 u(k, t)= , ∂t (κ2 + kkk2)α/2 f so for each spatial frequency k, thee temporal evolution of u(k, t) is an Ornstein-Uhlenbeck process with covariance e 1 κ2 + kkk2 exp −|t| . 4πγ(κ2 + kkk2)α+1 γ

There is one more property we need to understand: Markov in space

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Generalised Spectrum Markov Markov properties on discrete and continuous domains

• - •→•→• [email protected]

:nd order Marker

Undirected Mahon •÷- • pcxtlxtt )=pl×d×Ei,K⇒ • .FI?w..gojE.DplxtkA 0 =pHtI×u+ ) all For up . seed : ¥9 , pE#x×*¥TIxn*

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Generalised Spectrum Markov

Markov in time, ﬁltration σ-algebras: σ a ∈ F(−∞,t] ≡ σ(u(s), s ≤ t) σ b ∈ F[t,∞) ≡ σ(u(s), s ≥ t) P(a ∩ b | u(t)) = P(a | u(t))P(b | u(t))

Markov for regular random ﬁelds on spatial and spatio-temporal domains: A, B, S ⊂ D, such that S separates A and B. σ s s FS ≡ σ(u( ), ∈ S), σ σ a ∈ FA, b ∈ FB , σ σ σ P(a ∩ b | FS )= P(a | FS )P(b | FS )

Markov for generalised random ﬁelds on spatial and spatio-temporal domains: σ FS ≡ σ(hf , uiS , f ∈ HR(S)), σ σ a ∈ FA, b ∈ FB , σ σ σ P(a ∩ b | FS )= P(a | FS )P(b | FS )

Markov properties S is a separating set for A and B: u(A) ? u(B) | u(S)

Solutions to α/2 κ2 −r·r u(s)=W(s) B are Markov when α is an integer. ! " (Generally, when the reciprocal of the S spectral density is a polynomial, Rozanov, 1977)

Σ−1 A be Discrete representations (Q = ): 0 QAB = QA|S,B = QAA − −1 − µA|S,B = µA QAAQAS (u S µS )

timibinnnamnaeii¥57 . :c ⇒

Rozanov (1977)

−1 u(s) is stationary Markov ⇐⇒ Su (k) ∝ P(k) where P(k) ≥ 0 is a symmetric polynomial

2 2 −α Matérn/Whittle: Su (k) ∝ (κ + kkk )

GMRF Covariance (R2)

SAR(1) ∝ κkuk K1(κkuk) Whittle (1954) ( CAR(2) 1 CAR(1) 2π K0(κkuk) Besag (1981)

1 ICAR(1) − 2π log(kuk) Besag & Mondal (2005) On lattices, classical CAR → Matérn models (limits of). Can extend to non-stationary SPDE models, on irregular triangulations.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Finite elements Examples Hierarchical Markov Laplace Hilbert space approximation

We want to construct ﬁnite dimensional approximations to the distribution of u(·), where

2 α/2 fi , (κ −∇·∇) u(·) D , i = 1,..., m =d hfi , W(·)iD , i = 1,..., m for all ﬁnite collections of test functions fi ∈ HR(D). A ﬁnite basis expansion

u(s)= ψj (s)uj j =1 X can only hope to achieve this for a subspace of size n. Two main approaches: ◮ Galerkin: {fi = ψi , i = 1,..., n} ◮ 2 α/2 Least squares: {fi =(κ −∇·∇) ψi , i = 1,..., n} We use least squares for α = 1, Galerkin for α = 2, and a recursion for α ≥ 3.

Stochastic Green’s ﬁrst identity On any sufﬁciently smooth manifold domain D,

hf , −∇·∇giD = h∇f , ∇giD −hf ,∂n gi∂D

holds, even if either ∇f or −∇·∇g are as generalised as white noise.

For now, we’ll impose deterministic Neumann boundary conditions, informally ∂n u(s) = 0 for all s ∈ ∂D. For α = 2 and Galerkin,

2 2 ψi , (κ −∇·∇) ψj uj = κ hψi ,ψj iD + h∇ψi , ∇ψj iD uj * j + j X D X =(κ2C + G)u

The covariance for the RHS of the SPDE is

Cov(hψi , WiD , hψj , WiD = hψi ,ψj iD = C

by the deﬁnition of W.

We seek u ∼N (0, Σ) such that Var{(κ2C + G)u} = C :

(κ2C + G)Σ(κ2C + G)= C Σ =(κ2C + G)−1C (κ2C + G)−1

If ψi are piecewise linear on a triangulation of D, then C and G are both very sparse, and in addition, C = diag(hψi , 1iD ) is a valid approximation. Then, the precision matrix is also sparse,

Q =(κ2C + G)C −1(κ2C + G)

and u is Markov on the adjacency graph given by the non-zero structure of Q. Least squares and Galerkin recursion gives precisions for all α = 1, 2,...: ◮ 2 Q1 =(κ C + G) ◮ 2 −1 2 4 2 −1 Q2 =(κ C + G)C (κ C + G)= κ C + 2κ G + GC G ◮ 2 −1 −1 2 Qα =(κ C + G)C Qα−2C (κ C + G) α ◮ 1/2 −1/2 2 −1/2 1/2 Any α ≥ 0: Qα = C C (κ C + G)C C (non-sparse for non-integern α) o

Basis deﬁnitions Finite basis set (k = 1,..., n) 2 −α Karhunen-Loève (κ −∇·∇) eκ,k (s)= λκ,k eκ,k (s) Fourier −∇·∇ek (s)= λk ek (s) 2 α/2 Convolution (κ −∇·∇) gκ(s)= δ(s) General ψk (s)

Field representations Field u(s) Weights Karhunen-Loève ∝ k eκ,k (s)zk zk ∼N (0,λκ,k ) 2 −α Fourier ∝ ek (s)zk zk ∼N (0, (κ + λk ) ) Pk Convolution ∝ k gκ(s − sk )zk zk ∼N (0, |cellk |) P −1 General ∝ ψk (s)uk u ∼N (0, Q ) Pk κ Note: Harmonic basis functionsP (as in the Fourier approach) give a diagonal

Qκ, but lead to dense posterior precision matrices.

(κ2 −∇·∇)u(s)= W(s), s ∈ D

(κ2 exp(iθ) −∇·∇)u(s)= W(s), s ∈ D, Re(u) independent of Im(u)

(κ(s)2 −∇·∇)u(s)= κ(s)W(s), s ∈ Ω

(κ2 −∇· H ∇)u(s)= W(s), s ∈ D

(κ2 −∇· H (s)∇)u(s)= W(s), s ∈ D

(κ2 −∇·∇)u(s)= W(s), s ∈ D

(κ2 −∇·∇)u(s)= W(s), s ∈ D = S2

(κ2 exp(iθ) −∇·∇)u(s)= W(s), s ∈ D = S2

(κ(s)2 −∇· H (s)∇)u(s)= κ(s)W(s), s ∈ Ω

Continuous Markovian spatial models (Lindgren et al, 2011)

Local basis: u(s)= k ψk (s)uk , (compact, piecewise linear) 0 −1 Basis weights: u ∼N (P, Q ), sparse Q based on an SPDE Special case: (κ2 −∇·∇)u(s)= W(s), s ∈ Ω 4 2 4 2 2 4 Precision: Q = κ C + 2κ G + G2 (κ + 2κ |ω| + |ω| )

Conditional distribution in a jointly Gaussian model

Cholesky decomposition (Cholesky, 1924)

Q = LL⊤, L lower triangular (∼O(n(d+1)/2) for d = 1, 2, 3) Q−1x = L−⊤L−1x, via forward/backward substitution

log det Q = 2 log det L = 2 log Lii i X

André-Louis Cholesky (1875–1918) "He invented, for the solution of the condition equations in the method of least squares, a very ingenious computational proce- dure which immediately proved extremely useful, and which most assuredly would have great beneﬁts for all geodesists, if it were published some day." (Euology by Commandant Benoit, 1922)

−1 p(u | θ) ∼N (µu , Qu ), y | u, θ ∼ p(y | u) −1 pG (u | y, θ) ∼N (µ, Q ) 0 = ∇u {ln p(u | θ)+ln p(y | u)}|u=µe e e 2 Q = Qu − ∇u ln p(y | u) u=µe

e Direct Bayesian inference with INLA (r-inla.org)

p(θ)p(u | θ)p(y | u, θ) p(θ | y) ∝ p (u | y, θ) G u=µe

pe(u i | y) ∝ pGG (u i | y, θ)p(θ | y ) dθ Z The main practicale limiting factors for the INLA methode are the number of latent variables and the number model parameters.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References LGCP Multiscale Preconditioning Mesh Example: Point process data

Log-Gaussian Cox processes Point intensity:

λ(s) = exp bi (s)βi + u(s) i ! X Inhomogeneous Poisson process log-likelihood: N

ln p({y k }| λ)= |D|− λ(s) ds + ln λ(y k ) D k Z X=1 The likelihood can be approximated numerically, e.g. n λ(s) ds ≈ λ(sj )wj , D j =1 Z X

where sj are mesh nodes, and wj = hψj , 1iD

Discretised ﬁeld and likelihood:

λ(s) = exp bi (s)βi + ψj (s)uj i j ! Xn Xn

ln p({y k }| λ) ≈|D|− λ(sj )wj + ln λ(y k ) j =1 k X X=1

Then, with λD = λ(si ) , AD = ψj (si ) , and Ay = ψj (yi ) ,

⊤ ⊤1 ∇u ln p({y k }| λ) ≈−AD diag(w)λD + Ay 2 ⊤ ∇u ln p({y k }| λ) ≈−AD diag(w) diag(λD )AD

2 and similarly for ∇β, ∇β, and ∇u ∇β.

4.0 4.5 5.0 5.5 6.0 0.0 0.2 0.4 0.6 0.8 1.0

◮ A temporally slow, simpliﬁed stochastic heat equation (non-separable) ∂ z(s, t) − γ ∇·∇z(s, t)= E(s, t) ∂t z (1 − γE ∇·∇)E(s, t)= WE (s, t)

◮ A temporally quick, spatially non-stationary SPDE/GMRF (separable) ∂ 2 ∂t + γt (κ(s) −∇·∇)(τ(s)a(s, t)) = Wa (s, t) ◮ Measurements yi = a(si , ti )+ z(si , ti )+ ǫi , discretised into 0 −1 y = A(a +(B ⊗ I )z )+ ǫ, ǫ ∼N ( , Qǫ ) where B maps from long-term basis functions to short-term, and A maps from short-term basis functions to the observations. The posterior precision can be formulated for (a + z, z)|y:

Q ⊗ Q + A⊤Q A −Q B ⊗ Q Q = t a ǫ t a (a+z,z)|y −B ⊤Q ⊗ Q Q + B ⊤Q B ⊗ Q t a z t a

Finite element construction of basis weight precision Non-stationary SPDE: (κ(s)2 −∇·∇)(τ(s)u(s)) = W(s)

The SPDE parameters are constructed via spatial covariates: p p τ τ κ κ log τ(s)= b0 (s)+ bj (s)θj , log κ(s)= b0 (s)+ bj (s)θj j =1 j =1 X X Finite element calculations give

T = diag(τ(si )), K = diag(κ(si ))

Cii = ψi (s) ds, Gij = ∇ψi (s) ·∇ψj (s) ds Z Z Q = T K 2CK 2 + K 2G + GK 2 + GC −1G T Combining this with an AR(1) discretisation of the temporal operator, we get Qt ⊗ Qa .

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References LGCP Multiscale Preconditioning Mesh

GMRF precision for simpliﬁed stochastic heat equation

(t) (s) (t) (s) (t) (s) Qz = M 2 ⊗ M 0 + M 1 ⊗ M 1 + M 0 ⊗ M 2 (s) M 0 = C + γE G (s) −1 M 1 = G + γE GC G (s) −1 −1 −1 M 2 = GC G + γE GC GC G

Ignoring the degenerate aspect of the model, the precision structure can be used to formulate sampling as 0 Qz z = Lz w, w ∼N ( , I )

where Lz is a pseudo Choleskye factor, (t) (t) (t) −⊤ Lze= L2 ⊗ LC , L1 ⊗ LG , L0 ⊗ GLC , hh 1/2 (t) (t) −⊤ (t) i −1 e γE L2 ⊗ LG , L1 ⊗ GLC , L0 ⊗ GC LG h ii

Write x =(a + z, z) for the full latent ﬁeld.

Q ⊗ Q + A⊤Q A −Q B ⊗ Q Q = t a ǫ t a x|y −B ⊤Q ⊗ Q Q + B ⊤Q B ⊗ Q t a z t a can be pseudo-Cholesky-factorised:

⊤ L ⊗ L 0 A⊤L Q = L L , L = t a ǫ x|y x|y x|y x|y −B ⊤L ⊗ L L 0 t a z e e e 0 Posterior expectation, samples, and marginal variances (withe A = A ): ⊤ Qx|y (µx|y − µx )= A Qǫ(y − Aµx ), e Q (x − µ )= L w, w ∼N (0, I ), or x|y x|y ex|y e ⊤ 0 Qx|y (x − µx )= Ae Qǫ(y − Aµx )+ Lx|y w, w ∼N ( , I ), Var(xi |y)= diag(inla.qinv(Qx y )) (requires Cholesky) e e e |

Solving Qx = b is equivalent to solving M −1Qx = M −1b. Choosing M −1 as an approximate inverse to Q gives a less ill-conditioned system. Only the action of M −1 is needed, e.g. one or more ﬁxed point iterations: Block Jacobi and Gauss-Seidel preconditioning

⊤ Matrix split: Qx|y = L + D + L Jacobi: x (k+1) = D−1 −(L + L⊤)x (k) + b Gauss-Seidel: x (k+1) =(L + D)−1 −L⊤x (k) + b Remark: Block Gibbs sampling for a GMRF posterior ⊤ With Q = Qx|y , b = A Qǫ(y − Aµx ) and x = x − µx ,

(k+1) −1 ⊤ (k) x =(L + D) −L x + b + eLD w , w ∼N (0, I ) Gauss-Seidele and Gibbs are both verye inefﬁcient on theire own. Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References LGCP Multiscale Preconditioning Mesh Finite element mesh

Triangulation mesh

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Theory 1D 2D All deterministic boundary conditions are ‘inappropriate’

● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

Normal derivatives at boundaryNormal derivatives ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

● ● ● −10● −5 0 5 10

−2 −1 0 1 2

Process at boundary

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Theory 1D 2D Stationary stochastic boundary adjustment (current work)

Recall the Matérn generating SPDE

(κ2 −∇·∇)α/2u(s)= W(s)

RKHS inner product/precision operator for Matérn ﬁelds on Rd :

α α 2α−2k k k hf , QRd gi = κ ∇ f , ∇ g D k D k=0 X Boundary adjusted precision operator on a compact subdomain, where P is a conditional expectation operator:

2 2 hf , QD giD = hf , QR giD − hPf , QR PgiD + hf , Q∂D gi∂D

2 = hf −Pf , QR (g −Pg)iD + hf , Q∂D gi∂D ,

where the boundary precision operator may involve normal derivatives of f and g.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Theory 1D 2D Covariances (D&N, Robin, Stoch) for κ =5 and 1 Covariance Covariance Covariance 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x Covariance Covariance Covariance 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Theory 1D 2D Derivative covariances (D&N, Robin, Stoch) for κ =5 and 1 Covariance Covariance Covariance 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x Covariance Covariance Covariance 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0 0.0 0.5 1.0 1.5 2.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Theory 1D 2D Process-derivative cross-covariances (D&N, Robin, Stoch)

●

● ●

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Cross−covariance Cross−covariance Cross−covariance

● −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x

● ● ● ● ● ●

● ● ●

●

● ● ● ● ● ● ● ● ● ● ● ● ● ●

●

Cross−covariance Cross−covariance ● Cross−covariance

● −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

−1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0 −1.0 −0.5 0.0 0.5 1.0

x x x

2.2

2.0

0.5

1.8

1.6 0.0

1.4

1.2 −0.5

1.0

0.8 −0.5 0.0 0.5

2.2

2.0

0.5

1.8

1.6 0.0

1.4

1.2 −0.5

1.0

0.8 −0.5 0.0 0.5

2.2

2.0 0.5 1.8

1.6 0.0

1.4

1.2 −0.5

1.0

0.8 −1.0 −0.5 0.0 0.5 1.0

2.2

2.0 0.5 1.8

1.6 0.0

1.4

1.2 −0.5

1.0

0.8 −1.0 −0.5 0.0 0.5 1.0

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References Excursion sets for random ﬁelds

Excursion sets Let u(s), s ∈ D be a random process. The positive and negative level u excursion sets with probability 1 − α are

+ + Eu,α(x) = argmax{|D| : P(D ⊆ Au (x)) ≥ 1 − α}. D − − Eu,α(x)= argmax{|D| : P(D ⊆ Au (x)) ≥ 1 − α}. D

These are sets with high probability for excursions in the entire set.

Excursion functions The positive and negative u excursion functions are given by

+ + Fu (s)= sup{1 − α; s ∈ Eu,α}, − − Fu (s)= sup{1 − α; s ∈ Eu,α}.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References

PM10 exceedances in Piemonte, January 30, 2006

+ Marginal probabilities F50(s)

5150 5150 1.0 1.0

5100 5100 0.8 0.8

5050 5050 0.6 0.6

5000 5000 0.4 0.4

4950 4950 0.2 0.2

4900 4900 0.0 0.0

350 400 450 500 350 400 450 500

Model estimated with INLA, result passed onward to excursions(), evaluating high dimensional GMRF probabilities and ﬁnding credible regions. Latest version has user friendly options for continuous domain interpretations.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References References (2/3)

◮ F. Lindgren, H. Rue and J. Lindström (2011), An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach (with discussion), JRSSB, 73(4):423–498. Code available in R-INLA, see http://r-inla.org/ ◮ Preliminary draft book chapter on GMRF computation basics: http://people.bath.ac.uk/fl353/tmp/gmrf.pdf ◮ R. Ingebrigtsen, F. Lindgren, I. Steinsland (2014), Spatial models with explanatory variables in the dependence structure, Spatial Statistics, 8:20–38. http://dx.doi.org/10.1016/j.spasta.2013.06.002 ◮ G-A. Fuglstad, F. Lindgren, D. Simpson, H. Rue (2015), Exploring a new class of non-stationary spatial Gaussian random fields with varying local anisotropy, Statistica Sinica, 25:115–133. http://arxiv.org/abs/1304.6949 ◮ D. Bolin, F. Lindgren (2015), Excursion and contour uncertainty regions for latent Gaussian models, JRSSB, 77(1):85–106. Accepted version at http://arxiv.org/abs/1211.3946 CRAN package: excursions

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References References (2/3)

◮ J. Besag (1974), Spatial interaction and the statistical analysis of latice systems (with discussion), JRSSB, 36(2):192–225. ◮ J. Besag and C. Kooperberg (1995), On conditional and intrinsic autoregressions, Biometrika, 82(4):733–746. ◮ J. Besag (1981), On a system of two-dimensional recurrence equations, JRSSB, 43(3):302–309. ◮ J. Besag and D. Mondal (2005), First-order intrinsic autoregressions and the de Wij process, Biometrika, 92(4):909–920. ◮ Y. A. Rozanov (1977), Markov random ﬁelds and stochastic partial differential equations, Mathematics of the USSR – Sbornik, 32(4):515–534 ◮ Y. A. Rozanov (1977), On the paper “Markov random ﬁelds and stochastic partial differential equations”, Mathematics of the USSR – Sbornik, 35(1):157–164 ◮ Y. A. Rozanov (1982, Russian original 1980), Markov Random Fields, Springer Verlag, New York.

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications Intro SPDE GMRF Examples Boundaries Excursions References References (3/3)

◮ Y. A. Rozanov (2010, reprint of 1st ed. 1998), Random ﬁelds and stochastic partial differential equations, Springer Verlag, Mathematics and its Applications. ◮ R. J. Adler and J. Taylor (2007), Random Fields and Geometry, Springer Monographs in Mathematics ◮ P. Whittle (1954), On stationary processes in the plane, Biometrika, 41(3/4):434–449. ◮ P. Whittle (1963), Stochastic processes in several dimensions, Bull. Inst. Internat. Statist., 40:974–994. ◮ B. Matérn (1960), Spatial variation. Stochastic models and their application to some problems in forest surveys and other sampling investigations. (Swedish: Stokastiska modeller och deras tillämpning på några problem i skogstaxering och andra samplingundersökningar.), Stockholm University, PhD thesis in Mathematical Statistics. You may be lucky and ﬁnd someone who has a copy!

Finn Lindgren - [email protected] Stochastic PDEs and Markov random ﬁelds with ecological applications