Lecture 10: Random Fields in Uncertainty Quantification

Home , Random field

Expected Learning Outcomes • The participants can explain the application of random fields in the • context of UQ They understand the difference between random variables and • random fields They are able to sample from a random field given its eigenvalues • and -vectors using the Karhunen-Loève (KL) expansion or given its covariance function. They understand the concept of finding eigenpairs of the KL • expansion

Algorithms for Uncertainty Quantiﬁcation Lecture 10: Random Fields Lecture 10: ST 2018

Tobias Neckel Scientific Computing in Computer Science Random fields in Uncertainty TUM Quantification

Repetition from previous lecture Concept of Building Block:

Sensitivity analysis Time: 90 minutes categorisation: local vs. global SA • ≈ • Content: Random ﬁelds global SA can provide useful insight • • in time • variance-based global SA & Sobol’ indices (for gPC) in space • • example: damped linear oscillator •

T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 3 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 4 | | | | | |

Concept of Building Block: Agenda

Time: 90 minutes • ≈ Content: Random fields • Topic in time • Random fields and their approximation in space • Expected Learning Outcomes • Content The participants can explain the application of random fields in the • random fields context of UQ • They understand the difference between random variables and particular example: stochastic processes • • random fields approximation of random fields: the Karhunen-Loève expansion They are able to sample from a random field given its eigenvalues • • example: approximation of the Wiener process and -vectors using the Karhunen-Loève (KL) expansion or given its • summary covariance function. • They understand the concept of finding eigenpairs of the KL • expansion

T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 4 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 5 | | | | | |

Random Variables vs. Random Fields Random variables until now, we discussed only random • variables intuitively, a random variable (RV) is a • number Random Fields i.e., a quantity that varies at a single • location 1 i.e., a mapping from the event space to • R Random fields: intuition what about quantities that vary temporally • or spatially? modeled as a “continuum” of random ⇒ variables i.e., a random field (RF) ⇒ 1source: https://www.mathsisfun.com/data/images/random-variable-1.svg T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 7 | | | Random Fields: Examples Random Fields: Formal Definition Application examples finance • Reminder: Probability space engineering • biochemistry 3 A probability space is a triple (Ω, , ), where • F P fractal analysis in medical Ω: sample space; set of all possible outcomes • • imagining : σ algebra; set of events s.t. each event is a set containing zero or • F − forecasting more outcomes • ... : [0, 1] probability measure that satisfies: • • P F → 1. (∅) = 0 P 2. (Ω) = 1 P 3. if Ai and Ai Aj = ∅, then ( ∞ Ai ) = ∞ (Ai ) ∈ F ∩ P ∪i=1 i=1 P 4 P 2 Reminder: Random variables 2source: jimmyakin.com/2012/05/brownian-motion-explained.html A random variable is a function X :Ω R s.t. ω Ω X(ω) x 3source: www.businessinsider.com/1929-stock-market-crash-chart-is-garbage-2014-2?IR=T → { ∈ | ≤ } ⊂ F 4source: http://www.imm.dtu.dk/ pcha/UQ/OEUQLec.pdf T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 8 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 9 | | | | | |

Random Fields: Formal Deﬁnition Random Fields: Formal Deﬁnition

Random fields Reminder: covariance matrix An X-valued random field (RF) is a collection of random variables X : t { t ∈ T } Covariance matrix: measure of correlation strength of two RV X and Y indexed by t , being a topological space. ∈ T T Examples: cov(X, Y ) = E[(X E[X])(Y E[Y ])] = E[XY ] E[X]E[Y ] − − − Gaussian random field: RF involving Gaussian PDFs of the random • variables Markov random field: RF that satisfies the Markov property Covariance function • ... Covariance function: function describing spatial/temporal covariance of a • RF/SP ( , ) := ( , ) Stochastic processes C t s cov Xt Xs If t = time, random field stochastic Properties: ∈ T ⇒ process (SP) C(t, s) is symmetric • C(t, s) is positive semi-definite •

T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 10 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 11 | | | | | |

Covariance Functions Example

Exponential covariance Exponential Kernel 1.0 V = 5 V = 0.5 V = 1.0 0.8 V = 0.1

0.6

C(t, s) := exp ( d(X , X )/V ) 0.4 − t s 0.2 Brownian Motion: An

0.0

4 2 0 2 4 Xs Xt Illustrative Example

Squared exponential covariance Squared Exponential Kernel 1.0

0.8

0.6 V = 5 V = 0.5 V = 1.0

2 2 0.4 C(t, s) := exp ( d (X , X )/(2V )) V = 0.1 − t s 0.2

0.0

4 2 0 2 4 Xs Xt d(X , X ) = absolute distance between X and X • t s t s V is the correlation length and constant • T. Neckel Algorithms for Uncertainty Quantiﬁcation L10: Random ﬁelds in UQ ST 2018 12 | | |

Stochastic process example: Brownian motion Brownian motion: generalities (R. Brown, 1827)

particles undergo constant, small • random fluctuations in a fluid due to atomic collisions How to approximate random Follow path stochastic process • ⇒ fields? Brownian motion: mathematical definition (N. Wiener, 1930s) Brownian motion Wiener process • ⇔ Wiener process: stochastic process W : t [0, ) s.t. • { t ∈ ∞ } W = 0 • 0 W is continuous in t • t the increments W W , W W ,... are (stochastically) • 1 − 0 2 − 1 independent W W (0, u) • t+u − t ∼ N C(t, s) = min (t, s) T. Neckel •Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 14 | | | Approximation of random fields approximate a RF/SP: infinite dimensions finite dimensions • ⇒ we want an approximation that is • accurate • relatively cheap to compute • approximation methods, e.g.: • direct sampling based on Cholesky decomposition • circulant embedding • Karhunen-Loève expansion • remark: almost all available approximations are not designed for generic • random fields

Approximation of random fields Approximation of random fields Random fields/stochastic processes in practice Random fields/stochastic processes in practice Remember: RF/SP are infinite-dimensional objects Remember: RF/SP are infinite-dimensional objects • • Q: how to use them in computer simulations? Q: how to use them in computer simulations? • • A: via approximations A: via approximations • • Approximation of random fields approximate a RF/SP: infinite dimensions finite dimensions • ⇒ we want an approximation that is • accurate • relatively cheap to compute • approximation methods, e.g.: • direct sampling based on Cholesky decomposition • circulant embedding • Karhunen-Loève expansion • remark: almost all available approximations are not designed for generic • random fields T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 16 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 16 | | | | | |

The Karhunen-Loève (KL) Expansion Properties of the Karhunen-Loève Expansion

Setting

let (Ω, , ) be a probability space • F P let Y (ω) be a random field designed for (but not restricted to) Gaussian processes • t • t the KL expansion is bi-orthogonal • ∈ T • ω Ω ψ (ω) are orthogonal • ∈ • n assume that m(Y (ω)) := [Y (ω)] and cov(Y (ω), Y (ω)) := C(t, s) are ζn(ω) are orthogonal • t E t t s • available ζ (ω) are uncorrelated • n the KL expansion is optimal w.r.t. the mean-squared error The KL expansion • ∞ if the process is Gaussian, ζn(ω) are i.i.d. Gaussian random variables Yt (ω) = m(Yt (ω)) + λnψn(t)ζn(ω), • Xn=1 p where λ , ψ (t) - the eigenpairs of C(t, s) • n n ζ (ω): uncorrelated random variables • n T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 17 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 18 | | | | | |

How to ﬁnd the Eigenpairs of the KL Expansion? Numerical Karhunen-Loève Expansion

Remember: we need solve Spectral representation of the covariance function • C(t, s)ψ (t)dt = λ ψ (s) Mercer’s theorem (Assume: C(t, s) is continuous): n n n • ZT ∞ analytical solution only for few types of covariance functions defined on C(t, s) = λnψn(t)ψn(s) • rectangular domains n=1 X therefore, no analytical solution for most domains/covariance functions • λ > 0 – eigenvalues, ψ (x ) – orthogonal basis functions need to resort to numerical methods • n n 1 • find λn, ψn(t) ∞ via a continuous eigenvalue problem Quadrature-based methods • { }n=1 mesh-free approach 2nd kind Fredholm integral eq: C(t, s)ψ (t)dt = λ ψ (s) • ⇒ n n n use quadrature nodes and weights to solve the eigenvalue problem (e.g., ZT • however, analytical solution for only few cases numerical methods Nyström method) • ⇒ in practice, truncate the expansion at M terms s.t. a certain % of the • Expansion-based methods variance is retained each eigenfunction approximated by linear combination of chosen basis M M • functions C(t, s) = λnψn(t)ψn(s) Yt (ω) = m(Yt (ω)) + λnψn(t)ζn(ω) ⇒ resulting error minimized via e.g. Galerkin method, stochastic collocation Xn=1 Xn=1 • T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018p 19 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 20 | | | | | |

KL Expansion Example KL expansion example

Example with analytical solution Remember: Brownian motion/Wiener process W : t [0, ) • { t ∈ ∞ } for simplicity, restrict t [0, 1] • ∈ C(t, s) = min (t, s) • to find the eigenpairs needed for the KL expansion, solver • 1 min (t, s)ψn(t)dt = λnψn(s) Z0 after some fancy math, we get • 1 λ = , ψ (t) = √2 sin ((n + 0.5)πt), ζ (0, 1) n (n + 0.5)2π2 n n ∼ N M ∞ sin ((n + 0.5)πt) W = m(W (θ)) + λ ψ (t)ζ (θ) = √2 ζ t t n n n (n + 0.5)π n Xn=1 p Xn=1 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 21 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 22 | | | | | | KL expansion example KL expansion pros & cons

Pros Eigenvalues of the Wiener process' KL expansion separates the stochastic and deterministic parts (similar to the polynomial • M = 1000 chaos expansion!) 10 1 bi-orthogonal (random variables and the deterministic basis functions) • based on uncorrelated random variables the approximation is a • ⇒ decorrelated process

10 2 optimal approximation from a mean squared error point of view •

eigenvalue Cons assumption that the underlying process is Gaussian • 10 3 each expansion term = one stochastic dimension more terms • ⇔ ⇒ higher dimensionality no analytical solution for most covariance functions 0 200 400 600 800 1000 • M - number of KL terms computation of covariance’s eigenpairs can be expansive •

T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 23 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 24 | | | | | |

Bringing Everything Together Literature

R. Ghanem, P. Spanos: Stochastic Finite Elements: A Spectral ... • Approach, Springer New York, 1991

T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 25 T. Neckel Algorithms for Uncertainty Quantification L10: Random fields in UQ ST 2018 26 | | | | | |

Summary

random fields • motivation • application examples • formal definition • stochastic processes • definition • example: Brownian motion • approximation of random fields • the Karhunen-Loève expansion • example: approximation of the Wiener process •

T. Neckel Algorithms for Uncertainty Quantiﬁcation L10: Random ﬁelds in UQ ST 2018 27 | | |