Introduction Geostatistical Model Covariance structure Cokriging Conclusion Geostatistics for Gaussian processes Hans Wackernagel Geostatistics group — MINES ParisTech http://hans.wackernagel.free.fr Kernels for Multiple Outputs and Multi-task Learning NIPS Workshop, Whistler, July 2009 NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Introduction Geostatistical Model Covariance structure Cokriging Conclusion Introduction NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Geostatistics and Gaussian processes Geostatistics is not limited to Gaussian processes, it usually refers to the concept of random functions, it may also build on concepts from random sets theory. Geostatistics Geostatistics: is mostly known for the kriging techniques. Geostatistical simulation of random functions conditionnally on data is used for non-linear estimation problems. Bayesian inference of geostatistical parameters has become a topic of research. Sequential data assimilation is an extension of geostatistics using a mechanistic model to describe the time dynamics. In this (simple) talk: we will stay with linear (Gaussian) geostatistics, concentrate on kriging in a multi-scale and multi-variate context. A typical application may be: the response surface estimation problem eventually with several correlated response variables. Statistical inference of parameters will not be discussed. We rather focus on the interpretation of geostatistical models. Geostatistics: definition Geostatistics is an application of the theory of regionalized variables to the problem of predicting spatial phenomena. (G. MATHERON, 1970) We consider the regionalized variable z(x) to be a realization of a random function Z (x). Stationarity For the top series: we think of a (2nd order) stationary model For the bottom series: a mean and a finite variance do not make sense, rather the realization of a non-stationary process without drift. Second-order stationary model Mean and covariance are translation invariant The mean of the random function does not depend on x: h i E Z (x) = m The covariance depends on length and orientation of the vector h linking two points x and x0 = x + h: h i cov(Z (x), Z (x0)) = C(h) = E Z (x) − m · Z (x+h) − m Non-stationary model (without drift) Variance of increments is translation invariant The mean of increments does not depend on x and is zero: h i E Z (x+h) − Z (x) = m(h) = 0 The variance of increments depends only on h: h i var Z (x+h)−Z (x) = 2 g(h) This is called intrinsic stationarity. Intrinsic stationarity does not imply 2nd order stationarity. 2nd order stationarity implies stationary increments. The variogram With intrinsic stationarity: 1 h 2 i g(h) = E Z (x+h) − Z (x) 2 Properties - zero at the origin g(0) = 0 - positive values g(h) ≥ 0 - even function g(h) = g(−h) The covariance function is bounded by the variance: C(0) = s2 ≥ jC(h) j The variogram is not bounded. A variogram can always be constructed from a given covariance function: g(h) = C(0) − C(h) The converse is not true. What is a variogram ? A covariance function is a positive definite function. What is a variogram? A variogram is a conditionnally negative definite function. In particular: any variogram matrix G = [g(xa−xb)] is conditionally negative semi-definite, > > [wa] g(xa−xb) [wa] = w G w ≤ 0 for any set of weights with n ∑ wa = 0. a=0 Ordinary kriging n n ? Estimator: Z (x0) = ∑ wa Z (xa) with ∑ wa = 1 a=1 a=1 Solving: " n # ? arg min var (Z (x0) − Z (x0)) − 2m( ∑ wa − 1) w1,...,wn,m a=1 yields the system: 8 n > w g(x −x ) + m = g(x −x ) 8a > ∑ b a b a 0 < b=1 n > w = 1 > ∑ b : b=1 n 2 and the kriging variance: sK = m + ∑ wa g(xa−x0) a=1 Introduction Geostatistical Model Covariance structure Cokriging Conclusion Anisotropy of the random function Consequences in terms of sampling design Mobile phone exposure of children by Liudmila Kudryavtseva NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Phone position and child head Head of 12 year old child SAR exposure (simulated) Max SAR for different positions of phone The phone positions are characterized by two angles The SAR values are normalized with respect to 1 W. Regular sampling. Variogram: 4 directions Linear anisotropic variogram model The sample variogram is not bounded. The anisotropy is not parallel to coordinate system. Max SAR kriged map Prediction error Kriging standard deviations sK The sampling design is not appropriate due to the anisotropy. Prediction error Different sample design Changing the sampling design leads to smaller sK. Introduction Geostatistical Model Covariance structure Cokriging Conclusion Geostatistical Model NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Introduction Geostatistical Model Covariance structure Cokriging Conclusion Linear model of coregionalization The linear model of coregionalization (LMC) combines: a linear model for different scales of the spatial variation, a linear model for components of the multivariate variation. NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Two linear models S Linear Model of Regionalization: Z (x) = ∑ Yu(x) u=0 h i E Yu (x+h) − Yu (x) = 0 h i E Yu (x+h) − Yu (x) · Yv (x+h) − Yv (x) = gu (h) duv N Linear Model of PCA: Zi = ∑ aip Yp p=1 h i E Yp = 0 cov Yp, Yq = 0 for p 6= q Linear Model of Coregionalization Spatial and multivariate representation of Zi (x) using p u uncorrelated factors Yu (x) with coefficients aip: S N u p Zi (x) = ∑ ∑ aip Yu (x) u=0 p=1 p Given u, all factors Yu (x) have the same variogram gu(h). This implies a multivariate nested variogram: S G(h) = ∑ Bu gu(h) u=0 Coregionalization matrices The coregionalization matrices Bu characterize the correlation between the variables Zi at different spatial scales. In practice: 1 A multivariate nested variogram model is fitted. 2 Each matrix is then decomposed using a PCA: N h u i h u u i Bu = bij = ∑ aip ajp p=1 yielding the coefficients of the LMC. LMC: intrinsic correlation When all coregionalization matrices are proportional to a matrix B: Bu = au B we have an intrinsically correlated LMC: S G(h) = B ∑ au gu(h) = B g(h) u=0 In practice, with intrinsic correlation, the eigenanalysis of the different Bu will yield: different sets of eigenvalues, but identical sets of eigenvectors. Regionalized Multivariate Data Analysis With intrinsic correlation: The factors are autokrigeable, i.e. the factors can be computed from a classical MDA on the variance-covariance matrix V =∼ B and are kriged subsequently. With spatial-scale dependent correlation: The factors are defined on the basis of the coregionalization matrices Bu and are cokriged subsequently. Need for a regionalized multivariate data analysis! Regionalized PCA ? Variables Zi (x) # no Intrinsic Correlation ? −! gij (h) = ∑ Bu gu (h) u # yes # PCA on B PCA on Bu ## Transform into Y Cokrige Y ? (x) u0p0 ## Krige Y ? (x) −! Map of PC p0 Introduction Geostatistical Model Covariance structure Cokriging Conclusion Multivariate Geostatistical filtering Sea Surface Temperature (SST) in the Golfe du Lion NIPS 2009 • Whistler, december 2009 Hans Wackernagel Geostatistics for Gaussian processes Modeling of spatial variability as the sum of a small-scale and a large-scale process Variogram of SST SST on 7 june 2005 The variogram of the Nar16 im- age is fitted with a short- and a long-range structure (with geo- metrical anisotropy). The small-scale components of the NAR16 image and of corresponding MARS ocean-model output are extracted by geostatistical filtering. Geostatistical filtering Small scale (top) and large scale (bottom) features 3˚ 4˚ 5˚ 6˚ 7˚ 3˚ 4˚ 5˚ 6˚ 7˚ 43˚ 43˚ 43˚ 43˚ 3˚ 4˚ 5˚ 6˚ 7˚ 3˚ 4˚ 5˚ 6˚ 7˚ −0.5 0.0 0.5 −0.5 0.0 0.5 NAR16 image MARS model output 3˚ 4˚ 5˚ 6˚ 7˚ 3˚ 4˚ 5˚ 6˚ 7˚ 43˚ 43˚ 43˚ 43˚ 3˚ 4˚ 5˚ 6˚ 7˚ 3˚ 4˚ 5˚ 6˚ 7˚ 15 20 15 20 Zoom into NE corner 3˚ 4˚ 5˚ 6˚ 7˚ 8001000 900 700 100 600 600 200500 400 200500 43˚ 700 1100 43˚ 300 9001000 14001200 400 100 8001600 15001300300 1700 2400 200018002300 2100 1900 2200 300200 800 900 500 1000 1200 400 600700 1300 Bathymetry 1100 1500 1400 1700 1600 1800 1900 2000 3˚ 4˚ 5˚ 6˚ 7˚ 0 500 1000 1500 2000 2500 Depth selection, scatter diagram Direct and cross variograms in NE corner Cokriging in NE corner Small-scale (top) and large-scale (bottom) components 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 43˚30' 43˚30' 43˚24' 43˚24' 43˚18' 43˚18' 43˚12' 43˚12' 43˚06' 43˚06' 43˚00' 43˚00' 42˚54' 42˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 −0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 NAR16 image MARS model output 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 43˚30' 43˚30' 43˚24' 43˚24' 43˚18' 43˚18' 43˚12' 43˚12' 43˚06' 43˚06' 43˚00' 43˚00' 42˚54' 42˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 6˚18' 6˚24' 6˚30' 6˚36' 6˚42' 6˚48' 6˚54' 19.0 19.5 20.0 20.5 21.0 21.5 22.0 19.0 19.5 20.0 20.5 21.0 21.5 22.0 Different correlation at small- and large-scale .