Cross-Covariance Functions for Multivariate Geostatistics

Home , Covariance, Covariance function, Covariance matrix, Geostatistics, Mathematical statistics

arXiv:1507.08017v1 [stat.ME] 29 Jul 2015 acG etnadWlimKleiber William and Genton G. Marc Geostatistics Multivariate for Functions Cross-Covariance S e-mail: 80309-0526, USA Colorado Boulder, Colorado, of University Mathematics, Applied of Department Professor, 05 o.3,N.2 147–163 2, DOI: No. 30, Vol. 2015, Science Statistical 10.1214/14-STS519 10.1214/14-STS517 [email protected] 35-90 ad rbae-mail: Arabia Thuwal Saudi Technology, 23955-6900, and Science of University Abdullah

acG etni rfso,CMEDvso,King Division, CEMSE Professor, is Genton G. Marc c nttt fMteaia Statistics Mathematical of Institute 1 icse in Discussed 10.1214/14-STS487 [email protected] 10.1214/14-STS515 . and metry. spati smoothness, separability, nonstationarity, fields, di phrases: and a words via with Key as closes well article as problems. The value unsolved studies. likelihood observati by co-kriging temperature models pressure, validation maximum compare and and we temperature minimum dataset; bivariate for a example bivari with output a on of model models review select climate a illustrate with We domains, models. spherical constrained and designe support addi those compact We try, including others. among constructions, these specialized no of cover Matérn and extensions cor multivariate space–time of and bu the ary model to methods, approaches linear convolution main the tion, the including nonnega review models, a We cross-covariance yields matrix. always covariance functions structure covariance inite second-order marginal the with that funct consistent way cross-covariance be these to relatio the chosen multi Indeed, for model variables. responsible function to cro distinct the the commonplace is, specifying become in that is function, it difficulty key has The jointly. decades processes process few individual an last for the behavior model spatial developed the capturing scientists at years, many to For challenges analysis statisticians. substantial pose and sciences, environ mate ecological, geophysical, the in ubiquitous become Abstract. 10.1214/14-STS518 ila lie sAssistant is Kleiber William . 2015 , otnosyidxddtst ihmlil aibe hav variables multiple with datasets indexed Continuously , 10.1214/14-STS516 eone at rejoinder ; . smer,c-rgn,mliait random multivariate co-kriging, Asymmetry, , 1 . Motivation 1.1 aesine,mntr olc nomto nmul- on cli- information and collect environmental monitors re- sciences, in mate in instance, For statistics years. in cent interest sustained applications prompted of number has large a in coordinates spatial ern iesfo h rgnli aiainand pagination in detail. original typographic the from differs reprint ttsia Science article Statistical original the the of by reprint published electronic an is This h curneo utvraedt nee by indexed data multivariate of occurrence The 1 .INTRODUCTION 1. lsaitc,sym- statistics, al nttt fMteaia Statistics Mathematical of Institute , 05 o.3,N.2 147–163 2, No. 30, Vol. 2015, o asymme- for d etladcli- and mental si between nship osms be must ions cetssand scientists ss-covariance htaimed that s nywithin only ; t regional ate csinof scussion egionaliza- nsc a such in nstation- iedef- tive physics- tionally cross- ilding along onal ple e This . in 2 M. G. GENTON AND W. KLEIBER tiple variables such as temperature, pressure, wind The covariance matrix Σ of the random vector speed and direction and various pollutants. Simi- Z(s )T,..., Z(s )T T Rnp: { 1 n } ∈ larly, the output of climate models generate multi- C(s , s ) C(s , s ) C(s , s ) ple variables, and there are multiple distinct climate 1 1 1 2 · · · 1 n C(s2, s1) C(s2, s2) C(s2, sn) models. Physical models in computer experiments Σ =  . . ·. · · .  , often involve multiple processes that are indexed by . . .. . not only space and time, but also parameter set-  C(sn, s1) C(sn, s2) C(sn, sn)   · · ·  tings. With the increasing availability and scientific (2)   T interest in multivariate processes, statistical science should be nonnegative definite: a Σa 0 for any Rnp ≥ faces new challenges and an expanding horizon of vector a , any spatial locations s1,..., sn, and ∈ opportunities for future exploration. any integer n. Fanshawe and Diggle (2012) reviewed Geostatistical applications mainly focus on inter- approaches for the bivariate case p = 2, although polation, simulation or statistical modeling. Inter- most techniques can be readily extended to p> 2, polation or smoothing in spatial statistics usually is and Alvarez,´ Rosasco and Lawrence (2012) reviewed synonymous with kriging, the best linear unbiased approaches for machine learning. prediction under squared loss (Cressie, 1993). With A multivariate random field is second-order sta- multiple variables, interpolation becomes a multi- tionary (or just stationary) if the marginal and variate problem, and is traditionally accommodated cross-covariance functions depend only on the sepa- via co-kriging, the multivariate extension of krig- ration vector h = s1 s2, that is, there is a mapping Rd R − ing. Co-kriging is often particularly useful when one Cij : such that → variable is of primary importance, but is correlated cov Z (s ),Z (s ) = C (h), h Rd. with other types of processes that are more read- { i 1 j 2 } ij ∈ ily observed (Almeida and Journel (1994); Wacker- Otherwise, the process is nonstationary. Stationarity nagel (1994); Journel (1999); Shmaryan and Journel can be thought of as an invariance property under (1999); Subramanyam and Pandalai (2008)). Much the translation of coordinates. A test for the station- expository work has been developed on co-kriging, arity of a multivariate random field can be found in see Myers (1982, 1983, 1991, 1992), Long and Myers Jun and Genton (2012). (1997), Furrer and Genton (2011) and Sang, Jun and A multivariate random field is isotropic if it is Huang (2011) for discussion and technical details. stationary and invariant under rotations and reflec- Consider a p-dimensional multivariate random tions, that is, there is a mapping Cij : R+ 0 R ∪ { }→ field Z(s) = Z (s),...,Z (s) T defined on Rd, such that { 1 p } d 1, where Zi(s) is the ith process at location d cov Zi(s1),Zj(s2) = Cij( h ), h R , s,≥ for i = 1,...,p. If Z(s) is assumed to be a { } k k ∈ Gaussian multivariate random field, then only its where denotes the Euclidean norm. Other- k · k mean vector µ(s)=E Z(s) and cross-covariance wise, the multivariate random field is anisotropic. { } matrix function C(s1, s2) = cov Z(s1), Z(s2) = Isotropy or even stationarity are not always realistic, p { } especially for large spatial regions, but sometimes Cij(s1, s2) i,j=1 composed of functions { } are satisfactory working assumptions and serve as d (1) Cij(s1, s2) = cov Zi(s1),Zj(s2) , s1, s2 R , basic elements of more sophisticated anisotropic and { } ∈ for i, j = 1,...,p, need to be described to fully spec- nonstationary models. ify the multivariate random field. Authors typically In the univariate setting, variograms are often the main focus in geostatistics, and are defined as refer to Cij as direct- or marginal-covariance functions for i = j, and cross-covariance functions for i = the variance of contrasts. Variograms can be ex- j. Here, we assume that Z(s) is a mean zero process.6 tended to multivariate random fields in two ways: A covariance-based cross-variogram (Myers, 1982) The quantities ρij(s1, s2)= Cij(s1, s2)/ Cii(s1, s1) { · defined as C (s , s ) 1/2 are the cross-correlation functions. jj 2 2 } Our goal is then to construct valid and flexible (3) cov Z (s ) Z (s ),Z (s ) Z (s ) , { i 1 − i 2 j 1 − j 2 } cross-covariance functions (1), that is, the matrix- Rd d d s1, s2 , and a variance-based cross-variogram valued mapping C : R R M × , where M × p p p p (Myers,∈ 1991), also coined pseudo cross-variogram, is the set of p p real-valued× → matrices, must be nonnegative definite× in the following sense. (4) var Z (s ) Z (s ) , s , s Rd. { i 1 − j 2 } 1 2 ∈ CROSS-COVARIANCE FUNCTIONS 3

The corresponding stationary versions are immedi- (2010) used separable covariances in the context ate. Cressie and Wikle (1998) reviewed the differ- of computer model calibration. In the past, sep- ences between (3) and (4), and argued that (4) is arable cross-covariance structures were sometimes more appropriate for co-kriging because it yields the called intrinsic coregionalizations (Helterbrand and same optimal co-kriging predictor as the one ob- Cressie, 1994). tained with the cross-covariance function Cij in (1); With a large number of processes, detecting struc- see also Ver Hoef and Cressie (1993) and Huang, tures of the multivariate random process such as Yao, Cressie and Hsing (2009). Unfortunately, the symmetry and separability can be difficult via ele- interpretation of cross-variograms is difficult, and mentary data analytic techniques. Li, Genton and so most authors favor working with covariance and Sherman (2008) proposed an approach based on cross-covariance formulations. the asymptotic distribution of the sample cross- covariance estimator to test these various structures. 1.2 Properties of Cross-Covariance Matrix Their methodology allows the practitioner to assess Functions the underlying dependence structure of the data and Because the covariance matrix Σ in (2) must to suggest appropriate cross-covariance functions, be symmetric, the matrix functions must satisfy an important part of model building. T T C(s1, s2) = C(s2, s1) , or C(h) = C( h) under In the special case of stationary matrix-valued co- stationarity. − variance functions, there is an intimate link between Therefore, cross-covariance matrix functions are not the cross-covariance matrix function and its spec- symmetric in general, that is, tral representation. In particular, define the cross- d spectral densities fij : R R as Cij(s1, s2) = cov Zi(s1),Zj(s2) → { } 1 hT ω −ι ω h h ω Rd s s s s fij( )= d e Cij( )d , , = cov Zj( 1),Zi( 2) = Cji( 1, 2), (2π) Rd ∈ 6 { } Z Rd s1, s2 , unless the cross-covariance functions √ ∈ where ι = 1 is the imaginary number. A nec- themselves are all symmetric (Wackernagel, 2003). essary and sufficient− condition for C( ) to be a However, the collocated matrices C(s, s), or C(0) valid (i.e., nonnegative definite), stationary· matrix- under stationarity, are symmetric and nonnegative valued covariance function is for the matrix func- definite. p tion f(ω0)= fij(ω0) i,j=1 to be nonnegative def- The marginal and cross-covariance functions sat- { } inite for any ω0 (Cramér, 1940). While Cramér’s 2 isfy Cij(s1, s2) Cii(s1, s1)Cjj(s2, s2), or original result is stated in terms of measures of C (h)|2 C (0)|C ≤(0) under stationarity. How- | ij | ≤ ii jj bounded variation, in practice using spectral den- ever, Cij(s1, s2) need not be less than or equal sities is preferred. This can be viewed as a mul- to C |(s , s ), or| C (h) need not be less than or ij 1 1 | ij | tivariate extension of Bochner’s celebrated theo- equal to Cij(0) under stationarity. This is because rem (Bochner, 1955). The analogue of Schoen- the maximum value of Cij(h) is not restricted to berg’s theorem for multivariate random fields, that occur at h = 0, unless i = j, and in fact this some- is, Bochner’s theorem for isotropic cross-covari- times occurs in practice (Li and Zhang, 2011). Thus, ance functions, has recently been investigated by 2 there are no similar bounds between Cij(s1, s2) Alonso-Malaver, Porcu and Giraldo (2013, 2015). and C (s , s )C (s , s ), or between C| (h) 2 and| ii 1 2 jj 1 2 | ij | 1.3 Estimation of Cross-Covariances Cii(h)Cjj(h) under stationarity. A cross-covariance matrix function is separable if The empirical estimator of the cross-covariance matrix function of a stationary multivariate random (5) C (s , s )= ρ(s , s )R , s , s Rd, ij 1 2 1 2 ij 1 2 ∈ field is for all i, j = 1,...,p, where ρ(s1, s2) is a valid, non- 1 C(h)= Z(s ) Z¯ stationary or stationary, correlation function and h k N( ) h { − } R = cov(Z ,Z ) is the nonspatial covariance be- | | (k,l)∈N( ) ij i j (6) X tween variables i and j. Mardia and Goodall (1993) b T Z(s ) Z¯ , introduced and used separability to model multivari- · { l − } d ate h R , where N(h)= (k, l) sk sl = h , N(h) ∈ { ¯ | 1 − n } | | spatio-temporal data, and Bhat, Haran and Goes denotes its cardinality, and Z = n k=1 Z(sk) is the P 4 M. G. GENTON AND W. KLEIBER sample mean vector. A valid parametric model is a p r full rank matrix. When r = 1, the cross- then typically fit by least squares methods to the covariance× function (7) is separable as in (5). The empirical estimates in (6). Alternatively, one can use allure of this approach is that only r univariate co- likelihood-based methods or the Bayesian paradigm variances ρk(h) must be specified, thus avoiding di- (Brown, Le and Zidek, 1994). In any case, valid and rect specification of a valid cross-covariance matrix flexible cross-covariance models are needed. Künsch, function. The LMC can additionally be built from a Papritz and Bassi (1997) studied generalized cross- conditional perspective (Royle and Berliner (1999); covariances and their estimation. Gelfand et al. (2004)). Note that the discrete sum Papritz, Künsch and Webster (1993) discussed representation (7) can also be interpreted as a scale empirical estimators of the cross-variogram (3) and mixture (Porcu and Zastavnyi, 2011). (4). Unlike the pseudo cross-variogram, the cross- With a large number of processes, the number of variogram (3) has the disadvantage that it cannot parameters can quickly become unwieldy and the re- be estimated when the variables are not observed sulting estimation difficult. Zhang (2007) described at the same spatial locations. Lark (2003) proposed maximum likelihood estimation of the spatial LMC two outlier-robust estimators of the pseudo cross- based on an EM algorithm, whereas Schmidt and variogram (4) and applied them in a multivari- Gelfand (2003) proposed a Bayesian coregionaliza- ate geostatistical analysis of soil properties. Fur- tion approach with application to multivariate pol- rer (2005) studied the bias of the empirical cross- lutant data. A second drawback of the LMC is that covariance matrix C(0) estimation under spatial de- the smoothness of any component of the multivari- pendence using both fixed-domain and increasing- ate random field is restricted to that of the roughest domain asymptotics. Lim and Stein (2008) investi- underlying univariate process. gated a spectral approach based on spatial cross- periodograms for data on a lattice and studied their 2.2 Convolution Methods properties using fixed-domain asymptotics. Convolution methods fall into the two categories 2. CROSS-COVARIANCES BUILT FROM of kernel and covariance convolution. The kernel UNIVARIATE MODELS convolution method (Ver Hoef and Barry (1998); Ver Hoef, Cressie and Barry (2004)) uses The most common approach to building cross- covariance functions is by combining univariate co- Cij(h) variance functions. The three main options in this vein are the linear model of coregionalization, var- = ki(v1)kj(v2)ρ(v1 v2 + h)dv1 dv2, Rd Rd − ious convolution techniques and the use of latent Z Z d dimensions. s1, s2 R , where the ki are square integrable kernel functions∈ and ρ( ) is a valid stationary correla- 2.1 Linear Model of Coregionalization tion function. This approach· assumes that all the Probably the most popular approach of com- spatial processes Zi(s), for i = 1,...,p, are gener- bining univariate covariances is the so-called lin- ated by the same underlying process, which is very ear model of coregionalization (LMC) for station- restrictive in that it imposes strong dependence be- ary random fields (Bourgault and Marcotte (1991); tween all constituent processes Zi(s). Overall, this Goulard and Voltz, 1992; Grzebyk and Wackernagel approach and its parameters can be difficult to in- (1994); Vargas-Guzmán, Warrick and Myers (2002); terpret and, except for some special cases, requires Schmidt and Gelfand (2003); Wackernagel, 2003). numerical integration. It consists of representing the multivariate random Covariance convolution for stationary spatial ran- field as a linear combination of r independent uni- dom fields (Gaspari and Cohn (1999); Gaspari et al. variate random fields. The resulting cross-covariance (2006); Majumdar and Gelfand (2007)) yields functions take the form r d Cij(h)= Ci(h k)Cj(k)dk, h R , (7) C (h)= ρ (h)A A , h Rd, Rd − ∈ ij k ik jk ∈ Z k=1 X where Ci are square integrable functions. Although for an integer 1 r p, where ρk( ) are valid sta- some closed-form expressions exist, this method usu- ≤ ≤ · p,r tionary correlation functions and A = (Aij)i,j=1 is ally requires numerical integration. A particularly CROSS-COVARIANCE FUNCTIONS 5 useful example of a closed form solution is when the 3. MATERN´ CROSS-COVARIANCE Ci are Matérn correlation functions with common FUNCTIONS scale parameters. In this setup, Matérn correlations The Matérn class of positive definite functions has are closed under convolution and this approach re- become the standard covariance model for univari- sults in a special case of the multivariate Matérn ate fields (Gneiting and Guttorp, 2006). The pop- model (Gneiting, Kleiber and Schlather, 2010). ularity in large part is due to the work of Stein 2.3 Latent Dimensions (1999) who showed that the behavior of the covari- Another approach to build valid cross-covariance ance function near the origin has fundamental impli- functions based on univariate (p = 1) spatial covari- cations on predictive distributions, particularly pre- ances was put forward by Apanasovich and Gen- dictive uncertainty. The key feature of the Matérn ton (2010) (see also Porcu and Zastavnyi (2011)). is the inclusion of a smoothness parameter that di- Their idea was to create additional latent dimen- rectly controls correlation at small distances. The sions that represent the various variables to be mod- Matérn correlation function is eled. Specifically, each component i of the multi- 21−ν M(h ν, a)= (a h )νK (a h ), h Rd, variate random field Z(s) is represented as a point | Γ(ν) k k ν k k ∈ T Rk ξi = (ξi1,...,ξik) in , i = 1,...,p, for an integer 1 k p, yielding the marginal and cross- where Kν is a modified Bessel function of order ν, covariance≤ functions≤ a> 0 is a length scale parameter that controls the rate of decay of correlation at larger distances, while Rd (8) Cij(s1, s2)= C (s1, ξi), (s2, ξj) , s1, s2 , ν> 0 is the smoothness parameter that controls be- { } ∈ where C is a valid univariate covariance function on havior of correlation near the origin. The smooth- Rd+k; see Gneiting, Genton and Guttorp (2007) for ness parameter is aptly named as it implies levels of a review of possible univariate covariance functions. mean square differentiability of the random process, It is immediate that the resulting cross-covariance with large ν yielding very smooth processes that matrix Σ in (2) is nonnegative definite because are many times differentiable, and small ν yield- its entries are defined through a valid univariate ing rough processes; in fact there is a direct con- covariance. If the covariance C is from a station- nection between the smoothness parameter and the ary or isotropic univariate random field, then so is Hausdorff dimension of the resulting random process also the cross-covariance function (8); for instance, (Goff and Jordan, 1988). Cij(h)= C(h, ξi ξj). Due to its popularity for univariate modeling, As an example− of the aforementioned construc- there is interest in being able to simultaneously tion, Apanasovich and Genton (2010) suggested model multiple processes, each of which marginally has a Matérn correlation structure. To this end, σiσj α h Cij(h)= exp − k k Gneiting, Kleiber and Schlather (2010) introduced ξ ξ + 1 ( ξ ξ + 1)β/2 k i − jk k i − jk the so-called multivariate Matérn model, where each (9) constituent process is allowed a marginal Matérn + τ 2I(i = j)I(h = 0), h Rd, ∈ correlation, with Matérns also composing the cross- where I( ) is the indicator function, σi > 0 are correlation structures. In particular, the multivari- · marginal standard deviations, τ 0 is a nugget ef- ate Matérn implies fect, and α> 0 is a length scale. Here,≥ β [0, 1] controls the nonseparability between space∈ and vari- ρii(h) = M(h νi, ai) and (10) | ables, with β = 0 being the separable case. The ρ (h)= β M(h ν , a ), h Rd. parameters of the model are estimated by maxi- ij ij | ij ij ∈ mum likelihood or composite likelihood methods. Of course, this correlation structure can be coerced Apanasovich and Genton (2010) provided an appli- to a covariance structure by multiplying Cii(h) by 2 cation to a trivariate pollution dataset from Cali- σi and Cij(h) by σiσj. Here, βij is a collocated cross- fornia. Further use of latent dimensions for multi- correlation coefficient, and represents the strength of variate spatio-temporal random fields are discussed correlation between Zi and Zj at the same location, in Section 7.2. The idea of latent dimensions was h = 0. recently extended to modeling nonstationary pro- The difficulty in (10) is deriving conditions on cesses by Bornn, Shaddick and Zidek (2012). model parameters νi,νij, ai, aij and βij that result 6 M. G. GENTON AND W. KLEIBER in a valid, that is, a nonnegative definite multivari- or land use type, or dynamical environments such as ate covariance class. In the original work, Gneiting, prevailing winds. In either case, the evolving nature Kleiber and Schlather (2010) described two main of spatial dependence is not well captured by sta- models, the parsimonious Matérn and the full bi- tionary models, and thus the availability of nonsta- variate Matérn. The parsimonious Matérn is a re- tionary constructions is desired, that is, models such duction in complexity over (10) in that ai = aij = a that the marginal and cross-covariance functions are are held at the same value for all marginal and now dependent on the spatial location pair, not just cross-covariances, and the cross-smoothnesses are the lag vector, cov Zi(s1),Zj(s2) = Cij(s1, s2). { } set to the arithmetic average of the marginals, νij = Many of the aforementioned models have been (νi + νj)/2. The model is then valid with an easy-to- extended to the nonstationary setup, including the check condition on the cross-correlation coefficient original stationary models as special cases. The first βij . natural extension to allowing the LMC to be non- The flexibility of the parsimonious Matérn is in stationary is to let the latent univariate correlations allowing each process to have a distinct marginal be nonstationary, so that smoothness behavior, and thus allowing for simulta- r neous modeling of highly smooth and rough fields. C (s , s )= ρ (s , s )A A , s , s Rd, ij 1 2 k 1 2 ik jk 1 2 ∈ The natural extension to allow distinct process- k=1 X dependent length scale parameters a turns out to be i where now ρk are nonstationary univariate correla- more involved. The full bivariate Matérn of Gneit- tion functions. The onus of deriving a matrix-valued ing, Kleiber and Schlather (2010) allows for distinct nonstationary covariance function is then alleviated smoothness and scale parameters for two processes in favor of opting for univariate nonstationary corre- (and in fact results in a characterization for p = 2). lations, of which there are many choices (e.g., Samp- A second set of authors, Apanasovich, Genton and son and Guttorp (1992); Fuentes (2002); Paciorek Sun (2012), were able to overcome the deficiencies of and Schervish (2006); Bornn, Shaddick and Zidek the parsimonious formulation for p> 2, introducing (2012)). Although this extension seems straightfor- the flexible Matérn. The flexible Matérn works for ward, we are unaware of any authors who have im- any number of processes, allowing for each process to plemented such an approach. The second way to ex- have distinct smoothness and scale parameters, and tend the LMC to a nonstationary setup is to al- is as close in spirit to allowing entirely free marginal low the coefficients to be spatially varying (Gelfand Matérn covariances with some level of cross-process et al., 2004), so that dependence as is currently available. A number of r simpler sufficient conditions are available by us- C (s , s )= ρ (s s )A (s )A (s ), ij 1 2 k 1 − 2 ik 1 jk 2 ing scale mixtures (Reisert and Burkhardt (2007); k=1 Gneiting, Kleiber and Schlather (2010); Schlather X s , s Rd. This type of approach can be useful if (2010); Porcu and Zastavnyi (2011)). 1 2 ∈ It is worth pointing out that the experimental the observed multivariate process is linked in a vary- results of both sets of authors, Gneiting, Kleiber ing way to some underlying and unobserved pro- and Schlather (2010) and Apanasovich, Genton and cesses. Guhaniyogi et al. (2013) combined a low Sun (2012), highlighted the importance of allowing rank predictive process approach with the nonsta- for highly flexible and distinct marginal covariance tionary LMC for computationally feasible modeling structures, while still allowing for some degree of with large datasets. cross-process correlation, and indeed the improve- The multivariate Matérn was extended to the ment over an independence assumption was sub- nonstationary case by Kleiber and Nychka (2012). stantial. The basic idea is to allow the various Matérn parameters, variance, smoothness and length scale, 4. NONSTATIONARY CROSS-COVARIANCE to be spatially varying (Stein (2005); Paciorek FUNCTIONS and Schervish (2006)), using normal scale mixtures (Schlather, 2010). For example, temperature fields Geophysical, environmental and ecological spatial exhibit longer range spatial dependence over the processes often exhibit spatial dependence that de- ocean than over land due to terrain driven nonsta- pends on fixed geographical features such as terrain tionarity, and a nonstationary Matérn with spatially CROSS-COVARIANCE FUNCTIONS 7

− varying length scale parameter can capture this type n n 1 of dependence without resorting to using disjoint Kλ( x sk )Kλ( y sℓ ) , · k − k k − k ! models between ocean and land. In particular, the Xk=1 Xℓ=1 nonstationary multivariate Matérn supposes d x, y R , where Kλ(r)= K(r/λ) is a positive kernel function∈ with bandwidth λ. The displayed equation ρii(s1, s2) M(s1, s2 νi(s1, s2), ai(s1, s2)), ∝ | (11) is set up for the case when Zi is mean zero ρ (s , s ) β (s , s )M(s , s ν (s , s ), a (s , s )), ij 1 2 ∝ ij 1 2 1 2| ij 1 2 ij 1 2 for i = 1,...,p, for instance representing residuals d after a mean trend has been removed; the estima- s1, s2 R . An additional point here is that βij(s, s) is proportional∈ to the collocated cross-correlation tor can also be applied to centered residuals such as Zi(sk) Z¯i. This type of estimator can capture sub- coefficient cor Zi(s),Zj(s) , that is, the strength − of relationship{ between variables} at the same loca- stantial nonstationarity that may be difficult to pick tion. This strength often varies spatially, for exam- up parametrically (Kleiber, Katz and Rajagopalan, ple minimum and maximum temperature are less 2013). The nonparametric approach to estimation is correlated over highly mountainous regions than primarily useful when replications of the multivari- over plains where they exhibit greater dependence. ate random field are available. Although it can be Kleiber and Genton (2013) considered an approach applied when only a single field realization is available, we caution against its use given the well-known to allowing this correlation coefficient to vary with variability of empirical estimates in small samples. location in such a way that it can be included with The two methods of covariance and kernel convo- any arbitrary multivariate covariance choice, as long lution can also be extended to result in nonstation- as each process has a nonzero nugget effect (which ary matrix functions (Calder, 2007, 2008; Majum- is not usually restrictive, as most processes exhibit dar, Paul and Bautista (2010)). As with the uni- small scale dependence that are typically modeled variate case, the convolution integrals are often in- as nugget effects). Other authors have noted similar tractable and must be estimated numerically, and phenomena with other scientific data (Fuentes and parametric interpretations are sometimes ambigu- Reich (2013); Guhaniyogi et al. (2013)). ous. Owing to the increasing complexity of nonstationary and multivariate models and the expertise 5. CROSS-COVARIANCE FUNCTIONS WITH required to decide on a framework as well as im- SPECIAL FEATURES plement an estimation scheme, a few authors have considered nonparametric approaches to estimation. 5.1 Asymmetric Cross-Covariance Functions Extending Oehlert (1993) and Guillot, Senoussi and All the stationary models described so far are Monestiez (2001) to the multivariate case, Jun et al. symmetric, in the sense that C (h) = C (h), or (2011) and Kleiber, Katz and Rajagopalan (2013) ij ji equivalently, C (h)= C ( h). Although C (h)= worked with a nonparametric estimator of multivari- ij ij ij C ( h) by definition, the− aforementioned proper- ate covariance that is free from model choice and is ji ties− may not hold in general. Li, Genton and Sher- available throughout the observation domain. The man (2008) proposed a test of symmetry of the underlying idea is to kernel smooth the empirical cross-covariance structure of multivariate random method-of-moments estimate of spatial covariance fields based on the asymptotic distribution of its em- in a way that retains nonnegative definiteness and pirical estimator. If the test rejects symmetry, then yields covariance estimates at any arbitrary location asymmetric cross-covariance functions are needed. pairs, not only those with observations. Their non- Li and Zhang (2011) proposed a general approach parametric estimators are variations on the form to render any stationary symmetric cross-covariance Cîj(x, y) function asymmetric. The key idea is to notice that h n n if Cij( ) is a valid symmetric cross-covariance func- = K ( x s ) tion, then λ k − kk k=1 ℓ=1 (12) Ca (h)= C (h + a a ), h Rd, X X ij ij i − j ∈ is a valid asymmetric cross-covariance function for (11) Kλ( y sℓ )Zi(sk)Zj(sℓ) · k − k any vectors a Rd, i = 1,...,p, such that a = ! i ∈ i 6 8 M. G. GENTON AND W. KLEIBER a . Indeed, if Z(s)= Z (s),...,Z (s) T has cross- or variations on this theme (Reisert and Burk- j { 1 p } covariance functions Cij(h), then Z1(s a1),..., hardt (2007); Porcu and Zastavnyi (2011)). Here, T { − a p Zp(s ap) has cross-covariance functions C (h) ν (d + 1)/2, and g (x) forms a valid cross- − } ij ij i,j=1 given by (12), i, j = 1,...,p. In particular, the con- covariance≥ matrix{ function.} The generality of this struction (12) can be used to produce asymmetric construction gives rise to many interesting exam- γ versions of the LMC and the multivariate Matérn ples. For instance, with g (x)= xν(1 x/b) ij where ij − + models. The vectors a1,..., ap introduce delays that γij = (γi + γj)/2 and γi > 0 for all i = 1,...,p we generate asymmetry in the cross-covariance struc- have the multivariate Askey taper ture. Because only the differences ai aj matter, one ν+γij +1 − ν+1 h can impose a constraint such as a1 + + ap = 0 or C (h)= b B(γ + 1,ν + 1) 1 k k , · · · ij ij − b a1 = 0 to ensure identifiability. Li and Zhang (2011) proposed to first estimate the marginal parame- h < b, and 0 otherwise, where B is the beta a k k ters of Cij(h) in (12), and then estimate the cross- function (Porcu et al., 2013). Kleiber and Porcu parameters and p 1 of the ai’s. Their simulations (2015) provided a nonstationary extension of this and data examples− showed that asymmetric cross- model, while Porcu et al. (2013) considered simi- covariance functions, when required, can achieve re- lar ideas for Buhmann functions and B-splines. Da- markable improvements in prediction over symmet- ley, Porcu and Bevilacqua (2015) obtained multi- ric models. Apanasovich and Genton (2010) used variate Askey functions with different compact sup- a similar strategy to produce asymmetric spatio- ports bij and the multivariate analogue of Wend- temporal cross-covariance models based on latent land functions. The latter provide a tool for taper- dimensions; see Section 7.2. Inducing asymmetry in ing cross-covariance functions such as the multivari- a nonstationary model is yet an open problem. ate Matérn. Recent results on equivalence of Gaus- sian measures of multivariate random fields by Ruiz- 5.2 Compactly Supported Cross-Covariance Medina and Porcu (2015) will allow for assessing the Functions statistical properties of multivariate tapers. Du and Computational issues in the face of large datasets Ma (2013) derived compactly supported classes of is a major problem in any spatial analysis, includ- the Pólya type. Although there has been a flurry ing likelihood calculations and/or co-kriging; see the of recent activity, much additional work remains review by Sun, Li and Genton (2012, Section 3.7). in implementing these models in real world appli- Especially, if the observation network is very large cations, exploring covariance tapering and under- (even on the order of thousands), likelihood calcu- standing limitations of stationary constructions. lations and co-kriging equations are difficult or im- 5.3 Cross-Covariance Functions on the Sphere possible to solve with standard covariance models, Many multivariate datasets from environmental due to the dense unstructured observation covari- and climate sciences are collected over large portions ance matrix. One approach to overcoming this dif- of the Earth, for example, by satellites and, there- ficulty is to induce sparsity in the covariance ma- fore, cross-covariance functions on the sphere S2 in trix, either by using a compactly supported covari- R3 are in need. Consider a multivariate process on ance function as the model, or by covariance taper- the sphere for which the ith variable is described ing, that is, multiplying a compactly supported non- by Zi(L, l), i = 1,...,p, with L denoting latitude negative definite function against the model covari- and l denoting longitude. Jun (2011) constructed ance (Furrer, Genton and Nychka (2006); Kaufman, cross-covariance functions by applying differential Schervish and Nychka (2008)). Then sparse matrix operators with respect to latitude and longitude to methods can be used to invert the covariance ma- the process on the sphere. Furthermore, Jun (2011) trix, or find the determinant thereof. studied nonstationary models of cross-covariances Only recently have authors begun to consider this with respect to latitude, so-called axially symmet- problem for multivariate random fields. Most of the ric, and longitudinally irreversible cross-covariance currently available models are based on scale mix- functions for which tures of the form cov Zi(L1, l1),Zj(L2, l2) ν d { } Cij(h)= (1 h /x) gij(x)dx, h R , − k k + ∈ = cov Zi(L1, l2),Zj(L2, l1) , Z 6 { } CROSS-COVARIANCE FUNCTIONS 9

2 2 (L1, l1) S , (L2, l2) S . All the models described temperature data that illustrates spatially irregu- in Jun∈ (2011) are valid∈ for the chordal distance, larly located data. R3 that is, the Euclidean distance in between points 6.1 Climate Model Output Data on S2. Castruccio and Genton (2014) relaxed the assumption of axial symmetry for univariate ran- The specific reanalysis dataset in use is a Na- dom fields on the sphere and the extension of their tional Centers for Environmental Protection-driven work to multivariate random fields on the sphere (NCEP) run of the updated Experimental Climate remains an open problem. Gneiting (2013) pro- Prediction Center (ECP2) model, which was origi- vided a very thorough study of positive definite nally run as part of the North American Regional functions on a sphere that can be used as covari- Climate Change Assessment Program (NARCCAP) ances. Du, Ma and Li (2013) developed a char- climate modeling experiment (Mearns et al., 2009). acterization of isotropic and continuous variogram Reanalysis data can be thought of as an estimate matrix functions on the sphere, extending some of of the true state of the atmosphere for a given pe- the ideas of Ma (2012) who characterized contin- riod. The variables we use are average summer tem- uous and isotropic covariance matrix functions on perature and cube-root precipitation (summer be- the sphere using Gegenbauer polynomials. Because ing comprised of June, July and August; JJA) over the great circles are the geodesics on the sphere, a region of the midwest United States that is largely they are the natural metric to measure distances in an agricultural region with relatively constant ter- this context. Porcu, Bevilacqua and Genton (2014) rain. The cube-root transformation reduces skew- developed cross-covariance functions of the great ness in the precipitation output and brings the dis- circle distances on the sphere. In particular, they tribution closer to Gaussian. For each grid cell, we studied multivariate Matérn models as functions of calculate a pointwise spatially varying mean as the the great circle distance on the sphere. Recently, arithmetic average of all 24 years of model output Jun (2014) developed nonstationary Matérn cross- from 1981 through 2004. The data considered then covariance models whose smoothness parameters are 24 years of residuals, having removed this spa- vary over space and with large-scale nonstationarity tially varying mean from each year’s reanalysis out- obtained with the aforementioned differential oper- put for the two variables of temperature and cube- ators. root precipitation. The residuals are assumed to be independent between years, and are additionally as- 6. DATA EXAMPLES sumed to be realizations from a mean zero bivariate Gaussian process (both assumptions are supported We illustrate a selection of the above cross-covari- by exploratory analysis). ance models on two data examples. First, a set of re- Figure 1 contains an example set of reanaly- analysis climate model output that represents spa- sis residuals for the year 1989. By eye, it ap- tially gridded data. Second, a set of observational pears that temperature residuals are smoother over

Fig. 1. Example residuals from 1989 after removing a spatially varying mean from NCEP-driven ECP2 regional climate model run for the variables of average summer temperature and precipitation. Units are degrees Celsius for temperature and centimeters for precipitation. 10 M. G. GENTON AND W. KLEIBER space, while precipitation is apparently rougher, where Z1 and Z2 are independent mean zero spa- while both seem to have similar correlation length tial processes with Matérn covariances. We opt for scales. The two variables are strongly negatively this formulation since temperature is expected to correlated, with an empirical correlation coefficient be smoother than precipitation, and our goal is to of 0.67. This situation, with negative and strong preserve this feature within the statistical model. cross-correlation− and both variables exhibiting dis- Parameters are estimated by maximum likelihood. tinct levels of smoothness, provides numerous chal- Finally, we additionally consider two latent dimen- lenges to available cross-correlation models. Call sional models. The first is parameterized by (9), ex- T (s,t) and P (s,t) the temperature and precipita- cept without a nugget effect, and the second is built tion residual at location s in year t, respectively (re- via calling that, although indexed by year, the processes T (s,t)= b11Z(s,t)+ b12Z1(s,t), are viewed as temporally-independent). P (s,t)= b Z(s,t)+ b Z (s,t), Of the above models, we compare six to an in- 21 22 2 dependence assumption, that is, where temperature where Z(s,t) has a latent dimensional covariance of and precipitation residuals are assumed to be inde- the form pendent; for the independence model, each variable 1 α h 2 h is assumed to follow a Matérn covariance, and pa- C( )= β exp − k k β , ( ξi ξj + 1) ( ξi ξj + 1) rameters are estimated by maximum likelihood. The k − k k − k h R2, and Z ,Z are independent with Matérn first nontrivial bivariate model is the parsimonious ∈ 1 2 Matérn, whose parameters we estimate by maxi- correlations. This choice for Z allows the temper- mum likelihood. The second model is a nearly full ature process to retain smoother behavior at the bivariate Matérn, where we set the cross-covariance origin than precipitation, whereas the model of (9) smoothness νTP , T representing temperature and forces exponential-like behavior at the origin. P precipitation, to be the arithmetic average of Table 1 contains the parsimonious and full bivari- the marginal smoothnesses. For the full bivariate ate Matérn parameter estimates. Note the smooth- Matérn, we set marginal parameters to be those of ness parameter of the temperature field is approximately 1.3, indicating a relatively smooth field, the independence model, and conditional on these, which supports the theoretical analysis of North, estimate the remaining cross-covariance length scale Wang and Genton (2011); on the other hand, pre- aTP and cross-correlation coefficient ρTP by maxi- cipitation has a smoothness of approximately 0.55, mum likelihood. We additionally consider two varia- suggesting an exponential model may work well. tions on the bivariate parsimonious Matérn, one us- Both variables have similar length scale parameters, ing a lagged covariance of Li and Zhang (2011) (see which suggests the assumptions of the parsimonious Section 5.1), and a nonstationary Matérn with spa- Matérn model may be reasonable for this particu- tially varying variances for both variables. Spatially lar dataset. The cross-correlation coefficient is es- varying variances are estimated empirically at each timated to be strongly negative in both cases, with grid cell, and conditional on these, the remaining pa- the full Matérn slightly closer to the empirical cross- rameters are estimated by maximum likelihood. We correlation. also consider a linear model of coregionalization, Table 2 contains log likelihood values for the various models considered. Evidently, the parsimonious, T (s,t)= a Z (s,t), 11 1 full and parsimonious lagged Matérn all have likeli- P (s,t)= a12Z1(s,t)+ a22Z2(s,t), hood values on the same order, which are all superior

Table 1 Maximum likelihood estimates of parameters for full and parsimonious bivariate Mat´ern models, applied to the NARCCAP model data. Units are degrees Celsius for temperature, centimeters for precipitation, and kilometers for distances

Model σT σP νT νP 1/aT 1/aP 1/aTP ρTP

Full 1.63 0.19 1.31 0.55 384.3 361.6 420.1 −0.60 Parsimonious 1.61 0.19 1.33 0.54 367.1 – – −0.49 CROSS-COVARIANCE FUNCTIONS 11

Table 2 Comparison of log likelihood values and pseudo cross-validation scores averaged over ten cross-validation replications for various multivariate models on the NARCCAP model data residuals for temperature (T) and precipitation (P)

Loglikelihood RMSE(T ) CRPS(T ) RMSE(P ) CRPS(P )

Nonstationary parsimonious Matérn 53564.5 0.168 0.084 0.085 0.047 Parsimonious lagged Matérn 52563.7 0.179 0.090 0.087 0.048 Full Matérn 52560.1 0.178 0.090 0.087 0.048 Parsimonious Matérn 52556.9 0.179 0.090 0.087 0.048 Latent dimension 52028.8 0.180 0.091 0.088 0.049 LMC 51937.0 0.179 0.091 0.090 0.050 Independent Matérn 50354.5 0.180 0.091 0.088 0.049 Latent dimension of (9) 48086.3 0.195 0.100 0.088 0.048

to the LMC, independent Matérn and latent dimen- the case with all datasets (Gneiting, Kleiber and sional models. We remark that, given the smooth na- Schlather, 2010). ture of the temperature field, the latent dimensional 6.2 Observational Temperature Data model of (9) is not expected to perform as well, as it fixes the smoothness of the temperature field at The second example we consider is a bivariate ν = 0.5, while on the other hand the latent dimen- minimum and maximum temperature observational sional model using a shared process with squared ex- dataset. Observations are available at stations that ponential covariance performs nearly as well as the are part of the United States Historical Climatology Matérn alternatives. The nonstationary extension of Network (Peterson and Vose, 1997) over the state of the parsimonious Matérn exhibits the largest log Colorado. Stations in the USHCN form the highest likelihood, improving the next best model by over quality observational climate network in the United 1000. This suggests that the bivariate field indeed States; observations are subject to rigorous quality exhibits nonstationarity, and there may be other control. modeling improvements that can be explored with We consider bivariate daily temperature residu- new nonstationary cross-covariance developments. als (i.e., having removed the state-wide mean) on Finally, we perform a small pseudo cross-validation September 19, 2004, a day which has good network study. We hold out the bivariate model output at a coverage with observations being available at 94 sta- randomly chosen 90% of spatial locations consistent tions. Exploratory Q–Q plots suggest the residuals over all time points. We then co-krige the remaining are well modeled marginally as Gaussian processes; 10% (62 locations) to the held out grid cells using we suppose the bivariate process is a realization from parameter estimates based on the entire dataset. As a bivariate Gaussian process with zero mean. the residual process is assumed to be independent We entertain the same set of bivariate models as between years, co-kriging is performed separately in the previous example subsection. Due to the fact for each year. Root mean squared error (RMSE) that the data are observational, we augment each and the continuous ranked probability score (CRPS) process’ covariance with a nugget effect. We be- are used to validate interpolation quality, averaged gin by estimating the independent Matérn model over all held out locations and years. We repeat separately for both minimum and maximum tem- this experiment ten times for different randomly perature residuals by maximum likelihood. Since chosen sets of held out spatial locations and av- the nugget effect is tied to marginal process be- erage the resulting scores; the results are displayed havior, we fix the estimated nugget effects at their in Table 2. Generally speaking, all models are ef- marginal estimates, and estimate all other covari- fectively equivalent in terms of predictive ability, ance parameters from the remaining bivariate mod- except for the nonstationary extension to the par- els by maximum likelihood, conditional on these simonious Matérn, which appears to improve both marginal nugget estimates. We remove both the bi- predictive quantities for temperature especially. Per- variate Matérn and nonstationary model from con- haps surprisingly, the independent Matérn performs sideration, as these are both difficult to estimate as well for interpolation, although this has not been given a single realization of the spatial process. 12 M. G. GENTON AND W. KLEIBER

Table 3 Comparison of log likelihood values and pseudo cross-validation scores averaged over 100 cross-validation replications for various multivariate models on the USHCN observed temperature residuals for maximum temperature (max) and minimum temperature (min)

Loglikelihood RMSE(min) CRPS(min) RMSE(max) CRPS(max)

Parsimonious lagged Matérn −414.0 3.18 1.83 3.14 1.79 Parsimonious Matérn −414.9 3.22 1.85 3.16 1.80 LMC −415.7 3.22 1.85 3.16 1.80 Latent dimension −416.2 3.23 1.86 3.18 1.81 Latent dimension of (9) −419.1 3.24 1.86 3.17 1.81 Independent Matérn −427.6 3.41 1.94 3.35 1.91

On top of comparing in sample log likelihood dependence, as well as having substantial cross- values, we additionally consider a pseudo cross- correlation between variables (with possibly oppos- validation study, leaving out a randomly selected ing short/long range dependence); their construc- 25% of locations, and co-krige the remaining bivari- tion is a special case of a multivariate generalization ate observations to these held out locations. This of the univariate Cauchy class of covariance (Gneit- pseudo cross-validation procedure is repeated 100 ing and Schlather, 2004). Hristopoulos and Porcu times, and Table 3 contains the averaged scores from (2014) defined the multivariate analogue of Spartan this study. Contrasting with the results of the NAR- Gibbs random fields, obtained through using Hamil- CCAP example, we now see the predictive bene- tonian functionals. fit of considering multivariate second-order struc- Ma (2011b) also studied various approaches to tures. Generally, predictive RMSE and CRPS are produce valid cross-covariance functions based on improved by between 6–7% when co-kriging using differentiation of univariate covariance functions the parsimonious lagged Matérn, as compared to and on scale mixtures of covariance matrix func- marginally kriging each variable. A potential expla- tions. Alternatively, Ma (2011d) provided construc- nation for the improvement here as compared to the tions of variogram matrix functions, and Du and NARCCAP example is that in the current study, the Ma (2012) introduced an approach to building var- observations are subject to measurement error, and iogram matrix functions based on a univariate vari- thus the greater uncertainty in estimating the bi- ogram model. variate surface is more readily quantified using an We close this section by pointing out a recent novel approach to generating valid matrix covari- appropriate bivariate covariance model. ances by considering stochastic partial differential equations (SPDEs); Hu et al. (2013) used systems 7. DISCUSSION of SPDEs to simultaneously model temperature and 7.1 Specialized Cross-Covariance Functions humidity, yielding computationally efficient means to analysis by approximating a Gaussian random The models introduced so far cover the broad field by a Gaussian Markov random field. majority of usual datasets requiring multivariate models. However, specialized scenarios sometimes 7.2 Spatio-Temporal Cross-Covariance Functions arise, and call for novel developments. For instance, So far, the cross-covariance models that we de- some constructions involve modeling variables that scribed were aimed at spatial multivariate random exhibit long range dependence. Ma (2011c) exam- fields. When adding the time dimension, the re- ined a construction for all variables having long sulting spatio-temporal multivariate random field, or short range dependence utilizing univariate vari- Z(s,t), has stationary cross-covariance functions ograms; and Ma (2011a) explored the relationship Cij(h, u), where u denotes a time lag. All the previ- between multivariate covariances and variograms. ous spatial cross-covariance models can be straight- Kleiber and Porcu (2015) derived a nonstationary forwardly extended to the spatio-temporal setting, construction that allows individual variables to be a for example, Rouhani and Wackernagel (1990), Choi spatially varying mixture of short and long range et al. (2009), Berrocal, Gelfand and Holland (2010) CROSS-COVARIANCE FUNCTIONS 13 and De Iaco et al. (2013), De Iaco, Palma and Posa 7.3 Physics-Constrained Cross-Covariance (2013) developed space–time versions of the linear Functions model of coregionalization. Gelfand, Banerjee and Especially for geophysical processes, often there Gamerman (2005) used a dynamic approach for mul- are physical constraints on a system of variables that tivariate space–time data using coregionalization. must be obeyed by any stochastic model. For in- Based on the concept of latent dimensions de- stance, Buell (1972) explored valid covariance mod- scribed in Section 2.3, Apanasovich and Genton els for geostrophic wind that must satisfy physical (2010) have extended a class of spatio-temporal relationships for isotropic geophysical flow includ- covariance functions for univariate random fields ing geopotential, longitudinal wind components and due to Gneiting (2002) to the multivariate setting. transverse wind components. Specifically, if ϕ (t), t 0, is a completely monotone 1 In a similar vein, a number of physical processes, function and ψ (t), ψ≥(t), t 0, are positive func- 1 2 especially in fluid dynamics, involve fields with spe- tions with completely monotone≥ derivatives, then cialized restrictions such as being divergence free. σ2 Scheuerer and Schlather (2012) developed matrix- C(h, u, v)= 2 2 d/2 2 1/2 [ψ1 u /ψ2( v ) ] ψ2( v ) valued covariance functions for divergence-free and (13) { k k } { k k } curl-free random vector fields, which are based on 2 h combinations of derivatives of a specified variogram ϕ1 2 k k 2 , · ψ1 u /ψ2( v ) and extend earlier work by Narcowich and Ward { k k } (1994). is a valid stationary covariance function on Rd+1+k Constantinescu and Anitescu (2013) introduced that can be used to model cross-covariance func- a framework for building valid matrix-valued co- tions with v = ξ ξ . When ψ (t) 1, Gneiting’s i j 2 variance functions when the constituent processes class is retrieved.− The case v = 0 yields≡ a common have known physical constraints relating their be- spatio-temporal covariance function for each vari- havior. By approximating a nonlinear physical rela- able that can be made different through a LMC-type tionship between variables through series expansions construction. Also judicious choices of the functions and closures, the authors develop physically-based in (13) allow one to control nonseparability between matrix covariance classes. They explored large-scale space and time, between space and variables, and geostrophic wind as a case study, and illustrated between time and variables; see Apanasovich and that physically motivated cross-correlation models Genton (2010) for various illustrative examples. can substantially outperform independence models. To further introduce asymmetry in spatio-temporal North, Wang and Genton (2011) studied spatio- cross-covariance functions, Apanasovich and Genton temporal correlations for temperature fields arising (2010) have proposed two approaches based on la- from simple energy-balance climate models, that tent dimensions. Using the notation of Section 2.3, is, white-noise-driven damped diffusion equations. the first type of asymmetric spatio-temporal cross- The resulting spatial correlation on the plane is of covariance is Matérn type with smoothness parameter ν = 1, al- a T (14) Cij(h, u)= C(h, u λξ (ξi ξj), ξi ξj), though rougher temperature fields are expected due − − − to terrain irregularities for example. Derivations for Rd R h , u , where C is a valid covariance function temperature fields on a uniform sphere were pre- ∈Rd+k ∈ Rk on of a univariate random field and λξ , sented as well. Whether these results can be ex- 1 k p, controls the delay in time that creates∈ ≤ ≤ tended to other variables such as pressure and wind asymmetry. There is no time delay if and only if fields, and possibly lead to Matérn cross-covariance λξ = 0 or i = j. The second type of asymmetric models of type (10), is an open question. spatio-temporal cross-covariance is 7.4 Open Problems a (15) Cij(h, u)= C(h γhu, u, ξi ξj γξu), − − − Finally, there are many open problems that call Rd R Rd h , u , where the velocity vectors γh for more research. The most fundamental question ∈ R∈k ∈ and γξ are responsible for the lack of symme- is the theoretical characterization of the allowable try. When∈ u = 0, this model is spatially anisotropic. classes of multivariate covariances. For instance, Combinations6 of models (14) and (15) are possible. given two marginal covariances, what is the valid 14 M. G. GENTON AND W. KLEIBER class of possible cross-covariances that still results in There is also a need for valid multivariate cross- a nonnegative definite structure? Such a character- covariance functions for spatial data on a lattice. Al- ization is an unsolved problem. Additional to char- though one can apply any of the models mentioned acterization, the companion theoretical question is in this manuscript to lattice data, the extension of the utility of cross-covariance models. Given the two univariate Markov random field models is another data examples in this review, a natural question is: route. For instance, Gelfand and Vounatsou (2003) for the purposes of co-kriging, in what situations are have studied proper multivariate conditional autore- the use of nontrivial cross-covariances beneficial? Al- gressive models. Daniels, Zhou and Zou (2006) pro- though it is traditional to focus on kriging and co- posed a class of conditionally specified space–time kriging in the geostatistical literature, we wish to models for multivariate processes geared to situa- additionally emphasize the utility of these models tions where there is a sparse spatial coverage of one for simulation of multivariate random fields. Indeed, of the processes and a much more dense coverage of without flexible cross-covariance models, it is im- the other processes. This is motivated by an appli- possible to simulate multiple fields with nontrivial dependencies. cation to particulate matter and ozone data. Sain The power exponential class of covariances is a and Cressie (2007) also developed Markov random useful marginal class of covariances, but to the best field models for multivariate lattice data. of our knowledge, a characterization of parameters Many additional open questions remain, includ- for the validity of the multivariate version ing theoretical development of estimation in the multivariate context (Pascual and Zhang, 2006). h κij ρ (h)= β exp k k , h Rd, Vargas-Guzmán, Warrick and Myers (1999) looked ij ij − φ ∈ ij at the relationship between support size and rela- is not known. Although we believe that the mul- tionship between variables, but relatively few have tivariate Matérn model (10) has more flexibility, explored this phenomenon in the multivariate case. this is still an interesting question, especially as this Finally, there is a need to better understand and ex- set of covariances requires no calculations involving plore the intimate connection between multivariate Bessel functions. spline smoothers, co-kriging and multivariate nu- The extension of spatial extremes to the case of merical analysis (Beatson, zu Castell and Schrödl multiple variables has not been explored yet except (2011); Fuselier (2008); Narcowich and Ward (1994); for the recent proposal of Genton, Padoan and Sang Reisert and Burkhardt, 2007). (2015) who considered multivariate max-stable spatial processes. The aim of that research is to de- scribe the behavior of extreme events of several vari- REFERENCES ables across space, such as extreme rainfall and ex- Almeida, A. S. and Journel, A. G. (1994). Joint simulation treme temperature for example. This requires flexi- of multiple variables with a Markov-type coregionalization ble and physically-realistic cross-covariance models model. Math. Geol. 26 565–588. and therefore the families described herein may play Alonso-Malaver, C., Porcu, E. and Giraldo, R. (2013). an important role for such applications. Multivariate versions of walks through dimensions and Recently, there has been some new interest in Schoenberg measures. Technical report, Univ. Tecnica Fed- other types of random fields than the usual Gaus- erico Santa Maria, Valparaiso, Chile. sian case. Mittag–Leffler fields contain the Gaussian Alonso-Malaver, C., Porcu, E. and Giraldo, R. (2015). Multivariate and multiradial Schoenberg measures with case as a subset, but are specified in terms of an their dimension walk. J. Multivariate Anal. 133 251–265. infinite series expansion that is unwieldy for appli- MR3282029 cations (Ma, 2013b). Another option is a multivari- Alvarez,´ M. A., Rosasco, L. and Lawrence, N. D. (2012). ate extension of the Student’s t distribution, a t- Kernels for vector-valued functions: A review. Found. vector distribution (Ma, 2013a); these seem to be Trends Mach. Learn. 3 195–266. more promising for applications, and some explo- Apanasovich, T. V. and Genton, M. G. (2010). Cross- ration of the utility of these types of models is called covariance functions for multivariate random fields based on latent dimensions. Biometrika 97 15–30. MR2594414 for. Finally, hyperbolic vector random fields contain Apanasovich, T. V., Genton, M. G. and Sun, Y. (2012). the Student’s t as a limiting case, although model in- A valid Matérn class of cross-covariance functions for mul- terpretation, estimation and implementation remain tivariate random fields with any number of components. J. unexplored (Du et al. (2012)). Amer. Statist. Assoc. 107 180–193. MR2949350 CROSS-COVARIANCE FUNCTIONS 15

Beatson, R. K., zu Castell, W. and Schrodl,¨ S. J. Daniels, M. J., Zhou, Z. and Zou, H. (2006). Conditionally (2011). Kernel-based methods for vector-valued data with specified space–time models for multivariate processes. J. correlated components. SIAM J. Sci. Comput. 33 1975– Comput. Graph. Statist. 15 157–177. MR2269367 1995. MR2831042 De Iaco, S., Palma, M. and Posa, D. (2013). Prediction Berrocal, V. J., Gelfand, A. E. and Holland, D. M. of particle pollution through spatio-temporal multivari- (2010). A bivariate space–time downscaler under space ate geostatistical analysis: Spatial special issue. Adv. Stat. and time misalignment. Ann. Appl. Stat. 4 1942–1975. Anal. 97 133–150. MR3045764 MR2829942 De Iaco, S., Myers, D. E., Palma, M. and Posa, D. Bhat, K., Haran, M. and Goes, M. (2010). Computer (2013). Using simultaneous diagonalization to identify a model calibration with multivariate spatial output: A case space–time linear coregionalization model. Math. Geosci. study. In Frontiers of Statistical Decision Making and 45 69–86. MR3018750 Bayesian Analysis (M.-H. Chen, D. K. Dey, P. Muller¨ , Du, J. and Ma, C. (2012). Variogram matrix functions for D. Sun and K. Ye, eds.) 168–184. Springer, New York. vector random fields with second-order increments. Math. MR2766461 Geosci. 44 411–425. MR2925032 Bochner, S. (1955). Harmonic Analysis and the Theory of Du, J. and Ma, C. (2013). Vector random fields with com- Probability. Univ. California Press, Berkeley and Los An- pactly supported covariance matrix functions. J. Statist. geles. MR0072370 Plann. Inference 143 457–467. MR2995107 Bornn, L., Shaddick, G. and Zidek, J. V. (2012). Modeling Du, J., Ma, C. and Li, Y. (2013). Isotropic variogram nonstationary processes through dimension expansion. J. matrix functions on spheres. Math. Geosci. 45 341–357. Amer. Statist. Assoc. 107 281–289. MR2949359 MR3107159 Bourgault, G. and Marcotte, D. (1991). Multivariable Du, J., Leonenko, N., Ma, C. and Shu, H. (2012). Hy- variogram and its application to the linear model of core- perbolic vector random fields with hyperbolic direct and gionalization. Math. Geol. 23 899–928. cross covariance functions. Stoch. Anal. Appl. 30 662–674. Brown, P. J., Le, N. D. and Zidek, J. V. (1994). Multi- MR2946043 Fanshawe, T. R. and Diggle, P. J. (2012). Bivariate geo- variate spatial interpolation and exposure to air pollutants. statistical modelling: A review and an application to spatial Canad. J. Statist. 22 489–509. MR1321471 variation in radon concentrations. Environ. Ecol. Stat. 19 Buell, C. E. (1972). Correlation functions for wind and 139–160. MR2946168 geopotential on isobaric surfaces. J. Appl. Meteorol. 11 51– Fuentes, M. (2002). Spectral methods for nonstationary spa- 59. tial processes. Biometrika 89 197–210. MR1888368 Calder, C. A. (2007). Dynamic factor process convolution Fuentes, M. and Reich, B. (2013). Multivariate spatial non- models for multivariate space–time data with application parametric modelling via kernel processes mixing. Statist. to air quality assessment. Environ. Ecol. Stat. 14 229–247. Sinica 23 75–97. MR3076159 MR2405328 Furrer, R. (2005). Covariance estimation under spatial de- Calder, C. A. (2008). A dynamic process convolution ap- pendence. J. Multivariate Anal. 94 366–381. MR2167920 proach to modeling ambient particulate matter concentra- Furrer, R. and Genton, M. G. (2011). Aggregation- tions. Environmetrics 19 39–48. MR2416543 cokriging for highly multivariate spatial data. Biometrika Castruccio, S. Genton, M. G. and (2014). Beyond axial 98 615–631. MR2836410 symmetry: An improved class of models for global data. Furrer, R., Genton, M. G. and Nychka, D. (2006). Co- Stat 3 48–55. variance tapering for interpolation of large spatial datasets. Choi, J., Reich, B. J., Fuentes, M. and Davis, J. M. J. Comput. Graph. Statist. 15 502–523. MR2291261 (2009). Multivariate spatial–temporal modeling and pre- Fuselier, E. J. (2008). Improved stability estimates and diction of speciated fine particles. J. Stat. Theory Pract. 3 a characterization of the native space for matrix-valued 407–418. MR2751608 RBFs. Adv. Comput. Math. 29 269–290. MR2438345 Constantinescu, E. M. and Anitescu, M. (2013). Physics- Gaspari, G. and Cohn, S. E. (1999). Construction of corre- based covariance models for Gaussian processes with lation functions in two and three dimensions. Quarterly J. multiple outputs. Int. J. Uncertain. Quantif. 3 47–71. Roy. Meteorol. Soc. 125 723–757. MR3044910 Gaspari, G., Cohn, S. E., Guo, J. and Pawson, S. (2006). Cramer,´ H. (1940). On the theory of stationary random pro- Construction and application of covariance functions with cesses. Ann. of Math. (2) 41 215–230. MR0000920 variable length-fields. Quarterly J. Roy. Meteorol. Soc. 132 Cressie, N. A. C. (1993). Statistics for Spatial Data. Wiley 1815–1838. Series in Probability and Mathematical Statistics: Applied Gelfand, A. E., Banerjee, S. and Gamerman, D. (2005). Probability and Statistics. Wiley, New York. MR1239641 Spatial process modelling for univariate and multivari- Cressie, N. and Wikle, C. K. (1998). The variance-based ate dynamic spatial data. Environmetrics 16 465–479. cross-variogram: You can add apples and oranges. Math. MR2147537 Geol. 30 789–799. MR1646242 Gelfand, A. E. and Vounatsou, P. (2003). Proper multi- Daley, D. J., Porcu, E. and Bevilacqua, M. (2015). variate conditional autoregressive models for spatial data Classes of compactly supported correlation functions for analysis. Biostatistics 4 11–25. multivariate random fields. Stoch. Environ. Risk Assess. Gelfand, A. E., Schmidt, A. M., Banerjee, S. and Sir- To appear. mans, C. F. (2004). Nonstationary multivariate process 16 M. G. GENTON AND W. KLEIBER

modeling through spatially varying coregionalization. Test Journel, A. G. (1999). Markov models for cross-covariances. 13 263–312. MR2154003 Math. Geol. 31 955–964. MR1717033 Genton, M. G., Padoan, S. and Sang, H. (2015). Multi- Jun, M. (2011). Non-stationary cross-covariance models for variate max-stable spatial processes. Biometrika 102 215– multivariate processes on a globe. Scand. J. Stat. 38 726– 230. MR3335107 747. MR2859747 Gneiting, T. (2002). Nonseparable, stationary covariance Jun, M. (2014). Matérn-based nonstationary cross-covariance functions for space–time data. J. Amer. Statist. Assoc. 97 models for global processes. J. Multivariate Anal. 128 134– 590–600. MR1941475 146. MR3199833 Gneiting, T. (2013). Strictly and non-strictly positive Jun, M. and Genton, M. G. (2012). A test for stationarity definite functions on spheres. Bernoulli 19 1327–1349. of spatio-temporal random fields on planar and spherical MR3102554 domains. Statist. Sinica 22 1737–1764. MR3027105 Gneiting, T. Genton, M. G. Guttorp, P. , and (2007). Jun, M., Szunyogh, I., Genton, M. G., Zhang, F. and Geostatistical space-time models, stationarity, separabil- Bishop, C. H. (2011). A statistical investigation of the ity and full symmetry. In Statistics of Spatio-Temporal sensitivity of ensemble-based Kalman filters to covariance Systems. Monographs in Statistics and Applied Probability filtering. Mon. Weather Rev. 139 3036–3051. (B. Finkenstaedt, L. Held and V. Isham, eds.) 151–175. Kaufman, C. G., Schervish, M. J. and Nychka, D. W. Chapman & Hall/CRC Press, Boca Raton, FL. (2008). Covariance tapering for likelihood-based estimation Gneiting, T. and Guttorp, P. (2006). Studies in the his- tory of probability and statistics. XLIX. On the Matérn in large spatial data sets. J. Amer. Statist. Assoc. 103 1545– correlation family. Biometrika 93 989–995. MR2285084 1555. MR2504203 Gneiting, T., Kleiber, W. and Schlather, M. (2010). Kleiber, W. and Genton, M. G. (2013). Spatially vary- Matérn cross-covariance functions for multivariate random ing cross-correlation coefficients in the presence of nugget fields. J. Amer. Statist. Assoc. 105 1167–1177. MR2752612 effects. Biometrika 100 213–220. MR3034334 Gneiting, T. and Schlather, M. (2004). Stochastic models Kleiber, W., Katz, R. W. and Rajagopalan, B. that separate fractal dimension and the Hurst effect. SIAM (2013). Daily minimum and maximum temperature sim- Rev. 46 269–282 (electronic). MR2114455 ulation over complex terrain. Ann. Appl. Stat. 7 588–612. Goff, J. A. and Jordan, T. H. (1988). Stochastic model- MR3086432 ing of seafloor morphology: inversion of sea beam data for Kleiber, W. and Nychka, D. (2012). Nonstationary model- second-order statistics. J. Geophys. Res. 93 13589–13608. ing for multivariate spatial processes. J. Multivariate Anal. Goulard, M. and Voltz, M. (1992). Linear coregional- 112 76–91. MR2957287 ization model: Tools for estimation and choice of cross- Kleiber, W. and Porcu, E. (2015). Nonstationary matrix variogram matrix. Math. Geol. 24 269–282. covariances: Compact support, long range dependence and Grzebyk, M. and Wackernagel, H. (1994). Multivariate quasi-arithmetic constructions. Stoch. Environ. Res. Risk analysis and spatial/temporal scales: Real and complex Assess. 29 193–204. models. In Proceedings of XVIIth International Biometric Kunsch,¨ H. R., Papritz, A. and Bassi, F. (1997). Gener- Conference, Hamilton, Ontario, Canada 1 19–33. alized cross-covariances and their estimation. Math. Geol. Guhaniyogi, R., Finley, A. O., Banerjee, S. and 29 779–799. MR1470669 Kobe, R. K. (2013). Modeling complex spatial dependen- Lark, R. M. (2003). Two robust estimators of the cross- cies: Low-rank spatially varying cross-covariances with ap- variogram for multivariate geostatistical analysis of soil plication to soil nutrient data. J. Agric. Biol. Environ. Stat. properties. European J. Soil Sci. 54 187–201. 18 274–298. MR3110894 Li, B., Genton, M. G. and Sherman, M. (2008). Test- Guillot, G. Senoussi, R. Monestiez, P. , and (2001). ing the covariance structure of multivariate random fields. A positive definite estimator of the non stationary covari- Biometrika 95 813–829. MR2461213 ance of random fields. In GeoENV 2000: Third European Li, B. and Zhang, H. (2011). An approach to modeling asym- Conference on Geostatistics for Environmental Applica- metric multivariate spatial covariance structures. J. Multi- tions (P. Monestiez, D. Allard and R. Froidevaux, variate Anal. 102 1445–1453. MR2819961 eds.) 333–344. Kluwer Academic, Dordrecht. Lim, C. Y. and Stein, M. (2008). Properties of spatial cross- Helterbrand, J. D. and Cressie, N. (1994). Universal co- kriging under intrinsic coregionalization. Math. Geol. 26 periodograms using fixed-domain asymptotics. J. Multi- 205–226. MR1267008 variate Anal. 99 1962–1984. MR2466546 Hristopoulos, D. and Porcu, E. (2014). Vector Spartan Long, A. E. and Myers, D. E. (1997). A new form of the spatial random field models. Probabilistic Engineering Me- cokriging equations. Math. Geol. 29 685–703. MR1460688 chanics 37 84–92. Ma, C. (2011a). Vector random fields with second-order mo- Hu, X., Steinsland, I., Simpson, D., Martino, S. and ments or second-order increments. Stoch. Anal. Appl. 29 Rue, H. (2013). Spatial modelling of temperature and hu- 197–215. MR2774237 midity using systems of stochastic partial differential equa- Ma, C. (2011b). Covariance matrices for second-order vec- tions. Available at arXiv:1307.1402v1. tor random fields in space and time. IEEE Trans. Signal Huang, C., Yao, Y., Cressie, N. and Hsing, T. (2009). Process. 59 2160–2168. MR2816489 Multivariate intrinsic random functions for cokriging. Ma, C. (2011c). Vector random fields with long-range depen- Math. Geosci. 41 887–904. MR2557237 dence. Fractals 19 249–258. MR2803242 CROSS-COVARIANCE FUNCTIONS 17

Ma, C. (2011d). A class of variogram matrices for vector ran- of the great circle distance on a sphere. Unpublished dom fields in space and/or time. Math. Geosci. 43 229–242. manuscript. MR2764536 Porcu, E., Gregori, P. and Mateu, J. (2006). Nonsepara- Ma, C. (2012). Stationary and isotropic vector random fields ble stationary anisotropic space–time covariance functions. on spheres. Math. Geosci. 44 765–778. MR2956272 Stoch. Environ. Res. Risk Assess. 21 113–122. MR2307631 Ma, C. (2013a). Student’s t vector random fields with power- Porcu, E. and Zastavnyi, V. (2011). Characterization the- law and log-law decaying direct and cross covariances. orems for some classes of covariance functions associated Stoch. Anal. Appl. 31 167–182. MR3007889 to vector valued random fields. J. Multivariate Anal. 102 Ma, C. (2013b). Mittag–Leffler vector random fields with 1293–1301. MR2811618 Mittag–Leffler direct and cross covariance functions. Ann. Porcu, E., Daley, D. J., Buhmann, M. and Bevilac- Inst. Statist. Math. 65 941–958. MR3105803 qua, M. (2013). Radial basis functions with compact sup- Majumdar, A. and Gelfand, A. E. (2007). Multivariate port for multivariate geostatistics. Stoch. Environ. Res. spatial modeling for geostatistical data using convolved co- Risk Assess. 27 909–922. variance functions. Math. Geol. 39 225–245. MR2324633 Reisert, M. and Burkhardt, H. (2007). Learning equivari- Majumdar, A., Paul, D. and Bautista, D. (2010). A gen- ant functions with matrix valued kernels. J. Mach. Learn. eralized convolution model for multivariate nonstationary Res. 8 385–408. MR2320676 spatial processes. Statist. Sinica 20 675–695. MR2682636 Rouhani, S. and Wackernagel, H. (1990). Multivariate Mardia, K. V. and Goodall, C. R. (1993). Spatial– geostatistical approach to space–time data analysis. Wa- temporal analysis of multivariate environmental monitor- ter Resour. Res. 26 585–591. ing data. In Multivariate Environmental Statistics. North- Royle, J. A. and Berliner, L. M. (1999). A hierarchical Holland Ser. Statist. Probab. 6 347–386. North-Holland, approach to multivariate spatial modeling and prediction. Amsterdam. MR1268443 J. Agric. Biol. Environ. Stat. 4 29–56. MR1812239 Mearns, L. O. Gutowski, W. J. Jones, R. Le- , , , Ruiz-Medina, M. D. and Porcu, E. (2015). Equivalence ung, A. M. McGinnis, B. Nunes, Y. Qian, Y. , , and of Gaussian measures of multivariate random fields. Stoch. (2009). A regional climate change assessment program for Environ. Res. Risk Assess. 29 325–334. North America. Eos, Transactions, American Geophysical Sain, S. R. and Cressie, N. (2007). A spatial model for Union 90 311–312. multivariate lattice data. J. Econometrics 140 226–259. Myers, D. E. (1982). Matrix formulation of co-kriging. J. MR2395923 Internat. Assoc. Math. Geol. 14 249–257. MR0660192 Sampson, P. D. and Guttorp, P. (1992). Nonparametric Myers, D. E. (1983). Estimation of linear combinations and estimation of nonstationary spatial covariance structure. co-kriging. J. Internat. Assoc. Math. Geol. 15 633–637. J. Amer. Statist. Assoc. 87 108–119. MR0719602 Sang, H., Jun, M. and Huang, J. Z. (2011). Covariance Myers, D. E. (1991). Pseudo-cross variograms, positive- approximation for large multivariate spatial data sets with definiteness, and cokriging. Math. Geol. 23 805–816. an application to multiple climate model errors. Ann. Appl. MR1121438 Stat. 5 2519–2548. MR2907125 Myers, D. E. (1992). Kriging, co-kriging, radial basis func- Scheuerer, M. Schlather, M. tions and the role of positive definiteness. Comput. Math. and (2012). Covariance Appl. 24 139–148. MR1190311 models for divergence-free and curl-free random vector Narcowich, F. J. and Ward, J. D. (1994). Generalized Her- fields. Stoch. Models 28 433–451. MR2959449 mite interpolation via matrix-valued conditionally positive Schlather, M. (2010). Some covariance models based on definite functions. Math. Comp. 63 661–687. MR1254147 normal scale mixtures. Bernoulli 16 780–797. MR2730648 North, G. R., Wang, J. and Genton, M. G. (2011). Cor- Schmidt, A. M. and Gelfand, A. E. (2003). A Bayesian relation models for temperature fields. J. Climate 24 5850– coregionalization approach for multivariate pollutant data. 5862. J. Geophys. Res. 108 1–9. Oehlert, G. W. (1993). Regional trends in sulfate wet de- Shmaryan, L. E. and Journel, A. G. (1999). Two Markov position. J. Amer. Statist. Assoc. 88 390–399. models and their application. Math. Geol. 31 965–988. Paciorek, C. J. and Schervish, M. J. (2006). Spatial mod- MR1717034 elling using a new class of nonstationary covariance func- Stein, M. L. (1999). Interpolation of Spatial Data: Some tions. Environmetrics 17 483–506. MR2240939 Theory for Kriging. Springer Series in Statistics. Springer, Papritz, A., Kunsch,¨ H. R. and Webster, R. (1993). On New York. MR1697409 the pseudo cross-variogram. Math. Geol. 25 1015–1026. Stein, M. L. (2005). Nonstationary spatial covariance func- MR1246957 tions. Technical Report 21, Univ. Chicago, CISES. Pascual, F. and Zhang, H. (2006). Estimation of linear Subramanyam, A. and Pandalai, H. S. (2008). Data config- correlation coefficient of two correlated spatial processes. urations and the cokriging system: Simplification by screen Sankhy¯a 68 307–325. MR2303086 effects. Math. Geosci. 40 425–443. MR2401515 Peterson, T. C. and Vose, R. S. (1997). An overview of Sun, Y., Li, B. and Genton, M. G. (2012). Geostatistics for the Global Historical Climatology Network temperature large datasets. In Space–Time Processes and Challenges database. Bull. Amer. Meteorol. Soc. 78 2837–2849. Related to Environmental Problems 207 (E. Porcu, J. Porcu, E., Bevilacqua, M. and Genton, M. G. (2014). M. Montero and M. Schlather, eds.) 55–77. Springer, Spatio-temporal covariance and cross-covariance functions Berlin. 18 M. G. GENTON AND W. KLEIBER

Vargas-Guzman,´ J. A., Warrick, A. W. and My- tial prediction. Math. Geol. 25 219–240. MR1206187 ers, D. E. (1999). Multivariate correlation in the frame- Ver Hoef, J. M., Cressie, N. and Barry, R. P. (2004). work of support and spatial scales of variability. Math. Flexible spatial models for kriging and cokriging using Geol. 31 85–104. moving averages and the fast Fourier transform (FFT). Vargas-Guzman,´ J. A., Warrick, A. W. and My- J. Comput. Graph. Statist. 13 265–282. MR2063985 ers, D. E. (2002). Coregionalization by linear combina- Wackernagel, H. (1994). Cokriging versus kriging in re- tion of nonorthogonal components. Math. Geol. 34 405–419. gionalized multivariate data analysis. Geoderma 62 83–92. MR1951789 Wackernagel, H. (2003). Multivariate Geostatistics: An In- Ver Hoef, J. M. and Barry, R. P. (1998). Constructing troduction with Applications, 3rd ed. Springer, Berlin. and ﬁtting models for cokriging and multivariable spa- Zhang, H. (2007). Maximum-likelihood estimation for mul- tial prediction. J. Statist. Plann. Inference 69 275–294. tivariate spatial linear coregionalization models. Environ- MR1631328 metrics 18 125–139. MR2345650 Ver Hoef, J. M. and Cressie, N. (1993). Multivariable spa-