Sequential Simulations of Mixed Discrete-Continuous Properties: Sequential Gaussian Mixture Simulation

Dario Grana, Tapan Mukerji, Laura Dovera, and Ernesto Della Rossa

Abstract We present here a method for generating realizations of the posterior probability density function of a Gaussian Mixture linear inverse problem in the combined discrete-continuous case. This task is achieved by extending the sequential simulations method to the mixed discrete-continuous problem. The sequential approach allows us to generate a Gaussian Mixture random ﬁeld that honors the covariance functions of the continuous property and the available observed data. The traditional inverse theory results, well known for the Gaussian case, are ﬁrst summarized for Gaussian Mixture models: in particular the analytical expression for means, covariance matrices, and weights of the conditional probability density function are derived. However, the computation of the weights of the conditional distribution requires the evaluation of the probability density function values of a multivariate Gaussian distribution, at each conditioning point. As an alternative solution of the Bayesian inverse Gaussian Mixture problem, we then introduce the sequential approach to inverse problems and extend it to the Gaussian Mixture case. The Sequential Gaussian Mixture Simulation (SGMixSim) approach is presented as a particular case of the linear inverse Gaussian Mixture problem, where the linear operator is the identity. Similar to the Gaussian case, in Sequential Gaussian Mixture Simulation the means and the covariance matrices of the conditional distribution at a given point correspond to the kriging estimate, component by component, of the mixture. Furthermore, Sequential Gaussian Mixture Simulation can be conditioned by secondary information to account for non-stationarity. Examples of applications

D. Grana () · T. Mukerji Stanford University, 397 Panama Mall, Stanford, CA 94305, USA e-mail: [email protected] T. Mukerji e-mail: [email protected]

L. Dovera · E. Della Rossa Eni E&P, Via Emilia 1, Milan 20097, Italy L. Dovera e-mail: [email protected] E. Della Rossa e-mail: [email protected]

P. Abrahamsen et al. (eds.), Geostatistics Oslo 2012, 239 Quantitative Geology and Geostatistics 17, DOI 10.1007/978-94-007-4153-9_19, © Springer Science+Business Media Dordrecht 2012 240 D. Grana et al. with synthetic and real data, are presented in the reservoir modeling domain where realizations of facies distribution and reservoir properties, such as porosity or net- to-gross, are obtained using Sequential Gaussian Mixture Simulation approach. In these examples, reservoir properties are assumed to be distributed as a Gaussian Mixture model. In particular, reservoir properties are Gaussian within each facies, and the weights of the mixture are identiﬁed with the point-wise probability of the facies.

1 Introduction

Inverse problems are common in many different domains such as physics, engineering, and earth sciences. In general, solving an inverse problem consists of estimating the model parameters given a set of observed data. The operator that links the model and the data can be linear or nonlinear. In the linear case, estimation techniques generally provide smoothed solutions. Kriging, for example, provides the best estimate of the model in the least-squares sense. Simple kriging is in fact identical to a linear Gaussian inverse problem where the linear operator is the identity, with the estimation of posterior mean and covariance matrices with direct observations of the model space. Monte Carlo methods can be applied as well to solve inverse problems [12] in a Bayesian framework to sample from the posterior; but standard sampling methodologies can be ineffi- cient in practical applications. Sequential simulations have been introduced in geostatistics to generate high resolution models and provide a number of realizations of the posterior probability function honoring both prior information and the observed values. References [3] and [6] give detailed descriptions of kriging and sequential simulation methods. Reference [8] proposes a methodology that applies sequential simulations to linear Gaussian inverse problems to incorporate the prior information on the model and honor the observed data. We propose here to extend the approach of [8] to the Gaussian Mixture case. Gaussian Mixture models are convex combinations of Gaussian components that can be used to describe the multi-modal behavior of the model and the data. Ref- erence [14], for instance, introduces Gaussian Mixture distributions in multivariate nonlinear regression modeling; while [10] proposes a mixture discriminant analysis as an extension of linear discriminant analysis by using Gaussian Mixtures and Expectation-Maximization algorithm [11]. Gaussian Mixture models are common in statistics (see, for example, [9] and [2]) and they have been used in different domains: digital signal processing [13] and [5], engineering [1], geophysics [7], and reservoir history matching [4]. In this paper we first present the extension of the traditional results valid in the Gaussian case to the Gaussian Mixture case; we then propose the sequential approach to linear inverse problems under the assumption of Gaussian Mixture distribution; and we finally show some examples of applications in reservoir modeling. If the linear operator is the identity, then the methodology provides an extension of the traditional Sequential Gaussian Simulation (SGSim, see [3], and [6])toanew Sequential Gaussian Mixture Simulation 241 methodology that we call Sequential Gaussian Mixture Simulation (SGMixSim). The applications we propose refer to mixed discrete-continuous problems of reservoir modeling and they provide, as main result, sets of models of reservoir facies and porosity. The key point of the application is that we identify the weights of the Gaussian Mixture describing the continuous random variable (porosity) with the probability of the reservoir facies (discrete variable).

2 Theory: Linearized Gaussian Mixture Inversion

In this section we provide the main propositions of linear inverse problems with Gaussian Mixtures (GMs). We ﬁrst recap the well-known analytical result for posterior distributions of linear inverse problems with Gaussian prior; then we extend the result to the Gaussian Mixtures case. In the Gaussian case, the solution of the linear inverse problem is well-known ∼ [15]. If m is a random vector Gaussian distributed, m N(μm, Cm), with mean μm and covariance Cm; and G is a linear operator that transforms the model m into the observable data d

d = Gm + ε, (1) where ε is a random vector that represents an error with Gaussian distribution N(0, Cε) independent of the model m; then the posterior conditional distribution of m|d is Gaussian with mean and covariance given by = + T T + −1 − μm|d μm CmG GCmG Cε (d Gμm) (2) T T −1 Cm|d = Cm − CmG GCmG + Cε GCm. (3) This result is based on two well known properties of the Gaussian distributions: (A) the linear transform of a Gaussian distribution is again Gaussian; (B) if the joint distribution (m, d) is Gaussian, then the conditional distribution m|d is again Gaus- sian. These two properties can be extended to the Gaussian Mixtures case. We assume that x is a random vector distributed according to a Gaussian Mixture with = Nc ; k k Nc components, f(x) k=1 πkN(x μx, Cx), where πk are the weights and the ; k k k distributions N(x μx, Cx) represent the Gaussian components with means μx and k covariances Cx evaluated in x. By applying property (A) to the Gaussian components of the mixture, we can conclude that, if L is a linear operator, then y = Lx is distributed according to a Gaussian Mixture. Moreover, the pdf of y is given by = Nc ; k k T f(y) k=1 πkN(y Lμx, LCxL ). Similarly we can extend property (B) to conditional Gaussian Mixture distributions. The well-known result of the conditional multivariate Gaussian distribution has already been extended to multivariate Gaussian Mixture models (see, for exam- 242 D. Grana et al. ple, [1]). In particular, if (x1, x2) is a random vector whose joint distribution is a Gaussian Mixture

Nc f(x1, x2) = πkfk(x1, x2), (4) k=1 where fk are the Gaussian densities, then the conditional distribution of x2|x1 is again a Gaussian Mixture

Nc f(x2|x1) = λkfk(x2|x1), (5) k=1 and its parameters (weights, means, and covariance matrices) can be analytically derived. The coefﬁcients λk are given by

πkfk(x1) λk = , (6) Nc =1 πf(x1) where f (x ) = N(x ; μk , Ck ); and the means and the covariance matrices are k 1 1 x1 x1 − μk = μk + Ck Ck 1 x − μk (7) x2|x1 x2 x2,x1 x1 1 x1 − Ck = Ck − Ck Ck 1 Ck T , (8) x2|x1 x2 x2,x1 x1 x2,x1 where Ck is the cross-covariance matrix. By combining these propositions, the x2,x1 main result of linear inverse problems with Gaussian Mixture can be derived.

Theorem 1 Let m be a random vector distributed according to a Gaussian Mixture ∼ Nc k k k m k=1 πkN(μm, Cm), with Nc components and with means μm, covariances k = :M →N Cm, and weights πk, for k 1,...,Nc. Let G be a linear operator, and ε a Gaussian random vector independent of m with 0 mean and covariance Cε, such that d = Gm + ε, with d ∈N , m ∈M , ε ∈N , then the posterior conditional distribution m|d is a Gaussian Mixture. Moreover, the posterior means and covariances of the components are given by k = k + k T k T + −1 − k μm|d μm CmG GCmG Cε d Gμm (9) k = k − k T k T + −1 k Cm|d Cm CmG GCmG Cε GCm, (10) k k th where μm and Cm, are respectively the prior mean and covariance of the k Gaus- sian component of m. The posterior coefﬁcients λk of the mixture are given by

πkfk(d) λk = , (11) Nc =1 πf(d) k = k k = where the Gaussian densities fk(d) have means μd Gμm and covariances Cd k T + GCmG Cε. Sequential Gaussian Mixture Simulation 243

3 Theory: Sequential Approach

Based on the results presented in the previous section, we introduce here the sequential approach to linearized inversion in the Gaussian Mixture case. We ﬁrst recap the main result for the Gaussian case [8]. The solution of the linear inverse problem with the sequential approach requires th some additional notation. Let mi represent the i element of the random vector m, and let ms represent a known sub-vector of m. This notation will generally be used to describe the neighborhood of mi in the context of sequential simulations. Finally we assume that the measured data d are known having been obtained as a linear transformation of m according to some linear operator G.

∼ Theorem 2 Let m be a random vector, Gaussian distributed, m N(μm, Cm) with mean μm and covariance Cm. Let G be a linear operator between the model m and the random data vector d such that d = Gm + ε, with ε a random error vector independent of m with 0 mean and covariance Cε. Let ms be the subvector with direct th observations of the model m, and mi the i element of m. Then the conditional distribution of mi|(ms, d) is again Gaussian. Moreover, if the subvector ms is extracted from the full random vector m with the th linear operator A such that ms = Am, where the i element is mi = Aim, with Ai again linear, then the mean and variance of the posterior conditional distribution are: − T T −1 ms Aμm μm |(m ,d) = μm + AiCmA AiCmG (C(m ,d)) (12) i s i s d − Gμ m T − ACmA σ 2 = σ 2 − A C AT A C GT (C ) 1 i , (13) mi |(ms,d) mi i m i m (ms,d) T GCmAi = 2 = T where μmi Aiμm, σm AiCmAi , and i AC AT AC GT = m m C(ms,d) T T . (14) GCmA GCmG + Cε

To clarify the statement we give the explicit form of the operators Ai and A.In particular, Ai is written as

Ai =[00... 1 ... 0], (15) th ={ with the one in the i column. If the sub-vector ms has size n, ms mi1 ,mi2 , T ...,mi } , and m has size M; then the operator A is given by n ⎡ ⎤ 00... 1 ... 0 ⎢ ⎥ ⎢0 ... 10... 0⎥ A = ⎢ ...... ⎥ , (16) ⎣ ...... ⎦ 01... 000 244 D. Grana et al. where A has dimensions n × M and the ones are in the i1,i2,...,in columns. The- orem 2 can be proved using the properties (A) and (B) described in Sect. 2 (see [8]). Then, by using Theorem 1, we extend the result to the Gaussian Mixture case.

Theorem 3 Let m be a random vector distributed according to a Gaussian Mixture, ∼ Nc k k k m k=1 πkN(μm, Cm), with Nc components and with means μm, covariances k = = Cm, and weights πk, for k 1,...,Nc. Let G a linear operator such that d Gm+ε, with ε a random error vector independent of m with 0 mean and covariance th Cε. Let ms be the sub-vector with direct observations of the model m, and mi the i element of m. Then the conditional distribution of mi|(ms, d) is again a Gaussian Mixture. Moreover, the means and variances of the components of the posterior conditional distribution are: − k k k k T k T k −1 ms Aμm μ | = μ + AiC A AiC G C (17) mi (ms,d) mi m m (ms,d) − k d Gμm k T − AC A σ 2(k) = σ 2(k) − A Ck AT A Ck GT Ck 1 m i , (18) mi |(ms,d) mi i m i m (ms,d) k T GCmAi where μk = A μk σ 2(k) = A Ck AT and mi i m, mi i m i , ACk AT ACk GT Ck = m m . (19) (ms,d) k T k T + GCmA GCmG Cε The posterior coefﬁcients of the mixture are given by

πkfk(ms, d) λk = , (20) Nc =1 πf(ms, d) where the Gaussian components fk(ms, d) have means Aμk μk = m , (21) (ms,d) k Gμm and covariances Ck . (ms,d)

In the case where the linear operator is the identity, the associated inverse problem reduces to the estimation of a Gaussian Mixture model with direct observations of the model space at given locations. In other words, if the linear operator is the identity, the theorem provides an extension of the traditional Sequential Gaussian Simulation (SGSim) to the Gaussian Mixture case. We call this methodology Se- quential Gaussian Mixture Simulation (SGMixSim), and we show some applications in the next section. Sequential Gaussian Mixture Simulation 245

4 Application

We describe here some examples of applications with synthetic and real data, in the context of reservoir modeling. First, we present the results of the estimation of a Gaussian Mixture model with direct observations of the model space as a spe- cial case of Theorem 3 (SGMixSim). In our example, the continuous property is the porosity of a reservoir, and the discrete variable represents the corresponding reservoir facies, namely shale and sand. This means that we identify the weights of the mixture components with the facies probabilities. The input parameters are then the prior distribution of porosity and a variogram model for each component of the mixture. The prior is a Gaussian Mixture model with two components and its parameters are the weights, the means, and the covariance matrices of the Gaussian components. We assume facies prior probabilities equal to 0.4 and 0.6 respectively, and for simplicity we assume the same variogram model (spherical and isotropic) with the same parameters for both. We then simulate a 2D map of facies and porosity according to the proposed methodology (Fig. 1).Thesimulationgridis70× 70 and the variogram range of porosity is 4 grid blocks in both directions. The simulation can be performed with or without conditioning hard data; in the example of Fig. 1, we introduced four porosity values at four locations that are used to condi- tion the simulations, and we generated a set of 100 conditional realizations (Fig. 1). When hard data are assigned, the weights of the mixture components are determined by evaluating the prior Gaussian components at the hard data location and discrete property values are determined by selecting the most likely component. As we previously mentioned, the methodology is similar to [8], but the use of Gaussian Mixture models allows us to describe the multi-modality of the data and to simulate at the same time both the continuous and the discrete variable. SG- MixSim requires a spatial model of the continuous variable, but not a spatial model of the underlying discrete variable: the spatial distribution of the discrete variable only depends on the conditional weights of the mixture (20). However, if the mixture components have very different probabilities and very different variances (i.e. when there are relatively low probable components with relatively high variances), the simulations may not accurately reproduce the global statistics. If we assume, for instance, two components with prior probabilities equal to 0.2 and 0.8, and we assume at the same time that the variance of the first component is much bigger than the variance of the second one, then the prior proportions may not be honored. This problem is intrinsic to the sequential simulation approach, but it is emphasized in case of multi-modal data. For large datasets or for reasons of stationarity, we often use a moving searching neighborhood to take into account only the points closest to the location being simulated [6]. If we use a global searching neighborhood (i.e. the whole grid) the computational time, for large datasets, could significantly increase. In the localized sequential algorithm, the neighborhood is selected according to a fixed geometry (for example, ellipsoids centered on the location to be estimated) and the conditioning data are extracted by the linear operator (Theorem 3) within the neighborhood. When no hard data are present in the searching neighborhood and the sample value is drawn from the prior distribution, the algorithm could generate 246 D. Grana et al.

Fig. 1 Conditional realizations of porosity and reservoir facies obtained by SGMixSim. The prior distribution of porosity and the hard data values are shown on top.Thesecond and third rows show three realizations of porosity and facies (gray is shale, yellow is sand). The fourth row shows the posterior distribution of facies and the ensemble average of 100 realizations of facies and porosity. The last row shows the comparison of SGMixSim results with and without post-processing isolated points within the simulation grid. For example, a point drawn from the ﬁrst component could be surrounded by data, subsequently simulated, belonging to the second component, or vice versa. This problem is particularly relevant in the case of multi-modal data especially in the initial steps of the sequential simulation (in other words when only few values have been previously simulated) and when the searching neighborhood is small. To avoid isolated points in the simulated grid, a post-processing step has been included (Fig. 1). The simulation path is ﬁrst revisited, and the local conditional Sequential Gaussian Mixture Simulation 247

Fig. 2 Linearized sequential inversion with Gaussian Mixture models for the estimation of porosity map from acoustic impedance values. On top we show the true porosity map and the acoustic impedance map; on the bottom we show the inverted porosity and the estimated facies map probabilities are re-evaluated at all the grid cells where the sample value was drawn from the prior distribution. Then we draw again the component from the weights of the re-evaluated conditional probability. Finally, we introduce a kriging correction of the continuous property values that had low probabilities in the neighborhood. Next, we show two applications of linearized sequential inversion with Gaussian Mixture models obtained by applying Theorem 3. The first example is a rock physics inverse problem dealing with the inversion of acoustic impedance in terms of porosity. The methodology application is illustrated by using a 2D grid representing a synthetic system of reservoir channels (Fig. 2). In this example we made the same assumptions about the prior distribution as in the previous example. As in traditional sequential simulation approaches, the spatial continuity of the inverted data depends on the range of the variogram and the size of the searching neighborhood; however, Fig. 2 clearly shows the multi-modality of the inverted data. Gaussian Mixture models can describe not only the multi-modality of the data, but they can better honor the data correlation within each facies. The second example is the acoustic inversion of seismic amplitudes in terms of acoustic impedance. In this case, in addition to the usual input parameters (prior distribution and variogram models), we have to specify a low frequency model of impedance, since seismic amplitudes only provide relative information about elastic contrasts and the absolute value of impedance must be computed by combining the estimated relative changes with the low frequency model (often called prior model in seismic modeling). Once again, the discrete variable is identified with the reservoir facies classification. In this case shales are characterized by high impedance values, 248 D. Grana et al.

Fig. 3 Sequential Gaussian Mixture inversion of seismic data (ensemble of 50 realizations). From left to right: acoustic impedance logs and seismograms (actual model in red, realization 1 in blue, inverted realizations in gray, dashed line represents low frequency model), inverted facies profile corresponding to realization 1, maximum a posteriori of 50 inverted facies profiles and actual facies classification (sand in yellow, shale in gray) and sand by low impedances. The results are shown in Fig. 3. We observe that even though we used a very smoothed low frequency model, the inverted impedance log has a good match with the actual data (Fig. 3), and the prediction of the discrete variable is satisfactory compared to the actual facies classification performed at the well. In particular, if we perform 50 realizations and we compute the maximum a posteriori of the ensemble of inverted facies profiles, we perfectly match the actual classification (Fig. 3). However, the quality of the results depends on the separability of the Gaussian components in the continuous property domain. Finally we applied the Gaussian Mixture linearized sequential inversion to a layer map extracted from a 3D geophysical model of a clastic reservoir located in the North Sea (Fig. 4). The application has been performed on a map of P-wave velocity corresponding to the top horizon of the reservoir. The parameters of the variogram models have been assumed from existing reservoir studies in the same area. In Fig. 4 we show the map of the conditioning velocity and the corresponding histogram, two realizations of porosity and facies, and the histogram of the posterior distribution of porosity derived from the second realization. The two realizations have been performed using different prior proportions: 30 % of sand in the first realization and 40 % in the second one. Both realizations honor the expected proportions, the multi-modality of the data, and the correlations with the conditioning data within each facies. Sequential Gaussian Mixture Simulation 249

Fig. 4 Application of linearized sequential inversion with Gaussian Mixture models to a reservoir layer. The conditioning data is P-wave velocity (top left). Two realizations of porosity and facies are shown: realization 1 corresponds to a prior proportion of 30 % of sand, realization 2 corresponds to 40 % of sand. The histograms of the conditioning data and the posterior distribution of porosity (realization 2) are shown for comparison

5Conclusion

In this paper, we proposed a methodology to simultaneously simulate both continuous and discrete properties by using Gaussian Mixture models. The method is based on the sequential approach to Gaussian Mixture linear inverse problem, and it can be seen as an extension of sequential simulations to multi-modal data. Thanks to the sequential approach used for the inversion, the method is generally quite efﬁcient from the computational point of view to solve multi-modal linear inverse problems and it is applied here to reservoir modeling and seismic reservoir characterization. We presented four different applications: conditional simulations of porosity and facies, porosity-impedance inversion, acoustic inversion of seismic data, and inversion of seismic velocities in terms of porosity. The proposed examples show that we 250 D. Grana et al. can generate actual samples from the posterior distribution, consistent with the prior information and the assigned data observations. Using the sequential approach, we can generate a large number of samples from the posterior distribution, which in fact are all solutions to the Gaussian Mixture linear problem.

Acknowledgements We acknowledge Stanford Rock Physics and Borehole Geophysics Project and Stanford Center for Reservoir Forecasting for the support, and Eni E&P for the permission to publish this paper.

References

1. Alspach DL, Sorenson HW (1972) Nonlinear Bayesian estimation using Gaussian sum ap- proximation. IEEE Trans Autom Control 17:439Ð448. doi:10.1109/TAC.1972.1100034 2. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc, Ser B, Methodol 39(1):1Ð38. doi:10.2307/2984875 3. Deutsch C, Journel AG (1992) GSLIB: geostatistical software library and user’s guide. Oxford University Press, London 4. Dovera L, Della Rossa E (2011) Multimodal ensemble Kalman filtering using Gaussian mixture models. Comput Geosci 15(2):307Ð323. doi:10.1007/s10596-010-9205-3 5. Gilardi N, Bengio S, Kanevski M (2002) Conditional Gaussian mixture models for environ- mental risk mapping. In: Proc. of IEEE workshop on neural networks for signal processing, pp 777Ð786. doi:10.1109/NNSP.2002.1030100 6. Goovaerts P (1997) Geostatistics for natural resources evaluation. Oxford University Press, London 7. Grana D, Della Rossa E (2010) Probabilistic petrophysical-properties estimation integrat- ing statistical rock physics with seismic inversion. Geophysics 75(3):O21ÐO37. doi:10.1190/ 1.3386676 8. Hansen TM, Journel AG, Tarantola A, Mosegaard K (2006) Linear inverse Gaussian theory and geostatistics. Geophysics 71:R101ÐR111. doi:10.1190/1.2345195 9. Hasselblad V (1966) Estimation of parameters for a mixture of normal distributions. Techno- metrics 8(3):431Ð444. doi:10.2307/1266689 10. Hastie T, Tibshirani R (1996) Discriminant analysis by gaussian mixtures. J R Stat Soc B 58(1):155Ð176. doi:10.2307/2346171 11. Hastie T, Tibshirani R, Friedmann J (2009) The elements of statistical learning. Springer, Berlin 12. Mosegaard K, Tarantola A (1995) Monte Carlo sampling of solutions to inverse problems. J Geophys Res 100:12431Ð12447. doi:10.1029/94JB03097 13. Reynolds DA, Quatieri TF, Dunn RB (2000) Speaker verification using adapted Gaussian mixture models. Digit Signal Process 10(1Ð3):19Ð41. doi:10.1006/dspr.1999.0361 14. Sung HG (2004) Gaussian mixture regression and classification. PhD thesis, Rice University 15. Tarantola A (2005) Inverse problem theory. SIAM, Philadelphia