Using Polynomial Chaos Expansions

Analysis of covariance (ANCOVA) using polynomial chaos expansions

B. SUDRET ETH Zurich,¨ Institute of Structural Mechanics, Chair of Risk, Safety & Uncertainty Quantiﬁcation Wolfgang-Pauli-Strasse 15, CH-8093 Zurich¨ Y. CANIOU Phimeca Engineering, Centre d’Affaires du Zenith,´ 34 Rue de Sarlieve,` 63800 Cournon, France

ABSTRACT: Global sensitivity analysis aims at quantifying the uncertainty of the output of a computer model that may be attributed to each input parameter or combination thereof. Variance decomposition techniques that lead to the well-known Sobol’ indices are now well established. However this classical framework only holds for independent input parameters. Moreover, when the computational model under consideration is costly to evaluate, it is not possible to resort to crude Monte Carlo simulation to evaluate sensitivity indices. In this paper we extend the polynomial chaos-based Sobol’ indices derived by Sudret (2006, 2008) to the case of dependent input parameters using the covariance decomposition recently proposed by Li et al. . The functional decomposition which is natively given by the polynomial chaos expansion is taken advantage of, and the proposed approach is consistent with the classical Sobol’ indices when the input variables are independent. The methodology is illustrated on a tolerance analysis problem.

1 INTRODUCTION that the input parameters of the computational model are statistically independent. Computer simulation has become an inescapable tool In engineering problems though, many situations of modern engineering. Computational models of exhibit correlations between model parameters, e.g. ever increasing complexity are built by engineers and between loads applied onto a structural system or scientists in order to describe in the most refined between material parameters describing a constitu- way complex natural or man-made systems. However tive law. Deriving sensitivity indices that are able computer models always represent an idealized vision to account for the input correlation structure is thus of the real world. In this context it is important to as- of practical importance. Until recently not much at- sess the uncertainty in the model predictions that are tention has been devoted to this topic. Distribution- caused by the natural variability (aleatory uncertainty) based sensitivity indices have been introduced by and lack of knowledge (epistemic uncertainty) in the Borgonovo (2007), Borgonovo et al. (2011), which model input parameters. are equally defined when the model input parame- In the field of uncertainty quantification, sensitiv- ters are dependent or independent. The interpretation ity analysis aims at determining what are the in- of the resulting δ-indices is not easy though since put parameters or combinations thereof that have the they do not sum up to 1 as in the case of Sobol’ largest impact onto the model predictions, i.e. those indices. In contrast several approaches to generalize which explain at best the response variability. A re- the Sobol’ indices to the case of dependent parame- view of classical sensitivity analysis can be found in ters have been recently proposed, see Xu and Gertner Saltelli et al. (2004). In this paper we focus on so- (2008), Da Veiga et al. (2009), Mara and Tarantola called variance-based sensitivity indices originally in- (2012), Kucherenko et al. (2012). However no con- troduced by Sobol’ (1993). This approach is based on sensus has been attained so far. the decomposition of the variance of the model re- In this paper the covariance decomposition origi- sponse into a sum of contributions that can be related nally proposed by Li et al. (2010) is exploited. In the to each single model input parameter, pairs of param- original paper the authors first build a so-called High- eters, triplets, etc. The main drawback of these so- Dimensional Model Representation (HDMR) of the called Sobol’ indices is that there definition assumes model. In the present paper we propose to make use of polynomial chaos expansions (Ghanem & Spanos For instance, the truncation scheme may corre- 2003) which have already proven efficient for sensi- spond to all the multivariate polynomials whose tivity analysis in the case of independent input param- PM total degree (computed as ||α||1 = i=1 αi) is eters, see Sudret (2008), Blatman and Sudret (2010). smaller than or equal to a prescribed p, namely As a summary the goal of the paper is to provide n M o A = α : P α ≤ p . Note that other trunca- a computationally efficient framework for sensitiv- M,p i=1 i ity analysis in case of dependent parameters which tion schemes such as the hyperbolic sets may be more is based on a covariance decomposition of the model appropriate for large dimensional problems, see Blat- output and polynomial chaos expansions. The paper is man & Sudret (2011). organized as follows: in Section 2 the basics of poly- The set of PC coefficients {yα, α ∈ A} in Eq.(4) nomial chaos expansions is summarized. In Section 3 are interpreted as the coordinates of random variable the tools of variance-based sensitivity analysis are re- Y in the orthonormal basis made of the Ψα’s. Many called. In Section 4 the proposed polynomial chaos / computational methods have been proposed for their covariance-based sensitivity indiced are derived. The evaluation. In practical engineering applications, so- approach is illustrated by two application examples in called non intrusive methods are most suited, in the Section 5. sense that they only use a set of runs of the original model M using selected points in the domain of definition of the input parameters, namely a so-called 2 POLYNOMIAL CHAOS EXPANSIONS experimental design denoted by X = {x1,..., xN }. Such non intrusive methods include stochatic colloca- Let us consider a computational model M : x ∈ M tion approaches based on sparse grids (Xiu 2007), re- DX ⊂ R 7→ y = M(x) ∈ R. The uncertain input gression (Berveiller, Sudret, & Lemaire 2006) or vari- parameters are modelled by a random vector X with able selection techniques such as Least Angle Regres- DX through its probability density function (PDF) sion (Blatman & Sudret 2011). In this paper we focus denoted by fX . We assume that the model output on classical least-square regression for the sake of il- has a finite variance, i.e. Var [M(X)] < +∞. In this lustration. case Y = M(X) belongs to the Hilbert space of sec- The idea behind least-square regression is to con- ond order random variables. Thus Y may be repre- sider the infinite series in Eq.(1) as a sum of the sented by its coordinates onto a countably infinite ba- truncated series in Eq.(4) and a residual. The coef- sis (Soize & Ghanem 2004): def ficients y = {yα , α ∈ A} are obtained by mini- X MS Y = M(X) = yα Ψα(X) (1) mizing the empirical mean-square error between the α∈ M original model response and its approximation by PC N expansion over the selected experimental design: M where the construction of the basis Ψα , α ∈ N is now described. Assume now that the input random N " #2 vector X has independent components, i.e. fX (x) = X (i) X (i) M y = arg min M(x ) − yα Ψα(x ) Q MS A i=1 fXi (xi) where fXi is the marginal distribution y∈ card R i=1 α∈A of the i-th component Xi. It is possible to define an Hilbertian basis made of multivariate polynomials in the input random vector. More precisely, for any M- (5) tuple α = {α1, . . . , αM }, one defines: Typical choices of experimental designs are Latin Hy- percube designs M or low-discrepancy sequences, and Y the size of the experimental design is selected by the Ψ (X) = P (i)(X ) (2) α αi i thumb rule N = 2 − 3 cardA. Note that robust error i=1 estimates based on cross-validation techniques may n (i) o be derived, see Blatman & Sudret (2010). where P , n ∈ is the set of orthonormal poly- n N As a conclusion, based on an experimental de- nomials with respect to the probability measure sign X of appropriate size and the model responses P (dxi) = fXi (xi) dxi. By construction the multivari- {M(xi) i = 1,...,N}, a surrogate model of the ate basis is orthonormal (δαβ is the Kronecker sym- original model is obtained, which is nothing but a bol): polynomial response surface with specific orthogo- nality properties. E [Ψα(X)Ψβ(X)] = δαβ (3) For computational purpose, the series is truncated 3 SENSITIVITY ANALYSIS by selected a truncation set A, which lead to the polynomial chaos approximation: 3.1 Sobol’ indices def X Considering a computational model M with inde- YA = MA(X) = yα Ψα(X) (4) α∈A pendent uncertain input parameters gathered in X, the Sobol’ decomposition (also known as functional proposed, see e.g. Janon et al. (2012) for recent devel- ANOVA decomposition) reads: opments including asymptotic properties. Whatever the selected estimator, the computational cost in or- M X der to get an accurate result is still large due to the M(x) = M0 + Mi(xi) sampling procedure. i=1 In contrast polynomial chaos-based Sobol’ indices (6) provide an exact result for the sensitivity indices once X + Mij(xi, xj) + ··· + M12...M (x) the PC expansion is available. As originally shown in 1≤iorthogonality property: ing of Eq.(4) reads:

[Mu(xu) Mv(xv)] = 0 ∀ u, v ⊂ {1,...,M} (8) X X E MA(x) = y0 + yα Ψα(x) (13)

u⊂{1, ... ,M} α∈Au the above decomposition is unique. As a consequence u6=∅ the variance of the model response: As the Sobol’ decomposition is unique, the compar- Var [M(X)] = (M(X) − M )2 (9) ison of Eqs.(7) and (13) readily provides the expres- E 0 sion of each summand for the PC expansion: can be computed by substituting for Eq.(6) into X Eq.(9). It reduces to the sum of the variances of the Mu(xu) = yα Ψα(X) (14) summands since the cross terms cancel out due to α∈Au Eq.(8): Due to the orthonormality of the polynomial chaos basis (see Eq.(3)) the variance of the truncated PC ex- X Var [M(X)] = Var [Mu(X)] (10) pansion (resp. of a given summand) reads: u⊂{1, ... ,M} X 2 Var [YA] = yα The Sobol’ indices are then defined by the normalized α∈A , α6=0 variances, i.e. : (15) X 2 Var [Mu(Xu)] = yα Su = Var [Mu(X)] /Var [M(X)] (11) α∈Au By construction they sum up to 1. The first or- and the associated Sobol’ indices in Eq.(11) is the der indices {Si, i = 1,...,M} characterize the por- mere ratio of these two quantities. tion of the output variance that can be attributed to As a conclusion, once a PC expansion is available, each single parameter Xi. The second order indices the computation of the Sobol’ indices reduces to ele- {Sij, 1 ≤ i < j ≤ M} correspond to the second-order mentary operations on the coefficients of the expan- interaction effects, etc. sion.

3.2 Sobol’ indices using polynomial chaos 4 COVARIANCE-BASED SENSITIVITY expansions INDICES USING PC EXPANSIONS

The traditional approach to evaluating the Sobol’ in- 4.1 Introduction dices defined above is based on Monte Carlo simulation. Following the original estimator developed in As shown in the previous section, a truncated poly- Sobol’ (1993), several improved estimators have been nomial chaos expansion not only provides a surrogate model MA of the original computational model M, where each term can in turn be decomposed as follow: it also yields a functional decomposition in which the first order effects as well as the second and higher or- Cov [Mu(Xu),, M(X)] = Var [Mu(Xu)] der interaction terms readily appear. When the input variables in X are independent the polynomial chaos X (18) + Cov [M (X ); M (X )] basis is orthonormal by construction so that the vari- u u v v ous variances that must be computed to evaluate the v6=u Sobol’ indices are easily obtained. Consider now the general case where the input ran- By renormalizing the above equation we introduce the covariance-based total sensitivity index dom vector has dependent components. Introducing : the marginal CDF and PDF of each component X , i [M (X ) , M(X)] respectively denoted by F and f , the joint PDF (cov) Cov u u Xi Xi Su = (19) of X may be represented using the copula theory Var [M(X)] (Nelsen 1999) as follows: This covariance index is the sum of a structural (or fX (x) = c(FX1 (x1),...,FXM (xM ))· (U) uncorrelated) sensitivity index Su and a correlative (C) M (16) sensitivity index Su which are defined by: Y f (x ) Xi i Var [M (X )] i=1 S(U) = u u u Var [M(X)] where c(.) is the copula density function (which re- (20) duces to c = 1 in the independent case). S(C) = S(cov) − S(S) Due to the statistical dependence between the Xi’s u u u it is not possible anymore to derive a unique Sobol’- like decomposition in terms of orthogonal summands In the case of independent input variables, if the func- of increasing order, as Eq.(7) in the independent case, tional approximation is a truncated polynomial chaos see e.g. Chastaings, Gamboa, & Prieur (2012). expansion then the correlative indices vanish due to However, provided the computational model M Eq.(8) and the structural indices reduce to the classi- may be given an approximate functional decompo- cal Sobol’ indices. The above definition is thus con- sition, the variance of the output may be cast as a sistent with the classical approach. covariance decomposition as originally shown in Li et al. (2010). The approach proposed in this paper thus 4.3 Estimation of the covariance-based indices consists in: • building a functional approximation of M in Once the functional approximation is available from terms of a polynomial chaos decomposition as the PC expansion (see Eqs.(13)-(14)), Monte Carlo if the input vector X had independent compo- estimates of variances and covariances in Eqs.(19)- nents; (20) are easily obtained. Denoting by XMCS = {x1,..., xn} a sample set drawn from the original • using this functional approximation in order to distribution of the input vector Eq.(16), the empirical evaluate the variance of the output under the real mean value and variance of the response reads: input distribution that includes some dependence n between the parameters. 1 X y = M (x ) A n A i 4.2 Covariance decomposition i=1 (21) Assume that the PC expansion of the computational n 1 X 2 model Eq.(4) has been constructed under the hypoth- Var\[YA] = (MA(xi) − yA) n − 1 esis of independent input random variables. Consider i=1 now the true input random vector X. The variance of the model response reads: Note that yA 6= y0 since the expectations of each basis polynomial E [Ψα] under the actual distribution Var [M(X)] fX are non zero. For any subset u ∈ {1,...,M}, the subvector of a sample xi ∈ XMCS made of the com-   ponents that are labelled by u is simply denoted by X = Cov M0 + Mu(Xu) , M(X) xu,i. As in Eq.(21) the expected value E [Mu(xu)] is (17) u⊂{1, ... ,M} estimated by:

X n = [M (X ) , M(X)] def 1 X Cov u u , yu = b [Mu(xu)] = Mu(xu,i) (22) E n u⊂{1, ... ,M} i=1 The estimate of the covariance-based sensitivity index 5.2 Problem statement then reads: The assembly under consideration corresponds to an electrical connector for which a contact clearance is Pn of interest. More speciﬁcally the axial deviation of the [(cov) i=1 (MA(xi) − yA)(Mu(xu,i) − yu) Su = n 2 connector pin is considered which shall not be greater P (M (x ) − y ) i=1 A i A than a given threshold for the sake of correct assembly (see Figure 1). (23)

Similarly the uncorrelated contribution is estimated by:

Pn (M (x ) − y )2 Sd(U) = i=1 u u,i u (24) u Pn 2 i=1 (MA(xi) − yA)

5 APPLICATION EXAMPLE: TOLERANCE ANALYSIS

5.1 Introduction In industrial mass production, assembled products are made of individual parts that are prone to uncertainty in their dimensions due to the manufacturing pro- cesses. Non-assembly problems may occur when the real dimensions of some parts differ (even slightly) from their nominal values, even if they remain in the prescribed tolerance ranges. The customer qual- ity requirements are usually deﬁned in terms of number of out-of-tolerance assemblies (e.g. expressed in ppm, i.e. parts per million). Predicting this probability of non-assembly prior to the mass production is of crucial importance since it may avoid a large wastage during production. Probabilistic methods de- Figure 1: Electric connector – after Gayton et al. (2011) rived from structural reliability analysis have been recently proposed by Gayton, Beaucaire, Bourinet, This deviation is computed as a non linear func- Duc, Lemaire, & Gauvrit (2011) in order to address tion of the dimensions of the various components and such problems. parameters describing their respective positions (see Within the tolerance analysis it is also impor- Gayton et al. (2011) for details): tant to detect the key dimensions, i.e. those dimen- cm + cm + cm c = 4 8 7 (25) sions whose uncertainty may affect the most the 2 non-assembly. This question can be solved using global sensitivity analysis. The functional require- ima + ima + ima i = 17 19 20 (26) ment which is typically a function of the various di- 2 mensions of the parts in the assembly is considered as the output Y = M(X) where X is a random vector h = ima2 + ima3 (27) whose components model the tolerance range of the parts respective dimensions. The key dimensions cor- c i respond to those dimensions which exhibit the largest α = arccos √ − arccos √ sensitivity indices. i2 + h2 i2 + h2 When complex parts are manufactured it is com- (28) mon that several surfaces are machined during the ima + ima /2 cm + cm /2 same operation. The corresponding dimensions (e.g. 19 20 − 8 7 hole or axle diameters) are then strongly correlated: 2 2 cos(α) r = (29) if one dimension is smaller than its nominal value 2 tan(α) (e.g. due to the wear of the tool) the others will most probably also be. Thus the covariance-based sensitiv- r cm + cm /2 cm ity indices are relevant in this situation. In the sequel z = 2 + 9 10 + 7 tan(α) we present an application example taken from Caniou cos(α) 2 4 (2012). (30) 2. Due to the dimension of the problem, i.e. M = 14, the number of coefﬁcients to be computed by mean J1 = (cm1 − cm3 − z) sin(α) (31) square regression is high, i.e. P = 120. The experimental design a Latin Hypercube sample of size cm7 J2 = cos(α) (32) N = 2 P = 240. The various covariance terms are 4 computed using n = 104 in the Monte Carlo simu- lations. cm5 + cm6 J3 = cos(α) (33) 2 (cov) U C Parameter Si Si Si

Y = J1 + J2 + J3 (34) cm1 0.00 0.00 0.00 cm 0.00 0.00 0.00 The 14 parameters appearing in the above equa- 3 cm 0.02 0.01 0.01 tions are considered as Gaussian random variables, 4 whose parameters are reported in Table 1. cm5 0.15 0.15 0.00 cm6 0.02 0.02 0.00 Parameter Mean µ σ cm7 0.52 0.51 0.01 cm1 10.530 0.200/6 cm8 0.03 0.02 0.01 cm3 0.750 0.040/6 cm9 0.00 0.00 0.00 cm4 0.643 0.015/6 cm5 0.100 0.200/6 cm10 0.00 0.00 0.00 cm6 0.000 0.060/6 ima2 0.00 0.00 0.00 cm 0.000 0.200/6 7 ima3 0.00 0.00 0.00 cm8 0.720 0.040/6 ima17 0.11 0.05 0.06 cm9 1.325 0.050/6 ima19 0.07 0.02 0.05 cm10 0.000 0.040/6 ima2 3.020 0.060/6 ima20 0.09 0.03 0.06 ima3 0.400 0.060/6 Σ 1.00 0.81 0.19 ima17 0.720 0.040/6 Table 2: Results of the sensitivity analysis for the electrical con- ima19 0.970 0.040/6 nectors. ima20 0.000 0.040/6 Table 1: Electric connector – probabilistic modelling of the dimensions and their tolerances (Gaussian distributions, all dimen- The first order sensitivity indices S , i = 1,...,M sions in mm). i attached to each input variable are reported in Table 2. It appears that the main contributor to the variability Because different surfaces are machined during the of Y are the dimension cm7 and cm5. These param- (cov) same manufacturing operation, their dimension are eters are not correlated to others such that Scmk ≈ SU , k = 5, 7 highly correlated. The different Spearman correlation cmk . coefficients ρS are estimated as follows: In contrast the three dimensions imak, k = 17, 19, 20, which contribute to 27% of the output vari- ρS(cm1, cm3) = 0.8 ance are correlated. It is clear from the table that their C correlative sensitivity indices Si are greater than the ρS(cm4, cm8) = ρS(cm4, cm9) uncorrelated counterpart, which can be explained by the large correlation coefficients between these pa- = ρS(cm8, cm9) = 0.8 (35) rameters in Eq.(35).

ρ (ima , ima ) = ρ (ima , ima ) S 17 19 S 17 20 6 CONCLUSIONS = ρ (ima , ima ) = 0.8 S 19 20 Taking into account the statistical dependence be- The joint distribution of this parameters is built as tween model input parameters in global sensitivity in Eq.(16) from the marginal Gaussian distributions analysis is an important problem that has not been and a Gaussian copula (see Nelsen (1999) for details) given a definitive solution so far. In the context of which is is parameterized by a correlation matrix R complex models such as those appearing in mechan- derived from Eq.(35): ical engineering the limited number of runs of the π computational model that is affordable precludes the Rij = 2 sin ρS,ij (36) use of simulation-based estimators. 6 In this paper we use the so-called covariance de- A covariance-based sensitivity analysis is carried composition in order to define sensitiviy indices that out using polynomial chaos expansion of degree p = have both a structural (also called uncorrelated) and a correlative contribution. These indices are computed Sudret, B. (2006). Global sensitivity analysis using polyno- by first carrying out a polynomial chaos expansion of mial chaos expansions. In P. Spanos and G. Deodatis (Eds.), the computational model under the assumption of in- Proc. 5th Int. Conf. on Comp. Stoch. Mech (CSM5), Rhodos, Greece. dependent input parameters. The expansion is then Sudret, B. (2008). Global sensitivity analysis using polynomial split into functional summands of increasing order, chaos expansions. Reliab. Eng. Sys. Safety 93, 964–979. which in turn are used to compute the required co- Xiu, D. (2007). Efficient collocational approach for parametric variances by Monte Carlo simulation. uncertainty analysis. Comm. Comput. Phys. 2(2), 293–309. An application example that is taken from tolerance Xu, C. & G. Gertner (2008). Uncertainty and sensitivity analysis for models with correlated parameters. Reliab. Eng. Sys. analysis in mechanical engineering is used in order Safety 93, 1563–1573. to illustrate the approach and shows that some important parameters (i.e. notably contributing to the output variance) have significant correlative contributions.

REFERENCES

Berveiller, M., B. Sudret, & M. Lemaire (2006). Stochastic finite elements: a non intrusive approach by regression. Eur. J. Comput. Mech. 15(1-3), 81–92. Blatman, G. & B. Sudret (2010). Efficient computation of global sensitivity indices using sparse polynomial chaos expansions. Reliab. Eng. Sys. Safety 95, 1216–1229. Blatman, G. & B. Sudret (2011). Adaptive sparse polynomial chaos expansion based on Least Angle Regression. J. Com- put. Phys 230, 2345–2367. Borgonovo, E. (2007). A new uncertainty importance measure. Reliab. Eng. Sys. Safety 92, 771–784. Borgonovo, E., W. Castaings, & S. Tarantola (2011). Moment- independent importance measures: new results and analytical test cases. Risk Analysis 31(3), 404–428. Caniou, Y. (2012). Global sensitivity analysis for nested and multiscale models. Ph. D. thesis, Universite´ Blaise Pascal, Clermont-Ferrand. Chastaings, G., F. Gamboa, & C. Prieur (2012). Generalized Hoeffding-Sobol decomposition for dependent variables – application to sensitivity analysis. Electronic Journal of Statistics 6, 2420–2448. Da Veiga, S., F. Wahl, & F. Gamboa (2009). Local polynomial estimation for sensitivity analysis on models with correlated inputs. Technometrics 51(4), 452–463. Gayton, N., P. Beaucaire, J.-M. Bourinet, E. Duc, M. Lemaire, & L. Gauvrit (2011). APTA: advanced probability-based tolerance analysis of products. Mecanique´ & Industries 12(2), 71–85. Ghanem, R. & P. Spanos (2003). Stochastic Finite Elements : A Spectral Approach. Courier Dover Publications. Janon, A., T. Klein, A. Lagnoux-Renaudie, M. Nodet, & C. Prieur (2012). Asymptotic normality and efficiency of two Sobol’ index estimators. Technical report, INRIA. Kucherenko, S., A. Tarantola, & P. Annoni (2012). Estimation of global sensitivity indices for models with dependent variables. Comput. Phys. Comm. 183, 937–946. Li, G., H. Rabitz, P. Yelvington, O. Oluwole, F. Bacon, C. Kolb, & J. Schoendorf (2010). Global sensitivity analysis for systems with independent and/or correlated inputs. J. Phys.Chem. 114, 6022–6032. Mara, T. & S. Tarantola (2012). Variance-based sensitivity indices for models with dependent inputs. Reliab. Eng. Sys. Safety 107, 125–131. Nelsen, R. (1999). An introduction to copulas, Volume 139 of Lecture Notes in Statistics. Springer. Saltelli, A., S. Tarentola, F. Campolongo, & M. Ratto (2004). Sensitivity analysis in practice – A guide to assessing scien- tific models. J. Wiley & Sons. Sobol’, I. (1993). Sensitivity estimates for nonlinear mathemati- cal models. Math. Modeling & Comp. Exp. 1, 407–414. Soize, C. & R. Ghanem (2004). Physical systems with random uncertainties: chaos representations with arbitrary probability measure. SIAM J. Sci. Comput. 26(2), 395–410.