Proceedings of Building Simulation 2011: 12th Conference of International Building Performance Simulation Association, Sydney, 14-16 November. SAMPLING BASED ON SOBOL0 SEQUENCES FOR MONTE CARLO TECHNIQUES APPLIED TO BUILDING SIMULATIONS Sebastian Burhenne1;∗, Dirk Jacob1, and Gregor P. Henze2 1Fraunhofer Institute for Solar Energy Systems, Freiburg, Germany 2University of Colorado, Boulder, USA ∗Corresponding author. E-mail address: [email protected] ABSTRACT numerically expensive when the number of analyzed Monte Carlo (MC) techniques are commonly used to parameters (k) is large as its volume increases dramat- perform uncertainty and sensitivity analyses. A key el- ically with k. ement of MC methods is the sampling of input param- In this paper different sampling techniques are ana- eters for the simulation, where the goal is to explore lyzed with respect to the estimator of the mean of the the entire input space with a reasonable sample size result and how quick this estimator converges to the (N). The sample size determines the computational true mean (i.e., expected value) with respect to the cost of the analysis since N is equal to the required sample size. Another analyzed measure of the perfor- number of simulation runs. Quasi-random (QR) se- mance of the sampling strategy is its robustness. Ro- quences such as the Sobol0 sequences are designed to bustness can be measured via the standard error of the generate a sample that is uniformly distributed over estimated mean. This is done using multiple MC sim- the unit hypercube. In this paper, sampling based ulations and analyzing their results. A way to visualize on Sobol0 sequences is compared with other standard the robustness is to compare the empirical cumulated sampling procedures with respect to typical building density functions (CDFs) of several repetitions of the simulation applications. The work revealed that for MC simulation (Helton and Davis, 2003). the most of the analyzed aspects the sampling based Macdonald analyzed the performance of random sam- on Sobol0 sequences performs better than the other in- pling, stratified sampling and Latin hypercube sam- vestigated sampling techniques. pling applied to the evaluation of a building model INTRODUCTION (Macdonald, 2009). This paper extends his work by applying a sampling technique based on Sobol0 se- Due to the substantial influence of uncertain param- quences to a building simulation model. Furthermore, eters on building performance, uncertainty and sensi- models with different properties than the model used tivity analyses will become an important part of the by Macdonald are analyzed. As a test case, a simple building performance simulation and the design pro- mathematical model and a typical building simulation cess of low energy buildings. In an uncertainty analy- model are used. sis the modeler quantifies the uncertainty in the model output given the uncertainty in the model input. This SAMPLING TECHNIQUES goes often hand in hand with a sensitivity analysis Sampling is the process of exploring the domain of in- where the aim is to apportion the uncertainty in the terest (e.g., x1). That can be done randomly, where model output to the uncertainty in the model input the random numbers are independent realizations of a (Saltelli et al., 2008, pg. 1). Both analyses give in- random variable (Sobol’ and Levitan, 1999). In com- sights to the driving parameters or variables of the puter experiments, pseudo-random numbers or quasi- model and the model structure. random numbers are used. These numbers are gener- Several examples of Monte Carlo based uncertainty ated using an algorithm or a sequence of numbers that and sensitivity analyses applied to building sim- fulfill requirements as if they were true random num- ulations exist (Lomas and Eppel, 1992; de Wit, bers. The properties of the samples can be analyzed by 2003; Mara and Tarantola, 2008; Macdonald, 2009; the use of statistical tests (Sobol’ and Levitan, 1999). Burhenne et al., 2010a,b). Compared to sampling In the context of this paper the language and environ- methods already applied in various building perfor- ment R for statistical computing is used to generate the mance simulation applications (e.g., random sam- samples (R Development Core Team, 2010). pling, stratified sampling and Latin Hypercube sam- Random sampling pling) the sampling based on Solbol0 quasi-random se- quences is expected to be more effective in exploring A random sample can be generated by a pseudo- the input parameter space. This space is a unit hy- random number generator which is available in many percube (Ω) with k dimensions. Exploring the unit software packages. A sample is randomly distributed hypercube with a sufficient sample density becomes in a defined interval according to some distribution - 1816 - Proceedings of Building Simulation 2011: 12th Conference of International Building Performance Simulation Association, Sydney, 14-16 November. (e.g., uniform distribution in the interval [0,1], hence Xi ∼ U(0; 1) with i = 1; 2; :::; N). For small sample sizes (N), the samples can contain clusters and gaps as 1.0 shown in Figure 1 on line a. Regions with gaps are not taken into account in the statistical analyses for any 0.8 uncertainty or sensitivity analysis and function values 0.6 in the regions with clusters are overemphasized in the , random 3 x 0.4 calculations (Saltelli et al., 2008, pg. 83). The sample 1.0 0.8 , random on line b was drawn using the same pseudo-random 2 0.6 x 0.2 0.4 number generator but shows a better coverage of the 0.2 0.0 interval. 0.00.0 0.2 0.4 0.6 0.8 1.0 x1, random a Figure 2: Three-dimensional plot of the pseudo- b randomly sampled points in the parameter space x1, x2 and x3. The color of the points varies from red to black depending on the value of x2. That color 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 variation allows a better interpretation of the three- Figure 1: Two examples of sampling with a pseudo- dimensional plot. random number generator. The unbiased mean and variance of the model output way. For this reason Figure 3 shows the variables can be calculated by the following equations (Saltelli x1, x2 and x3 plotted against each other in two- et al., 2008, pg. 59): dimensional plots. The plots for random sampling show clusters and gaps. N Stratified sampling 1 X Y¯ = y (1) Figure 1 showed that a random sample may contain N i i=1 clusters and gaps. Using a stratified sampling tech- N nique can solve that problem. In a scheme which ap- 1 X ¯ 2 Var(Y ) = (yi − Y ) : (2) plies stratified sampling, the domain of x is divided N − 1 i i=1 into subintervals. Each of the subintervals contains the same number of sample points. These points are sam- The mean and the variance resulting from the sample pled randomly within each subinterval using a pseudo- and calculated with these two equations are uncertain. random number generator. If one compares Figure 1 Based on the central limit theorem, the uncertainty in with Figure 4 it is obvious that the stratified sampling the estimate of the mean can be quantified with the technique ensures the avoidance of clusters and gaps standard error at a certain resolution. r Var(Y ) SE(Y¯ ) = : (3) N a This equation shows that the uncertainty decreases b slowly when N increases since it depends on the square root of N. 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 In the following example a three-dimensional param- eter space is used to illustrate the properties of the Figure 4: Two examples of stratified sampling. The sampling methods. However, the reader should keep position of the points within each subinterval is chosen in mind that exploring the parameter space becomes randomly. harder as the number of analyzed parameters (k) in- creases. The sampling is performed according to a uni- The mean and variance can be calculated in the same form distribution in the interval [0,1]. Figure 2 shows way as for pseudo-random sampling (see Equations 1 a three-dimensional plot of the parameter space x1, x2 and 2). and x3 with N = 128. The number was chosen be- In multivariate stratified sampling the same technique cause of the properties of the sampling based on Sobol0 is applied. Figure 5 shows a two-dimensional param- sequences which will be explained later. For pseudo- eter space with a stratified sampling with 10 strata for random sampling any N can be chosen but for the sake each parameter. That results in 100 cells where one of comparability N = 128 was used. point is in each cell. For a given resolution stratified With a three-dimensional plot it is a difficult task to sampling results in less uncertain mean and variance check if the parameter space is explored in a proper estimates than pseudo-random sampling (Saltelli et al., - 1817 - Proceedings of Building Simulation 2011: 12th Conference of International Building Performance Simulation Association, Sydney, 14-16 November. 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 , random , random , random 0.4 0.4 0.4 2 3 3 x x x 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x1, random x1, random x2, random 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 , stratified , stratified , stratified 0.4 0.4 0.4 2 3 3 x x x 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x1, stratified x1, stratified x2, stratified 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 , LHS , LHS , LHS 2 3 3 0.4 0.4 0.4 x x x 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x1, LHS x1, LHS x2, LHS 1.0 1.0 1.0 0.8 0.8 0.8 0.6 0.6 0.6 , Sobol' , Sobol' , Sobol' 2 3 3 0.4 0.4 0.4 x x x 0.2 0.2 0.2 0.0 0.0 0.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 x1, Sobol' x1, Sobol' x2, Sobol' Figure 3: Three sampled parameters plotted against each other in pairs using different sampling techniques.