Comparison of Latin Hypercube and Quasi Monte Carlo Sampling Techniques

. Sergei Kucherenko1, Daniel Albrecht2, Andrea Saltelli2

1Imperial College London, UK Email: [email protected] 2The European Commission, Joint Research Centre, ISPRA(VA), ITALY

1 Outline

Monte Carlo integration methods

Latin Hypercube sampling design

Quasi Monte Carlo methods. Sobol’ sequences and their properties

Comparison of sample distributions . generated by different techniques

Do QMC methods loose their efficiency in higher dimensions ?

Global Sensitivity Analysis and Effective dimensions

CiComparison resu lts

2 Monte Carlo integration methods

If[]= fxdx () ∫H n see as an expp[][()]ectation: I[]fff= E [()]f x 1 N Monte Carlo : IfNi [ ]= ∑ fz ( ) N i=1 n {zHi }− is a sequence of random points in

Error:εε ε =−I[]fffI N [f ] . σ ()f = ( E (21/2 )) =→ N N 1/2 Convergence does not depent on dimensionality but it is slow Imppgyg()rove MC convergence by decreasing σ ()f Use variance reduction techniques: antithetic variables; control variates; stratified sampling → LHS sampling 3 Latin Hypercube sampling

.

Latin Hypercube sampling is a type of Stratified Sampling.

To sample N points in d-dimensions Divide each dimension in N equal intervals => Nd subcubes. Take one point in each of the subcubes so that being projected to lower dimensions points do not overlap

4 Latin Hypercube sampling

{{}π k }, kn= 1,..., - independent random permut ations of {1 ,..., N} each uniformly distributed over all N! possible permutations π ()i − 1+U k LHS coordinates: xiNknk ===ki , 1,..., , 1,..., i N k . Ui ∼ U(0,1) LHS is built by superimposing well stratified one-dimensional samples. It cannot be expected to ppgrovide good uniformityyp pro perties in a n-dimensional unit hypercube.

5 Deficiencies of LHS sampling

.

1) Space is ba dly exp lore d (a ) 2) Possible correlation between variables (b) 3) Points can not be sampled sequentially => Not suited for integration

6 Discrepancy. Quasi Monte Carlo.

Discrepancy is a measure of deviation from uniformit y: n Defintions: Qy() ( )∈ H, Qy() ( ) =××× [0, y12)[0 ) [0, y) ... [0 , yn ) , mQ( )− volume of Q

N DmQ* =−supQ()y ( ) N Qy()∈ Hn N

. *1/21/2 Random sequences: DNNNN → (ln ln ) / ~ 1/ (ln N )n Dcd* ≤ ( ))Low− Low discrepancy sequences (LDS) N N * Convergence: εQMC=−If[ ] I N [ f ] ≤ VfD ( ) N , ON(ln )n ε = QMC N

Assymptotically εQMC ~(1/) ~ON (1/ )→ much higher than

ε MC ~(ON1/ ) 7 QMC. Sobol’ sequences

ON(ln )n Convergence: ε =− for all LDS N ON(ln )n−1 For Sobol' LDS: ε ==− ,if Nk 2k , integer N

Sobol' LDS: . 1. Best uniformity of distribution as N goes to infinity. 2. Good distribution for fairly small initial sets. 3A3. A very fas t compu ttitational alg lorith m.

"Preponderance of the experimental evidence amassed to date points to Sobol' sequences as the most effective quasi-Monte Carlo method for application in financial engineering." Paul Glasserman , Monte Carlo Methods in Financial Engineering, Springer, 2003 8 Sobol LDS. Property A and Property A’

A low-discrepancy sequence is said to satisfy Property A if for any binary segment (not an arbitrary subset) of the n-dimensional sequence of length 2n there is exactly one point in each 2n hyper-octant that results from subdividing the unit hypercube along each of its length extensions into half. A low-discrepancy sequence is said to satisfy Property A’ if for any binary segment (not an arbitrary subset) of the n-dimensional sequence of length 4n there is exactly one point in each 4n hyper-octant that results from subdividing the unit hypercube along each of its length extensions into four equal parts.

.

Property A Property A’ 9 Distributions of 4 points in two dimensions

Property A

MC -> No

.

LHS -> No

Sobol’ -> Yes

10 Distributions of 16 points in two dimensions

Property A’

MC -> No

.

LHS -> No

Sobol’ -> Yes

11 Comparison of Discrepancy I. Low Dimensions

0.1 MC LHS QMC-Sobol 0.01 Use standard MC and , LHS generators 0.001 og(DN_L2)

LL Sobol' sequence generator:

0.0001 SobolSeq:

1e-05 *. Sobol' seqqyuences satisfy 128 256 512 1024 2048 4096 8192N 16384 32768 Log2(N) Properties A and A’

0.01 MC LHS www.broda.co.uk QMC-Sobol

0.001 (DN_L2) gg Lo Result: QMC in low dimensions shows 0. 0001 128 256 512 1024 2048 4096 8192 16384 32768 muchllh smaller discrepancy than Log2(N) MC and LHS 12 Comparison of Discrepancy II High Dimensions

1e-07 MC LHS QMC-Sobol

1e-08

og(DN_L2) . LL

1e- 09 128 256 512 1024 2048 4096 8192 16384 32768 Log2(N)

All sampling methods in high-dimensions have comparable discrepancy

13 Do QMC methods loose their efficiency in higher dimensions ?

ON(ln )n ε = QMC N

Assymptotically εQMC ~ON (1/ ) . * but εQMC increseas with NN until ≈ exp( n ) nN=≈50, 5 1021 − not achievable for practical appl ications Is QMC better than MC and LHS in higher dimensions (≥ 20)

14 ANOVA decomposition and Sensitivity Indices

Consider a model Yfx= () x is a vector of input variables xxxx= ()( , ,..., ) f(x) is integrable 12 k

01≤≤xi ANOVAdecompositionANOVA decomposition: k Yfxf==+( )0∑∑∑ fxii() + fxx ijij() , ++ ... f 1,2,..., k() xxx 1 , 2 ,..., k , iiji=>1 > . 1

fxii... (,,...,) xdx i =∀ 0,,1 k ≤≤ ks ∫ 11isisk 0 22 22 Variance decomposition: σσ=+ σσ +… ∑ iiiij∑ ,1,2,...,j n

k Sobol’ SI: 1 = ∑ S i + ∑ S ij + ∑ S ijl + ... + S 1, 2 ,..., k i =1 i < j i < j < l 15 Sobol’ Sensitivity Indices (SI)

„ Definition: S = σ 22/σ ii11...ss ii ... 1 σ 22= f ()xxdxx,..., ,..., - partial variances ii1111...ss∫ ii ... i is i is 0 1 σ 2 = fx− f2 dx - total variance ∫( ( ) 0 ) 0 „ Sensitivity indices for subsets. of variables: xyz= ( , ) m σ 22= σ yii∑∑ 1 ,,… s σ sii=〈〈∈Κ1 ( 1 ... s ) tot 2 σ 2 2 Total variance for a subset: ()y = −σ z

Corresponding global sensitivity indices:

2 2 tot tot 2 2 S y = σ y /σ , S y = (σ y ) /σ .

16 Effective dimensions

Let u be a cardinality of a set of variables u . The effective dimension of f (x ) in the superposition sense ithis the small lltitest integer d S such htht that

∑ Su ≥−(1εε ), << 1 0<

It means that f (x ) is almost a sm u of d S -dimensional functions. ______. ______

The function fx ( ) has effective dimension in the truncation sense dT if

∑ Su ≥−(1 ε ), ε <<1 ud⊆{1,2,...,T }

Important property: ddS ≤ T

n Example: fx( )= ∑ xiT→= dS 1, d =n i=1 17 Classification of functions

Type A. Variables are Type B ,C. Variables are not equally important equally important T T S y S z >> ↔dT << n SSij≈↔↔ d T ≈ n nnyz

.

Type B. Type C. Dominant Dominant low order indices higher order indices n n Sn<<↔1 d ≈ ∑Sni ≈↔1 dS << ∑ i S i=1 i=1

18 When LHS is more effective than MC ?

ANOVA: fx ( )=+ f0 ∑ fxii ( ) + rx ( ) i rx( )− high order interactions terms

2211 LHS: ErxdxO (ε LHS )=+ [ ( )] ( ) (Stein, 1987) NN∫H n

. 222111 MC: EfxdxrxdxO (ε MC )= [ i ( i )]++ [ ( )] ( ) ∑∫∫HHnn NNNi 2 if [()] [rx ( )]dx i s small ⇔ dS ( Type B func tions ) ∫H n

22 →< EE (εεLHS ) ( MC )

19 Classification of functions

Function Description Relationship dT dS QMC is LHS is type between more more tot Si and Si efficient efficient than MC than MC AA few << n << n Yes No tot to dominant Sy /ny >> Sz / nz variables BNo ≈ n << n Yes Yes unimportant subsets; only . S ≈ S , ∀ i, j low-order i j S / S tot ≈ 1, ∀ i interaction i i terms are present CNo ≈ n ≈ n No No unimportant subsets; high- S ≈ S , ∀ iji, j order i j S / S tot << 1, ∀ i interaction i i terms are present

20 How to monitor convergence of MC, LHS and QMC calculations ?

The root mean square error is defined as

K 1/2 ⎛⎞1 k 2 ε =−⎜⎟∑()IIdN ⎝⎠K k =1 K is a number of independent runs

MC and LHS: all runs should be statisticall. yyp independent ( use a different seed point ). QMC: for each run a different part of the Sobol' LDS was used ( start from a different index number ). The root mean square error is approximated by the formula cN −α , 0 <<α 1 MC: α ≈ 0.5 QMC: α ≤ 1 LHS: α ∼ ? 21 Integration error vs. N. Type A n i i s (a) f(x) = ∑ j=1(-1) Π j=1 xj, n = 360, (b) f(x) = Π i=1 │4xi-2│/(1+a i), n = 100

0 MC (-0.50) QMC-SOB (-0.94) T T 1/2 -5 LHS (- 0.52) S S ⎛⎞1 K y z ε =−()IIk 2 >> ↔dT << n ⎜⎟∑ N -10 nn ⎝⎠K k =1 yz (error) 2 -15 log

-20

-25 εα~,01N −α << 8 11141720 log2(N) (a) . -2 MC (0.49)

QMC-SOB (0.65)

LHS (0.50) -6 r) oo (err 2

log -10

(b) -14 8 11141720

log2(N)

22 Integration error. Type A

S T T y Sz K 1/2 >> ↔dT << n ⎛⎞1 k 2 ε =−⎜⎟∑()IIN nnyz ⎝⎠K k =1 εα~,01N −α <<

Index Function Dim n Slope Slope Slope MC QMC LHS

n i . i ∑(−1) ∏ x j 1Ai=1 j = 1 360 0.50 0.94 0.52

n 42xaii−++ ∏ 1+ a 2A i = 1 i 100 0.49 0.65 0.50

a1 = a2 = 0 a3 ==a= … = a100 =652= 6.52

23 Integration error vs. N. Type B

n Dominant low order indices ∑ Si ≈↔1 ↔<

MC (0.52)

-8 QMC-SOB (0 .96)

LHS (0.69) -12 n n − x f (x) = i -16 ∏ i=1 n − 0.5 (a) g2(error) lo -20 n = 360

-24 8 11141720 log2(N).

MC (0.50) -8 QMC-SOB (0.87) LHS (0.62) n 1/ n f (x) = ∏(1+1 n)xi -12 i=1 (b) n = 360 og2(error) l -16

-20 811141720

log2(N)

24 Integration error. Type B functions Dominant low order indices

n ∑ Sni ≈1 ↔<

Index Function Dim n Slope Slope Slope

. MC QMC LHS

n nx− i 1B∏ 30 0.52 0.96 0.69 i = 1 n − 050.5 1 n n ⎛⎞n 2B⎜⎟1 + ∏ xi 30 0.50 0.87 0.62 ⎝⎠n i = 1 n 42xii− + a ∏ 1 + a 3B i =1 i 30 0.51 0.85 0.55

ai = 6.52

25 The integration error vs. N. Type C n Dominant higher order indices: ∑ Sni <<1 ↔dS ≈ i=1

-2 MC (0.47)

QMC-SOB (0.64) -4 LHS (0.50)

-6 n 42xa−+ fx()= ii , a= 0 -8 ∏ i

log2(error) 1+ a (a) i=1 i -10 n →−∏ 42xi -12 i=1 8 11141720n =10 log2(N).

-2 MC (0.49) QMC-SOB (0.68) -4 LHS (0.51)

-6

-8 n 1/n log2(error) (b) -10 f ()xx= (1/2) ∏ i i=1 -12

-14 n =10 811141720

log2(N)

26 Integration error for type C functions Dominant higher order indices

n ∑ Sni <<↔1 dS ≈ i=1

Index Function Dim n Slope Slope Slope . MC QMC LHS n 1C∏ 42xi − 10 0.47 0.64 0.50 i=1 n n 2C(2) ∏ xi 10 0.49 0.68 0.51 i =1

27 The integration error vs. N. Function 1A

MC QMC LHS Theoretical Value Value

-0.315

n i -0.320 i ∑(−1) ∏ x j , i=1 j = 1 -0.325 n = 360 -0.330 - integral AA -0.335

-0.340 Function 1 Function -0.345 .

-0.350

-0.355 0 2 4 6 8 10 12 log2 (N)

QMC: convergence is monotonic MC and LHS: convergence curves are oscilla ting

QMC is 30 times faster than MC and LHS

LHS: it is not possible to incrementally add a new point while keeping the old LHS design 28 Evaluation of quantiles I. Low quantile

n Distribution 2 dimension n = 5. f ()xx= ∑ i , i=1 xNi ∼ (0,1) are independent standard normal variates

0.0100 "MC" "QMC" "LHS" Expon. ("QMC") Expon. ("MC")

) Expon. ("LHS")

ww .

0.0010 g(MeanError Lo g(MeanError oo L

0.0001 10 11 12 13 14 15 16 17 18 Log2(N)

Low quantile (percentile for the cumulative distribution function) = 0.05 A superior convergence of the QMC method 29 Evaluation of quantiles II. High quantile

n Distribution 2 dimension n = 5. f ()xx= ∑ i , i=1 xNi ∼ (0,1) are independent standard normal variates

0.0100 "MC" "QMC" "LHS" Expon. ("MC") Expon. ("QMC") Expon. ("LHS")

igh) .

0.0010 Log(MeanError H Log(MeanError

0.0001 10 11 12 13 14 15 16 17 18 Log2(N)

Hig h quantile (percentil e for th e cumul a tive dis trib u tion func tion ) = 0950.95 QMC convergences faster than MC and LHS 30 Summary

Sobol’ sequences possess additional uniformity properties which MC and LHS techniques do not have (Properties A and A’).

Comparison of L2 discrepancies shows that the QMC method has the lowest discrepancy in low dimensions ( up to 20).

QMC method outperforms MC and .LHS for types A and B functions (problems with low effective dimensions)

LHS met ho d outper forms MC on ly for type B funct ions.

QMC remains the most efficient method among the three techniques for non-uniform distributions

31