Centered L2-Discrepancy of Random Sampling and Latin Hypercube Design, and Construction of Uniform Designs

Home , Latin hypercube sampling

MATHEMATICS OF COMPUTATION Volume 71, Number 237, Pages 275–296 S 0025-5718(00)01281-3 Article electronically published on October 16, 2000

CENTERED L2-DISCREPANCY OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN, AND CONSTRUCTION OF UNIFORM DESIGNS

KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

Abstract. In this paper properties and construction of designs under a centered version of the L2-discrepancy are analyzed. The theoretic expectation and variance of this discrepancy are derived for random designs and Latin hypercube designs. The expectation and variance of Latin hypercube designs are signiﬁcantly lower than that of random designs. While in dimension one the unique uniform design is also a set of equidistant points, low-discrepancy designs in higher dimension have to be generated by explicit optimization. Op- timization is performed using the threshold accepting heuristic which produces low discrepancy designs compared to theoretic expectation and variance.

1. Introduction Many problems arising in industry, statistics, physics, and finance require mul- tivariate integration, the canonical form of which can be expressed as Z (1.1) I(f)= f(x)dx, Cs s s where C =[0, 1) and f(x)=f(x1, ··· ,xs). The sample mean method has been recommended to give an approximation to I(f)by 1 Xn (1.2) Iˆ(f,P)= f(x ), n i i=1 s where P = {x1, ··· , xn} is a set of points on C .Ifx1, ··· , xn are i.i.d. uniformly distributed on Cs, the set is called simple random sampling or simple random design (SRD) and is denoted by Rn,s.ItisknownthatIˆ(f,Rn,s)isunbiased and has an asymptotic variance O(n−1). This rate of convergence is too slow for the applications. Therefore, McKay, Beckman and Conover [MBC79] proposed the so-called Latin hypercube design (LHD), which also provides an unbiased estimate Iˆ(f,P) with smaller asymptotic variance than that of SRD. LHD has been widely used in conducting computer experiments. A systematic study on LHD and various modified versions of LHD that can significantly reduce the asymptotic variance of LHD is given by [Owen92], [Owen94], [Owen95], [KO96], and [Tang93]. In this

Received by the editor July 20, 1999 and, in revised form, February 25, 2000. 2000 Mathematics Subject Classiﬁcation. Primary 68U07; Secondary 65D17, 62K99. Key words and phrases. Uniform design, Latin hypercube design, threshold accepting heuristic, quasi-Monte Carlo methods. This work was partially supported by a Hong Kong RGC-grant and SRCC of Hong Kong Baptist University.

c 2000 American Mathematical Society 275

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 276 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

article, we only consider a special case of LHD, where x1, ··· , xn are i.i.d. uniformly distributed on the lattice set 2ai−1 xi = 2n ,i=1,... ,n, T = x =(x1,... ,xn) . (a1,... ,an)isapermutationof{1,... ,n}

We denote this LHD by Ln,s. There exist diﬀerent measures to assess the performance of various designs on Cs. The Koksma-Hlawka inequality gives an upper bound for the approximation error

(1.3) |I(f) − Iˆ(f,P)|≤D(P)V (f),

where D(P) is the discrepancy of P that will be defined in (1.4), and V (f)isa measure of the variation of f [Nie92]. In fact, we can find a number of other pairs {D(P),V(f)} satisfying the Koksma-Hlawka inequality, where D(P)isameasure of nonuniformity of P and V (f) is a measure of variation of f. An excellent study −1/2 on this topic is given by [Hic98]. For SRD, D(Rn,s)=O(n )asn →∞.The determination of the order of convergence of D(P)forLHDP is still an open problem. It will be answered for the centered L2-discrepancy in this paper. s s Let P = {x1, ··· , xn} be a set of n points on C =[0, 1) .ThestarLp- discrepancy (Lp-discrepancy for simplicity) has been widely used in quasi-Monte Carlo methods (or number-theoretic methods) as well as in uniform design theory (cf. [Nie92] and [FWa94]). It is defined as Z p 1/p N(P, [0, x)) (1.4) Dp(P)= − Vol([0, x)) dx , Cs n

where [0, x) denotes the interval [0,x1) × ··· × [0,xs), N(P, [0, x)) the number of points of P falling in [0, x), and Vol(A)thevolumeofA.AmongtheLp- discrepancies, the D2(P)andD(P)=D∞(P) (called discrepancy for short) are used most frequently. Hickernell [Hic98] pointed out some weakness of the Lp-discrepancy and proposed several modified Lp-discrepancies, among which the centered L2-discrepancy (CL2) seems most interesting. Ma and Fang [MF98] and Fang and Mukerjee [FM00] found some connections between CL2 and orthogonality, minimum aberration, and confounding for a certain class of designs. The centered Lp-discrepancy is a modification of the Lp-discrepancy by the requirement that it becomes invariant under reflections of P about any plane xj =0.5. It is defined by Z X N(P ,J ) p P p u xu − (1.5) (Dp( )) = Vol(Jxu ) du, u n u=6 ∅ C where u is a nonempty subset of the set of coordinate indices S = {1, ··· ,s}, |u| denotes the cardinality of u, Cu is the |u|-dimensional unit cube involving the coordinates in u, Jx is an s-dimensional interval uniquely determined by x, Pu is P u u As the projection of to C ,andJxu is the projection of Jx on C .Let denote s s s the set of 2 vertices of the cube C and α =(a1,... ,as) ∈A be the closest one to x. Define

s Jx = {y ∈ C | min(aj,xj ) ≤ yj < max(aj ,xj), for 1 ≤ j ≤ s}.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 277

For CL2 Hickernell [Hic98] derived an analytical expression 13 s 2 Xn Ys 1 1 CL (P)2 = − (1 + |x − 0.5|− |x − 0.5|2) 2 12 n 2 kj 2 kj k=1 j=1 1 Xn Xn Ys 1 1 1 (1.6) + 1+ |x − 0.5| + |x − 0.5|− |x − x | , n2 2 ki 2 ji 2 ki ji k=1 j=1 i=1

where xk =(xk1, ··· ,xks) ∈P. From the deﬁnition (1.5) the centered Lp- discrepancy takes into account not only the uniformity of P over Cs, but also uniformity of all the projections of P over Cu. In Sections 2 and 3 we shall derive the expectation and variance for square CL2 of simple random designs Rn,s and Latin hypercube designs Ln,s,andgive comparisons of these statistics. It will be shown that the LHD has much lower expected CL2-value and variance than SRD. Our results are consistent with the results of comparing variance of Iˆ(f,P) between SRD and LHD [Owen92]. Note that the LHD Ln,s can be deﬁned in terms of U-type designs.

Definition 1.1. AU-typedesignUn,qs is an n × s matrix U =(uij ) of which each column has q entries 1,... ,q appearing equally often. The induced matrix of U, − denoted by XU =(xij ), is defined by xij =(uij 0.5)/q.Whenq = n,weusethe notation Un,s instead of Un,ns .LetUn,qs and Un,s be the set of all Un,qs and the set of all Un,s, respectively. Any induced matrix XU defined in Definition 1.1 corresponds to a set of n s P P points on C , denoted by U . Each row of XU corresponds to a point of U on s P L C .TheCL2(U) is defined as CL2( U ). The LHD n,s is a design XU ,where ∈U ∈U U n,s. The design XU , where U n,qs , can be considered as an extension of LHD and is denoted by Ln,qs . In Sections 2 and 3 we also derive the expectation 2 and variance of CL2(Ln,qs ) . The U-type design is the basis of the uniform design. The latter is one of “space filling” designs (Cheng and Li (1995) [CL95], and Koehler and Owen (1996) [KO96]). The uniform design allocates experimental points uniformly scattered on the domain in the sense of low-discrepancy [FWa94]. Any discrepancy mentioned before can be used as a measure of nonuniformity. In the past, most uniform designs are obtained in terms of the discrepancy and the L2-discrepancy. Fang and Winker [FW98] found that both discrepancy and L2-discrepancy are not suitable measures of nonuniformity for searching the UD, since the discrepancy is not sensitive enough for identifying different designs while the L2-discrepancy ignores differences | N(P,[0,x)) − |2 n Vol([0, x)) on any low-dimensional subspace. Therefore, they recom- mend the use of the three modified L2-discrepancies proposed by [Hic98]. In this paper we concentrate on the centered L2-discrepancy for construction of uniform designs. s ∗ Let Pn be the class of sets of n points on C .AsetP ∈Pn is called a uniform design if it has the smallest CL2-value over Pn, i.e., ∗ (1.7) CL2(P )= min CL2(P). P∈Pn In Section 4 we propose a heuristic optimization algorithm for the construction of uniform designs based on U-type designs under CL2. The results obtained in Sections 2 and 3 provide information which can be used to reduce the computing time of searching uniform designs and low-discrepancy designs.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 278 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

The paper is organized as follows. The expectation and variance of CL2(Rn,s) and CL2(Ln,qs ) are derived in Sections 2 and 3. Part of the proofs are put into an appendix. Some numerical comparisons of these expectations and variances are also given. In Section 4 we discuss how to construct uniform designs. An algorithm based on threshold accepting is proposed and some modiﬁcations are suggested. The performance of this algorithm will be discussed in Section 5, which also provides numerical results for low-discrepancy designs. The last section contains concluding remarks.

2. The expectation of square CL2 of Rn,s and Ln,s

In this section we derive the expectation of square centered L2-discrepancy for simple random designs Rn,s and Latin hypercube designs Ln,qs . Their diﬀerence is also given.

Theorem 2.1. The average square centered L2-discrepancy of the random design Rn,s is given by 5 s 13 s (2.1) E(CL (R )2)= − /n. 2 n,s 4 12

The average square centered L2-discrepancy of Latin hypercube design Ln,qs is given by (2.2) 2 E(CL (L s ) )= 2 n,q  2  13 s − 13 − 1 s 1 5 − 1 s − 1 13 n−q − 1 s ( 12 ) 2( 12 12q2 ) + n ( 4 4q2 ) +(1 n )( 12 + 6q2(n−1) 4q2 ) ,qodd,  2  13 s − 13 1 s 1 5 s − 1 13 n−q s ( 12 ) 2( 12 + 24q2 ) + n ( 4 ) +(1 n )( 12 + 6q2(n−1) ) ,qeven.

Proof. From formula (1.6) and the fact that x1, ··· , xn are i.i.d. uniformly distributed on Cs,wehave 13 2 Xn Ys 1 1 E(CL (R )2)=( )s − E(1 + |x − 0.5|− |x − 0.5|2) 2 n,s 12 n 2 kj 2 kj k=1 j=1 1 Xn Ys + E(1 + |x − 0.5|) n2 ki k=1 i=1 − 2 Xn kX1 Ys 1 1 1 + E 1+ |x − 0.5| + |x − 0.5|− |x − x | n2 2 ki 2 ji 2 ki ji k=1 j=1 i=1 Z s 13 1 1 1 =( )s − 2 1+ (1 + |x − 0.5|− |x − 0.5|2)dx 12 0 2 2 Z s 1 1 + (1 + |x − 0.5|)dx n 0 Z Z s 2 n(n˙− 1) 1 1 1 1 1 + (1 + |x − 0.5| + |y − 0.5|− |x − y|)dxdy n2 2 2 2 2 0 0 5 s 13 s = − /n. 4 12

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 279

Thus, equation (2.1) is proved. For giving a proof of formula (2.2) we need the following lemma. Its proof is straightforward and is omitted. Lemma 2.2. q 1 X 2i − 1 − q 1 − 1 ,qis odd, | | = 4 4q2 q 2q 1 ,qis even; i=1 4

q 1 X 2i − 1 − q 1 1 ( )2 = − ; q 2q 12 12q2 i=1

q q 1 X X |j − k| q2 − 1 = . q q 3q j=1 k=1

Now, we come back to prove formula (2.2). Xn Ys 2 13 s 2 1 1 2 E(CL (L s )=( ) − E(1 + |x − 0.5|− |x − 0.5| ) 2 n,q 12 n 2 kj 2 kj k=1 j=1 1 Xn Ys + E(1 + |x − 0.5|) n2 ki k=1 i=1 − 2 Xn kX1 Ys 1 1 1 + E 1+ |x − 0.5| + |x − 0.5|− |x − x | n2 2 ki 2 ji 2 ki ji k =1 j=1 i=1 ! q s 13 1 X 1 2k − 1 1 2k − 1 =( )s − 2 (1 + | − 0.5|− | − 0.5|2) 12 n 2 2q 2 2q k=1 ! q s 1 1 X 2k − 1 + (1 + | − 0.5|) n q 2q k=1 2 n(n − 1) 1 1 1 s + E(1 + |x − 0.5| + |x − 0.5|− |x − x |) . n2 2 2 ki 2 ji 2 ki ji Note that 1 1 1 E(1 + |x − 0.5| + |x − 0.5|− |x − x |) 2 ki 2 ji 2 ki ji q q X X 1 2k − 1 1 2j − 1 1 k − j = (1 + | − 0.5| + | − 0.5|− | |)P (k, j), 2 2q 2 2q 2 q k=1 j=1 where P (k, j) is the probability that the ﬁrst two elements of a speciﬁc column of U,whereU is uniformly distributed on Un,qs ,arek and j, respectively. Let r = n/q be the duplicate number of each level in each column of U. Obviously, we r2 6 r(r−1) have P (k, j)= n(n−1) for k = j,andP (k, k)= n(n−1) . Put these results into the above formula and (2.2) is proved. From this theorem we immediately have the following corollaries. Corollary 1. We have 2 −1 −2 E(CL2(Ln,qs ) )=O(n )+O(q ) as q →∞and n = qs →∞.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 280 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

Corollary 2. The average square discrepancy of Ln,s is (2.3) E(CL (L )2)= 2 n,s   13 s − 13 − 1 s 1 5 − 1 s − 1 13 − 1 − 1 s ( 12 ) 2( 12 12n2 ) + n ( 4 4n2 ) +(1 n )( 12 6n 4n2 ) ,nodd,   13 s − 13 1 s 1 5 s − 1 13 − 1 s ( 12 ) 2( 12 + 24n2 ) + n ( 4 ) +(1 n )( 12 6n ) ,neven.

From (2.1) and (2.3) we can ﬁnd the diﬀerence of average square CL2-value between simple random designs Rn,s and Latin hypercube designs Ln,s. Corollary 3. 2 2 E(CL2(Rn,s)) − E(CL2(Ln,s)) − − (2.4) 13 s 1 s 13 s 1 2s2 +11s = − + O(n−3), 12 6n 12 156n2 in particular, when s =1, 1 1 E(CL (R ))2 − E(CL (L ))2 = − . 2 n,1 2 n,1 6n 12n2

This corollary shows that LHDs have a lower expectation of square CL2-value than that of simple random designs. This result is consistent with the result that the Latin hypercube designs beat random designs in the sense of small-variance 2 [Owen92]. It is not easy to ﬁnd a sharp upper bound for CL2(PU ) over U ∈Un,qs . The following example gives a bad U-type design in the sense of uniformity. There- fore, its CL2 can be considered as an upper bound of CL2(PU ), U ∈Un,qs . P { 2i−1 2i−1 } Example 2.3. The CL2 of the U-type design ∗ = ( 2n ,... , 2n ),i=1,... ,n in [0, 1]s,wherethen points are on a line, is Xn − − − − 2 13 2 2k 1 n 2k 1 n 2 s (CL (P∗)) = − (1 + | |−| | /2) 2 12 n 4n 2n k=1 1 Xn Xn 2k − 1 2j − 1 |k − j| (2.5) + (1 + | | + | |− )s = O(1). n2 4n 4n 2 k=1 j=1

For s =2, 3 and 4, formula (2.5) can be simpliﬁed:    1 + 1 ( 50 − 11 ),s=2,  90 240 n2 n4 2 2321n6+2390n4−10206n2+360 (2.6) (CL2(P∗)) = ,s=3,  60480n6   64427n8+491400n6−322056n4+37080n2+3024 725760n8 ,s=4.

3. The variance of square CL2 of Rn,s and Ln,s 2 2 We have calculated the expectation of CL2(R) and CL2(Ln,qs ) in the previous section. In this section, by a similar, but more complicated procedure, we derive the variance of square CL2 for the random design and Latin hypercube design. As the related proof is too long, we put it into an appendix (found at the end of this article).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 281

Theorem 3.1. For the random design Rn,s we obtain 2 19 s 47 s 169 s Var(CL (R )2)= − 2 + 2 n,s n2 16 40 144 1 19 s 25 s 19 s + − − 2 n3 12 16 16 65 s 87 s 47 s 169 s (3.1) +4 − 4 +8 − 6 . 48 64 40 144

For the Latin hypercube design Ln,qs we ﬁnd

s2 2 2 13 4 s s Var(CL (L s ) )=− E(CL (L s )) − + (A +(n − 1)A ), 2 n,q 2 n,q 12 n 1 2 4 − (Bs +2(n − 1)Bs +(n − 1)Bs +(n − 1)(n − 2)Bs) n2 1 2 3 4 1 + (Cs +4(n − 1)Cs +(n − 1)Cs +2(n − 1)Cs +2(n − 1)(n − 2)Cs n3 1 2 3 4 5 − − s − − − s (3.2) +4(n 1)(n 2)C6 +(n 1)(n 2)(n 3)C7 ),

where δq =1for odd q and 0 for even q,and

47 7+80q2 1+16q2 A = + − δ , 1 40 960q4 q 64q4 169 21 + 240q2 +4q4 − 5n(1 + 52q2) 3+n +48q2 − 52nq2 A = − + δ , 2 144 2880(−1+n)q4 q 192(−1+n)q4 87 1 1+24q2 B = + − δ , 1 64 32q2 q 64q4 47 4n − 10q2 +70nq2 − 59q4 3+72q2 − 2n(2 + 37q2) B = + + δ , 2 40 320(−1+n)q4 q 192(−1+n)q4 65 6 − 10n + q2 3+72q2 + n(4 − 82q2) B = − + δ , 3 48 192(−1+n)q2 q 192(−1+n)q4 169 5n2(2 + 65q2)+5q2(18 + 107q2) − 3n(12 + 235q2 +88q4) B = + 4 144 1440(−2+n)(−1+n)q4 −3+2n − 72q2 + 115nq2 − 39n2q2 + δ , q 96(−2+n)(−1+n)q4 19 1 1 C = − − δ , 1 12 12q2 q 2q2 87 16 + 36n − 43q2 32q2 − n(1 + 34q2) C = + + δ , 2 64 192(−1+n)q2 q 64(−1+n)q4 25 −4+q2 n +8q2 − 10nq2 C = − + δ , 3 16 48(−1+n)q2 q 16(−1+n)q4 19 4+18n − 19q2 8q2 − n(1 + 8q2) C = + + δ , 4 16 48(−1+n)q2 q 16(−1+n)q4

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 282 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

65 −16 + 20n2 +44q2 − 3n(12 + 7q2) C = + 5 48 96(−2+n)(−1+n)q2 −96q2 + n2(2 − 56q2)+3n(−1+54q2) + δ , q 96(−2+n)(−1+n)q4 47 8q2(−10 + 49q2) − 3nq2(120 + 61q2)+n2(16 + 170q2) C = + 6 40 480(−2+n)(−1+n)q4 −96q2 − 2n2(2 + 25q2)+3n(3 + 50q2) + δ , q 96(−2+n)(−1+n)q4 15q2(12−59q2)+10n3(1+13q2)+5nq2(162+137q2)−12n2(4+55q2+11q4) C = 7 360(−3+n)(−2+n)(−1+n)q4 48q2 − 3n(1 + 26q2)+n2(1 + 26q2) 169 − δ + . q 48(−2+n)(−1+n)q4 144

Corollary 4. ThevarianceofsquareCL2(Ln,s) is given by (3.2) with the following Ai’s, Bj’s and Ck’s.

47 7+80n2 1+16n2 A = + − δ , 1 40 960n4 n 64n4 169 21 + 16n + 256n2 − 4n3 3+4n +52n2 A = + − δ , 2 144 2880n4 n 192n4 87 1 1+24n2 B = + − δ , 1 64 32n2 n 64n4 47 4 − 10n +70n2 − 59n3 3 − 4n +72n2 − 74n3 B = + + δ , 2 40 320(−1+n)n3 n 192(−1+n)n4 65 6 − 10n + n2 3+4n +72n2 − 82n3 B = − + δ , 3 48 192(−1+n)n2 n 192(−1+n)n4 169 18 − 41n + 332n2 − 264n3 −3+2n − 72n2 + 115n3 − 39n4 B = + + δ , 4 144 1440(−1+n)n3 n 96(−2+n)(−1+n)n4 19 1 1 C = − − δ , 1 12 12n2 n 2n2 87 16 + 36n − 43n2 −1+32n − 34n2 C = + + δ , 2 64 192(−1+n)n2 n 64(−1+n)n3 25 −4+n2 1+8n − 10n2 C = − + δ , 3 16 48(−1+n)n2 n 16(−1+n)n3 19 4+18n − 19n2 −1+8n − 8n2 C = + + δ , 4 16 48(−1+n)n2 n 16(−1+n)n3 65 8+22n − 21n2 3+94n − 162n2 +56n3 C = + − δ , 5 48 96(−1+n)n2 n 96(−2+n)(−1+n)n3 47 32 + 196n − 183n2 9 − 100n + 150n2 − 50n3 C = + + δ , 6 40 480(−1+n)n2 n 96(−2+n)(−1+n)n3 169 22 + 155n − 132n2 3 − 49n +78n2 − 26n3 C = + + δ . 7 144 360(−1+n)n2 n 48(−2+n)(−1+n)n3

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 283

−2 −3 s=2 s=3 s=4 10 s=5 10 −3 10 −3 10 −3 −4 10 10 −4 10 −4 −5 10 −4 10 10 −5 10 −5 −6 10 10 −5 −6 10 10 5 10 20 30 5 10 20 30 5 10 20 30 5 10 20 30

0.04 −1 −2 s=6 s=7 0.05 s=8 10 s=9 10 0.01 0.01 −2 −3 10 10 0.001 0.001 −3 −4 10 10 0.0001 0.0001 −4 −5 10 10 5 10 20 30 5 10 20 30 5 10 20 30 5 10 20 30

−1 0.4 0.8 10 s=10 −1 s=11 s=12 s=13 10 0.3 0.1 −2 0.1 10 −2 10 0.01 −3 0.01 10 −3 10 0.001 −4 0.001 10 −4 10 5 10 20 30 5 10 20 30 5 10 20 30 5 10 20 30

Figure 1. Comparison of variance

Formulas (3.1) and (3.2) are too complicated for an intuitive understanding. Therefore, Figure 1 gives numerical comparisons of variance between SRD and LHD Ln,s for s =2,... ,13, where “points” is for SRD and “stars” for LHD. The plots show the variance against the number of runs n on a double logarithmic scale. It becomes obvious that LHD Ln,s has much smaller variance of square CL2 than SRD. The cases of s = 2 and 3 are particularly interesting in practice. For these cases again much simpler formulas can be obtained. Example 3.2. (a) s =2 2 2 E(CL2(Ln,q2 ) )andVar(CL2(Ln,q2 ) )forq even are 5 − 34q2 +4q4 +n(−1+26q2) and 144(−1+n)q4 (−1+q2)2(−225q4 +2n3(7+2q2)2 +6n(−4+62q2 +41q4) − n2(124+328q2 +79q4)) , 32400(−3+n)(−2+n)(−1+n)2 nq8 2 2 respectively. E(CL2(Ln,q2 ) )andVar(CL2(Ln,q2 ) )forq odd are 1 − 68q2 +8q4 + n(7 + 52q2) and 288(−1+n)q4 (−1+q2)2(−225q4 +2n3(7 + 2q2)2 +6n(−4+62q2 +41q4) − n2(124 + 328q2 +79q4)) , 32400(−3+n)(−2+n)(−1+n)2nq8 respectively.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 284 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

(b) s =2andq = n 2 2 E(CL2(Ln,2) )andVar(CL2(Ln,2) )forn even are

−5 − 4n +30n2 +4n3 (−1+n)(1 + 8n)(2 + n − n2)2 and , 144n4 32400n8

2 2 respectively. E(CL2(Ln,2) )andVar(CL2(Ln,2) )forn odd are

−1 − 8n +60n2 +8n3 (−2+n)2(1 + n)2(−1 − 7n +8n2) and , 288n4 32400n8

respectively. (c) s =3andq = n 2 2 E(CL2(Ln,3) )andVar(CL2(Ln,3) )forn even are

29 + 36n − 223n2 − 200n3 + 663n4 + 164n5 and 1728n6 1 (−6016 + 3968n − 67904n2 + 607296n3 − 1337752n4 2985984000(−1+n)2n12 −17983064n5 + 28836416n6 + 19647736n7 − 48223786n8 +10671771n9 + 18977848n10 − 13190707n11 + 2610944n12),

2 2 respectively. E(CL2(Ln,3) )andVar(CL2(Ln,3) )forn odd are

−1 − 46n2 − 656n3 + 2652n4 + 656n5 and 6912n6 (1 + n)2 (229241 + 1266861n − 3213532n2 2985984000(−2+n)2n12 +19720300n3 − 48811906n4 − 423698n5 + 119671988n6 −153693140n7 + 86277425n8 − 23634483n9 + 2610944n10),

respectively.

4. Searching uniform designs under CL2 In this section we discuss the construction of uniform designs and low-discrepancy designs for the centered L2-discrepancy. When s = 1, the following theorem shows that the set of equidistant points is a unique uniform design on [0, 1]. This result is consistent with the ﬁndings for the star discrepancy [FWa94, Example 1.2].

Theorem 4.1. When s =1, the unique uniform design on [0, 1] under CL2 is 2i − 1 ,i=1,... ,n , 2n

2 1 and its CL2 is 12n2 .

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 285

Proof. Let P = {x1,x2,... ,xn} be a set in [0, 1]. Without loss of generality, we suppose x1 ≤ x2 ≤···≤xn and let yk = xk − 1/2, k =1, 2,... ,n. 13 2 Xn |y | |y |2 (CL (P))2 = − (1 + k − k ) 2 12 n 2 2 k=1 1 Xn Xn |y | |y | |y − y | + (1 + k + j − k j ) n2 2 2 2 k=1 j=1 13 1 Xn Xn |y | |y | |y − y | = + (1 + k + j − k j 12 n2 2 2 2 k=1 j=1 |y | |y |2 |y | |y |2 −(1 + k − k ) − (1 + j − j )) 2 2 2 2 1 1 Xn 1 X = + ( y2 − (y − y )) 12 n k n k j k=1 j>k 1 1 Xn 2k − 1 − n 1 Xn 2k − 1 − n = + (y − )2 − ( )2 12 n k 2n n 2n k=1 k=1 1 1 Xn 2k − 1 − n = + (y − )2. 12n2 n k 2n k=1 P 2 2k−1−n (CL2( )) achieves its minimum if and only if yk = 2n , i.e., if and only if 2k−1 xk = 2n , k =1,... ,n. The proof is completed. Clearly, the solution of (1.7) is not unique when s>1. To search uniform designs for given n and s(s>1) is probably an NP hard problem when n and s increase. Furthermore, even for moderate values of n and s, it is an almost intractable problem to ﬁnd a uniform design, because the domain is too large. Therefore, Fang and Hickernell [FH95] suggested considering only a subset of designs, the so-called U-type designs, as the domain for searching uniform designs. U-type designs were deﬁned in Section 1. If one chooses CL2 as measure of nonuniformity, a design ∗ s U ∈Un,qs is called U-uniform design, denoted by Un(q ), if it has the smallest CL2-value over Un,qs . Figure 2 gives plots of uniform designs and U-uniform designs for s =2and n =2,... ,9. These uniform designs are obtained by the Nelder-Mead simplex method, which in dimension two can be directly applied for n ≤ 6. For n =7, 8, 9,

0.062283 0.029578 0.016262 0.011051 0.0076276 0.0058239 0.0044751 0.0035828

0.059886 0.028844 0.016185 0.010937 0.007612 0.0057821 0.0044558 0.0035755

Figure 2. Plots of uniform designs and U-uniform designs

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 286 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

the U-uniform designs have to be used as initial values in order to obtain convergence to a low-discrepancy design. Thus, this method is not generally applicable, i.e., for dimension s>2 or large n. The upper plots on Figure 2 are of U-uniform designs and the lower plots are of uniform designs. The CL2-value of each design is put on the top of its plot. Obviously, U-uniform designs are very close to their corre- sponding uniform designs. However, when n and s increase, designs obtained by the application of the simplex method often exhibit poor uniformity, while U-uniform designs or U-type designs with low discrepancy can still be obtained. Therefore, we shall construct only U-uniform designs in this paper and call them uniform designs for simplicity throughout the rest of the paper. Winker and Fang [WF98] applied an implementation of the threshold accepting heuristic to search uniform designs under the discrepancy D defined in (1.4). They found a number of uniform designs and very low-discrepancy designs. In this section, we use a similar implementation for searching uniform designs under CL2. We also discuss some modifications of the algorithm for improving the results. As the application of the threshold accepting heuristic to uniform design problems is described in some more detail in [WF97], [FW98], and [WF98], we restrict ourselves to a sketch of the main ideas. Threshold accepting tackles the optimization problem given in (1.7) restricted to U-uniform designs by a refined local search technique. It starts with some arbitrarily chosen U-type design P∈Un,qs . Then, 2 in each iteration step the CL2 of the current design is compared with that of some neighbouring design. If the new design is better than the current one, it becomes the new candidate solution. The same holds true if it is not much worse. The threshold up to which a worsening is accepted is defined by the threshold sequence which decreases to zero as the algorithm proceeds (a data driven method for the generation of this threshold sequence is described in [WF97]). If the new design is not accepted based on this criterion, a new trial design is selected in the neighbourhood of the current design. The final crucial step of the implementation of threshold accepting consists in the definition of neighbourhoods on the set Un,qs . The approach followed in [WF98] consisted in selecting one or several columns of the induced matrix and exchanging randomly two elements in each of the selected columns. Thus, it is guaranteed that the new candidate design is also a U-type design. A first modification of this neighbourhood concept restricts the exchange within a column to the k next neighbours, i.e., i can be exchanged only with i−k,...,i+k mod q. This restriction was already used in [WF98]. For the purposes of this paper we tested yet another modification, which uses much larger neighbourhoods and is introduced as a ruin-and-recreate approach by [SSSD99]. Instead of exchanging a small number of elements, a complete submatrix of the current design is first eliminated and then reconstructed. The reconstruc- tion step consists in testing all possible permutations of the deleted elements in one column and selecting the permutation which results in the smallest discrepancy together with the unchanged part of the design and the already reconstructed columns. There are still some open questions of how to integrate ruin-and-recreate steps in the most efficient way in the refined local search strategy of threshold accepting. Hence, it is not too surprising that at the current stage this modification rarely outperforms our standard threshold accepting method for the instances analyzed in this section. However, for larger instances, it may exhibit significant improvements which will be assessed in further research.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 287

Given the high complexity of ﬁnding uniform designs, in general, it cannot be guaranteed that some threshold accepting implementation will ﬁnd a uniform design using a reasonable amount of computing resources, although an asymptotic convergence property was proven by [AK91]. Thus, our standard proceeding is to use a limited number of iterations of the algorithm and to assume that the result- ing design is a low-discrepancy, if not uniform design. In fact, for the case s =2, where uniform designs under the discrepancy have been obtained by [LF95], our implementation could reproduce these results. The results on the theoretic expectation and variance of CL2 for LHD provided in the previous sections allow for an alternative approach. Instead of searching low-discrepancy designs on an absolute scale, we can do so relative to the theoretic expectation and variance. Then, the threshold accepting implementation is run until a design is obtained which has CL2 lower than the expected value for LHD or lower than the expected value forp LHD minus some multiple c of its standard 2 2 deviation, i.e., E(CL2(Ln,qs ) ) − c Var(CL2(Ln,qs ) ). The results presented in the next section indicate that c can be chosen from 1 to about 8 as s increases. Finally, we can contrast the designs optimized with regard to CL2 with those obtained by [WF98] for the discrepancy.

5. Remarks on new uniform designs Using the methods described in the previous section, we have obtained a number of low-discrepancy U-type designs under CL2. Although the modiﬁed threshold accepting algorithm is powerful, there can be no guarantee that these designs have minimum CL2 in the class Un,qs or Un,s. Therefore, we have to check whether these designs have low CL2-value from several angles.

−1 10 s=2 −1 s=3 0.2 s=4 0.2 s=5 10 0.1 0.1 −2 10 −2 10 −3 0.01 10 0.01

5 10 20 30 5 10 20 30 5 10 20 30 10 15 20 2530 0.3 0.3 0.4 0.5 0.2 s=6 s=7 0.3 s=8 s=9 0.2 0.3 0.1 0.2 0.1 0.1 0.1

0.01 10 15 20 2530 10 15 20 2530 10 15 20 25 30 10 15 20 25 30 0.6 s=10 0.6 s=11 0.8 s=12 0.9 s=13 0.4 0.6 0.4 0.7 0.2 0.4 0.5 0.2 0.2 0.3

15 20 25 30 15 20 25 30 15 20 25 30 15 20 25 30

Figure 3. The comparisons among uniform designs by optimization (denoted by point), random designs (by star) and random U-type designs (by x-mark).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 288 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

0.01 s=2 0.025 s=3 0.06 s=4 s=5 0.008 0.02 0.06 0.04 0.015 0.006 0.04 0.01 0.004 0.02 0.002 0.005 0.02

10 20 30 10 20 30 10 20 30 10 20 30

Figure 4. Centered L2 for designs obtained by optimization (circles) and designs from [WF98] (stars)

1.6 s=2 s=3 s=4 s=5 2.6 3.5 4.5 1.5 2.4 4 2.2 3 1.4 2 3.5 1.8 2.5 1.3 3 1.6 10 20 30 10 20 30 10 20 30 10 20 30

5.5 s=6 s=7 7 s=8 s=9 6 5 7 6 4.5 5 6 4 5 5 3.5 4 4 10 20 30 10 20 30 10 20 30 10 20 30

8 s=10 s=11 s=12 s=13 8 8 8 7 7 7 7 6 6 6 6 5 5 5 5 15 20 25 30 15 20 25 30 15 20 25 30 15 20 25 30

Figure 5. The plots of c against n for s = 2–13

2 Firstly, Figure 3 gives plots of CL2(P) against the number of runs n on a double logarithmic scale. The values for the designs obtained by explicit optimization using threshold accepting are marked by a “point”; a “star” denotes the expectation for simple random designs, and an “x-mark” the expectation for Latin hypercube designs. The plots of each design for given s are close to linear lines. The slope −β of the lines, denoted by −β, stands for the convergence rate n of E(CL2(P)). From Theorem 2.1, the slope of the lines for simple random designs and for Latin hypercube designs should be close to −0.5. The slope of the lines for the optimized U-type designs, which serve as upper bound for the uniform designs, can be esti- mated by least squares. They vary between −0.9563 and −0.6436 for 2 ≤ s ≤ 13. The slope increases as the dimension s increases. Secondly, when s = 2, [LF95] proposed an algorithm by which one can ﬁnd uniform designs with minimum discrepancy. These designs should also be good designs in the sense of low-CL2. Figure 4 plots the square CL2 against n for these designs (stars) and the designs obtained in this paper (circles). It shows that

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 289

1 −0.0225 0.00912 −0.0113 −0.0719 −0.0251 0.09722 0.01179 −0.0718 0.01935 x 0.58867 0.50085 0.46998 0.60809 0.52916 0.5443 0.54587 0.55714 0.52225 10 0 1 −0.0131 0.01357 −0.0598 0.01402 0.03181 0.06563 0.00334 0.00467 x 0.56546 0.51179 0.5276 0.48825 0.57566 0.49048 0.47583 0.4592 9 0 1 −0.0340 0.01313 −0.0069 −0.0167 −0.0269 −0.01401 0.04249 x 0.54817 0.55262 0.48216 0.52356 0.5899 0.55344 0.52085 8 0 1 0.02425 −0.0211 −0.0398 −0.0255 0.02513 −0.0126 x 0.53476 0.45624 0.52076 0.56488 0.55015 0.55533 7 0 1 0.01179 −0.0020 −0.0073 0.02202 −0.0362 x 0.5215 0.50562 0.55294 0.47163 0.53739 6 0 1 −0.0011 −0.0086 0.04694 0.02425 x 0.52348 0.46043 0.56768 0.63393 5 0 1 −0.0451 0.02335 0.02692 x 0.49081 0.57187 0.51434 4 0 1 0.01490 0.01891 x 0.54455 0.5378 3 0 1 −0.0611 x 0.50134 2 0 1 x 1 0

Figure 6. Uniform design

the new designs have lower CL2 than the designs obtained by [LF95]. The same comparison can be performed with regard to the low-discrepancy designs obtained by [WF98], which coincide with the designs of [LF95] in dimension s =2forn ≤ 23. The results of the comparison are also shown in Figure 4. How can we assess the new uniform designs for s ≥ 2tohavealowCL2?Let 2 ∗ 2 E(CL2(Ln,s)) − (CL2(P )) c = p n,s , 2 Var((CL2(Ln,s)) ) P∗ where n,s is the new design for given n and s. Figure 5 provides plots of c against n for s =2tos =13andn = s + 1 to 30. It shows that c increases from 1.25 to 8.75 when n or/and s increase. We ﬁnd that the distribution of square CL2 of Ln,s can be approximated by a rescaled beta distribution for given n and s. Then, the results on c in Figure 5 show that the new designs have, in fact, a signiﬁcantly lower CL2 than the expectation for LHD.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 290 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

1 0.40189 0.19249 0.026295 0.098674 −0.28834 0.14138 −0.33774 −0.0683 0.11287 x 15.4493 15.8948 29.1739 16.8983 17.7036 33.2247 23.7278 14.5088 17.9446 10 0 1 0.1192 0.1922 0.21919 0.31093 0.15158 0.18073 0.17581 0.02099 x 7.5061 8.3247 21.837 10.9926 10.3685 27.528 15.8492 8.0915 9 0 1 −0.0607 −0.0215 −0.1690 −0.3079 0.0316 −0.1954 −0.1312 x 4.6636 5.3791 19.5054 8.2723 6.9128 24.5233 11.897 8 0 1 −0.0829 0.2241 0.06095 0.03106 0.09249 0.14125 x 13.2187 12.3202 25.707 13.7832 14.3659 31.6451 7 0 1 −0.0691 0.29238 −0.1223 0.27174 −0.0142 x 24.741 24.0437 38.4832 27.1914 26.2416 6 0 1 0.18759 −0.21465 −0.21217 0.17245 x 7.7445 7.552 20.9431 9.6373 5 0 1 0.06903 −0.0426 0.03716 x 6.8047 8.0301 21.1842 4 0 1 −0.10527 0.27649 x 18.8101 19.2413 3 0 1 −0.25611 x 5.7675 2 0 1 x 1 0

Figure 7. Random design

Thirdly, let us look at the projection properties of our new designs. For a uni- s form design Un(q ), we wish that it has good uniformity in all subdimensions, in particular, in all marginal two-dimensional subspaces. For illustration purposes, 10 we choose a new design U30(30 ) obtained by threshold accepting. Figure 6 gives plots for all marginal two-dimensional subspaces. For each marginal case we list its 2 ∗ correlation coeﬃcient and CL2 1000. The former has been paid much attention by many authors including, e.g., [Owen94]. Similar plots for a simple random design and a Latin hypercube design are given in Figures 7 and 8, respectively. From these plots, the CL2 values and the correlation coeﬃcients, it becomes clear that the new uniform or low-discrepancy design also has better properties in projection uniformity.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 291

1 −0.26051 0.076307 0.020245 −0.24138 0.10834 0.05762 −0.14972 −0.42336 −0.0665 x 1.3022 0.68513 0.78315 1.2785 0.67179 1.2313 0.75311 2.2448 0.7969 10 0 1 0.28454 0.08921 0.073192 −0.0149 −0.16885 0.15818 −0.30857 0.084761 x 1.475 0.87303 0.91607 0.97352 1.7946 1.241 1.5815 1.2019 9 0 1 0.21913 −0.31791 −0.09277 0.25473 −0.19377 −0.13593 0.18265 x 0.98924 1.4683 1.1008 1.0878 0.90356 1.1241 1.0058 8 0 1 0.00200 −0.0033 −0.12792 0.010456 −0.17731 −0.196 x 0.74397 0.67393 1.036 0.82891 1.0406 1.6191 7 0 1 −0.11057 0.25517 −0.0407 −0.1119 0.20044 x 1.2538 1.3427 0.74653 0.88546 0.942 6 0 1 −0.28676 0.1733 0.11012 −0.0069 x 1.4771 0.9922 0.95385 0.90521 5 0 1 −0.08743 −0.13593 −0.0705 x 1.0109 1.0019 0.92076 4 0 1 −0.0963 0.063404 x 1.1241 1.1079 3 0 1 −0.0443 x 0.78801 2 0 1 x 1 0

Figure 8. A Latin hypercube design randomly chosen

6. Conclusions In this paper we derived some theoretical results on the expectation and variance of random designs and Latin hypercube designs under the centered L2-discrepancy (CL2). It turned out that Latin hypercube designs have a lower expected discrepancy and variance compared to simple random designs. Using these theoretical findings, we proposed an algorithm for the explicit construction of low-discrepancy designs based on a modified version of the global search heuristic threshold accepting. The results indicate that this approach has the potential to generate designs with significantly lower discrepancy than the expectation for Latin hypercube designs. Further research will aim at extending the numerical results for designs with the number of levels q being smaller than the number of runs n. Also a more detailed analysis of the threshold accepting implementation, in particular with regard to

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 292 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

the recent ruin-and-recreate feature may allow for the extension of the approach to larger problem instances.

Appendix Proof of Theorem 3.1. We prove two cases by the same procedure, but diﬀerent s arguments. For any set P of n points on C the three terms of its CL2 in (1.6) are denoted by a, b,andc, respectively. Then,

2 2 2 2 2 2 E(CL2(P) ) =2aE(CL2(P)) − a + Eb +2Ebc + Ec

and

2 2 2 2 2 Var(CL2(P) )=E(CL2(P) ) − (E(CL2(P) )) .

Let x, y, z, u be i.i.d. uniformly distributed on [0, 1), and let

1 1 f (x)=1+|x − 0.5|− |x − 0.5|2, 1 2 2 1 2i − 1 − q 1 2i − 1 − q g (i)=1+| |− | |2, 1 2 2q 2 2q

f2(x)=1+|x − 0.5|, 2i − 1 − q g (i)=1+| |, 2 2q 1 1 1 f (x, y)=1+|x − 0.5| + |y − 0.5|− |x − y|, 3 2 2 2 1 2i − 1 − q 1 2j − 1 − q i − j g (i, j)=1+| | + | |−| |, 3 2 2q 2 2q q i, j =1,... ,q.

In the following statement, we use SRD= for random designs and LHD=forLatin hypecube designs Ln,qs .

n2 Xn Xn Ys E(b2)=E( f (x )f (x )) 4 1 kj 1 lj k=1 l=1 j=1 Xn Ys Xn Xi−1 Ys 2 = Ef1(xkj ) +2 Ef1(xkj )f1(xlj ) k=1 j=1 k=1 l=1 j=1 Xn Xn Xi−1 SRD 2 s s = (Ef1(x) ) +2 (Ef1(x)f1(y)) k=1 k=1 l=1 q q q 1 X X X LHD= n( g (k)2)s + n(n − 1)( g (i)g (j)P (i, j))s, q 1 1 1 k=1 i=1 j=1

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 293

and

  n3 Xn Xn Xn Ys − E(bc)=E  f (x )f (x ,x ) 2 1 kj 3 lj ij k=1 l=1 i=1 j=1 Xn Ys = E(f1(xkj )f3(xkj ,xkj )) k=1 j=1 Xn Ys +2 E(f1(xkj )f3(xkj ,xij )) k=6 i j=1 Xn Ys + E(f1(xkj )f3(xij ,xij )) k=6 i j=1 Xn Ys + E(f1(xkj )f3(xlj ,xij )) k=6 i=6 l=6 k j=1 Xn Xn SRD s s = (E(f1(x)f3(x, x))) +2 (E(f1(x)f3(x, y))) k=1 k=6 i Xn Xn s s + (E(f1(x)f3(y,y))) + (E(f1(x)f3(y,z))) 6 6 6 6 k=i ! k=i=l=k q s 1 X LHD= n g (k)g (k) q 1 2 k=1   s Xq Xq   +2n(n − 1) g1(i)g3(i, j)P (i, j) i=1 j=1  s Xq Xq   +n(n − 1) g1(i)g2(j)P (i, j) i=1 j=1  s Xq Xq Xq   +n(n − 1)(n − 2) g1(i)g3(j, l)P (i, j, l) . i=1 j=1 l=1

Similarly, we have

4 2 SRD 2 s s n Ec = n(Ef2(x) ) +4n(n − 1)(Ef3(x, y)f2(x))

s 2 s +n(n − 1)(Ef2(x)f2(y)) +2n(n − 1)(Ef3(x, y) )

s +2n(n − 1)(n − 2)(Ef2(x)f3(y,z))

s +4n(n − 1)(n − 2)(Ef3(x, z)f3(y,z))

s +n(n − 1)(n − 2)(n − 3)(Ef3(x, y)f3(z,u))

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 294 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER ! q s 1 X LHD= n g (k)2 q 2 k=1  s Xq Xq   +4n(n − 1) g3(k, j)g2(k)P (k, j) k=1 j=1  s Xq Xq   +n(n − 1) g2(k)g2(j)P (k, j) k=1 j=1   s Xq Xq  2  +2n(n − 1) g3(k, j) P (k, j) k=1 j=1  s Xq Xq Xq   +2n(n − 1)(n − 2) g2(k)g3(j, l)P (k, j, l) k=1 j=1 l=1  s Xq Xq Xq   +4n(n − 1)(n − 2) g3(k, l)g3(j, l)P (k, j, l) k=1 j=1 l=1 +n(n − 1)(n − 2)(n − 3)   s Xq Xq Xq Xq   × g3(k, j)g3(l, m)P (k, j, l, m) , k=1 j=1 l=1 m=1

where P (k, j) is defined in the proof of Theorem 2.1 and P (k, j, l) is the probability that the first three elements of a specific column of the design matrix are k, j,and l, respectively. The notation P (k, j, l, m) can be similarly defined. It is easy to find that   3  r if i, j, andk are not equal to each other,  n(n−1)(n−2) P (i, j, k)= r2(r−1) 6 6 6  n(n−1)(n−2) if i = j = k, or i = k = j, or j = k = i,   r(r−1)(r−2) n(n−1)(n−2) if i = j = k

and

P (i, j, k, l)   r4  − − − if i, j, k, and l are not equal to each other,  n(n 1)(n 2)(n 3)  3 −  r (r 1) if only two of i, j, k, and l are equal to each other,  n(n−1)(n−2)(n−3) = r2(r−1)2 6 6 6  n(n−1)(n−2)(n−3) if i = j = k = l;ori = k = j = l;ori = l = j = k,   2  r (r−1)(n−2)  n(n−1)(n−2)(n−3) if only three of i, j, k, and l are equal to each other,   r(r−1)(r−2)(r−3) n(n−1)(n−2) if i = j = k = l. The theorem is proved by some straightforward but complicated calculations.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE CL2 OF RANDOM SAMPLING AND LATIN HYPERCUBE DESIGN 295

Acknowledgments We thank the Associate Editor and the referee for their valuable comments.

References

[AK91] I. Alth¨ofer and K.-U. Koschnick, On the convergence of “Threshold Accepting”, Applied Mathematics and Optimization 24 (1991), 183-195. MR 92i:90081 [CL95] C. S. Cheng and K. C. Li, A study of the method of principal Hessian direction for analysis of data from design experiments, Statistics Sinica 5 (1995), 617-639. MR 96f:62129 [FH95] K. T. Fang and F. J. Hickernell, The uniform design and its applications, Bull. Inst. Internat. Statist., 50th Session, Book 1 (1995), 333-349. [FM00] K. T. Fang and R. Mukerjee, Connection between uniformity and aberration in regu- lar fractions of two-level factorials, Technical Report MATH–216, Hong Kong Baptist University; Biometrika 87 (2000), 193–198. [FWa94] K. T. Fang, and Y. Wang, Number Theoretic Methods in Statistics, Chapman and Hall, London, 1994. MR 95g:65189 [FW98] K. T. Fang and P. Winker, Uniformity and orthogonality, Technical Report MATH–176, Hong Kong Baptist University, 1998. [Hic98] F. J. Hickernell, A generalized discrepancy and quadrature error bound,Math.Comp. 67 (1998), 299–322. MR 98c:65032 [KO96] J. R. Koehler and A. B. Owen, Computer experiments, Handbook of Statistics, Vol. 13, S. Ghosh and C. R. Rao eds., Elsevier Science B. V., 1996, 261-308. [LF95] W. Li and K. T. Fang, A global optimum algorithm on two factor uniform design, Workshop on quasi-Monte Carlo methods and their applications (1995), 147-159. [MF98] C. X. Ma and K. T. Fang, Applications of uniformity to orthogonal fractional factorial designs, Technical Report MATH–193, Hong Kong Baptist University, 1998. [MBC79]M.D.McKay,W.J.ConoverandR.J.Beckman,A comparison of three methods for selecting values of input variables in the analysis of output from a computer code, Technometrics 21 (1979), 239-245. MR 80k:65012 [Nie92] H. Niederreiter, Random number generation and quasi–Monte Carlo methods. SIAM CBMS–NSF Regional Conference Series in Applied Mathematics, Philadelphia, 1992. MR 93h:65008 [Owen92] A. B. Owen, A central limit theorem for Latin hypercube sampling, Journal of the Royal Statistical Society Ser. B 54 (1992), 541-551. MR 93j:62047 [Owen94] A. B. Owen, Controlling correlations in Latin hypercube samples,J.Amer.Statist. Assoc. 89 (1994), 1517-1522. [Owen95] A. B. Owen, Randomly permuted (t, m, s)-nets and (t, s)-sequences, Niederreiter, Har- ald (ed.) et al., Monte Carlo and quasi-Monte Carlo methods in scientiﬁc computing. Berlin: Springer-Verlag. Lect. Notes Stat., Springer-Verlag. 106 (1995), 299-317. MR 97k:65013 [SSSD99] G. Schrimpf, J. Schneider, H. Stamm-Wilbrandt and G. Dueck, Record breaking optimization results using the ruin & recreate principle, submitted to Journal of Compu- tational Physics, 1999. [Tang93] B. Tang, Orthogonal array-based Latin hypercube, J. Amer. Statist. Assoc. 88 (1993), 1392-1397. MR 94e:62075 [WF97] P. Winker and K. T. Fang, Application of Threshold Accepting to the Evaluation of the Discrepancy of a Set of Points, SIAM Journal on Numerical Analysis 34 (1997), 2028–2042. MR 98i:65015 [WF98] P. Winker and K. T. Fang, Optimal U–Type Designs, Niederreiter, Harald (ed.) et al., Monte Carlo and quasi-Monte Carlo methods 1996. Berlin: Springer-Verlag. Lect. Notes Stat., Springer-Verlag. 127 (1998), 436–448.

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 296 KAI-TAI FANG, CHANG-XING MA, AND PETER WINKER

Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong; and Chinese Academy of Sciences, Beijing, China E-mail address: [email protected] Department of Statistics, Nankai University, Tianjin, China E-mail address: [email protected] Department of Economics, University of Mannheim, 68131 Mannheim, Germany E-mail address: [email protected]

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use