中国科技论文在线 http://www.paper.edu.cn

Vol. 45 No. 8 SCIENCE IN CHINA (Series A) August 2002

Uniform supersaturated design and its construction

¢¡¢£ 1 2 3 ¨¢© ( ) , GE Gennian ( ¤¦¥¢§ ) & LIU Minqian ( ) 1. Department of , Hong Kong Baptist University, Hong Kong, China; 2. Department of Mathematics, Suzhou University, Suzhou 215006, China; 3. Department of , Nankai University, Tianjin 300071, China Correspondence should be addressed to Fang Kaitai (email: [email protected])

Received February 2, 2002

Abstract Supersaturated designs are factorial designs in which the number of main effects is greater than the number of experimental runs. In this paper, a discrete discrepancy is proposed as a measure of uniformity for supersaturated designs, and a lower bound of this discrepancy is obtained as a benchmark of design uniformity. A construction method for uniform supersaturated designs via resolvable balanced incomplete block designs is also presented along with the investigation of properties of the resulting designs. The construction method shows a strong link between these two different kinds of designs.

Keywords: discrepancy, resolvable balanced incomplete block design, supersaturated design, uni- formity.

Recently, supersaturated designs have aroused increasing interest, as their potential in saving run size and the technical novelty have begun to be realized. A supersaturated design is a factorial design whose run size is not large enough for estimating all the main effects represented by the columns of the design matrix. If many factors are to be investigated (e.g. in a screening study) and runs are very expensive, economic considerations may compel the investigators to adopt a supersaturated design. For example, a physical experiment may require the making of expensive prototypes; a computer experiment using finite elements analysis can be time-consuming and expensive. In practice, the data collected by supersaturated designs are analyzed under the assumption of effect sparsity, i.e. the response of interest depends mainly on the effects of a few dominant factors, and the interactions and the effects of the remaining factors are relatively negligible. Various fields of research may benefit from the use of supersaturated designs, including computer and medical experiments[1], industrial and engineering experiments[2,3], and so on. The problem of constructing supersaturated designs was originally considered by Satterthwa- ite[4]. Booth and Cox[5] first examined these designs systematically. Since then, many sesearchers[1−3,6−14] have investigated into two-level supersaturated designs. These two-level de- signs can be used for screening the factors in simple linear models. However, designs with multi- levels are often requested in many situations in exploring nonlinear effects of the factors. Such works include refs. [15—17] for three-level supersaturated designs, and refs. [18, 19] for multi-level supersaturated designs. A common characteristic of the existing q-level supersaturated designs (q > 3) except those of Yamada and Lin[15] is that these designs are constructed from given orthogonal designs through

转载 中国科技论文在线 http://www.paper.edu.cn

No. 8 UNIFORM SUPERSTRUCTURED DESIGNS 1081

computer search, while those of Yamada and Lin[15] are generated from two-level orthogonal or supersaturated designs. All existing supersaturated designs are evaluated based on some measures such as E(s2), aveχ2 and E(d2). In fact, supersaturated design is a kind of U-type design[20,21]. An important criterion for evaluating U-type designs is the uniformity criterion, which has gained popularity in recent years and has been shown to be intimately connected to many other design criteria. For example, Fang and Mukerjee[22] provided an analytic connection between uniformity and orthogonality in two-level factorial designs. When the experimental domain is a unit hyper-

cube, many measures of uniformity, such as the star discrepancy, the centered L2-discrepancy and [23−25] the wrap-around L2-discrepancy have been proposed . The main purpose of this paper is to provide a class of multi-level supersaturated designs from the uniformity viewpoint. In sec. 1, the discrepancy measure of uniformity is introduced, a discrete discrepancy for measuring the uniformity of factorial designs is defined by using a reproducing kernel in Hilbert space, and a lower bound of the discrete discrepancy and a sufficient and necessary condition for achieving it are also obtained. This lower bound can be used as a benchmark of design uniformity. Some justification for using the discrete discrepancy as a design criterion is also provided in sec. 1. In sec. 2, a new construction method for multi-level supersaturated designs via resolvable balanced incomplete block designs (RBIBDs) is proposed. The RBIBD is an important object both in experimental design theory and combinatorial design theory. It has many good statistical properties as well as combinatorial properties and has played a crucial role in the construction of other combinatorial configurations[26]. The resulting designs are shown to be uniform supersaturated designs. Thus the construction method serves as an important bridge between these two different kinds of designs, i.e. uniform supersaturated designs and RBIBDs. With this method, some infinite classes for the existence of uniform supersaturated designs are also obtained in this section without any computer search. 1 The discrepancy measure

First let us introduce some knowledge related to supersaturated designs. A U-type symmetric design (a U-type design for simplicity) X is an n × m matrix with symbols {1, ··· , q} such that the q symbols in each column appear equally often. A U-type design can be regarded as a design with n runs and m factors, each having q levels. Obviously, the number of levels q should be a divisor of n. The set of all such designs is denoted by U(n; qm)[27]. A U-type design is called an orthogonal design if every pair of design columns with all of its level-combinations appears equally often. In this case, m(q − 1) 6 n − 1. When m(q − 1) = n − 1, the design is saturated and when m(q − 1) > n − 1, the design has no enough degrees of freedom for estimating all the factorial effects simultaneously, the design is called a supersaturated design, denoted by S(n; qm).

1.1 Definition of the discrepancy Note that a U-type design may not be a good design. An important and popular measure of uniformity of U-type designs is the discrepancy. The discrepancy can be defined in terms of a kernel function. Let X be a measurable subset of Rm. A kernel function K(x, w) is a real-valued 中国科技论文在线 http://www.paper.edu.cn

1082 SCIENCE IN CHINA (Series A) Vol. 45

function defined on X × X and is symmetric in its arguments and non-negative definite, K(x, w) = K(w, x), for any x, w ∈ X , (1) n aiaj K(xi, xj ) > 0, for any ai ∈ R, xi ∈ X , i = 1, ··· , n. (2) i,j=1 X In fact K(x, w) is the reproducing kernel for some unique Hilbert space of real-valued functions on X , but that property is not needed here. For a more detailed discussion of reproducing kernels, see refs. [28, 29].

Let F∗ denote the uniform distribution function over X . Let P = {z1, ··· , zn} ⊆ X be a set

of design points and Fn denote the associated empirical distribution, where 1 F (x) = 1 z x . n n { 6 } z P X∈ 1A is the indicator function of A, and z = (z1, ··· , zm) 6 x = (x1, ··· , xm) means that zj 6 xj for all j. For a given kernel function K(x, w), the discrepancy of P is defined by[30] 1 x w x w w w 2 D(P ; K) = X 2 K( , )d[F∗( ) − Fn( )]d[F∗( ) − Fn( )] x w x w 2 x z x = RX 2 K( , )dF∗( )dF∗( ) − n z∈P X K( , )dF∗( ) 1 1 z z0 2 R+ n2 z,z0∈P K( , ) . P R (3) From this definition, it is clearP that the discrepancyo measures far away from the empirical distri- bution Fn is from F∗. From a uniformity point of view, for a fixed number of points, n, a design with low discrepancy is preferable[31]. For any q-level design X ∈ U(n; qm), X = {1, ··· , q}m comprising all possible level combina- −m tions of the m factors, F∗ just assigns probability q to each member of X . Let a if x = w, K˜ (x, w) = for x, w ∈ {1, ··· , q}, a > b > 0, (4) ( b if x 6= w, m Kd(x, w) = K˜ (xj , wj ), for any x, w ∈ X . (5) j=1 Y And then Kd(x, w) is a kernel function satisfying conditions (1) and (2). The corresponding discrete discrepancy can be used for measuring the uniformity of factorial design points. Note that the existing discrepancies mentioned above are defined on a unit hypercube and are used for measuring the uniformity of points corresponding to continuous variables, while for a factorial design, the factorial levels are not continuous; they are discrete, and a q-level factor needs q − 1 degrees of freedom in the estimation of its main effects, not one degree of freedom such as for a continuous variable. The discrete discrepancy defined here is partly from this consideration.

Given the kernel Kd(x, w) defined by (4) and (5), from (3) the discrete discrepancy D(P ; Kd) can be computed as follows: n m a + (q − 1)b m 1 D2(P ; K ) = − + K˜ (z , z ), (6) d q n2 kj lj k,l j=1   X=1 Y where zk = (zk1, ··· , zkm). 中国科技论文在线 http://www.paper.edu.cn

No. 8 UNIFORM SUPERSTRUCTURED DESIGNS 1083

2 1.2 The lower bound of D (P ; Kd) m For any X ∈ U(n; q ), let λkl be the number of coincidences between the kth and lth rows x x m k and l, i. e. λkl = j=1 1{xkj =xlj }. It is obvious that λkk = m. The following analytical expression and lower boundP of the corresponding discrete discrepancy can be easily derived. Theorem 1. For any design X ∈ U(n; qm), m m m a + (q − 1)b a b a λkl D2(X; K ) = − + + , (7) d q n n2 b 6k l6n   1 X6=   a + (q − 1)b m am n − 1 a λ > − + + bm , (8) q n n b     and the lower bound on the right-hand side of (8) can be achieved if and only if λ = m(n/q −

1)/(n − 1) is a positive integer and all the λkl’s for k 6= l are equal to λ. Proof. Formula (7) just comes from (6) and m Kd(xk, xl) = K˜ (xkj , xlj ) = K˜ (xkj , xlj ) K˜ (xkj , xlj ) j=1 x =x x x Y kjY lj kjY6= lj a λkl =aλkl bm−λkl = bm , b where xk and xl are two runs and λkk = m.   For a U-type design, it is obvious that n λkl = m(n/q − 1), k = 1, ··· , n. (9) l ,l k =1X6= Then from (7) and the well-known arithmetic-geometric means inequality, inequality (8) holds 2 under constraint (9), and because λkl’s are integers, the lower bound of D (X; Kd) on the right-

hand side of (8) can be achieved if and only if λ is a positive integer and all the λkl’s for k 6= l are equal to λ. 2 Based on this theorem, a U-type design is said to be uniform if its D (X; Kd) value achieves its lower bound in (8), which can be used as a benchmark of design uniformity. In most cases, uniform U-type designs are supersaturated. So this kind of U-type designs are also called the uniform supersaturated designs. Such a design requires its design points to be uniformly scattered on the experimental domain X . The uniform design possesses several advantages. It can explore relationships between the response and the factors with a reasonable number of runs and is shown to be robust to the underlying model specification. For a comprehensive review of uniform design one can refer to Fang et al.[27] and Fang and Wang [31]. 2 Before ending this section, let us provide some justification for using D (X; Kd) as a criterion m for supersaturated designs. For an S(n; 2 ) design X with two levels 1 and −1, let sij be the (i, j)-element of X0X. The popular E(s2) criterion (Booth and Cox[5]) is to minimize 2 2 E(s ) = sij /[m(m − 1)/2]. 6i

1084 SCIENCE IN CHINA (Series A) Vol. 45

(ij) 2 [15] where nuv is the number of (u, v)-pairs in the ith and jth columns of X. The aveχ criterion is to minimize 2 2 aveχ = χij /[m(m − 1)/2]. 6i

2 2 Hence from (10) and (11), we will only show the relation between D (X; Kd) and E(d ) in the following. Note that for an S(n; qm), where n is a multiple of q2, Lu and Sun[19] also obtained a lower bound of E(d2), n2(q − 1) m(q − 1) E(d2) > − 1 , (12) q2(m − 1) n − 1   and showed a necessary condition to achieve the lower bound: λ = m(n/q −1)/(n−1) is a positive 2 2 integer. Thus for any such design X, E(d ) shares the same necessary condition as D (X; Kd) to achieve their lower bounds respectively. In fact, it is easy to show that the sufficient and necessary 2 2 condition in Theorem 1 for D (X; Kd) to achieve its lower bound is also true for E(d ) to achieve its lower bound, i.e. m 2 Corollary 1. Let X ∈ U(n; q ). Then D (X; Kd) achieves its lower bound in (8) if and only if E(d2) achieves its lower bound in (12). 2 These discussions provide a strong justification for using D (X; Kd) as a design criterion for supersaturated designs, which measures the optimality of such designs from the uniformity point of view. 2 Design construction

In this section we will present a method of constructing uniform multi-level supersaturated designs from resolvable balanced incomplete block designs and study the properties of the result- ing designs.

2.1 The construction method Let us recall some knowledge of resolvable balanced incomplete block designs. A balanced incomplete block design (BIBD) with parameters (n, s, m, t, λ), denoted by BIBD(n, s, m, t, λ), is an arrangement of n treatments into s blocks of size t, where t < n, such that each treatment appears in m blocks, and every pair of treatments appears in exactly λ blocks. The five parameters must satisfy the following relations: nm = st and λ(n − 1) = m(t − 1). (13) Hence there are only three independent parameters in the definition. A BIBD(n, s, m, t, λ) is said to be resolvable, denoted by RBIBD(n, s, m, t, λ), if its blocks can be partitioned into m sets of 中国科技论文在线 http://www.paper.edu.cn

No. 8 UNIFORM SUPERSTRUCTURED DESIGNS 1085

blocks, called parallel classes, such that every treatment appears in each parallel class precisely once. The RBIBD is an important kind of block designs. And there is a comprehensive study on the existence and construction of RBIBDs in literature. For example, readers can refer to refs. [26, 32, 33] for such results. Let X ∈ U(n; qm). For any column xj of X, there exist q blocks of size n/q that partition {1, 2, ··· , n}, such that if the kth element of xj takes level u, then k is contained in the uth block. Obviously, these q blocks form a parallel class. Take n = 12, q = 4, and a column xj = (1 2 3 4 2 3 4 1 1 3 2 4)0 as an example. This column corresponds to four blocks of size 3, {1, 8, 9}, {2, 5, 11}, {3, 6, 10}, and {4, 7, 12}, which form a parallel class. Corresponding to the m columns of X, there are altogether mq blocks of size n/q. Let B be a collection of these mq blocks of size n/q. When the number of coincidences between any pair of distinct rows of X is a constant λ = m(n/q − 1)/(n − 1), then for any k and l, 1 6 k, l 6 n, the pair {k, l} appears in exactly λ blocks of B. Hence it is easy to see B is an RBIBD(n, mq, m, n/q, λ). On the contrary, given an RBIBD(n, mq, m, n/q, λ), where the n treatments are denoted by

1, ··· , n, and the m parallel classes are denoted by P1, ··· , Pm, each of which consists of q disjoint blocks, the construction method can be carried out as follows:

Step 1. Give a natural order 1, ··· , q to the q blocks in each parallel class Pj , j = 1, ··· , m. j Step 2. For each Pj, construct a q-level column x = (xkj ) as follows:

Set xkj = u, if treatment k is contained in the u-th block of Pj, u = 1, 2, ··· , q. m For this RBIBD, the m q-level columns constructed from Pj (j = 1, ··· , m) form an S(n; q ). We call this method the RBIBD method. Take the RBIBD(10, 45, 9, 2, 1) in table 1 as an example. We have 9 parallel classes (denoted

by P1, ··· , P9) in this design. Note that the block {1, 10} appears as the “first” one in P1. So we put “1” in the cells located in rows 1 and 10 of column 1 in table 2. Similarly, we put “2” in the cells located in rows 8 and 9 of that column, and so on. Thus we obtain a 5-level column, i. e. column 1 in table 2. In this way, 9 columns are then constructed from these 9 parallel classes, which form a supersaturated design S(10; 59) as shown in table 2.

Table 1 An RBIBD (10, 45, 9, 2, 1)

P1 P2 P3 P4 P5 P6 P7 P8 P9 j b1 {1,10}{2,10}{4,9}{3,7}{2,8}{5,7}{5,6}{1,7}{1,6} j b2 {8,9}{5,8}{3,10}{4,10}{6,9}{2,4}{3,4}{2,5}{2,7} j b3 {4,5}{3,6}{7,8}{1,2}{5,10}{1,9}{1,8}{4,6}{4,8} j b4 {6,7}{7,9}{2,6}{5,9}{1,3}{3,8}{7,10}{3,9}{3,5} j b5 {2,3}{1,4}{1,5}{6,8}{4,7}{6,10}{2,9}{8,10}{9,10} bj i j i , th block in the th parallel class.

2.2 Properties As we know, there are very rich results on RBIBDs in literature. If there exists an RBIBD (n, mq, m, n/q, λ), a q-level design S(n; qm) can be constructed by the RBIBD method. This design has the following properties. 中国科技论文在线 http://www.paper.edu.cn

1086 SCIENCE IN CHINA (Series A) Vol. 45

Table 2 S(10; 59) derived from table 1

Row 1 2 3 4 5 6 7 8 9 1 1 5 5 3 4 3 3 1 1 2 5 1 4 3 1 2 5 2 2 3 5 3 2 1 4 4 2 4 4 4 3 5 1 2 5 2 2 3 3 5 3 2 5 4 3 1 1 2 4 6 4 3 4 5 2 5 1 3 1 7 4 4 3 1 5 1 4 1 2 8 2 2 3 5 1 4 3 5 3 9 2 4 1 4 2 3 5 4 5 10 1 1 2 2 3 5 4 5 5

Theorem 2. Given an RBIBD(n, mq, m, n/q, λ), the S(n; qm) obtained by the RBIBD

method is a uniform design in the sense of D(X; Kd). And if λ = 1, then any of the possible level-combinations between any two columns appears at most once. Proof. Let X be the design obtained by the RBIBD method. It is easy to see that each element appears in any column of X exactly n/q times. Hence, X is a U-type design. For the BRIBD(n, mq, m, n/q, λ), from (13) we have λ = m(n/q − 1)/(n − 1). Now, we need to show

that the number of coincidences between any two distinct rows xk and xl of X is λ. In fact, the

elements in rows xk and xl of X are coincident in a certain position if and only if in the RBIBD there exists a block containing both k and l. Since the pair {k, l} occurs in exactly λ blocks, the uniformity of the resulting design then follows from Theorem 1. If λ = 1, then any of the possible level-combinations between any two columns appears at most once; otherwise if a certain level-combination appears twice (e.g. in the kth and lth runs) in some two columns, then from the construction method, there exist a pair (e.g. {k, l}) which occurs in two blocks, a contradiction. Base on this theorem, we can use an RBIBD to construct a uniform supersaturated design with equal-level factors. From the rich results for RBIBDs, a number of uniform supersaturated designs can be obtained as follows. Please refer to the corresponding references for the related RBIBDs. These results are also tabulated in Appendix. Corollary 2 [32]. n λ(n−1) If n is even, and λ is any positive integer, then a uniform S(n;( 2 ) ) exists. Corollary 3 [26]. n−1 n 2 (a) If n ≡ 3 (mod 6), then a uniform S(n;( 3 ) ) exists. n n−1 (b) If n ≡ 0 (mod 3) and n 6= 6, then a uniform S(n;( 3 ) ) exists. n−1 n 3 (c) If n ≡ 4 (mod 12), then a uniform S(n;( 4 ) ) exists. n n−1 (d) If n ≡ 0 (mod 4), then a uniform S(n;( 4 ) ) exists. n n−1 (e) If n ≡ 0 (mod 6), then a uniform S(n;( 6 ) ) exists except possibly n ∈ {174, 240}. n 2(n−1) (f) If n ≡ 0 (mod 6), then a uniform S(n;( 6 ) ) exists. Corollary 4 [33]. 中国科技论文在线 http://www.paper.edu.cn

No. 8 UNIFORM SUPERSTRUCTURED DESIGNS 1087

n−1 n 4 (a) If n ≡ 5 (mod 20), then a uniform S(n;( 5 ) ) exists except possibly n ∈ {45, 225, 345,465, 645}. n−1 n 2 (b) If n ≡ 5 (mod 10) and n 6= 15, then a uniform S(n;( 5 ) ) exists except possibly n ∈ {45,115,135,195,215,225,235,295,315,335,345,395}. n n−1 (c) If n ≡ 0 (mod 5) and n 6= 10, 15, then a uniform S(n;( 5 ) ) exists except possibly n ∈ {70, 90, 135, 160, 190, 195}. 3 Concluding remarks

In this paper we have proposed a discrete discrepancy as a measure of uniformity for fac- torial designs, and obtained a lower bound of the discrepancy for U-type designs. The discrete discrepancy is invariant at permuting levels of the factors. The construction method for uniform supersaturated designs from RBIBDs sets up an important bridge between these two different kinds of design. Nguyen[3], Cheng[7], Liu and Zhang[14] discussed the construction of 2-level supersaturated designs from BIBDs. The construction method proposed in this paper can be regarded as an extension of their methods. In a factorial design, two columns are called fully aliased if one column can be obtained from the other by permuting levels. It is necessary that all the columns should not be fully aliased, as we cannot use two fully aliased columns to accommodate two different factors. Theorem 2 tells us that in any of the uniform supersaturated designs constructed from RBIBDs with λ = 1, there are no fully aliased factors and any of the possible level combinations between any two factors appears at most once. The latter is a desirable property when the orthogonality between any two columns cannot be satisfied in a supersaturated design. In particular, when n = q2 and an RBIBD(n, mq, m, n/q, 1) exists, the resulting uniform S(q2; q(q+1)) reduces to a saturated orthog- onal design, i. e. the saturated orthogonal design is also a uniform design. For λ > 2, what are the properties of the resulting uniform supersaturated designs? This needs further investigations.

Appendix Index table of uniform S(n; qm)

Table 2 S(10; 59) derived from table 1 q m Conditions Reference n/2 λ(n − 1) n ≡ 0 (mod 2), λ is any positive integer [32] n/3 (n − 1)/2 n ≡ 3 (mod 6) [26] n/3 n − 1 n ≡ 0 (mod 3), n =6 6 [26] n/4 (n − 1)/3 n ≡ 4 (mod 12) [26] n/4 n − 1 n ≡ 0 (mod 4) [26] n/5 (n − 1)/4 n ≡ 5 (mod 20), n 6∈ {45, 225, 345, 465, 645} [33] n/5 (n − 1)/2 n ≡ 5 (mod 10), n 6∈ {15, 45, 115, 135, 195, 215, 225, [33] 235, 295, 315, 335, 345, 395} n/5 n − 1 n ≡ 0 (mod 5), n 6∈ {10, 15, 70, 90, 135, 160, 190, 195} [33] n/6 n − 1 n ≡ 0 (mod 6), n 6∈ {174, 240} [26] n/6 2(n − 1) n ≡ 0 (mod 6) [26]

Acknowledgements This work was partially supported by the Hong Kong RGC (HKUB RC/98-99/Gen- 370) the YNSFC (10001026), and the National Natural Science Foundation of China (Grant No. 10171051). The 中国科技论文在线 http://www.paper.edu.cn

1088 SCIENCE IN CHINA (Series A) Vol. 45

research was carried out while the last two authors were visiting Hong Kong Baptist University in 2001, and they wish to express many thanks to the Department of Mathematics for their hospitality. References

1. Lin, D. K. J., Generating systematic supersaturated designs, Technometrics, 1995, 37: 213. 2. Wu, C. F. J., Construction of supersaturated designs through partially aliased interactions, Biometrika, 1993, 80: 661. 3. Nguyen, N. -K., An algorithmic approach to constructing supersaturated designs, Technometrics, 1996, 38: 69. 4. Satterthwaite, F., Random balance experimentation (with discussion), Technometrics, 1959, 1: 111. 5. Booth, K. H. V., Cox, D. R., Some systematic supersaturated designs, Technometrics, 1962, 4: 489. 6. Lin, D. K. J., A new class of supersaturated designs, Technometrics, 1993, 35: 28. 7. Cheng, C. -S., E(s2)-optimal supersaturated designs, Statist. Sinica, 1997, 7: 929. 8. Li, W. W., Wu, C. F. J., Columnwise-pairwise algorithms with applications to the construction of supersat- urated designs, Technometrics, 1997, 39: 171. 9. Tang, B., Wu, C. F. J., A method for constructing supersaturated designs and its E(s2) optimality, Canad. J. Statist., 1997, 25: 191. 10. Yamada, S., Lin, D. K. J., Supersaturated designs including an orthogonal base, Canad. J. Statist., 1997, 25: 203. 11. Deng, L. Y., Lin, D. K. J., Wang, J. N., A resolution rank criterion for supersaturated designs, Statist. Sinica, 1999, 9: 605. 12. Lu, X., Meng, Y., A new method in the construction of two-level supersaturated designs, J. Statist. Plann. Inference, 2000, 86: 229. 13. Liu, M. Q., Zhang, R. C., The equivalence between E(s2) optimal supersaturated designs and BIB designs, Chinese Sci. Bull., 1998, 43: 2053. 14. Liu, M. Q., Zhang, R. C., Construction of E(s2) optimal supersaturated designs using cyclic BIBDs, J. Statist. Plann. Inference, 2000, 91: 139. 15. Yamada, S., Lin, D. K. J., Three-level supersaturated designs, Statist. Probab. Lett., 1999, 45: 31. 16. Yamada, S., Ikebe, Y. T., Hashiguchi, H. et al., Construction of three-level supersaturated design, J. Statist. Plann. Inference, 1999, 81: 183. 17. Liu, M. Q., Zhang, R. C., A criterion and an algorithm on constructing 3-level factorial supersaturated designs, Acta Math. Appl. Sinica, 2000, 23(2): 161. 18. Fang, K. T., Lin, D. K. J., Ma, C. X., On the construction of multi-level supersaturated designs, J. Statist. Plann. Inference, 2000, 86: 239. 19. Lu, X., Sun, Y. Z., Supersaturated design with more than two levels, Chin. Ann. of Math., 2001, 22B(2): 183. 20. Wang, Y., Fang, K. T., A note on uniform distribution and experimental design, Kexue Tongbao, 1981, 26: 485. 21. Fang, K. T., Hickernell, F. J., The uniform design and its applications, Bull. Inst. Internat. Statist., 50th Session, Book 1, 1995, 333—349. 22. Fang, K. T., Mukerjee, R., A connection between uniformity and aberration in regular fractions of two-level factorials, Biometrika, 2000, 87: 193. 23. Hua, L. K., Wang, Y., Applications of Number Theory to Numerical Analysis, Berlin-Beijing: Springer and Science Press, 1981. 24. Hickernell, F. J., A generalized discrepancy and quadrature error bound, Math. Comp., 1998: 67: 299. 25. Hickernell, F. J., Lattice rules: How well do they measure up? in Random and Quasi-Random Point Sets, Vol. 138 of Lecture Notes in Statistics (eds. Hellekalek, P., Larcher, G.), New York: Springer-Verlag, 1998, 109—166. 26. Colbourn, C. J., Dinitz, J. H., CRC Handbook of Combinatorial Designs, Boca Raton: CRC Press, 1996, (New results are reported at http://www.emba.uvm.edu/˜dinitz/newresults.html). 27. Fang, K. T., Lin, D. K. J., Winker, P. et al., Uniform design: theory and applications, Technometrics, 2000, 42: 237. 28. Saitoh, S., Theory of Reproducing Kernels and Its Applications, Essex, England: Longman Scientific and Technical, 1988. 29. Wahba, G., Spline Models for Observational Data, Philadelphia: SIAM, 1990. 30. Hickernell, F. J., Goodness-of-fit statistics, discrepancies and robust designs, Statist. Probab. Lett., 1999, 44: 73. 31. Fang, K. T., Wang, Y., Number Theoretic Methods in Statistics, London: Chapman and Hall, 1994. 32. Rees, R., Stinson, D. R., Frames with block size four, Canad. J. Math., 1992, 44: 1030. 33. Abel, R. J. R., Ge, G., Greig, M. et al., Resolvable balanced incomplete block designs with a block size of 5, J. Statist. Plann. Inference, 2001, 95: 49.