sign in sign up
<<
Home , Optimal design

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975

D-Optimality for Regression Designs: A Review

R. C. St. John and N. . Draper Bowling Green State University University of Wisconsin Department of Quantitative Department Analysis and Control 1210 W. Dayton Street Bowling Green, Ohio 43403 Madison, Wisconsin 53706

After stating the model and the design problem, we briefly present the results for regression design prior to the work of Kiefer and Wolfowitz. We then review the major results of Kiefer and Wolfowitz, particularly those on the theory of design, as well as the way the criterion has been extended to non-linear models. Finally, we discuss algorithms for constructing D-optimum designs.

KEY WORDS The design problem consists of selecting vectors

D-Optimum xi 7 i = 1, 2, *** , n from x such that the design Design defined by these n vectors is, in some defined sense, optimal. By and large, solutions to this problem consist of developing some sensible criterion based 1. MODEL AND NOTATION on the model (2), and using it to obtain optimal In this section we introduce our notation for the designs. model, and the assumptions about the model and the design region. We consider the 2. BACKGROUND Smith (1918) was one of the first to state a criterion and obtain optimal experimental designs yi = f’(x,)@ + ei , i = 1, 2, * * * ) n 7 (1) for regression problems. For which we express in notation as of order p - 1 in one variable over the design region x = [-1, 11, she proposed the criterion y= x0+&. (2) min max z@(x)). (7) The vector y is an n X 1 vector of observations; ,x;,i=1.2 ,..., n, =cx X is an n X p matrix, with row i containing f’(x;); xi is a Q X 1 vector of predictor variables; f’(xi) is a This criterion was later called G-optimality by p X 1 vector which depends on the form of the Kiefer and Wolfowitz (1959), and it has taken on response function assumed; 0 is a p X 1 vector of considerable importance in the theory and construc- unknown ; e is an n X 1 vector of inde- tion of optimal designs. Using this criterion Smith pendently and identically distributed random vari- obtained designs for various values of p. Guest ables, with zero and a2. The experi- (1958) showed that, for a polynomial of degree mental region is denoted by x, and we assume that p - 1, the allocation of points according to Smith’s x is compact and that fi(xi)‘s are continuous on x. min max criterion (7) could be obtained by finding We assume that estimates of the the zeroes of the derivative of a Legcndre poly- parameters @are to be obtained. These are given by nomial. Wald (1943) proposed the criterion of maximizing 6 = (x’x)-‘X’y, (3) the of X’X as a of maximizing and the variance- matrix of 0 is the local power of the F-ratio for testing a linear hypothesis on the parameters of certain fixed-effects V(O) = g”(x’x)-‘. (4) models. Mood (1946) proposed Then, at point xex, the predicted response is the same criterion for obtaining weighing designs. Kiefer and Wolfowitz (1959) later called this B(x) = f’(x>e^, (5) criterion D-optimality and extended its use to . with variance regression models in general. De la Garza (1954) showed that, for a polynomial of degree p - 1, v(Q(x)) = ~“f’(x)(x’x)-‘f(X). (6) there exists a design with p distinct points which has the same X’X matrix as a design with more than Received July, 1974; revised October, 1974 p distinct points (Kiefer (1959) subsequently 15 16 R. C. ST. JOHN AND N. R. DRAPER corrected De la Garza’s proof). Hoe1 (1958) obtained Dejnition 2. .$* is G-optimal if and only if the optimal allocation for a polynomial of degree min max d(x, l) = max d(x, l*). (13 p - 1 using both the min max criterion of Smith and t x*x x=,-i the determinant criterion of Wald. He showed that A sufficient condition for {* to satisfy (12) is the two criteria gave the same results, thus hinting at the equivalence theorem of Kiefer and Wolfowitz max d(x, .$*) = p. (13) (1959), which we discuss in the next section. xex Various other properties of the X’X matrix were The equivalence of D- and G-optimality is es- suggested as being an appropriate criterion for tablished in the general equivalence theorem of design. Elfving (1952) and Chcrnoff (1953) mini- Kiefer and Wolfowitz, which can be formally stated mized the of (X’X)-’ to obtain regression as follows: designs. Ehrenfeld (1955) suggested that maximizing Theorem 1. Conditions (ll), (12), and (13) are the minimum eigenvalue of X’X be used as a equivalent, and the set of all 5 satisfying these design criterion. conditions is convex. The implication of t,his result is that we can 3. DESIGN THEORY use (13) to verify whether or not a specific design Kiefer (1959, 1961a, 1961b, 1962 and 1970a) and is D-optimal. That is, if (13) is satisfied, then the Kiefer and Wolfowitz (1959, 1960) made a number design is D-optimal. Note that D-optimality is of contributions to the theory of regression design. essentially a estimation criterion, whereas We discuss the two main results: the idea of design G-optimality is a response estimation criterion. The measure and their general equivalence theorem. equivalence theorem says these two design criteria A design can be viewed as a measure E on x. Suppose are identical when the design is expressed as a measure we have an n-point design with n; observations at on x. the site xi (note that Zni = n). Let [ be a probability As defined above, the criterion of D-optimality is measure on x such that E(xi) = 0 if there are to be applicable when all p parameters are to be estimated. no observations at point xi , and such that t(x;) = However, in many cases one is interested in only ni/n if there are to be ni > 0 observations at s < p parameters, the other parameters being point x, . For a discrete n-point design, E takes on nuisance parameters. Kiefer and Wolfowitz (1959) values which are multiples of l/n, and defines an discussed this problem and Kiefer (1961a) gave exact design on x. Removing the restriction that [ be definitions of D,-optimality and G,-optimality (i.e., a multiple of l/n, we can extend this idea to a optimality when interest is in s < p parameters) design measure which satisfies, in general: which are analogous to the above definitions of D- and G-optimality. Kiefer (1961a, 1961b, and 1962) later extended the general equivalence theorem to this situation. ‘l(dx) = 1. (8) InI Karlin and Studden (1966a) gave various forms of the response model (I) for .which the D-optimal With respect to model (a), we can define a matrix designs were known. In addition, Karlin and Studden analogous to X’X for design 5. Let extended and simplified Kiefer’s results for D,- and G.-optimality. In particular they discuss the problem miCEI = / f~WiWG-W, i, j = 1, 2, . . . , P (9) x of estimating s < p parameters when M(t) is singular. and Atwood (1969) gave further results in the theory of D-optimality. In particular he gave more general definitions of D,- and G,-optimality, and he discussed Note that, for an exact design, nM([) = X’X. the role of symmetry in optimal design. Similarly, a generalization of the Silvey and Titterington (1973) have given a (6) is given by geometric interpretation of optimal design. Using d(x, 0 = f’(x)M-‘@f(x). (10) results from Strong Lagrangian Theory they established duality theorems which state the From this following general definitions of D-opti- Kiefer-Wolfowitz equivalence theorem in terms of mality and G-optimality. a design measure on the function space (the image Dejkition 1. [* is D-optimal if and only if of x under the model). Moreover, they suggest that M({*) is nonsingular and this duality mav lead to efficient algorithms- for constructing D-optimal designs, a point which we my det (M(Q) = dct (M(t*)). (11) discuss below.

TECHNOMETRICSO, VOL. 17, NO. 1. FEBRUARY 1975 D-OPTIMALITY FOR REGRESSION DESIGNS 17

Sibson (1973) and (1974), following his contribu- of obtaining D-optimal designs which are based on tion to the discussion of the paper of Wynn (1972), the same procedure as employed by Box and Hunter. is developing linear programming algorithms for Atkinson and Hunter (1968) extend the BOX- the explicit solution of D-opt,imality problems, based Lucas results to the case where the design is to have on these duality theorems. He has been able to avoid more than p points. Draper and Hunter (1967a and the unboundedness problems involved by imposing 1967b), discussed the problem of selecting prior extra constraints on the total mass allocation to distributions on the parameters to obtain designs points, namely, that no point should have mass for non-linear models. greater than l/p. M. J. BOX (1968, 1970, 1971) has given a number Kiefer (1974) has recently given further equi- of additional results for non-linear design, particu- valence results between D-optimality and other larly on the subjects of replicated design points design criteria. He also suggests construction (see also Wynn (1972)), model inadequacy, and methods based on these equivalence results. computing difficulties in obtaining D-optimal designs. For the sake of brevity we will hereafter discuss Sredni (1970) partitioned the X’X matrix into only the case of interest in all p parameters. Readers three components: (i) the determinant of X’X, interested in the case where s < p parameters are which is proportional to the volume of a Highest of interest should refer to the above papers (as well Posterior Distribution (H.P.D.) region on @; (ii) a as recent results by Atwood (1973), Wynn (1970 function of the eigenvalues of X’X, which describes and 1972), and Fedorov (1972)). the conditioning of this H.P.D. region; and, (iii) a function of the eigenvectors of X’X, which describes 4. DESIGN FOR NON-LINEAR MODELS. the orientation of the H.P.D. region relative to the The form of model (1) is somewhat restrictive axes of the parameter space. since it assumes that the response function is linear White (1973) recently extended the Kiefer- in the parameters. What can be done in the case of Wolfowitz equivalence theorem to design for non- non-linear response function? Chernoff (1953) linear- linear models. White developed the information ized the model using a first order Taylor series matrix (9) and the variance function (10) as functions expansion about a preliminary value for 0, and he of @(assumed values of the parameters) and proved applied the maximum trace criterion to obtain an equivalence using these new functions. locally optimum designs for this linearized model. G. E. P. Box and Lucas (1959) also linearized 5. ALGORITHMS FOR CONSTRUCTING models which are non-linear in the parameters. D-OPTIMAL DESIGNS. They applied the D-optimality criterion to obtain As we mentioned in section 1, the practical designs for the linearized models. Box and Lucas, design problem consists of selecting an exact design, using a geometric argument to obtain p-point namely vectors x1 , xZ , * . . , x, , which define the designs for p-parameter models, showed that the experimental conditions for n experimental runs. In locally D-optimal p-point design maximizes the section 3 we reviewed the concept of using a measure volume of the simplex defined by the design points f on the design region x to define a measure, or in the image (under the model) of the design region. approximate design. G. E. P. Box and Hunter (1965) showed that Measure designs are of interest primarily because locally D-optimal designs for non-linear models the D-optimal measure design provides the reference maximize the of the parameters against which exact designs can be evaluated, and at the least squares (maximum likelihood under also because the points in an optimal exact design normality) solution point. They extended this will often correspond to the points of support result to obtain sequential designs. Their design (points of positive measure) of the D-optimal sequence selects, at each step, the point which measure design. maximizes (over the design region) the variance of We first discuss algorithms for obtaining D-optimal the predicted response, based on a linearization of designs. Kiefer and Wolfowitz (1959) suggested a the non-linear model. Note that, at each st#epin the game theory approach for constructing designs. experimental sequence, the preliminary estimates However, this approach appears more useful for of the parameters must be updated as are verifying the optimality of given designs. Kiefcr obtained. That is, the posterior distribution on the (1961a and 1961b) gave examples of constructing parameters becomes the prior distribution for D-optimal designs, but his methods did not lead to obtaining the next design point. This design sequence a general algorithm. Fedorov and his co-workers is essentially an application of Wynn’s use of the [Fedorov (1969a, 1969b, 1972), Fedorov and Dubova Kiefer-Wolfowitz equivalence theorem to obtain (1968)] appear to have been the first to develop a designs. We will later discuss more general methods general algorithm for obtaining D-optimal designs.

TECHNOMETRICS@.‘, VOL. 17, NO. 1. FEBRUARY 1975 18 R. C. ST. JOHN AND N. R. DRAPER

Most of the original published results are in Russian. to Wynn’s algorithm.) Obviously, such a choice However, Fedorov’s (1969b) book was translated by for LYEdoes satisfy (17). Dykstra (1971), Hebble and Studden and Klimko (1972), and our discussion of Mitchell (1971), and Covey-Crump and Silvey Fedorov’s work is based on this translation. (1970) suggested algorithms similar to Wynn’s, Fedorov’s algorithm for obtaining a D-optimal but did not give convergence proofs. design allows the variance of the observation error Atwood (1973) has offered some improvements to (i.e., V@(X))) to be a function of the point x at which Fedorov’s algorithm. He made these suggestions: the observation is made. We shall assume that this 1. It may be possible to achieve a greater increase variance function is independent of x and y(x), in the value of the determinant by removing and present Fedorov’s algorithm in this simpler measure from a point already in the design .$( form. The steps of the algorithm are as follows: and distributing ,the measure removed among 1. Let En be a nondegenerate (det (M(Q) > 0) the remaining points in design fi . If such an n-point design on x, and let i = n. action would result in a greater increase in 2. Compute M(t;) and M-‘(Ei). det (M([,)) than would Fedorov’s method of 3. Find xi such that adding measure to the point satisfying (14), the point xI, from which measure is to be max d(x, ti) = d(x, , &). (14) x removed is given by

4. Let min d(Xi , [i> = d(x, , ii). w lxil&i(xi)>ol 4% , Fi) - p (15) cri = p(d(x, ) fi) - 1) 2. In the event that a number of different actions (for example, adding measure to a point of be the measure at point x, . maximum variance or removing measure from 5. Let a design point of minimum variance) would Ei+, = (1 - 4.5 + CriEXi (16) lead to identical increases in the value of det (M(li)), choose the new design to be the be the new design measure (where eXi places equal-proportion convex combination of these measure one at point xi). potential designs. This procedure can greatly 6. Repeat steps 2-5 until d(x, , ii) is sufficiently improve convergence when the D-optimal close to p. design is symmetric. In words, we can state these steps simply as follows: 3. The maximum measure for any design point 1. Select any non-degenerate starting design. in ,$* is l/p. If at any stage in the algorithm 2. Compute the dispersion matrix. the measure at a design point is greater than 3. Find the point of maximum variance. l/p, allocate the excess measure to the other 4. Add the point of maximum variance to the points in the design. design, with measure proportional to its St. John (1973) has modified Atwood’s first variance. suggestion by distributing the measure removed 5. Update the design measure. only among the points of support for which Fedorov also developed a numerical procedure for d(xi , ii) > p, and additionally, distributing the updating M(Ei) and M-‘(ii) such that the inverse removed measure among these points in a manner need be calculated only for M&). Fedorov proved proportional to d(zi , ii) - p. In this way the that the sequence of designs defined by steps l-6 removed measure will be distributed according to converges (as i --+ m) to the D-optimal design F*. “need”. Atwood’s second suggestion can also be The value of (Y< given in (15) maximizes the improved through a better choice of (Y~ when ties increase in the value of the determinant at each occur (St. John (1973)). step in the sequence. However, Fedorov (1972) These improvements by Atwood (1973) and indicated that any sequence of (Y; satisfying St. John (1973) can be especially useful in eliminating undesirable points of support which were included in the initial design. In such cases, the rate of convergence of Fedorov’s algorithm will be markedly will also guarantee convergence to the D-optimal improved. design E*. Silvey and Titterington (1973) have recently Wynn (1970) proved the convergence of a similar outlined an algorithm for obtaining a D-optimal algorithm, but with ai = l/(i + l), that is, with design measure on the dual space (i.e., on the space all points in the design sequence having equal spanned by the response function specified in model weight. (Pazman (1974) has given a different proof (1)). Their method of construction differs from that

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975 D-OPTIMALITY FOR REGRESSION DESIGNS 19 developed by Fedorov in that the iteration scheme Kiefer (1965) presented a counter-example to the consists of double-loop procedure (i.e., one iterative existence of such an equivalence theorem. procedure inside another iterative procedure). They A number of authors obtained D,-optimal designs are presently developing an efficient procedure for for specific models using mathematical programming implementing this double-loop algorithm. methods to optimize ] X,‘X,] . Designs using this Wynn (1972) extended his algorithm for con- approach were given by Box and Draper (1971), structing D-optimal designs when one is interested Atkinson (1969), Bloom, Pfaff enberger and Kochen- in efficient estimates for s < p parameters. His berger (1972), Box (1966), Hartley and Ruud (1969), algorithm is based on the extended equivalence Mahoney (1970)) and Newhardt and Bradley (197 1). theorem given by Kiefer (1961a), Karlin and Studden However, as general computing algorithms for (1966a, b) and Atwood (1969). Interested readers constructing designs, these methods involve opti- are referred to Wynn’s paper, and to the spirited mizing with respect to np variables and can give discussion that followed it. Further refinements of local solutions which are not optimal. this algorithm were given by Wynn (1973). Since no equivalence theorem exists, the algorithms Despite Fedorov’s algorithm, D-optimality is not for obtaining D,,-optimal designs are somewhat always an easy criterion to implement. Nalimov different than Fedorov’s D-optimal algorithm. There et al (1970) and M. J. Box and Draper (1971) point are two different algorithms for obtaining D,-optimal out that creating an exact design from the prob- designs. The first is due to Fedorov (1969b, 1972) ability measure may require an extremely large and Van Schalkwyk (1971). The second is due to number of experimental runs. It we restricted the Wynn (1972) and Mitchell and Miller (1970). design to a fixed number of experimental runs (that Suppose we have an arbitrary n-point design, and is, fixed n) it may be necessary to round off the we wish to find a D,-optimal design. Fedorov measures in order to implement the D-optimal suggested replacing one of the points in the design, design. However, there may be other n-point designs say x, , with a point from the design space x, say x, with det (X’X) larger than the value of det (X’X) such that the increase in the value of (X,‘X,( is given by this rounded off design. Kiefer (1970a) maximum. Let ti(n) represent the exact n-point suggested rounding-off procedures which yield n- design measure after exchange j. Define point designs with det (X’X) close to the best Ai(xi , x) = d,(x) - [&(x)&(xs) - di2(x, xi)] possible value of det (X’X) for an n-point design. A drawback of his procedure is that the number of - d,(Xi), (21) points in the design region with positive measure where (that is, the number of points of support (or sites) of the design) may be larger than the value of n. d,(x) = f’(x)M-‘(Sj(n))f(x) (22) In this case, the rounding-off procedure may elimi- and nate points with small measure, thereby changing the nature of the design. For these reasons, it is dj(x, xi) = f’(x)M-‘(tj(n))f(x,). (23) desirable to search for algorithms which yield the If Ai(xi , x) > 0, then, when point x, is replaced n-point design with maximum value of det (X’X). by point x, Let t(n) denote an exact design with n points. Then, we denote the information matrix for this n IMQi+I(4)I = IM(FAn>)ID + 4(xi , x)1. (24) point design as The algorithm is as follows: nM(t(n)) = X’X, , (19) 1. Begin with an arbitrary n-point design t”(n). 2. Calculate M&(n)) and M-‘(Ei(n)). and the variance function (assuming a2 = 1 without 3. Select xi and x which satisfy loss of generality) of the predicted response as max max Ai(xi , x) (25) Ixilti(xi)>ol XIIX v@(x)) = f’(x)(X”‘X,)-‘f(x). (20) and replace xi by x. Definition 3. F*(n) is D,-optimal if and only if 4. Repeat steps 2 and 3 until Ai(xi , X) is X,*/X,* is nonsingular and the value of det (X,,*‘X**) sufficiently close to zero. is the maximum value over all possible designs t(n). Fedorov also presented a numerical procedure for Definition 4. F*(n) is G,,-optimal if and only if updating M(Ei(n)) and M-‘(am). Van Schalkwyk max, f’(x) (X,*‘X,*)-‘f(x) is minimum over all (1971) presented a similar algorithm, except he possible designs E(n). assumed that the point to be removed from the design Unfortunately, an equivalence theorem compar- is the point xi with minimum variance (i.e., the able to Theorem 1 is not possible for n-point designs. point xi which minimizes (22)). This procedure does

TECHNOMETRICSt$ VOL. 17, NO. 1, FEBRUARY 1975 20 R. C. ST. JOHN AND N. R. DRAPER not always produce the maximum increase in the As we have previously mentioned, there is no value of t.he determinant. equivalence theorem for exact designs, and therefore The other exchange algorithm, due independently the value of max d(x, t(n)) cannot be used to prove to Wynn (1972) and to Mitchell and Miller (1970), that E(n) is D,-optimal. Atwood (1969) and Wynn does not attempt to maximize the increase in the (1970) obtained upper and lower bounds on the value of the determinant at each step, but rather of exact designs, based on the value of emulat,es the Wynn algorithm for finding a D-optimal IM(t*)I , (i* the D-optimal design) and the value of design. The algorithm is as follows: max, d( x, E(n)). Atwood, in the discussion of Wynn’s 1. Begin with an arbitrary n-point design to(n). (1972) paper, gave a different form of upper bound 2. Find x,,+~ such that on the efficiency of a D,-optimal design. It appears that these bounds are useful for determining that 4x”+, , L(n)> = max d(x, .5(n)> (26) xpx an exact design is not D,-optimal, but not for assuring that an exact design is D,-optimal. At present it and add x,+~ to the n-point design. appears that guaranteeing D,-optimality can only 3. Find xi* such that be done by an exhaustive search of all possible d(q*, t,(n + 1)) = min d(xi , Lb + 1)) (27) designs. l

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975 D-OPTIMALITY FOR REGRESSION DESIGNS 21

[2] ATKINSON, A. C. (1969). Constrained maximization estimating parameters. Ann. Math. Statist., 24, 586-602. and the design of . Technometrics, 11, [251 CLBRK, V. (1965). Choice of levels in polynomial regres- 616-618. sion with one or two variables. Technometrics, 7, 325- [3] ATKINSON, A. C. (1972). Planning experiments to 333. determine inadeqllate regression models. Biometrz’ka, [26] COVEY-CRUMP, P. A. K., and SILVEY, S. D. (1970). 59, 273-293. Optimal regression designs with previous observations. [4] ATKINSON, A. C. (1973). Multifactor second-order Biometrika, 67, 551-566. designs for cuboidal regions. Biometrika, 60, 15-19. [27] DE L:\ GARZ.~, A. (1954). Spacing of information in [5] ATKINSON, A. C., and HUNTER, W. G. (1968). The design polynomial regression. Ann. Math. Statist., 25, 123-130. of experiments for parameter extimation. Technometrics, [28] DRAPER, N. R. and HUNTER, W. G. (1966). Design of 10, 271-289. experiments for parameter estimation in multiresponse [6] ATwOOD, C. L. (1969). Optimal and efficient design of situations. Biometrika, 63, 525-533. experiments. Ann. Math. Statist., 40, 1570-1602. 1291 DRAPER, N. R. and HUNTER, W. G. (1967a). The use of [7] Arwooo, C. L. (1973). Sequences converging to D- prior distributions in the for optimal designs of experiments. The Annals of Statist., 1, parameter estimation in nonlinear situations. Bio- 342-352. metrika, 64, 147-153. [B] BISGGS, WILLI.IM J. (1966). The Kiefer approach to [30] DRAPER, N. R. and HUNTIJR, W. G. (1967b). The use experimental design. Master’s Essay, Department of of prior distributions in the design of experiments for Statistics, University of Wisconsin, Madison, Wisconsin. parameter extimation in nonlinear situations-multi- (91 BEHNKEN, D. W. (1964). Estimation of copolymer response case. Biometrika, 64, 662-665. reactivity rates: an example of nonlinear est,imation. [311 DRAPER, N. R. and ST. JOHN, R. C. (1974). Models J. of Polymer Science, Part A, 2, 645-668. and designs for experiments with mixtures: I, Back- [lo] BLOOM, T., PFAFFGNBERGER, R., and KOCHENBERGER, ground material. Mathematics Research Center Technical G. (1972). The efficient construction of optimal response Summary Report No. 1436, University of Wisconsin. surface designs based upon the IX’XI criterion. Pre- 1321 DUBOVA, I. S., PTSKIALADZE, T. V., and FEDOROV, V. V. sented at the 1972 American Statistical Assn. Meeting, (1970). Tables of D-optimum designs I (discrete D- Montreal, Canada, August 14-18. optimum designs). Preprint No. 16, Izd-vo Moscow [ll] Box, G. E. P. (1957). Use of statistical methods in the State University, Moscow, USSR. elucidation of basic mechanisms. Bull. Inter. Statist. [331 DYKSTRA, OTTO, JR. (1971). The augmentation of Inst., Part 3, 36, 215-225. experimental data to maximize IX’XI. Technometrics, [12] Box, G. E. P. and HILL, W. J. (1967). Discrimination 13, 682-688. among mechanistic models. Technometrics, 9, 57-71. 134) EHRENFKLD, S. (1955). On the efficiencies of experi- [13] Box, G. E. P., and HUNTER, W. G. (1965). Sequential mental designs. Ann. Math. Statist., 26, 247-255. design of experiments for nonlinear models. IBM [35] EHRENFELD, S. (1956). Complete class theorems in Scientijic Computing Symposium in Statist., 113-137. experimental design. Proceedings of the Third Berkeley [14] Box, G. E. P. and LUCAS, H. L. (1959). Design of Symposium on Mathematical Statist. and Probability, experiments in non-linear situations. Biometrika, 46, 1,57-68. 77-90. [36] ELFVING, G. (1952). Optimum allocation in linear [15] Box, M. J. (1966). A comparison of several current regression theory. Ann. Math. Statist., 23, 255-262. optimization methods, and the use of transformations [37] ELFVING, G., (1959). Design of linear experiments. in constrained problems. Computer Journal, 9, 67-77. Cramer Festschrift 68, Wiley, New York. [16] Box, M. J. (1968a). The use of designed experiments [38] FARRZLL, R. H., KIIZFF,R, J., and WALBRAN, A. (1967). in nonlinear model building. The Future of Statist., Optimum multivariate designs. Proceedings of the Fifth 241-268. Berkeley Symposium of Mathematical Statist. and Prob- [17] Box, M. J. (196813). The occurrence of replications in ability, 1, 113-138. opt,imal designs of experiments to estimate parameters [39] FEDOKOV, V. V. (1968). Design of experiments in the in nonlinear models. J. Roy. Statist. Sot., B30, 290-302. case of simultaneous measurements of several response [18] Box, M. J. (1969). The design of experiments and surfaces. Preprint No. 3 LSM, Izd-vo Moscow State nonlinear model-building under a variety of error University, Moscow, USSR. structures. Ph.D. Thesis, University of London, London, [40] FF,DOROV, V. V. (1969a). Properties and methods for England. construction of point optimal designs in regression [19] Box, M. J. (1970). Some experiences with a nonlinear experiments. Preprint No. 5 LSM, Id-vo MOSCOW State experimental design criterion. Technometrics, 12, 569- University, MOSCOW,USSR. 589. [41] FEDOROV, V. V. (1969b). Theory of optimal experiments. [20] Box, M. J. (1971a). Simplified experimental design. Preprint No. 7 LSM, Izd-vo Moscow State University, Technometrics, 13, 19-31. Moscow, USSR. [21] Box, M. J. (1971b). An experimental design criterion [42] FEDOROV, V. V. (1969c). Sequential met.hods for design for precise estimation of a subset of the parameters of experiments in the study of the mechanism of a in a nonlinear model. Biometrika, 68, 149-153. phehomenon. New Ideas in Experimental Design. Edited [22] Box, M. J. and DRAPER, N. R. (1971). Factorial designs, by V. V. Nalimov, Izd-vo “Nuuka”, MOSCOW, USSR. the IX’XJ criterion, and some related matters. Techno- [43] FEDOROV, V. V. (1971a). The design of eiperiments metrics, 13, 731-742. in the multiresponse case. Theory of Probability and [23] BRODSKY, V. Z., and GOLIKOVA, T. I. (1971). D-optimal its Applications, 16 (No. 2), 323-332. designs for a special case of linear regression. Preprint [45] FF,DOROV, V. V. (1971b). Design of experiments when No. 14 LSM, Izd-vo Moscow State University, MOSCOW, the response surface is nonlinear in the paramet)ers. USSR. Preprint No. 21, Izd-vo Moscow State University, [24] CHERNOFF, H. (1953). Locally optimum designs for Moscow, USSR.

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975 22 R. C. ST. JOHN AND N. R. DRAPER

I461 FEDOROV, V. V. (1972). Theory of Optimal Experiments. Theory and Related Topics, 109-118, Academic Press, Translated and edited by W. J. Studden, and E. M. New York. Klimko, Academic Press, New York. [69] KIEFER, J. (1970b). Optimum Experimental Designs. 1471 FEDOROV, V. V., and DUBOVA, I. S. (1968). Methods Proc. Intl. Congress Math., Nice, S, 249-254, Gauthier- for constructing optimal designs in regression experi- Villars, Paris. ments. Preprint No. 4 LSM, Izd-vo Moscow State Uni- 1701’ KIEFER, J. (1972). Exact optimality theory for designs. versity, Moscow, USSR. IMS Bull., 1, 182. [48] FEDOROV, V. V., and DUBOVA, I. S. (1972). Tables of [711 KIEFER, J. (1973). Panel discussion on optimality optimum designs II (saturated D-optimal designs on criteria for design. Proceedings of March 1973 Fort a cube). Issue No. 40, Izd-vo Moscow State University, Collins Conference. Moscow, USSR. [72] KIF,FER, J. (1974). General equivalence theory for 1491 FEDOROV, V. V., and MALYUTOV, M. B. (1972). Optimal optimum designs (approximate theory). Ann. Math. designs in regression problems. Mathematische Opera- Slalist., 2, 849-879. tionsfurschung und Statistik, 4, 237-324. [73] KIEFER, J., and WOLFOWITZ, J. (1959). Optimum [50] GAYLOR, D. W., and MERRILL, J. A. (1968). Augmenting designs in regression problems. Ann. Math. Statist., existing data in multiple regression. Technometrics, 10, so, 271-294. 73-81. [74] KIEFER, J. and WOLFOWITZ, J. (1960). The equivalence 1511 GOLIKOVA, T. I., and MIKESHINA, N. G. (1969). Prop- of two extremum problems. Canad. J. Math., 12, 363- erties of D-optimal designs and the structure of quasi- 366. D-optimal designs. Information Materials. Edited by 1751 KONO, K. (1962). Optimum designs for quadratic S. I. Samoilenko, 18-26. regression on k-cube. Mem. Fat. Sci., Kyushu University [52] GOLIKOVA, T. I., MIKESHINA, N. G., NALIMOV, V. V., of Science A, 114-122. and VESELAYA, G. N. (1967). Construction of nearly [76] LINDLEY, D. V. (1956). On a measure of the information D-optimal second order designs on a cube. (In Russian). provided by an . Ann. Math. Statist., 27, Zavodskaya Laboratoriya, XXXZIZ, 847-859. 968-1005. [53] GUEST, P. G. (1958). The spacing of observations in [77] MALYUTOV, M. B., and FEDOROV, V. V. (1971). On polynomial regression. Ann. Math. Statist., 29, 294-299. the design of experiments taking into account in- [54] GUEST, P. G. (1961). Numerical Methods of Curve adequacy of the assumed model. Preprint No. 18, Izd-vo Fitting. Cambridge University Press. Moscow State University, Moscow, USSR. 1551 HARTLEY, H. 0. and RUUD, P. G. (1969). Computer [78] MINNICH, G. M. (1972). Some considerations in the optimization of second order response surface designs. choice of experimental designs for dynamic models. Statistical Computing. Academic Press, New York, Ph.D. Thesis, Department of Statist., University of 441462. Wisconsin, Madison, Wisconsin. 1561 HEBBLE, T. L., and MITCHELL, T. J. (1971). Repairing [79] MITCHELL, T. J. (1974a). An algorithm for the construc- response surface designs. Technometrics, 14, 767-779. tion of D-optimal experimental designs. Technometrics, (571 HILL, W. J., HUNTER, W. G., and WICHERN, D. W. 16, 203-211. (1968). A joint design criterion for the dual problem of [80] MITCHELL, T. J. (197413). Computer construction of model discrimination and parameter estimation. Techno- D-optimal first-order designs. Technometrics, 16, 211- metrics, 10, 145-160. 220. [58] HILL, W. J., and HUNTER, W. G. (1974). Design of [81] MITCHELL, T. J., and MILLER, F. L., JR. (1970). Use experiments for subsets of parameters. Technometrics, of design repair to construct designs for special linear 16. 425434. models. Math. Div. Ann. Progr. Rept. (ORNL-4661), [59] HOEL, P. G. (1958). Efficiency problems in polynomial Oak Ridge National Laboratory, Oak Ridge, Tennessee, estimation. Ann. Math. Statist., 29, 1134-1145. 130-131. [60] HOEL, P. G. (1961). Some properties of optimal spacing [82] MAHONEY, F. (1970). The design of experiments for in polynomial estimation. Ann. Inst. Stat. Math., IS, l-8. by computer search procedures. (611 KARLIN, S., and STUDDEN, W. J. (1966a). Optimal Ph.D. Thesis, Rensselaer Polytechnic 1nstitut.e. experimental designs. Ann. Math. Statist., 37, 783-815. [83] MOOD, A. M. (1946). On Hotelling’s weighing problem. [62] KARLIN, S., and STUDDEN, W. J. (196613). Techebycheff Ann. Math. Statist., 17, 432446. Systems: With Applications in Analysis and Statist., [84] NALIMOV, V. V., editor. (1969). New Ideas in Experi- New York: Wiley Interscience. mental Design-A Collection of Articles. Izd-vo “Nauka”, [63] KIGFER, J. (1958). On the nonrandomized optimality Moscow, USSR. and randomized nonoptimality of symmetrical designs. [85] NALIMOV, V. V., and GOLIKOVA, T. I. (1971). Logical Ann. Math. Statist., 20, 675-699. foundations of the planning of experiments. Preprint [64] KIEFRR, J. (1959). Optimum experimental designs. J. No. 20, Zzd-vo Moscow State University, Moscow, USSR. Roy. Statist., Sot., B21, 272319. [86] NALIMOV, V. V., GOLIKA, T. I., and MIKESHINA, N. G. (651 KIEFER, J. (1961a). Optimum designs in regression (1970). On practical use of the concept of D-optimality problems, II. Ann. Math. Statist., 52, 298-325. Techn.ometrics, 12, 799-812. [SS] KIEFER, J. (1961b). Optimum experimental designs V, [87] NANDI, H. K. (1950). On the efficiency of experimental with applications to rotatable designs. Proceedings of designs. Calcutta Stat. Assn. Bull., S, 167-171. the Fourth Berkeley Symposium on Mathematical Statist. [88] NEUHARDT, J. B. and BRADLEY, H. E. (1971). Selection and Probability, 1, 381405. of multi-factor experimental arrangements with resource [67] KIEFER, J. (1962). Two more criteria equivalent to constraints. J. Amer. Statist. Assoc., 66, 618-621. D-optimality of designs. Ann. Math. Statist., SS, 792- [89] PAZMAN, A. (1974). A convergence theorem in the theory 796. of D-optimum experimental designs. Ann. Statist., 2, [SS] KIBFER, J. (1970a). The role of symmetry and approxi- 216-218. mation in exact design optimality. Statistical Decision [90] PESOTCHINSKY, L. L. (1972). D-optimum and quasi-D-

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975 D-OPTIMALITY FOR REGRESSION DESIGNS 23

optimum design for quadratic regression on a cube [103] TURING, A. M. (1948). Rounding-off errors in matrix and a ball. Preprint No. 4s LSM, Zzd-vo Moscow State processes. &uart. J. Mech. Applied Math., 1, 287-308. university, Moscow, USSR. [104] URANISI, J. (1964). Optimum design for the special [91] ST. JOHN, R. C. (1971). D-optimality, a review. Muster’s cubic regression on the q-simplex, Mathematical reports, Essay, Department of Statist., University of Wisconsin, General Education Department, Kyushu University, I, Madison, Wisconsin. 7-12. [92] ST. JOHN, R. C. (1973). Models and designs for experi- 11051 VAN SCHALKWYK, D. J. (1971). On the design of mixture ments with mixtures. Ph.D. Thesis, Department of experiments. Ph.D. Thesis, University of London, Statist., University of Wisconsin, Madison, Wisconsin. London, England. [93] S~nw, D. E. (1973). The effects of nonlinearity in [106] VIORT, B. (1972a). D-optimal designs for dynamic regression models. Ph.D. Thesis, Department of Statist., models, Part I. Theory. Technical Report No. 314, University of Wisconsin, Madison, Wisconsin. Department of Statist., University of Wisconsin, Madison, [94] SIBSON, R. (1973). DA-optimality and duality. Proc. Wisconsin. European Meeting , Budapest, 1972. (In [107] VIORT, B. (1972b). D-optimal designs for dynamic press). models, Part II. Applications. Technical Report No. [95] SIRSON, R. (1974). Cutting plane algorithms for D- 316, Department of Statist., University of Wisconsin, optimal designs. (To appear). Madison, Wisconsin. [96] SILVEY, S. D., and TITTERINGTON, D. M. (1973). A [108] WALD, A. (1943). On the efficient design of statistical geometric approach to optimal design theory. Bio- investigations. Ann. Math. Statist., 14, 134-140. metrika, 60, 21-32. [log] WHEELER, It. E. (1973). Efficient experimental desigu. (971 SKWISH, J. (1969). On the robustness of optimal designs Presented at the Gordon Research Conference on Statist. in polynomial regression. (Abstract). Ann. Math. in Chemistry and Chemical Engineering, New Hampton, Statist., 40, 1161. New Hampshire, July 9-13, 1973. 1981 SMITH, K. (1918). On the standard deviations of ad- [llO] WHITE, L. V. (1973). An extension of the general justed and interpolated values of an observed poly- equivalence theorem to nonlinear models. Biometrika, nomial function and its constants and the guidance 60, 345-348. they give towards a proper choice of the distribution [ill] WHITTLE, P. (1973). Some general points in the t)heory of observations. Biometrika, 12, l-85. of optimum experimental design. J. Roy. Statist. Sot., [99] SREDNI, J. (1970). Problems of design, estimation, BS6, 123-130. and lack of fit in model building. Ph.D. Thesis, Depart- [I121 WYNN, H. P. (1970). The sequential generation of ment of Statist., University of Wisconsin, Madison, D-optimum experimental designs. Ann. Math. Statist., Wisconsin. 41, 16.55-1664. [lOOI SRIVASTAVA, J. N., and MCDONALD, L. (1971). On the (1131 WYNN, H. P. (1972). Results in the theory and construc- costwise optimality of certain hierarchical and standard tion of D-optimum experimental designs. (With dis- multiresponse models under the determinant criterion. cussion). J. Roy. Statist. Sot., B34, 133-147 and 170-186. J. of i&&variate Analysis, 1, 118-128. [114] WYNN, H. P. (1973). Simple conditions for optimum [loll STIGLER, S. M. (1971). Optimal experimental design for experimental design algorithms. Presented at the Sym- polynomial regression. J. Amer. Statist. Assoc., 66, posium on Linear Models, Colorado Springs, Colorado, 311-318. March 23, 1973. [IO21 STONE, M. (1959). Application of a measure of infor- mation to the design and comparison of regression This is an updated version of a bibiography prepared experiments. Ann. Math. Statist., 30, 55-70. by R. C. St. John (1971).

TECHNOMETRICSO, VOL. 17, NO. 1, FEBRUARY 1975