Randomisation, Replication, Response Surfaces and Rosemary 1 A.C. Atkinson [email protected] Department of Statistics London School of Economics London WC2A 2AE, UK RAB AND ME ² One joint publication ² \One hundred years of the design of experiments on and o® the pages of Biometrika". Bka 88, 53{97 ² Perhaps this talk should be ² \Randomization, Replication, Response Surfaces and Rose- mary" ² We found that the subject of the \Design of Experiments" pretty much fell into two disjoint halves, corresponding to our own interests ² Here are some examples: AGRICULTURAL FIELD TRIAL ² Objective Compare 113 varieties of wheat ² Experimental Variables 4 replicates of each variety divided between blocks (¯elds, farms) ² Response Variables Yield " Rust resistance " Straw ? Resistance to flattening " PHOTOGRAPHIC INDUSTRY ² Objective Optimise sensitisation conditions for an emulsion (i.e. speed) ² Experimental Variables Sensitisor I, Sensitisor II Dye Reaction Time ² Response Variables Sensitivity " Contrast " Fog # Graininess # CLINICAL TRIALS ² Objective Compare several treatments (drugs, therapies) ² Experimental Variables Treatment allocation - A, B, C, ... Concomitant variables/ prognostic factors 9 History of => patient Measurements on ;> ² Response Variables Physical or chemical measurements \Quality of life" CONSUMER DURABLE: FRIDGE, CAR, CAMERA,.. ² Objective Make a \better" product ² Experimental Variables Conditions of use have also to be included Need \robustness" to abuse TWO KINDS OF EXPERIMENT `Agricultural' experiment: ² Physically distinct units; ² Complicated error structure associated with units, plots and blocks; ² Discrete treatments with no or little structure. Industrial experiments are often also of this type. `Industrial' experiment: ² Unit, for example, just another portion of chemical reagents, with no speci¯c properties; ² Simple error structure; ² Complicated treatment structure - maybe the setting of several con- tinuous variables. FISHERIAN PRINCIPLES `Agricultural' experiment (Fisher was at Rothamsted) ² Replication ² Randomization ² Local control ² Ideas of { Treatment Structure { Block Structure `Industrial' experiment ² Gossett (1917) and later. Days as blocks in laboratory testing ² Box (1974 in BHH and earlier). \Block what you can and randomize what you cannot" DESIGNING AN EXPERIMENT At least three phases: ² (a) choice of treatments; ² (b) choice of experimental units; and ² (c) deciding which treatment to apply to which experimental unit. The relative importance of the three phases depends on the application. Agriculture and medicine: insu±cient experimental units which are all alike, so some sort of blocking must be considered: divide experimental units into groups likely to be homogeneous. In an industrial experiment a block might be a batch of raw material. As soon as there is structure, phase (c) becomes important. In this context the `design' is the function allocating treatments to units. DESIGNING AN EXPERIMENT 2 (c). Allocation of treatments to units. Agricultural Experiment. 113 treatments £ 4 replicates. ² With large blocks replicate 113 treatments at each of four sites. These may be very di®erent - clay, sand, wet, marshy,.... ² Analyse with additive block e®ects ² Blocks are usually too small for complete replication ² Design so that, if possible, each treatment occurs in the same number of blocks - balance ² Allocate treatments at random, subject to design. If design has treat- ments A, B, C, ... and you have treatments 1, 2, 3, ... Randomize mapping from numbers to alphabet. ² Again analyse with additive block e®ects. DESIGNING AN EXPERIMENT 3 (c). Allocation of treatments to units. Industrial Experiment. Treatments are sensitisor, dye, reaction time ² Units are just another batch of chemical ² But there may be technical analytical variables: development condi- tions, time between manufacture and photographic testing, batches of material, shift of operators, ... ² Brien and Bailey (2006) \Multiple Randomizations" ² Should be allowed for in design as extra factors ² Randomize order of experiments if possible to avoid confounding with `lurking' variables COMMON STRUCTURE Concomitant Random variables error ² # z & Model Response y = f(x; z; ¯) + ² y x % 9 > Explanatory/ => Control/ variables > Design ;> ² Experimental Design Choose x ANALYSIS ² Second-order assumptions: yi = f(xi; ¯) + ²i with 2 E(²i) = 0 var(²i) = σ lead to least squares ² Additional normality of ²i gives distribution of estimates and test sta- tistics ² Randomization Analysis { BHH have yields of 11 tomato plants (?) six on treatment A, ¯ve on B. Calculatey ¹A ¡ y¹B for the data. Also for all 11!/5!6! permutations and compare observed value with this reference dis- tribution { Not much used for linear models, but may be for clinical trials. SMITH 1918 A remarkable paper, 35 years ahead of its time. K. Smith. \On the standard deviations of adjusted and interpolated values of an observed polynomial function and its constants and the guid- ance they give towards a proper choice of the distribution of observations". Bka 12, 1{85. ² Polynomials up to order six in one variable ² Design region X : ¡1 · x · 1 ² Additive errors of constant variance, so least squares giving predictions y^(x) ² \G-optimum" designs minimizing maximum, for x 2 X , of varfy^(x)g ² Also considered non-constant variance ² Showed the designs were optimum SMITH, PEARSON AND FISHER ² Kirstine Smith (b. 1878) was a Danish pupil of Hald who worked with K. Pearson ² Her 1916 paper \On the \best" values of the constants in frequency distributions" upset Fisher - he thought parameter estimates shouldn't depend on grouping: ML rather than minimum chi-squared ² Pearson twice refused Fisher's paper \I must keep the little space I have in Biometrika free from controversy" ² Fisher never again (?) published in Biometrika ² In 1924 Smith became a school teacher in Denmark - \because she felt a need to work more closely with people" OPTIMUM EXPERIMENTAL DESIGN 1 ² Uni¯ed Theory providing ² Numerical Methods for design construction and comparison ² Modern theory developed by Kiefer JACK KIEFER ² Kiefer (1959). \Optimum experimental designs (with discussion)" JRSSB ² Proposer of the vote of thanks (Tocher) \I think Dr Kiefer has shown conclusively just how useful mathematics is in the theory of design..." ² No comment from Box! ² Kiefer in reply \I must admit being somewhat disappointed to see that such a large proportion of the comments have been engendered by a careless reading of my paper..." ² David Cox arranged a conference in 1974 at Imperial College at which Kiefer was the main speaker ² Papers published in Biometrika 1975 OPTIMUM EXPERIMENTAL DESIGN 2 Requires: ² A model Many models are linear y = ¯T f(x) + ² f(x) is a p £ 1 vector of powers and products of x The errors ² are independent, with variance σ2 In matrix form E(y) = F ¯ Some models are nonlinear in the parameters y = 1 ¡ e¡θx + ² OPTIMUM EXPERIMENTAL DESIGN 3 Also requires: ² A design region X With two factors x1 and x2 the square ¡1 · x1 · 1; ¡1 · x2 · 1 ² The scaling is for convenience: for temperature could be: ¡1; 20oC ; 1; 40oC Figure 1: A square design region x_2 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x_1 OPTIMUM EXPERIMENTAL DESIGN 4 An experimental design is a set of n points in X ² Design Measure 8 9 < = x1 x2 : : : xm » = : ; w1 w2 : : : wm ² The design has { Support points x1 : : : xm { design weights w1 : : : wm ² Exact design wi = ri=n; ri # replicates at xi ² Continuous/ Approximate Design » is a measure over the design points. Removes dependence of design on n. Figure 2: A random design. Any good? x_2 -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x_1 INFERENCE ABOUT BETA ² The Least Squares Estimator of ¯ from the design » is ¯^ = (F T WF )¡1F T W Y; where W = diag wi ² The 100(1 ¡ ®)% Con¯dence Region for ¯ is the set of values for which ^ T T ^ 2 (¯ ¡ ¯) F WF (¯ ¡ ¯) · ps Fp;º;1¡® ² Con¯dence regions are ellipsoids ² Volume of con¯dence ellipsoids / jF T WF j¡1=2 THE INFORMATION MATRIX ² The Information Matrix F T WF is often written F T WF = M(») The information matrix depends on the design ² D-optimum designs max jF T WF j = det M(»¤) They minimise the volumes of the con¯dence region ² Other alphabetical criteria maximize other functions of M(») ² Often maximise » numerically to ¯nd »¤ THE PREDICTED RESPONSE ² Prediction at x is y^(x) = ¯^T f(x) ² The variance of prediction is varfy^(x)g = nσ2f T (x)M ¡1(»)f(x) ² The standardised variance is d(x; ») = f T (x)M ¡1(»)f(x) GENERAL EQUIVALENCE THEOREM 1 { Links var(¯^) varfy^(x)g parameters predictions { Provides Algorithm Kiefer and Wolfowitz (1959) (with a check on convergence) GENERAL EQUIVALENCE THEOREM 2 ² De¯nition »¤ maximises det M(») ² Minimax Equivalence »¤ minimises max x 2 X d(x; ») ² max x 2 X d(x; »¤) = p So we can check any purported optimum design ² Maximum is at points of support of »¤ SECOND-ORDER POLYNOMIAL ² Model 2 E(y) = ¯0 + ¯1x + ¯2x ² Design Region X x 2 [¡1; 1] ² Compare Two Designs Remember - designs are standardized for n Figure 3: Two designs. Which is better? Two designs -1.0 -0.5 0.0 0.5 1.0 -1.0 -0.5 0.0 0.5 1.0 x Figure 4: Variances for the two designs Variance for two designs 0 1 2 3 4 5 6 -1.0 -0.5 0.0 0.5 1.0 x COMPARISON OF DESIGNS ² Three-Point Design Clearly D-optimum max x 2 X d(x; ») = 3 and occurs at design points ² As found by Smith ² Six-Point Design Not D-optimum { d(x; ») > 3 at x = ¡1; 1 { < 3 at other 4 points OPTIMUM EXPERIMENTAL DESIGN 5 Dependence on n ² Can always (?)