University of Southampton Research Repository ePrints Soton Copyright © and Moral Rights for this thesis are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holders. When referring to this work, full bibliographic details including the author, title, awarding institution and date of the thesis must be given e.g. AUTHOR (year of submission) "Full thesis title", University of Southampton, name of the University School or Department, PhD Thesis, pagination http://eprints.soton.ac.uk UNIVERSITY OF SOUTHAMPTON FACULTY OF SOCIAL, HUMAN AND MATHEMATICAL SCIENCES Mathematical Sciences Optimal Design of Nonlinear Multifactor Experiments by Yuanzhi Huang Thesis for the degree of Doctor of Philosophy February 2016 UNIVERSITY OF SOUTHAMPTON ABSTRACT FACULTY OF SOCIAL, HUMAN AND MATHEMATICAL SCIENCES Mathematical Sciences Doctor of Philosophy OPTIMAL DESIGN OF NONLINEAR MULTIFACTOR EXPERIMENTS by Yuanzhi Huang Optimal design is useful in improving the efficiencies of experiments with respect to a specified optimality criterion, which is often related to one or more statistical models assumed. In particular, sometimes chemical and biological studies could involve multiple experimental factors. Often, it is convenient to fit a nonlinear model in these factors. This nonlinear model can be either mechanistic or empirical, as long as it describes the unknown mechanism behind the response surface. In this thesis, our main interest is in exact optimal design of experiments for nonlinear multifactor models. In order to search for optimal designs, we can use the conventional point or coordinate exchange approach, which however can incorporate a new continuous optimisation method. On the basis of this idea, we further develop and implement a multistage hybrid method to construct local and (pseudo-)Bayesian optimal designs. The recommended hybrid exchange algo- rithm overcomes the shortcomings of the modified Fedorov exchange algorithm and the coordinate exchange algorithm, contributing to improved properties of the experimental designs obtained. In addition, Bayesian optimal design with respect to an expected cri- terion function is based on the assumed parameter prior distributions for the nonlinear model. To limit the time for approximating expected criterion values in the algorithm, we use some efficient numerical integration methods (e.g. a Gauss-Hermite quadrature), which are much superior to the traditional pseudo-Monte Carlo method. We demonstrate the hybrid exchange algorithm by means of several examples relevant to Michaelis-Menten kinetics and other biochemical applications. Under some of these circumstances, we consider hybrid nonlinear models which can be adopted to be fitted to the data of new experiments, the tailor-made optimal designs of which are therefore found and compared with each other. In order to normalise the error structure of such a hybrid model, sometimes the Box-Cox transformation can be applied and the result would be a transform-both-sides (TBS) model. Optimal designs for either untransformed models or TBS models can be used for future experiments, as well as for comprehensive studies of complicated mechanisms. Contents Declaration of Authorship xiii Acknowledgements xv Nomenclature xvii 1 Introduction1 1.1 Experimentation and Response Surfaces...................1 1.2 Basics of Optimal Design of Experiments..................3 1.2.1 Motivation and Concepts.......................3 1.2.2 Computation and Exchange Algorithms...............6 1.3 Preliminaries and Miscellaneous Topics....................9 1.3.1 Nonlinear Experiments.........................9 1.3.2 Extensions of Basic Michaelis-Menten Kinetics........... 11 1.3.3 Parameter Prior Distribution..................... 14 1.4 Outline..................................... 16 2 Optimal Design of Experiments for General Nonlinear Response Sur- faces 19 2.1 Research Problem............................... 19 2.1.1 Nonlinear Multifactor Models..................... 19 2.1.2 Motivating Examples.......................... 20 2.1.3 Statistical D-Criterion for Optimal Design.............. 23 2.1.4 Local Optimal Design for Nonlinear Models............. 24 2.1.5 Need for Exact Design and Algorithm................ 25 2.2 Introduction to Computer Exchange Algorithms.............. 25 2.2.1 Point Exchange Approach....................... 25 2.2.2 Coordinate Exchange Approach.................... 28 2.3 A New Continuous Optimisation Method.................. 29 2.4 Numerical Results and Comparisons..................... 31 2.4.1 Example 1: A Multifactor Mechanistic Model............ 31 2.4.2 Example 2: A Special Empirical Model............... 35 2.5 A New Multistage Exchange Algorithm................... 38 2.5.1 Introduction to the New Method................... 38 2.5.2 General Applications of the Algorithm................ 42 2.6 Examples Revisited............................... 43 2.7 Discussion and Recommendations....................... 46 2.7.1 Nonlinear Multifactor Experiments.................. 46 v vi CONTENTS 2.7.2 Continuous Optimisation....................... 47 2.8 Optimal Design of Experiments in Blocks for Multifactor Nonlinear Models 48 2.9 Optimal Candidates for Optimal Design................... 53 2.10 Appendix: Derivation of the Nonlinear Multifactor Model......... 55 3 Hybrid Nonlinear Models: Applications to Michaelis-Menten Kinetics 59 3.1 Research Problem............................... 59 3.2 Models Based on Michaelis-Menten Kinetics................. 61 3.3 An Adapted Kinetics Example........................ 64 3.4 Local D-Optimal Design of Experiments................... 66 3.4.1 Model 1: the Additive Candidate Model............... 66 3.4.2 Model 2: the Exponential Candidate Model............. 70 3.4.3 Model 3: the Transformed Candidate Model............. 72 3.5 Complex Nonlinear Multifactor Models.................... 74 3.5.1 Practical Complications in Nonlinear Experiments......... 74 3.5.2 Model and Optimal Design of the Experiment............ 75 3.5.3 Modified Adjustment Algorithm................... 79 3.6 Local L-Optimal Design of Experiments................... 80 3.6.1 Numerical Results under the Local Weighted A-Criterion..... 80 3.6.2 General Applications of the Local L-Criterion............ 88 3.7 A Simple Compound Criterion........................ 89 3.8 A Nonlinear Model with a Categorical Variable............... 94 3.9 A General Discussion of Advanced Kinetics................. 97 4 Bayesian Optimal Design of Nonlinear Multifactor Experiments 101 4.1 Expectation of the D-criterion Function................... 101 4.1.1 Local Optimality Criterion...................... 101 4.1.2 The Pseudo-Bayesian Approach.................... 102 4.2 Deterministic Gauss-Hermite Approximation................ 105 4.3 Determination of a Reliable Gauss-Hermite Sample............. 109 4.4 Numerical Investigation............................ 113 4.4.1 Model and Parameters......................... 113 4.4.2 Initial Gauss-Hermite Quadrature Results.............. 114 4.4.3 Pseudo-Monte Carlo Approximation Results............. 118 4.5 Initial Optimal Design Results........................ 119 4.5.1 A Multistage Exchange Algorithm.................. 119 4.5.2 Mutual Comparisons and Robustness of Optimal Designs..... 122 4.6 A Combination of Normal and Lognormal Priors.............. 127 4.7 Final Results in the First Example...................... 131 4.8 Approximation of the Spherical-Radial Transformation........... 133 4.9 A Follow-Up Example in Optimal Design.................. 137 5 Model Transformation under Michaelis-Menten Mechanisms: Optimal Design of Experiments for Transform-Both-Sides Models 143 5.1 Review of the Michaelis-Menten Equation.................. 143 5.2 More Assumptions under Michaelis-Menten Kinetics............ 147 5.3 Other Factors in Association with the Initial Rate............. 151 CONTENTS vii 5.4 Measurement Errors in the Experiment................... 153 5.5 Residuals in Nonlinear Least Squares Estimation.............. 155 5.6 Empirical Kinetic Model and Box-Cox Transformation........... 160 5.7 Optimal Design of Experiments........................ 164 5.8 A Simple Hybrid Nonlinear Model Example................. 165 5.8.1 Estimation of Box-Cox Transformation Parameter......... 165 5.8.2 Lack of Fit and Pure Error...................... 168 5.8.3 Optimal Experimental Design Results................ 170 5.9 A Kinetic Model with a Categorical Variable................ 174 5.10 Precise Estimation of the Transformation Parameter............ 176 5.11 Discussion of Model Transformation..................... 180 6 Conclusion 181 6.1 State of the Art................................. 181 6.2 Areas for Future Development......................... 184 References 187 List of Figures 1.1 Defined Discrete Candidate Points in a Square Region...........7 1.2 Information to Know before Optimal Design for Nonlinear Models.... 11 4.1 Local and Bayesian D-Optimal Designs of the Same Experiment..... 126 4.2 Local D-Optimal Designs under Various Postulated Scenarios....... 128 5.1 Residuals Against Fitted Response Values Under: (a) the Untransformed Full Treatment Model (α = 1); (b) the Full Treatment Model....... 168