OPTIMAL DESIGN FOR MULTIPLE RESPONSES WITH VARIANCE DEPENDING ON UNKNOWN PARAMETERS Valerii Fedorov, Rob ert Gagnon, and Sergei Leonov GSK BDS Technical Rep ort 2001-03 August 2001 This pap er was reviewed and recommended for publication by Anthony C. Atkinson London Scho ol of Economics London, U.K. John Peterson Biomedical Data Sciences GlaxoSmithKline Pharmaceuticals Upp er Merion, PA, U.S.A. William F. Rosenb erger Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, MD, U.S.A. c Copyright 2001 by GlaxoSmithKline Pharmaceuticals Biomedical Data Sciences GlaxoSmithKline Pharmaceuticals 1250 South Collegeville Road, PO Box 5089 Collegeville, PA 19426-0989 Optimal Design for Multiple Resp onses with Variance Dep ending on Unknown Parameters Valerii FEDOROV, Rob ert GAGNON, and Sergei LEONOV Biomedical Data Sciences GlaxoSmithKline Pharmaceuticals Abstract We discuss optimal design for multiresp onse mo dels with a variance matrix that dep ends on unknown parameters. The approach relies on optimization of convex functions of the Fisher information matrix. We prop ose iterated estimators which are asymptotically equiv- alent to maximum likeliho o d estimators. Combining these estimators with convex design theory leads to optimal design metho ds which can b e used in the lo cal optimality setting. A mo del with exp erimental costs is intro duced which is studied within the normalized design paradigm and can be applied, for example, to the analysis of clinical trials with multiple endp oints. Contents 1 Intro duction 4 2 Regression Mo dels and Maximum Likeliho o d Estimation 4 3 Iterated Estimators and Combined Least Squares 7 3.1 Multivariate linear regression with unknown but constant covariance matrix . 9 4 Optimal Design of Exp eriments 10 4.1 Dose response model . 11 5 Optimal Designs Under Cost Constraints 12 5.1 Two response functions with cost constraints . 15 5.2 Linear regression with random parameters . 18 6 Discussion 20 7 App endix 22 1 INTRODUCTION 4 1 Intro duction In many areas of research, including biomedical studies, investigators are faced with multire- sp onse mo dels in whichvariation of the resp onse is dep endent up on unknown mo del parameters. This is a common issue, for example, in pharmacokinetics, dose resp onse, rep eated measures, time series, and econometrics mo dels. Many estimation metho ds have b een prop osed for these situations, see for example Beal and Sheiner (1988), Davidian and Carroll (1987), Jennrich (1969), and Lindstrom and Bates (1990). In these mo dels, as in all others, optimal allo cation of resources through exp erimental design is essential. Optimal designs provide not only statis- tically optimal estimates of mo del parameters, but also ensure that investments of time and money are utilized to their fullest. In many cases, investigators must design studies in which they are sub ject to some typ e of constraint. One example is a cost constraint, in which the total budget for conducting the study is limited, and the study design must b e adjusted not only to ensure that the budget is realized, but also to ensure that optimal estimation of parameters is achieved. In this pap er, we intro duce an iterated estimator which is asymptotically equivalentto the maximum likeliho o d estimator (MLE). This iterated estimator is a natural generalization of the traditional iteratively reweighted least squares algorithms. It includes not only the squared deviations of the predicted resp onses from the observations, but also the squared deviations of the predicted disp ersion matrix from observed residual matrices. In this way, our combined iterated estimator allows us to construct a natural extension from least squares estimation to the MLE. Weshowhow to exploit classic optimal design metho ds and algorithms, and provide the reader with several examples which include a p opular nonlinear dose resp onse mo del and a linear random e ects mo del. Finally, a mo del with exp erimental costs is intro duced and studied within the framework of normalized designs. Among p otential applications of this mo del is the analysis of clinical trials with multiple endp oints. The pap er is organized as follows. In Section 2, weintro duce the mo del of observations and discuss classic results of the maximum likeliho o d theory. In Section 3, the iterated estimator is intro duced. Section 4 concentrates on optimal design problems. In Section 5, the mo del with exp erimental costs is studied within the normalized design paradigm. We conclude the pap er with the Discussion. The App endix contains pro ofs of some technical results. 2 Regression Mo dels and Maximum Likeliho o d Estimation In this section, weintro duce the multiresp onse regression mo del, with variance matrix dep end- ing up on unknown mo del parameters. Mo dels of this typ e include rep eated measures, random co ecients, and heteroscedastic regression, among others. We also present a brief review of maximum likeliho o d estimation theory, concluding with the asymptotic normality of the MLE. Note that the MLE for the regression mo dels describ ed herein do es not yield closed form solu- 2 REGRESSION MODELS AND MAXIMUM LIKELIHOOD ESTIMATION 5 tions, except in the simplest of cases. It is necessary, therefore, to resort to iterative pro cedures, and to rely on the convergence and asymptotic prop erties of these pro cedures for estimation and inference. Let the observed k 1vector y have a normal distribution and E[y jx]= (x; ); Var [y jx]=S (x; ); (1) T where (x; )=( (x; );::: ; (x; )) ; S (x; )isak k matrix, x are indep endentvariables 1 k m (predictors) and 2 R are unknown parameters. In this case the score function of a single observation y is given by n o @ 1 T 1 log jS (x; )j +[y (x; )] S (x; )[y (x; )] ; R (y jx; )= 2 @ and the corresp onding Fisher information matrix is (cf. Magnus and Neudecker (1988, Ch. 6), or Muirhead (1982, Ch. 1)) m ; (2) (x; )=(x; ; ; S )=Var[R (y jx; )] = [ (x; )] ; =1 " # T @ (x; ) @S(x; ) @(x; ) 1 @S(x; ) 1 1 1 (x; )= tr S (x; ) S (x; ) + S (x; ) : @ @ 2 @ @ In general, the dimension and structure of y; , and S can vary for di erent x. To indicate this, we should intro duce a subscript k for every x , but we do not use it, retaining the traditional i i notation y ; (x ;) and S (x ;) if it do es not cause confusion. The log-likeliho o d function L i i i for N indep endent observations y ::: y can b e written as 1 N N n o X 1 T 1 log jS (x ;)j +[y (x ;)] S (x ;)[y (x ;)] ; (3) L ( )= i i i i i i N 2 i=1 and the information matrix is additive in this case, i.e. N X ( ) = (x ;): N i i=1 Anyvector which maximizes the log-likeliho o d function L ( ), N N = arg max L ( ) (4) N N 2 is called a maximum likeliho o d estimator (MLE). Weintro duce the following assumptions: Assumption 1. The set is compact; x 2X where X is compact, and al l components i in (x; ) and S (x; ) arecontinuous with respect to uniformly in ,with S (x; ) S where 0 S is a positive de nite matrix. The true vector of unknown parameters is an internal point 0 of . P Assumption 2. The sum f (x ;; )=N converges uniformly in to a continuous i i function (; ), N X 1 lim N f (x ;; )= (; ) ; (5) i N !1 i=1 2 REGRESSION MODELS AND MAXIMUM LIKELIHOOD ESTIMATION 6 where h i 1 f (x; ; ) = log jS (x; )j +tr S (x; ) S (x; ) + T 1 +[ (x; ) (x; )] S (x; ) [ (x; ) (x; )]; and the function (; ) attains its unique minimum at = . Following Jennrich (1969, Theorem 6), it can be shown that under Assumptions 1 and 2, the MLE is a measurable function of observations and is strongly consistent; see also Fedorov (1974), Heyde (1997, Ch. 12), Pazman (1993), Wu (1981). Condition (5) is based on the fact that n o T 1 E [y (x; )] S (x; )[y (x; )] = h i T 1 1 = [ (x; ) (x; )] S (x; )[ (x; ) (x; )] + tr S (x; ) S (x; ) ; and the Kolmogorovlaw of large numb ers; see Rao (1973, Ch. 2c.3). If in addition to the ab ove assumptions all comp onents of (x; ) and S (x; ) are twice di erentiable with resp ect to for all 2 , and the limit matrix N X 1 (x ; )= lim N M ( ) (6) i N !1 i=1 exists and is regular, then is asymptotically normally distributed, i.e. N p 1 N ( ) N(0; M ( )): (7) N N Note that the selection of the series fx g is crucial for consistency and precision of : i N N Remark 1. Given N and fx g , a design measure can b e de ned as i 1 N X 1 (x); (z )=f1 if z = x; and 0 otherwiseg : (x) = x x N i N i=1 If the sequence f (x)g weakly converges to (x), then the limiting function (; ) in the N \identi ability" Assumption 2 can b e presented as Z (; )= f (x; ; )d (x); cf. Malyutov (1988). Most often, within the optimal design paradigm, the limit design is a discrete measure, i.e. a collection of supp ort p oints fx ; j =1; :::; ng with weights p , suchthat j j P p = 1; see Section 4. j j 3 ITERATED ESTIMATORS AND COMBINED LEAST SQUARES 7 3 Iterated Estimators and Combined Least Squares If the disp ersion matrices of the observed y are known, i.e.