Volume 116, Number 1, January-February 2011 Journal of Research of the National Institute of Standards and Technology [J. Res. Natl. Inst. Stand. Technol. 116, 539-556 (2011)] Maximum Likelihood and Restricted Likelihood Solutions in Multiple-Method Studies Volume 116 Number 1 January-February 2011 Andrew L. Rukhin A formulation of the problem of considered to be known an upper bound combining data from several sources is on the between-method variance is National Institute of Standards discussed in terms of random effects obtained. The relationship between and Technology, models. The unknown measurement likelihood equations and moment-type Gaithersburg, MD 20899-8980 precision is assumed not to be the same equations is also discussed. for all methods. We investigate maximum likelihood solutions in this model. By representing the likelihood equations as Key words: DerSimonian-Laird simultaneous polynomial equations, estimator; Groebner basis; [email protected] the exact form of the Groebner basis for heteroscedasticity; interlaboratory studies; their stationary points is derived when iteration scheme; Mandel-Paule algorithm; there are two methods. A parametrization meta-analysis; parametrized solutions; of these solutions which allows their polynomial equations; random effects comparison is suggested. A numerical model. method for solving likelihood equations is outlined, and an alternative to the maximum likelihood method, the restricted Accepted: August 20, 2010 maximum likelihood, is studied. In the situation when methods variances are Available online: http://www.nist.gov/jres 1. Introduction: Meta Analysis and difficult when the unknown measurement precision Interlaboratory Studies varies among methods whose summary results may not seem to conform to the same measured property. This article is concerned with mathematical aspects Reference [6] provides some practical suggestions for of an ubiquitous problem in applied statistics: how to dealing with the problem. combine measurement data nominally on the same In this paper we investigate the ML solutions of the property by methods, instruments, studies, medical random effects model which is formulated in Sec. 2. By centers, clusters or laboratories of different precision. representing the likelihood equations as simultaneous One of the first approaches to this problem was sug- polynomial equations, the so-called Groebner basis gested by Cochran [1], who investigated maximum for them is derived when there are two sources. A likelihood (ML) estimates for one-way, unbalanced, parametrization of such solutions is suggested in heteroscedastic random-effects model. Cochran re- Sec. 2.1. The maxima of the likelihood function are turned to this problem throughout his career [2, 3], and compared for positive and zero between-labs variance. Ref. [4] reviews this work. Reference [5] discusses A numerical method for solving likelihood equations applications in metrology and gives more references. by reducing them to an optimization problem for a The problem of combining data from several sources homogeneous objective function is given in Sec. 2.2. is central to the broad field of meta-analysis. It is most An alternative to the ML method, the restricted maxi- 539 Volume 116, Number 1, January-February 2011 Journal of Research of the National Institute of Standards and Technology σ^ 2 mum likelihood is considered in Sec. 3. An explicit i = 0. In order to find these estimates one can replace formula for the restricted likelihood estimator is μ in (2) by discovered in Sec. 3.2 in the case of two methods. −1 Section 4 deals with the situation when methods ∑ x ()σσ22+ μ = i ii, (3) variances are considered to be known, and an upper σσ221+ − ∑ ()i bound on the between-method variance is obtained. i The Sec. 5 discusses the relationship between likeli- which reduces the number of parameters from p +2 to hood equations and moment-type equations, and p +1. Sec. 6 gives some conclusions. All auxiliary material Our goal is to represent the set of all stationary points related to an optimization problem and to elementary of the likelihood equations as solutions to simultaneous symmetric polynomials is collected in the Appendix. polynomial equations. To that end, note that −=μ 22 − 2 2. ML Method and Polynomial Equations ∑∑∑∑wi() x i wx ii ( wx ii )/() w i iiii(4) =−2 To model interlaboratory testing situation, denote by ∑∑wwij()/(). x i x j w i 1≤<ijp ≤ i ni the number of observations made in laboratory i, i = 1, ..., p, whose observations xij have the form This formula, which easily follows from the Lagrange identity [7, Sec. 1.3], will be used with xb=++με ij i ij . (1) σ 2 σ 2 −1 wi =( + i ) . We introduce a polynomial P of degree p in σ 2, Here μ is the true property value, b represents the i p method (or laboratory) effect, which is assumed to be PP==+=(,,,)σσ22 σ 2∏ ( σ 2 σ 2 )∑ σ 2 (5) σ 2 ε 1 pip− normal with mean 0 and unknown variance , and ij i =0 represent independent normal, zero mean random τ 2 ==σσ22 errors with unknown variances i . with (1 , ,p ), 0,1, ,p , denoting the -th ∑ For a fixed i, the i-th sample mean xi = j xij /ni is elementary symmetric polynomial. Another polynomial normally distributed with the mean μ and the variance of interest isQQ= (σσ22 , , , σ 2 ) having degree σ 2 + σ 2, where σ 2 = τ 2 /n . If the σ’s were known, then 1 p i i i i − σ 2 the best estimator of μ would be the weighted average p 2in , σ 2 σ 2 of xi with weights proportional to 1/( + i ). Since σσ22 σ 2=− 2 σ 2 + σ 2 these variances are unknown, the weights have to be Qxx(,,,)1 pijk∑ ( )∏ ( ) 2 σ 1≤<ijp ≤ kij≠ , estimated. Traditionally to evaluate i one uses the 2 ∑ − 2 p−2 classical unbiased statistic si = j (xij xi) /(nivi), 22 (6) 2 2 ==−σ v = n −1, which has the distribution σ χ (v )/v . Since ∑∑qxxPpijij−−2 (), i i i i i =≤<≤ 2 01ijp statistics xi , si , i = 1, ..., p, are sufficient, we use the ∂2 likelihood function based on them. wherePP= (σσ22 , , , σ 2 ) is a polynomial σ 2 σ 2 ij∂∂σσ22 1 p The ML solution minimizes in μ, , i , i = 1, ..., p, ij the negative logarithm of this function which is inσσσ222 of degreep − 2 which does not depend on , , proportional to ij = σσ22 σσ 22 andqq (11 , ,pp )isa multilinear formin , , . ()xvs− μ 22 L =++∑∑iii Since σσ22+ σ 2 ii (2) P′ ∂ 1 ii ==logP (σσ22 , , , σ 2 )∑ , σσ22++ σ 2 P ∂+σσσ2221 p (7) ∑∑log(iii )v log . k k ii the identity (4) can be written as σ^ 2 ()x − μ 2 QP/ Q It follows from (2) that if i is the ML estimator of ∑ k ==. (8) σ 2 σ^ 2 σσ22+ i , then i > 0. However, it is quite possible that k k P'/ P P' 540 Volume 116, Number 1, January-February 2011 Journal of Research of the National Institute of Standards and Technology σ 2 The negative log-likelihood function (2) in this variables, say, 2 . With n = n1 + n2, f = n + n2, σ 2 − 2 υ σ 2 − 2 2 − 2 notation is u = 1 /(x1 x2) , = 2 /(x1 x2) , z1 = v1s1 /(x1 x2) 2 − 2 and z2 = v2 s /(x1 x2) , under lexicographic order, 2 2 2 2 Q vs 2 σ σ LP=+log +∑∑ii + v logσ . (9) 1 > 2 , the Groebner basis for equations (13) written σ 2 ii P' iii in the form 22 2 +− +υ = Π σ +σ 2 uznuu()()0,11 Let Pi = k≠i ( k ) be the partial derivative of P (14) σ 2 with regard to i ; denote by Qi the same partial deriv- ′ ∑ 22 ative of Q and by Pi = j:j≠i Pij the derivative of υυυ+−()()0,zn u + = (15) ′ (σ2, σ2,…,σ 2 22 P p ). By differentiating (9), we see that the stationary points of (2) satisfy polynomial equations, consists of two polynomials, Gu(,υ )=++++ cu25 c υυυυ c 4 c 3 c 2 , σσ42+−+ σ 2 σ 4 2 101234 (16) iiiii()()()QP' QP' P' (10) cnzfz=++23[(1) nz ], +−vs()()0.σσσ222 + 2 P' = 012 1 12 ii i i =−23 + + 3 − 23 c121121222 nnfz nnn(3) nnz nnf , σ 2, =−32 + 32 + + 23 + Each of these polynomials has degree 4 in i cnnfznnnnnnnnzz22121121221222(442) 2 2 i =1,…, p. When σ = 0, P = ∏σ , and the equations 22222 i −+++nn(46) n n nn n z f n n z (10) simplify to 1121222 121 −++43 22234 + − n(5 n112121222 5 nn 8 nn 4 nn 4 n ) z 222 +++22 2 σσP()() Q P'−+ QP' P P' fn21(2 n nn 12 2), n 2 ii i ii (11) +−σ 22 2 = cfnznnnnnnnzz=−3332 +(4 + + 5 232 + 4) vsPP'ii()()0. ii 321211212212 −+43 + 22342 + + (2nnnnnnnnzz1121212212 8 12 7 2 ) These polynomials have degree 3 in σ 2. If σ2 > 0, in i +−+++33(3106)nnn23 z f n 32 z n 3 n n 22 n nn 3 zz 2 addition to (10) one has 12 2 21 1 2 1 2 12 12 ++−222 nn21(5 n 7 nn 1222 2 n ) z =+−′′′′′3 = (12) FP() ( QPQPP ) 0, ++++−−332233 3(264),fnz21 n 2 n 1 nn 1 2 nn 12 n 2 z 2 fn 2 2 =−22 + + and F has degree 3p−3 in σ . cnzzfznz4112112[(1) ], In both cases the collection of all stationary points and forms an affine variety whose structure can be studied υ=++++ υυυυυ5432(17) via the Groebner basis of the ideal of polynomials (11) Gu201234(, ) du d d d d , or (10) and (12) which vanish on this variety.