ESTIMATION FOR THE HERMITE DISTRIBUTION WITH SPECIAL REFERENCE TO TIME SERIES.
by R. Sym
Thesis presented for the degree of Doctor of Philosophy in the Faculty of Science of the University of London
Statistics Imperial College London.
February, 1971. The multivariate Hertite distribution'io'presented as a possible-, alternative- to the tultivariatp.Normaldistribution as a-time series because of - the lackof auchfalternativosat present, given some data tone -wonld.probably-prefer to.try several transformations ti to obtain as good a fit'-to_the.No#al distribution-se'possibley However,a singliaalternatiVe is not much use in itself. The importance of the _multivariate Fermite.distribution is as a'represoniativ of-the class' Of distributions defined- 1n the introduction by '(0.0.11). The problems of maximum likelihood'eetitation discussed here. would be encountered with any distribution of...this,class -for which a'recurrendo relation'provides - the most ConVenient'way.of calculating'the likelihood. It is shown -that maximum likelihood methods are feasiblelor this ''distributiOnot hen-thejean_is small_ and the covariance matrix- . has the foxes of a 9moVing2 aVerage prodessl.:Alowever, in these . circumstances, maximum.likelihbod.estimatesare-ore efficient than .moment estimatesdistribution ifvconetruoted which can be used to - :test al'series for independebee,without-the restriction:on the mean. Results:ars'also presented'on 'the effidiency of"" moment estimates when :using:the. Hermito distributionf-inmultivariste analysis situations, .because-thay, are ueeful'in'thetselves-and because they are helpfUl'in' 'explaining the, corresponding-results obtained for time series situations. Chapter The MUltivania e Hermits Distribution 1.1 Definition and notation 1.2 Moments 1 3 A recurrence relation
1.4 Asymptotio normality
1.5 A derivation of the multivariate He te distribution
1.6 The derivatives of the likelihood Chapter .2 Naltivariate Analysis 2.1 The method of maximum likelihood . an out- line 2.2 On rounding errors 2,3 Properties of the maximum likelihood esti— mates 2.4 , The effioienty of the method of moments Chapter 3 Time Series Analysis 3.1 The situation amenable,to simulation 3 • 2 The simulation experiment using the "moving average" model 3.3 A distribution with the'llarkov property 3.4 The simulation experiment using the Markov process model
Ohapter. 4 Some remarks 4.1 On the generalised l'oieson distribution 4.2 A aubromtine to compute the likelihood 4.3 Alternative distributions There of distributions of discrete valued °fuller r dox cables having probability generating functions of the form
(0.040)
and sno, a as generalised Poisson distributions; see Kemp and Kemp (1965). This class has a natural extension to multivariate distributions having p.g.f.'s of the form
exp
A meMber of this class he multivariate Berate distri- bution, the definition some properties of which are given in Chapter 1 The problems of parametric estimation for this distribution in multivariate and time series situations are investigated in Chapters 2 and 3 respe vely with speeial reference to comparing methods based on max- imum likelihood with, methods using sample moments. Chapter 1. The KultivariatawaggAgLaatatd191
1.1 Definitions and Notation
The multivariate Berate distribution is defined as the distribution of an n dimensional discrete valued random variable X with probability generating function (p.g.f.) r, ..- 11.4-T = G(a) = > > r ) r n=0
= exP [ aisi(1-zi)(1-°j) ] i=1 j=1 0.1.1) Here etc. are n dimensional column vectors with elements for example zi,...,an, and 2: is an nxn matrix with typical element cii. For future reference, 0, and e will denote nx1 vectors with elements 1, 0, and 6m+i respectively, and P(X = r ; ) will be written P(r) , or P(r i 4 o 2: ) .
Put then (1,1.2)
G( ) = exp E from which it can be seen that tie series expansion of G(g) in powers of zit...vs converges for all JE such that 1 I < a). Hence, from (1 .1)
(m G(4) = T1r) r1 4 r =0 and if all the elements of h and z are positive all the coefficients in the expansion will be positive too, so that G(& is a p.g.f. under these restrictions at least. Prom section 1.3 , P(e ) = )P(0), so the condition kl Z 0 m = is necessary; however in view of the derivation of this distribution given in section 1.4 the other condition oij > 0 itj = 1,...,40.1.4)) may'not be necessary. The coefficient of misj in the exponent of (1.1.1) is (0 + o ), so 2= is taken as symmetric to avoid 4 A non uniqueness of parameters, and it will be assumed in the folloWing that the elements of , a 21, are all natnegative. .5.
1.2 Moments
Because the Power series expansion tor G() converges for all Ai, all the moments exist. any factorial moment being given by
where 8
The distributions, moment generating function
ag G(e from which may be obtained
and Coy (x, j
It io interesting to see directly that the conditions (1.1.3) and (1.1.4) imply that the oovarianoe matrix (filiesij + crip) is non negative definite.
Per using •3)
InZ (11 i$4=1
11
with equality ;MIL xis 0) Its last ie non negative as a" 2 0 by 0.1.0 . 1.3 Rec Relation
ce careno P(z) la may I obtained as calms; put a(a) 0g G(a)
+ z'1 oittra Maio e
•*e n e(4)
• I en non 0) •
Now r P(1) C ai rt •en n a(a)] ao that (r T ( ) = itm• P(I) + isx1
040) for all r, where P(E) = 0 if any ri < tJ
1.4 Acymptt tic Normality
mn Let be a seamen** of independent, den oally- distributed tam variables, each with p.g.f.
vi(1.6z ) 4411 j( z Xi.z )1 14.0
t X,s1. X io ...n P
Then using the -d tonalcentral I
Limit theorem (Wilk. (1962) Emotion 9.2(o,)) X. i asymptotically formally distributed, ae m a) with mean m , and covariance matrix (( rlaij + or 4 )) • But has p.g.f /1 al(, )] exp I. vi( (1-z4 )(i i=1 which is (1.1.1) with g replaced by my., and replaced by m ( (ti )) . Thus if in (1.1.1). for a parameter the ratios =1,0,.in i and 11al, t j=1 tiosbtn , are fixed, and µ 4,011), the distribution approaohes normality. The importance of this result is that it suggests that for some range of the parameters the first and second order sample moments will provide estimates which will contain most of the information of the sample, and will thus' have good efficiency. -11-
1.5 A Derivation of the Multivariate Hermite Distribution
• Let have the Poisson distribution with mean vi i=ia•••tri , and I have the multivariato normal distribu- tion with mean lit covariance matrix r probability element ; w ), and suppose that Xi is=1 * 0.. ,n are statistically independ.ant for given / • Define
+00 +00 n.
i 1 00 vnas w 1.9.1) for re 0* 2# ... imei
Now for any r ,
+ CO n 2.( vi( = Tr (e i=1 v1= = where X < so that
I IP z1 1...
r rn=0 converges absolutely for all a such that zi < 1 _ • • •n o,
-1 2-0 and summation and integration may be interchanged in the following expression for G*(i) provided that the resultant integral converges:
+
' * • * • • r
+ CO + co n -v, TT (e dN( I ,Z) i=1 vnzi - co
dN(v
CO
exp [ which is the G(a) of (i.i.i) 0 Hence if(E) . is the probability distribution function of the multivariate Hermite distribution. -13-
The initial definition. of 'P*(z) :cannot taken as the . definition of' a probability :distributiOn, function because the variates vi, $ 40:1 tri on ihe"..one hand are normally diitributid.andvary between '4- ca,and. a)i but on the other band are taken as *ant of Poisson, variatos$ and thus ought to be nonivegativeiooimientiy it is. not' obvious that e(E)has.the properties- of a listribution function. -An alternative conitruction on these lines would. be to'use a runcatad nOrMaldiotributiOno - withwith i 0., , i=1,...in $ but this'Oes'nct give a very conVenient result. The univariate form of this derivation can.be,found in Kemp and ,temp (1966). 0.5.0 can be written in the form -
+ to P( z) = P(0) 12: cci n which will be of use later on. -14-
1 6 The Derivatives of:the Likelihood
Pu cap t tn 4. Ws) * the oo fio eat of 1.40 6, st 44A illpirom, then expressions or thederivatives of the `likelihood may be obtained formally by .(.1.6.1)- where is one of the parameters in it or 27 . can be established by an inductive argument using the ordering of the vectors r given in section 2.1, it is unlikely that a general theorem may be applied here as it not Possible to show that 8110
• • • converges Until (1.6.2) is stablished.
Now n acid 'Il. (1 14°1
rn in which the coefficient of zI . s
0P(z) Ogi Z-7n .84.5441[P(C-SCAP LP(z-ig (0) ow itP1
At = 0 9 8P(2)* P(9)
Suppooe now that (1.6.1) has been ea abliahed for all the Ple on the right hand side 'of (1..3.1)* then from (1.3.1)
8P(Et) 81)(02 bt.
ap(z) 11"Tr. — + P(r) EP(C"*.) - P(1)]
ap(r) -16-
)
.PO) +P (r ai)+(E.-
)P(z+sui )-timP(m.4.4).(rm+1 )P(z+sm)+IiraP(x)
p( E )
+1 )P(r+e P(r) 42+1 )P(z+1141-,ei)+1.1 P
(rmil )P(1.-1- +limp(
P(r) -17-
So (1.6,1) i.e eetabliehed for r replaced by rill, . It will be shown in section (2.1) that for given the vectors E such that 0 S xi 21 ti isgi,...,n can be put in an order starting from 0 so that the :vectors of the right hand side of (10'1) are always lower in the order than the vector of the left hand side. Hence the above argument establishes (1.6.1) for any =18-
Chapter 2* Fiultivariate Analysis
Given a simple random sample frowthe:multivariate. Hermits distributian'there are two main methods of estimating; the parameters, using sample moments or maxim imum likelihood. For the pUrpose of'oomparing these methods the first and second sample memente'were chosen because they provide unbiased setiMatwof thilvparaMeters of (,1.1.1)' , the sample means'are asymptotically fully efficient estimators of the population means, and because the result of section 1.4 suggests that for some,range of the parameters they will. have reasonable efficiency. If the efficiency of this method. of memento can be appre. eiably improved by using for example,' leading frequencies, then the efficiency improvement can be compared with the reeults obtained here.: .The method:ofrmaximum likelihood, .is. computationally the ,moredifficult so the maximum likelihood equations. do not seem to have cleied solutions, 'see Kemp, and Kemp (1965), however itean be shown that , these estimates are asymptotically fully efficient. A procedure by which maximum likelihood estimates may be. obtained is outlined in section 2.1 , 'in particular the possible use of,the reourrence'relation is investigated. -19-
Properties of the method of maximum likelihood are given in section 2.3 and in section 2.4 the effi— ciency of the method of moments is investigated with the univariate and bivariate cases as examples. -20-
2.1 The Method of Maximum Likelihood - an Outline
As solutions to the maximum likelihood equations cannot be found, it is necessary to'be able to cal- culate the value of the function P( ,5E) for • given E, and 'Z This can always be done using (1.1.1) or by evaluating the integral (165.2) however a method better suited to automatic computa.. tions is to use the - recurrence relation (1.3.1) . To see how the recurrence relation may be used , put
Yi = 1 i -1 170+r ) 1=2 3 • • • 01+1 J=1
1(1) = 1 + iYi i=1 -21- for a ouch that @ < ei < pi ..11401 ei.014) when
1(0 = I
1(X) = and it' 1(4) 1( EPor if e
riyi
Vence I(e) Pt 1( ) Similarly it may be shown that
and so on.] -22-
Now suppose that :for some, integer t ,,P(2) is known for every such that 1(s) 6 t Let st and ,m be such that P(Iimtl) = t+1 than in (1.3•1) With instead of r the right band oide involves 10e) 711) , for whioh
= :t4.1 yi
t
and 1(27-2i) t+i
< t =tipsiegn
Hence the function 1(s) defines an order in which the P(s)to are to be calculated which gives the value of P(E) after y124;i 'stews, starting from
P(0) exP (- >(11 )3 i=1 -23-
This may be implemented on a computer by writing a subprogramme oompute P(r I a, ) , for any n , r '5E" , in which an array of length g o say, has to be set aside for the' 1(s)'a" . Now the subprogramme can only be, Used for, an r -for which q yrol(x) * and thin is a restriction on the method because the array's length, g o cannot be indefinitely large. (A further restriotion is that in a long aeries of steps rounding errors could cause the method to give seriously inaccurate results, that this can. be controlled is shown in the next section.) Suppose for example that
i=lp•os•pn
are the (functionally independent) parameters in a par ticular applioationo and that the observations are
p • ••tn. Then the maximum likelihood equations are
i=1*.$10.*n
• • so Fi ; For - 2 4-
where
L = > log P( ; • • k=1 Using (1.6.2)
114-2.1) PLE ) ialip000tn
k=1
• t• • • ins (2.1.2)
In calculating P(Ek) k=1,...,m by the recurrence relation the values of P(Ek-li) and P(Ek-,¢rli) illjois•••011. k=1,...,m are also calculated, to the values of the first order derivatives of the log like- lihood may bps readily obtained. The values of the second order derivatives may be obtained similarly. Hence any of the standard numerical procednres of finding the max- imum of a function using either no derivatives, or first order, or first and second order derivatives may be used. -25-
The univariate case is considered by Kemp and Kemp (1965) and here it is recommended that the Newton- Rapheon method using first and second order derivatives be used. -26-
2.2 On Rounding\Etrors
The sequence of P(s)is that will be given by a numerical procedure using the recurrence relation as suggested in Emotion 2.4 can be defined by
i=i
where e(s+ Is the rounding error for this step. The value of e(s42m) will depend on the parameters
.and f the accuracy of the computer being used. and the, order, in which the computations are performed in (2.2.1). Also the magnitude of c(14-ew) will depend on the magnitude:: of 14(s4201), for on a computer "Rateni) will be recorded as a floating point number. of the form so if q is the number of operations 0.a a21• .a1 x 10- 1 (suoh as one multiplication) to be performed in (2.2.1) -27-
13*(1,141m)
So o obtain a bound or the errors, it will be assumed, that t(s) P PUT may be 'taken as a sequenoe of dependent random variables, with, expected value zero. These assumptions can be instified for the. 41)145 as some random number genera. tors are similar to (2.2.1), and in a single operation the rounding error will have expected value zero. To justify them for the t(2)10 it is noted that for a given , e(1) and c(& will be independent,' as e(g) is the iinsignificanti, unrecorded part of PI( ), If it is further assumed that the IWts 'are uniformly► distributed between 17 10_ I 2 t( )2
i0.41 however this need be used only to get some idea of the magnitude of the final bound. -28-
P0
testa*. of the error (2.2.4) and (1•3•41)
00 2- viz 414,)]
)]
) • Z T(Lwitti)21, 1.4, •013
(2 24
-29-
Choose k 3 1 , and suppose that
>
i=
(2.2.3)
then • E e < B 'if
2 (2.2..4 ) its +1) ,re 2.3) *20- - which implies that
which implies (2.2.4) whenever
Hence calculating p(a) using (1 .3.1) one could choose k such that B = (f)2( is an
acceptable upper bound. to E ka) and then at Pm ' every step for which < k /T. this bound would have
•to be increased to 40, Cf12. This gives a bound which can be calculated, and which'is•of the order of v2, and so in principle provides a method of controlling the. rounding errors. upper :'bound -31
2.3 Properties of the Maximum Likelihood Estimates
(a) Consistency The maximum likelihbod estimates will be shown to be consistent by demonstrating that the assumptions I to 8 of Wald (1949) are satisfied by this distribu— tion. Assumption i requires that the distribution is either discrete or absolutely continuous. With a the veotor of parameters of IL and 1 its true value and
)
if v(g t)>1
otherwise assumption 2 requires E log f (Xtg,p) aid E log AL,t) to be finite, where the expectations are with respect to go Beret
11-NE.1...P)2 = v (z,t) all 4rg ptt -32-
a least one (4 1 1) with for *II A $
at A implies that Ni
that
a. autaption 5 Lin i 44011111% 0 *Et* Qoaei o zero with respect to
Qthat to ilij, 440 (1,141,) = P(r;a) -33
Assumption 6 requires that ] log P(X;a0 Assume that for the true parameter point
hx > 0 i=itfo.,01 (2.3.1) (which is necessary to prove the asymptotic normality of the estimates) then using (1.1.1) and cij A 0 all i,j for every r
P(r) > cbefficient of TT zir i i=1 r4 Xi = P(0) . 1=1
Hence CO ) • • 0
log P(0) rilog Xi 2: log riliP( • * i=1
<
as log ri! i2 and the first and second moll:ants exist. -34--
Assumpt on 7 requires that the parameter space be closed; it is necessary to ensure that this assumption is satisfied in an application. Assumption 8 requires that the function f(r,a,p) introduced above is measureable, whiah is true for any discrete distribution. Hence, provided that (2.3.0 and assumptian1= 77. are satisfied, theorem 2 of Wald's paper gives for any e > 0 and 8 > 0 there exists 0) such that
6 ,.m=a,M+, > -' (2.3.2) where a is the vector of parameters in a and 7: a is the true parameter point, and is a point at which the likelihood of m observations is maximised.
(b) Asymptotic Normality
First tt will by shown that the first and seoond order derivatives of the log likelihood are regular, that is -35—
82P(r) = (2,3.4) 8T8y r.=n and
8log ( P(r) Oonvergeo, • • • av ay r1=0 rn=0 (2.3.31 where I and y are any of µi .
(2.3.3) and (2.3.4) can be proved by substituting (1.6.2) directly. Prom (1.3.1)
(rm OP(.. +e > NaP(r) any m,
'so that
P(E Ii) any P(Se) and • amY -3
so that, from (1.6.2) n n. 8logP(z) I 8µ4 r4 1 84.i. at i...i"It ( = + 1 ) r4r4 ri ± „i which gives (2.3.5') as all the moments of this distribution exist, if. Xi > 0 , all i . Now the usual argument, as in for example Wilke (1962) section 12.3(c), may be applied provided that the parameters of interest are functionally independent and that the points --ma of 2.3(b) lie strictly inside the parameter space so that the regularity just proved holds, and so that the maximum of the likelihood coincides with a solution of the maximum likelihood equations. In this case there exists h > 0 such that no a satisfying l< h is on the boundary -a—a of the parameter space: defintddby Xi. 2. 0 ► aij 2„ o r i,J=1,.. Hence choosing 6 < h in (2.3.2) gives the required result.
(0) Efficiency That these estimates are asymptotically fully efficient can now be shown as in Wilke (1962) section 12.3(d). -37-
To. see that the sample means are the maximum likelihood estimators of the parameters when pi aij are the parameters of interest, from (2.1.0-and (2.1.2)
811 • im o• • • on and 81 41114,, •-•.on • give
(2.3.4)
• • • •31 i (2.3.1) Further (1.3.1) may be written in the farm n rk jr( k) = r(4-2-4) aijP(Zit i=1 •4n -38-
Combining (2.3.4) (2.3.*) and 2.3.8) gives
) P(r *111j) ai (4d k=1 k=1 i=1 k=1
m > 1=1 using (1.1.2)
Hence kvi k=1 -39-
2.4 The fficienoy of the Method of Moments
The efficiency of the method of moments was investigated numerically for the univariate and bivariate forms of the Hermite distribution. The univariate Hermite distribution has p.g.f. 2 ' G(z) = exp [ l(1-z) + 0 (1-z) 2 ] for which the parameters p d a2 have moment estimators, m r = > rk k=
2 given a simple random sample of m observations ri p...orm . Only the efficiency of t need be considered' a® . 7 is asymptotically fully efficient. Now, with v = a2
a (a 4- 0.2 g + vi + v + m
-40-
and the lower bound to the. variance of an unbiased estimator of. v = o is.
a. m pp (Ipp Ivv - pv 2
where for example
Ipv = r=0:' The 2(014 were calculated using' the recurrence relation (1.3.1) and the derivatives given by (1.6.2). The infinite sums have to be approximated by finite sums over r=0, .,N say, numerical checks on the, accuracy of the results thus obtained were that
)L P(r) was close to unity, and that increasing r=0 did net change the results appreciably. . The bivariate Hermite distribution has p. . . 2 2 G(z) e p ..r 2- tl (i-zi) 4 2: ci (1 i=1 i j=i have for which the parameters pi (111,a12'a2 moment estimators
= m i=102 ri rkti k=1
=efteratoor
given a simple random sample ffi The measure of efficiently used. was 100. ( 1 41331)-1 (it where A is the covariance .matrix of the moment estimators,
0
2(t 1+011)2
+2011 2(µ2+(7223 012 2111112(//1 Li (112+°2 ) +a 2+a 12 12 2 0 2a 20 2024.022) 22 12 c'12(P2+ 22) 2 +2022 N o( (2.4.I)
and B entries for example asp elogP(r) eilogP( ) bp2 r2 -42--
co ad 8100(r) npg2(r) B = Z __w.a- 45 -12 1022 PIE) • r2=0 The P(r)ts were calculated using (1 3 i), the derivatives using (1.6.2), and similar, numerical ehecks to these in the univariate case:were used. In both, cases'the variance terms 0( 0) were m` dropped, so that the results hold for all m sufficiently large for this to be justified in particu— lar, the results may be used to compare the'efficieney of the method of momenta with the efficiency of the method of maximum likelihood at m = co Por the univariate ease the results are giVea in graphical form, with % efficiency tabulated against for p 1,2,500,20,50000 (table I). For the bivariate case the reaults are given in tabular form,. the % efficiency is given for various values of 0,114022 and 012 :ter' 111=42=20 (table II) o Ili=112=5(table III) and µi=20 42=5 (table IV). -43-
Conclusions
Looking first at the results for the'univariate case, table I, it can be seen that fora fixed value of v increasing µ increases the efficiency of the estimate It' of v to a figure arbitrarily close to 1007: . This agrees with the comment made in section 1.4 after it is shown that the distribution can be approximated by the normal distribution for large µ where garget depends somehow on . Thus table I suggests how this approximation behaves. Now comparing the results for gi=µ2=20 with those for gi=µ2=5 tables II and III' the same pattern can be 049 seen for each value of —4%. • Finally the results for 111
20 5 table IV, show haw the efficiency behaved when gi g Three firm conclusions may be drawn from these results. (i) For all values of 4 or 4.1 and 42, there will be values of v or all'a12'.and 022 for which the method of moments is inefficient. -44-
(ii)If V. or alit air and a22 , are small compart4g valth p , or pi and p2 . the method of momenta can be tolerably efficient. (iii)The recurrence relation may be used to calculate the value of the likelihood in the method of maximum likelihood outlined in section 2.1 for all the values of p or pi and p considered here. -45-
Chapter 3. • Pine Series Analo
Introduotion Strict etationarity and second order etationarity are equivalent for this distribution since the p.g.f. (1.1.1) is determined by the second order, moments. Hence (1.1.1) is the p.g.f. of a stationer* sequence of random variables if any and only if
Pi , i=1po**01 and
=2Pii-ji say, Mi 0 • • * 11,21 where . Equation (1.1.1) may now be written
G(z) = exg - 11 2- (1 ) Pti- )(1-zp 1=1 i,j=1 (3.0.1) In this chapter two possible uses of the distribu- tion defined by (3.0.1) in time series situations are investigated. First use of (3.0.1) as a model for a discrete valued time series is considered, and the efficiency of a method of estimation based on maximum -46- likelihood for a specialcase of (3.0.1). The method of moments chosen usei first and second order sample moments of untransformed data because the result of section 1.4 suggests that these will give estimates with high efficiency for some values of the parameters, and because there is a well established procedure based on them; if it is thought that this method can be improved by ming for example leading frequencies or higher order momenta, then the results of any improvement oan be compared with the results obtained here. The method of maximum likelihood has two major drawbacks. The likelihood is difficult to evaluate, and current theory does not provide the framework by which the consistency and asymptotic normality of the estimates can be established. On the first point, with observations ri pso*,r n the method of calculating the likelihood given in section 2.1 requires an array of length TE(1 r ) which can be exceedingly large i=1 in time series situations. -47-
However, in the special case when the t istribution has a p.g.f. of the form (3.0.i) with '? (3.0.2) pt=0 t > 1 for some 1 * it is possible to use the recurrence relation to calculate the likelihood when the ,mean of the prooess is small, and hence obtain point estimates from a particular data series and information about the distribution of ouch, estimates by simulation; this is shown in section 3.1. Nbw if one is prepared to aesume that the maximum likelihood estimates are approximately normally distributed with a covariance matrix the inverse of which has typioal element
rt 0 1 * * lip 82log PcI X .... X ) ] (3.0.3) n 400 8µ8pi then useful point estimates could always be obtained for a particular series of observations by obtaining the likelihood by expanding out (3.0.1). Unfortunately, (3.0.3) can be oalculated only numerically from the recurrence relation. Hence (3.0.1) can be used only to give useful maximum likelihood estimates of the parameters when the assumption (3.0.2) is appropriate and the•mean of the process is small. Now there will be situationswhen a distribution of the form (3.0.0 is appropriate as an alternative hypothesis against which the hypothesis. that the observations are independent is to be tested. It is possible to construct a distribution fvam (3.0.0 which could be useful in this case, and which does not suffer from the drawbacks of the distribution first considered. This other use of (3.0.1) is expounded in section 3.3. -49-
3.1 The Situation Amenable to Simulation
As mentioned in the introduction to this chapter, the obstacle to be overcome if this distri- bution is to be used in time series situations is the calculation of the likelihood. In the special case when the likelihood is of the form defined by (3.0.2) it is possible to split up the data series into a number of subseries, and to calculate the likelihood from the likelihoods of the subseries. This follows from the following result: with 1 = for example
P X r5-1 ( X8=0 X8+1=roi1 * 0 • III
,..., Xs-1 =r8-1 I Xs =0 (3.1.1)
For consider the .calcUlation of p = 13 X=x, using (1.3.1) when r8=0 , taking as starting value
P X =0 . X =0, /70+1 ***' Xern inbtead of P X=0 as in section 2.1. As cim=0 for li-ml >1 and r8=0, (1.3.1) never involves a term for which the last n-s places in the vectors r or r-ei, i=1,...,n are anything but r0+1,...,rn. -50-
So the sequence of operations which gives p from starting value q would give, for any r$+1 ,...
X WOOO.X8.01 4.1r0.4.1 04.5 0
Xn=i•n from starting alue
220,04,•0 X5=0 Xs+1 =r 0+1' 6 6 ° , ern so that
P = q
imilarly =0, Xs si1W2"6"_ P Xs=0, Xs+ =r
Hence, for r84-1"6 rn
/ =00 Xs l=rsil greip P 8
2. Xs 1;3=0, Xell+1 =r13444
....X3a7tn which is (3 -51
Bow if the aeries ri, 1=1," on has zeros at ictz ig .. • , P(r) may be calculated as k
J=0
where z0=1 and zk+i =n ; and this reduces the length of the array required:to calculate P(r) from
1 r to
(3.1.2)
For 1 > i the data series can be split up at strings of zeros of length 1, this can be shown by a similar argument to that given above for 1 = 1 Another result that, will be used is that the sample mean and the maximum likelihood estimate of g are asymptotically equivalent: taking 1 = 1 again and assuming that the parameters of interest are g a2
o+
iti4Z)a•
')4Ti ur ip It= (2)(TDIZ
ueq.447.14 eq Seta •csi.) M©U
,Z=T c? tp- 7 )o 1 7 A= 1 d
i.eer (x) a)az 'z )a1
7)ct)
TX (a 9, 0 taciax
-z5- -53-
On substitution, the ma'imum likelihood equations give AgA P(r71,1)+2(t-I) = SLan 2 ] where. µ, are the maximum likelihood estimates, and now P(r)mor p, 4;2, p) so that, as in section 2,2
for or xi giving
A tt 23
Hence, if Ap $A 2 and p are consistent estimates df. p a2 and, p 11. is, asymptotically equivalent to r
3,.2 The Simulation Experiment using the 'Moving Average' type Model
Obie_ctives The experiment was designed to investigate the following questions: (1)does the method of maximum likelihood give consistent estimates, (2)does this method give more effictrt estimates than a method based on first and se and order sample momenta?
:Programme The data was produced using a random number generator which provided a series of independent "obser - vations', each uniformly distributed in (OM ul,...,un say. These were then transformed to give a series of observations drawn from the distribution under investigation by finding riposoprn such that
r PCs 1r 4 P(si I r i.".1 000,09 i...1,441.•ri) 8 =0 s =0 i i -55-
The distribution has the covariance structure of a first order moving average process, so that the method using sample moments as given in Walker (1961) was appropriate. With the sample serial correlation coefficients of lags 1,...,k
tI4190.••9k 9
Walkerts method gives, an estimate of the distribution serial correlation coefficient lag one, R., of the form
at(R)Itt (3.2.1) t=2
For k=3
2R,( R4) 02(R) 1+4R4
03(R) and for k=4
c2(R) = 42R(1-2R +3R4-4 )/D 03(R) = R2(3-4R2-7R4)/D 2+R4 04 (R) = -40(1-3R )/D where
= 1-2R24-3R • k=3 aad k=4 being the two valuing used in the programme. This is an iterative method, R is calculated from (3.2.1) with the c (R)ts replaced by their estimates ot(R )Is uter6 R* is the previous estimate of, R.
The initial estimate of R was tw.ken to be R1' The reason for taking two values of k is that the method is shown in the reference to give estimates which are asymptotically fully efficient amongst the class of estimates derived from second order sample moments as both k and n become large: this is realised in practice for a particular n by taking successively larger k until the change in the estimates is small compared with their standard error. The parameter 2 was estimatedWlsg Z2=s2-4E , where
:3 2 = (ri i=1
The maximum likelihood estimates were obtained by maximising the likelihood with respect to a2 and -57-
—2 with 11.7 , taking as initial estimates CY r 2 and = to give estimates of o2 and r 0 and p say. The maximum likelihood estimate of R R , was obtained as
The numerical procedure used was as proposed in Powell (1964), being one not requiring derivatives. It does however assume that the variables are unconstrained, whereas the parameters 02 d p have to satisfy the constraints (1.1.3) d (1.,1.4), which now become
a2 > 0 and
02(1+2p) .
Variables b and b were introduced for the numerical to 1 2 procedure)work with:
2 a2 = Sin2b = Sin2b 2 a2 2 -58-
For each set of parameter values choaen the mean and variance of ^2a , ^R, -2a., -R were estimated by the sample means and variances of the results of 20 (sometimes fewer) independent trials. Now 62 and R do not necessarily satisfy the constraints imposed on the maximum likelihood estimates, so that for the 'purpose of comparing the estimates of their means and variances the moment estimates were corrected to satisfy the constraints
0 < Theconstraint '
R Se..".;2 "..r-r+a1- 1 was not imposed because it is not ear which of Z2 or R has to be corrected. The results are summarised for values of number of observations per series of 60,-100, and 140 in tables V, VI, and VII respeetively. The first two columns of thesetables contain the 'true values of the parameters a2 and R . The next two columns contain the results for the method of moments and the method of maximum likelihood estimates of a2. .59.
The next two columns contain the results for the method of moments and method of maximum `likelihood estimates of R. 'When R=O 'many of the method of moments and many of the method of maximum likelihood estimates of R were zero or very close to zero (after the contraint R a0 had been imposed), in these cases estimates of the means and variances are not appropriate statistics to describe the distribution of the estimates of R. Por the oases when R=0 then, the numbers of zeros (R, R < 0.00005) and the outliers (the three largest estimates of R) are also given. The true value of p was always 0.4. Results (t) Oft the availabilityof the-method of maximum ihood. • An array of length 15,000 was set aside for the computation of the likelihood, and this allowed the programme to be used to give maximum likelihood estimates for n as large as 306 and µ up to about , although for some data series with µ=1 n=200 or 300 (3.1.2) exceeded 15,000. In order that the results of the simulation experiment would not be biased by this, values were selected, g=0.4 n=60, 100, and 140 , well within the tadmissible range'. For these values of µ and n 'all the data series obtained in the experiment were able to be used to give maximum likelihood estimates. There is computing software which allows arrays very much larger than 15,000 to be used, so that the method proposed here could be Used over a wider range of the, parameters and for more general models, if the gain in efficiency over the method of moments is enough to justify the cost. .(2) On the numerical procedure. It was found that the 'hill olimbing" method used was able to locate a maximum of the likelihood for almost all the data series used, so that in general the likelihood has a well defined peak* Par a few of 'the trials with c2=0.i , (2, was very large; a typical result of this type was
true values moment estimates n 66 J=0,1, 4=4.1 '62= -0.0011, R=0.2975
maximum likelihood estimates a2-o•Do0O, 1)=17502.6208' where a particular data series lead to a very small , or negative moment4 estimate of 02, and. If2 iien5p P > 44 a but with a sensible estima of:.hil calculated from these values' of - a2:' and in:this example .4 = 0.1747* Although the location of the maximum of the, likelihood should not depend on the parametization 'chosen, it is probable that in, the sort of situation described above the maximum 'likelihood estimates suffer somewhat in precision* This would not occur if the likelihood was defined in terms of µ a2, and (o2p) say. -62-
The numerical procedure gave a report whenever the location of the maximum of the likelihood was not very well defined: these reports occurred less frequently for larger.values of n, indicating that the likelihood has a more sharply defined Ipeakli for larger values of n•
(3) On the consistency of the method of moments and the relative efficiency of the two methods con- sidered. The results in table I show that the two methods give estimates which have mean square errors of similar magnitude; this would be unlikely to occur if the maximum likelihood estimates were not consistent for the range of parameters considered, and it is reasonable to expect that similar results would be obtained for other values of the parameters, and alio for other models of the form (3.0.2) with 1 > 1' Concerning the relative efficiency of the two methods there is a pattern in the results, which is 'suggested by the results of section 2.4 , namely that the m3Irimum likelihood estimates are appreciably 2 more efficient when L. is close to one, and as 2 efficient when is aloe° to zero. The reason which explains this pattern, the result of section 1.4 -63- applies to both situations, so one would expect the difference in efficiencies of the two methods to decrease as p increases; however the method of maximum likelihood can be applied most readily in those situations when it is likely to give the greatest gain, when p is small. One cannot prove a mathematical property such as consistency by a simulation experiment, nor can one sum up the conclusions in a probabilistic manner (rider the hypothesis that the method of maximum likelihood gives inconsistent estimates, results as good, or better, than those have a probability of ...?) What one can do is present the results with such conclusions as seem reasonable as a guide.to other experimenters and as a spur to theoretiCal investigation. -64-
3.3 A -Distribution with the Markey Property
A situation could occur for which the distribu- tion defined by (3.0.1) mi#ht seem appropriate, and one would like to use maximum likelihood methods, but for which the special models considered in sections 3.1 and 3.2 would not be usable, either because the covariance Structure of Ithe process under conaidera- tionwoi.id not be likely to satisfy (3.0.'2) or because the numbers of the data would be too large. It would be possible to avoid the restrictions of the previous two sections if (3.0.0. could be the p.g.f. of a stationary tlarkov process of some order, however it will be, shown that this cannot be so. Hence. the best that can be done is to construct a distribution from (3.0.1) which will have a similar probability Structure to that distribution but which is numerically tractable. The distribution with p.g.f. (3.0,1) cannot be the p.g.f. • of a stationary first order Markov process, for this would require
P X.n=rri X13.... "IX1=r1 PIX.n=rxi I
for any n r (3.3.1)
-65-
n ) 2
= ex
Aso that using (3,3,1)
Patting n=3 gives c13 putting n=4 and using IstationaliY$ ives ale and so on. Hence
if 2, (3,3,2)
SimMarly, equating PkIl °J 1} and Pk 1
J.: =01 for n > P and na (3 3. ) and 3,3.2) new gives au t3 for any. > 2 Hence if (3,0.1) is the p.g.f. of a stationary Harkov process, it is the p.g.f. of a series 6f i dependent random variables. -66-
Now put
= X 1=200.,,M
q(r1) =
where Xi'Xii1 have a joint distribution with p.g,f.' defined by (3.0.1) with n=2 , and :X1 has a p,g.f. (3.0.1) with n=1 , Then the distribution of Xi,...,Xm defined by
(3.3.3) i=2 has the Markov property, and is such that any adjacent pair of random variables Xi_i,Xi has a joint distri- bution with p.g.f. (3,0.1) with n=2. In particular the mean, variance, and autocovariance 'lag one will be the same for (303.3) as for (3.0.1). Aut000variancee of lags greater than one do not have a convenient expression however: for example
to OD
E(Xt t+2)= > ret+2(1 *)q(rt+11rt)q(rt 2 t rt= r i= r=0 -67-
"'2 +021, .!1(rt+11) cAtt+il 32 q t 1 )
using (2.3.6)and whore q (r)=*=ri for X a random variable from a series with distribution defined either by (3.0.1) or by (3.3.3).
The likelihood of a series of observations r1 ,...,rm on the distribution defined by (3.3.3) oan be calculated as
jj .1.-1 2(ri) i=2 i=2 where *the r1' ri-1 are now pairs of observations on the distribution defined by (3.0.1). Hence the method given in section 2.1 is applicable, and the length of the array for this is max i=2, • • • 'VI (I 4...
-68-
It is, possible to justify theoretically the use of maximum likelihood methods here as the distribu- tion defined by (3.3.3) can be shown to satisfy the conditions 1.1 and 1.2 in Billingsley (1961). Par example, put 0 = (11,a2 ,p) , A the parameter space, and define transition probabilities r1s) = 14xt+1
=riXt=s where Xt,Xtil is a pair of random variables from a sequence with p.g.f. either (3.0.1) or (3.3.3). Then A is assumed to be open, so that one must assume that kr4,1-c2 (1p) > 0. Another of the condi- tions is that for every o , 0 there exists a neighbourhood N of 0 such that a) sup OP <0, . r=0
. Using (1.6.2)
al(rls) P(rls)
Arguments as in section 2.3(b) give r 1 2 J 11.-0- (3.3.5). -69--
Now the three parts of the right hand side of (3.3.5) have supreme. at points of N independent, of r for N e A eo that (3.3.4) follows. Then from the reference there is a consistent solution to the maximum likelihood equations, these maximum likelihood ootimatoo are asymptotically normal, and likelihood ratio tests based on these estimates have the usual properties. This justification is not entirely satisfactory because for the distribution considered here it cermet be shown that the maximum likelihood equations have a unique solution. In practice then, one would either look for a solution in a neighbourhood of some other consistent estimates, or dheck that there is only one, solution, In the next section a simulation experiment is described which uses data produced in such a way that a form of (3.0.1) would be thought to 'give a reasonable model. The, question of whether (3,3.3) provides a ueeful model is examined, and the behaviour of tests of the hypothesis :p=01 are investigated. 3.4 The Simulation Experiment using the Markov Process Model
Objectives The experiment was designed to investigate.a possible application of the distribution defined by (3,3.3) in the previous section. in view of the justification of the maximum likelihood methods given in that section it was thought that it would be more interesting to see how the tests considered behaved with data sampled from a distribution similar to (3.3.3) than to use data sampled from (3.3.3) exactly. Thus the experiment provides information on the robustness and power of the tests used: and on the problems that will arise in practical applications.
Programme The data was produced.in a manner suggested by the construction of the multivariate Hermits distribu- tion given in section. 1.5. A series of normally distributed •means' w re.tiroduced by the autoregre s on
41. = R I:1(1114.1-R) + a ei i ir• -71- where gl and p are the parameters as in ec ion 3.3,,with p < 14 and the si's, were a series of independent observations on a standard (univariate) normal distribution,.provided'hy a random number generator. .The first 100 pile" were discarded so that the remainder would have a stationary distribu. . tion These gits were then taken to be the means of a series of Poisson variates, and the data series ri ,...,rn was 'obtained by sampling uil.. ,un uniformly distributed-in (0i0-4 using a random number generator again, and finding the ri"s such that
S xi i=1,...,n
If any gi were negative, this would always give ri=0
If the mean g , of the gils is large compared, with their standard deviation, c, the ledge effects' of the two stage sampling procedure will be negligible, and rl'1-n. will have the distribution with p.g.f
(3.0.1) with (3.4.1) Pli-J1 J1 3ti=1,••001 • This distribution is not the game as (3.3.3) in general, but they do have the same mean, variance, and autocovariance of lag one. ?Luther if p=0 (3.3.3) and (3.4.1) are identical, so that significance levels produced in the experiment for tests of p=0 are estimates of the true significance levels for data drawn froth (3.4.1). An 'experiment' is a series of (60 or 100) 'trials' with a particular set of true parameter values. A 'trial' involves the maximisation of the likelihood of a single data aeries (of length n=100 observations): The programme was arranged so that for experiments with the same true values of µ and a2 the same series of µi's was produced for each value of p taken: ConSequently a 'peculiar' series of µits would effect a trial in each experiment with the same true values of µ and o in ,the same way. Two series of experiments were conducted. The
a2 first with true values of µ=10 , =5 and p=0.0(0.2)0.6 to see how the model fitted the data, The second, with true values of 11=5 , o2=2 and p=0.0(0.2)0.8 to also investigate the behaviour of the likelihood ratio test of tp=01. In the first series of experiments the likelihood was maximised with respect to g a2 , and p using as initial p stimates r , and 15- = R (notation as s —r in section 3.2, but with R , i instead of The maximisation was subject to the constraints
0 < g 0 < a2 - , and ai < p < au •- + p where 0, p. ) , and a=min
Thus the maximum likelihood estimat-e of p was constrained to lie within approximately three standard errors of the moments estimate of p . In the second series of experiments the likelihood was maximised first with respect to g and a2 with p=0 using the same initial estimates as before. The maximisation was then subject to the constraints
0 < p. 0 a =au=0 . -74- Secondly the likelihood was maximised with respect to p 2 and p 0 using initial estimates of p and a2 the maximum likelihood estimates of these para- meters just obtained, and with initial estimate of p=O.1 . The constraints for this maximisation were then 0 < p , 0 < a2 < and al < p au , where a1 =0.0and , au =1.0 These constraints were imposed by introducing variables b1 ob2 and b3 as in section 3.2 , where = 1312 2 b2 1+P 777 2 ) b3 4+b3 The crude bounds imposed on the maximum likelihood estimates of p in the second series of experiments is excusable when one is comparing tests of p = 0 , &nd -75- also this second method provides a useful comparison with the method adopted for the first series of experiments. For the purpose of comparison, the tmomentss estimates of a2 and R 2 a2 =s — g+a and R were recorded* The results are presented in tables VIII to XVI. For the estimates of a2, R, and p these tables are in the form of cumulative percentage frequencies plotted, on arithmetical imobability paper. Table VIII summarises the results on the moments estimate of a2 for each of the four experiments with g=10 , a2=5 — the scales are offset so that the lines are distinct. Table IX summarises the results for the corresponding maximum likelihood estimates of a2. Tables X and XI are similarly arranged for the, two sets of estimates of 0.2 for the experiments with 4=5 02=2. Tables XII to XV show the results for the estimates of R and p in a similar manner. Note that in each trial the data series was always 100 observations long, and that for the experiments with µ=10 a2=5 p=0.0(0.2)0.6 and for 11.5 , 02=2 p=0.0 100 trials were made -76- but for the experiments with u=5 (5=2 # p=0.2(0.2)0.8 only 60 trials per experiment were made. In table XVI cumulative percentage frequencies for the likelihood ratio test statistic are plotted on log 2 oycle by log 1 cycle paper. -77-- Results (1)On the programme. The 'hill climbing' method converged nearly always and seemed to behave itself reasonably well for both series of experiments irrespective of the initial values used or of the bounds imposed on p. This suggests that the likelihood has a well defined, unique maximum in most 'cases. In tables XIII and XV it can be seen that the points plotted ''tail offt from the straight lines drawn near the top of the tables. In the correspon- ding histograMs there were peaks caused by concentrations of estimates of p in the range 0.99 < p. < 1.00. This was caused by an effect similar to that noted in section 3.2 under Results (2): when a thigh' estimate of p is obtained the maximum likelihood estimate of R = calculated from it is tensible*, 11+a'2 (2)On the two estimates of a2 Comparing the slopes of the graphoof tables VIII and X with the corresponding graphs in tables IX axid XI, it can be seen that only for the pair with µ=5 a2=2 p=0.0 does the moments estimate give a steeper elope. Now from section 2,4 for p=0 the efficiency -78.. of the , moments estimate of a2 is 90% when u=i0 a2=5 and 94 when 11=5 , a2=2 , Hence the maximum likelihood estimate of a2 appears to have good efficiency. Using the 50% points as estimates of the means of the distributions it can be seen that there ie little difference in the bias of the two estimates. (3) On the tests for independence. The estimated powers of the various tests for independence of size 10% obtained from the graphs in tables XII to XVI are given in table XVII. The figure 0.118 is the upper 10% point of the N(— ) distribution when n=iOO and the figure 1.64 is the upper, 10% point of the ti + Xi 2t distribution (this is explained below). Ivor data drawn from the distribution defined by (3.3.3) the maximum likelihood tests for independence would be expected to be more powerful than the imomentst test considered here; the results in table AVII suggest that this property is - not seriously changed for data as considered here.. Note that for large n the power of the moments teat is not affected by the distribution of the data. -79- In tabled XIII and XV the 50% points of the graphs are near to the true values of the parameter.. Further the slopes of the graphs are fairly constant. Novt'for p=0 'the slopes. are estimatesof-the'vtriance. of the maximum likelihood estimate of 0 in the case when the data here has the distribution Assumed. Hence .the maximum likelihood estimate of p obtained by fitting the:distribution defined by (3.3.1) to data - as produced here maybe u*ed as an estimate of p (4) On the availability of the model. The points noted in (2) and (3) show that for' . the values of the parameters µ cy2 , and- p considered here the distribution defined by provides a:, reasonable model for the data used. Further some indications of the robustness of the methods Of.maximum likelihood are given. It should be noted that the model (3.3.3) would be expected tolit data.aa considered, here when is' ,is large compared with c2 This is ' the-circumstance whichthe results of section 1.4 indicate is such that climates usingsamplemoments could be tolerably efficient.' However,the,closer the data fit the model the better, in terms of efficiency or power', the methods of maximum likelihood will become. Zt has been pointed out previously that this mod does not suffer from the computational restrict on of the model considered in sections 3.1 and 3.2. -81- An explanation The distribution of the likelihood ratio statistic is obtained in Billingsley' (i961) assuming that under the null hypothesis Ho the parameter rt = (1020) lies in some region Ao say, and under the alternative hypothesis T lies in Ai , where A.01 is contained .n Ai • In the situation above however, it is , required to test the, hypothesis that p=0 against the alternative that p > 0. To see how this affects the distribution of the likelihood ratio statistic, T :, suppose for the moment that the test is, of the form H : p=0 verses H1 — 1 < p < .+ 1 In this case it oan be shown thit T will be approxi— mately distributed as a Aai 2'variate, and if the size of the test is to be one would reject Ho whenever. T where 1141 2 > Xifa( a , Further the maximum likelihood estimate of p will be approximately distributed as a normal variate with mean zero under Ho , and one would reject Ho using this test whenever n where i = a -82. It is reasonable to suppose that roost series of observations which give a significant result for the p test also give a significant result for the T test. Hence when H becomes < those series of observations -which would give /c:<- nu now give = 0 and appear not to be significant, and similarly those series mill now give a value of T olose to zero too. Consequentl the size of the T test becomes T > Xi: ,h ~ P jXi o S u. This argument suggests that the distribution of T under h, will now have a frequency proportional to X12 i,th a lump of probability of at the origin, i.e. Xi I The result is the same as obtained in Chernoff (1954) for an independent series of observations. Comparing -83- the observed frequencies (n.e) with the expected frequencies (et) based on this result for the experiment with 11=5 , 02=2 and p=O using the X goodness of fit test with five de ees'of freedom ' x5 i*f a value of X52 6.48 was obtained, which is approximately. the upper 26% point of the distribution. -84- Chapter 4. Some remarks 4.1 On the Ge aeralieed Poisson Distribution A distribution is one of the family of generalis ed Poisson. distributions 'if it has a p.g.f. of the form exp E 2Lar(z (4.1. row, Similar teohniques to those used in Chapter. give: a reourrenoe relation P(r) = iai (4.1.2) i=1 moment s r ar (4.1.3) '(I and derivativei aE(r) 4.1.4) eb4 i=1 where and b is a typical paramet rati -85- Further it can be shown as in section 2.3 that the method of maximum likelihood gives consistent and asymptotically normal estimates, however various infinite sums, such as those in the expressions for the moments (4.1.3), do not necessarily converge, and also the infinite set of parameters in (4.1.1) has to be avoided. If it is assumed that the ar s are functions of some finite set of parameters bi,...,bi then with b = (bit...o bi) p 10 the true value of and A the space of possible values of b, the follow- ing conditions are sufficient to prove the consistency of the maximum likelihood estimates of b : (a)a r(1) 0 r=1,2,... (b)a r(1) r=1,2 are continuous functions of b1t00111,101 (c) V(X) < cn (d)If ar(b) = ar(b ) for, r=1,2,... then oo (e)For any sequence [biji such that 4 co as i 4 co, h(bi) 4ce as i -0,co (f) A is a closed, finite dimensional, Oartesiau space. -86- Purther conditions suffioient to prove the asymptotio normality and full efficiency of the estimates are: ,°° Oa (g) 2 < im ....,1(implied by (k) r=4 and (0) 00 (h) > 1 81)2arb • 3w ok=111.6. r1 (k) There exists a constant K such that isi "!'!'"101" Pwilpivorde (1) is an interior point of A (m) 1 are functionally independent. I Condition (k) ie used to show that BE converges: for using (k) r P( x-.) -"VT- i a i=1 %mini (4.1.2) so that, use (4.1.4) + Kr for -87- and hence, using (o) E converges absolutely. If, for some m o a Or>m o only nditions (a), (1), and (m) need be retained. The following distributions were found to satisfy these conditions (they are those distributions given in Patil and Joshi (1968) which may be written naturally in the form (4.1.1)) provided that conditions (f) and (1) are imposed -► it is necessary in all these examples to dhange the parameters from those given in the reference op that the function gtven is a p.g.f. for all k t A . (1) Poisson binomial distribution This has p.g.f. exp w((q+pz) 1)], where vo poq are non negative, p+q=1 , and m is a given positive integer (m=2 given the univariate Hermits distribution) and T ra-r 0 r>m -88- (2) Poisson negative binomial distribution This has p.g.f.. exp [' tp(( 1,..! z)m - 1) ] , where w,p,(1 are non negative, p+q=1, and' m a given positive integer, and TP1241(-Or. (r r=4,2".. tmq(i+m ) p>0 0 p=0 X = TP(1 -Pm mp ] ar Bar = a IT" (3) Thomasi distribution .. This has p.g.f. exp [ y el(ze/z - 1) ] where Y and t are non negative, and Tr-i ar 17:1). V(X) = Y (t24.3w+1)ew X = y et -89- (4) Polya-Aepp i distribution This has exp 7(1t)' {1--zZ ) where d T are non negative, and v and ar = Y vr-i(i-T)2 , v(X) = y (Te+1) 7,4 = y 014) Guriandis distribution This has p.g f. exp a+p p r=1 ] where y and T are non negative, a and p are given parameters (for a=1, 0=0,1,2 this is the p.g.f. of Beymanis types AO), and 0 contagious distributions) aid • r=0 k=i02,... V(X) = T y el i+y . a+p+14.1 -90- a4+1). . (a+Of +r-i kat (k+i a The expressions gi*an for the arts , 'X and Tr may be used to verify the, conditions (a) - (m) for example, to show that (k) is satisfied by any one of Gurlandtddistributions note that Tr srit = -> 0 as k ->oo and that eak (k+1) a-101 ) W Erk = ak has a limit as k - co a o nvergense of kak k=1 implies that -91- The result of section 2.2 can be generalised to give a bound to the mean.square error in computing P(r) by the recurrence relation method, provided that the arta may be calculated without accumulative errors, this gives 1 B 2, max [(T)2 ('1 IC(0)2 ,s=0,4".. tr wher+ r > ii in the notation of ection 2.2. -92- 4.2 A Subroutine to Compute log P(rcu0 E. ) It is mentioned in section 2.1 that it is possible to write a Tsubprogrammet to oompute logP(rmo n for any n , provided that an array used is not too large. Such a subprogramme is given below, in Fortran IV. The variables of the eubprogramme oorrespord to the variables used elsewhore as follows Subprogramme Text PLOG log P(rtu, ) N IR(I) I=4,...,N U(I) r I=1,...,N V(I) I=1,...,N IY(I) S(I,41) I J=1,...,N The variables which have to be defined before the subprogramme is called are ',IR(I) e'', U(I)le .S(I0)113 * max and IC to ,where'maxia' the maximum length of the array G(I) • • rrOt+1 used to compute log P , allowed, aid K is such that -S(I07)#0 if , (K may be set at N- ). An array IS(I), I=1;...,N is also used. -93- The FUNCTION subprogramme is as follow FUNCTION PLOG(N,IR,U,S,K) DIMENSION IR (n) IS (233 ,Tr(n) T(n) Ir(n+1 max) DO 1 1=1M 1 '15(1)=0 IY(1 )=1 DO 2 I=1,N 2 XVI+1)=IVI)*(1+IXI)) DG 4 M=1 •Ii* V(14)=U(10 ILOW=MAX0 ( 1 $11-10 IHIGH=MINO(NoN+10 DO 3 I=ILQW,INIGH 3 V(11)=V(M) S(I,M) 4 CONTINUE Li1AX=IY(N+1) InliMAX.GT.max GQ TO 10 G(1)=1)0 DO 5 L=2 LMAX DO 6 14=1,11 IP(IS(M)-IR(M ?,6,6 6 I8(M)=0 — 4- 7 IS (M)=IS (M)+1 Lii=1IY(M) G(L)=V(11)*G(1410 IHIGH=MINO(N pt It) D‘) 8 I4-'14, ITIGH :nva=1,m-rt(i), IP(lava *GT .0) G(L) It14)10(1141) 5 G(L)=G(L)/Pi4A2(is(M)) PIA G=0 0 DO 9 I=1,1 9 PiGG=PLOG+V(i)+U( I ) GreALQG G IsMAX ).PliGG*0 5 10 ONT RETURN END -95- 4.3 Alternative Distributions A fundamental hazard which is always present when using maximum likelihood methods as in chapters 2 and 3 is that the data may not fit the particular distribution chosen. It is thus useful to have available alternative distributions to the one chosen, Two such alternatives are given here for the multivariate Hermite distribution. (1) The p.g.f. (0.0.2) gives (1.1.1) when terms are retained in the exponent of (0.0.2) for which > ri < 2 , i=1 so more g .al family is obtained by retaining terms in (0.0.2) for which rri 3. This gives a p.g.f. i=1 which may be written'in the form 'n G(z) = exp E ki(z ) + 012(zizi.4) i=1 i,j=1 Yi k zi zj zk -I) ] (4.3.1 where the are all non negative, -96-- and aii and yijt are symmetric. This distribution has recurrence relation: rm-11)111te = Xe( ) AM 4* 7L.. L— Tiara ) it=1 and, gith 33. 7ti 4. 7 Tisk =1 tk= T.l j = aij Yijk k=i moments: E(Xi) = Coy (Xia) = vij + pi8i E 8kelki64+1118ij8jOki -97- All the examples of the univariate generalised Poisson distribution given in section 4.1 have terms in the exponent of their p.g.f.ls for r -ix' with r=3,4,...' , except the Hermits distribution, so (4.3..1) will be useful as an alternative to the multivariate Hermite for any multivariate generalise. tion of these examples. The likelihood ratio test Calks section 13.6) provides a way to test the hypothesis that the yiskre are all zero. In (4.3.1) there are about r , -or if the distribution is stationary, about ! n2 , independent' parameters yisk , compared with about n,2 or n independent parameters hi and, ai3 it might be worthwhile making some simplifying assumption about the - Aisk's for example: Ii jk all • Ii-j1 141.41 Ox-il Tijk y a Pli. Plk-il or PijO kPki give forms with lees extra parameters to consider. -98- (2) The construction given in section 1.5 can be modified to give a multivariate form.of the negative binomial distribution: Put $110•01n where i 9 • • I 91 Y n ) j=1,...,m are independent normal variates, mean 0 covariance Suppose as in 1.5 that X1,...,Xn for given 11.1,...,4 are independent Poisson variates with means µ1,, p then +00 4i)riej" Y211 P(X = r 1. • If • TT E J=1 r i=1 Y./ ) J=i and Co Co n r4 •. zi = r100 rn=0 i=1 -99-- +0) p ) 3 say, where where Z has elements ( 4.3 . The bivaricite form of this distribution uao v cd ma paper by IT (1960 to it accidents to London. T ansport bun drivers in two consecutive years. Recurreneo relttions are given in this reference for the 2(0's of this bivariate case, suggesting that there may be generalisations of these recurrence relations in the case of (4.3.2) with m > 2 t - 14' i • i - ' 1 - • • , 1 • • ,..: . . . _ .. .._•_ - 1. c • t t , •_ I' . . . , .. . ., • .. ....r/ , • • r-. ...,-...... -... . 1 , . : : .. : : . : . , ; !1 ' ; : : . ' , ' 1 .... 1 :1!:".' .... . : :::: .'...1::. l • *•'-'1 1. ' !' ....,.,...... ,-,...... , . i - - ..t.i^.".. •- • ' - • • • .• - -LA.,' ' .:1_*. ....:i...... I --,' ...... _ . , , .. -',;•••-t-14..•••••;•-•-.•••,..1.,• ' " - - • " ".": t:: . • -1 r •••L '''' '''''''. :7..' • . _. • • 1 • .:::: .::. ' . i :14 : : ; :.. • 7: :. :: . : -- '-' ...14: 4. .. _..,..--. . 1 1 e ..__ ..... __ ...... • , ;-7.41.7 4'. . .. .-- . .-,...,...,.- ..•;;,...i,-..- •r: -.7 - r 7:1---. -4 - 7:-••_-r:-..—.-r -- -- ....- ••• . ,- ., I ._. . _ . - I • . 1 _ . ____. .,.... . - _. . , .___i - -..••..# :- - 1 .. • . . _•,•-•-= ,_ _ .. 1 i_ T.::: , : r . • -.I _ ---- • • . .--. -4:_"- • .147._-_.: • 'L-- ". • t:...2.- . . . . ,-. •_-.11' .7 f . . Eri-t-•:---t--• • --..-•-• . ... . • _ . . . I • ; ,- . •• , - I " ' • ...... t: . . ' I . - - r 1 :::.:1::-.:.• . ,-- -. • " -- : 1 , 4 , • . . . : .1 : . : 1 . , . .. • : • ...... : ,, 1 . : LI : Li;'.1.:-.::: .' - • . tr _ ..,. • - i . -... ::-:.: ' . - . -. ."•'• . • ..! t il .1 ,_:11 • : ...... • .... • I . ...-. • " • ...* ,_ I . i . • . - ... --- ..... e' - — .:-_-: - ,-._._,`' : • --'- I- . • . ,_:;:l="; : i - • " ' . • .. ' __ ._.. . ..1. . ..- :--4 ,•-•-• -- " 1 . . : : i, ' • :2..... tf: :2:: : - ' -- - " * _ - • f • : : ,:-.: ... __._ , . . - 6 . t - • • • -,..-:. :_.-... - __ .1:, . -- _ • _ . • • _ . : . :• - - --: . ; :::: • , • - . - -77- - . . :: :, " -. 1 • : • :•- • •" t:t.,_.,..r... _ _. : . . . ; 1.:. .., • • - . • .; . . -. - .. • • ._ . : . ___. • • . ::::. • ' • t 4 1 ,may " ..:."Tr:• - - : . -,._..— 1 ...... - . - . I ' . . ..r• r i " 1 "1- .. - . I :::: ' 4 ti I ---•-1. -: • • • T . '1 _ ---r• ,,•• — _ 1 . f.,- - • _- I ' : , .. 1 ,,, I 1 : • : - ,... . - , , 4 • . - , . :-..::: ,..:• - . _ :, ! - • - .._ _...... • , ' ' i : : • .. " _ __ — • ' :. - ' • t ' . . ::.t:.'_:...... —•"""._. - - •• , 1 I • . 7:".T.4'..;"' 4 • ;v.' I .- .441. : ' . '.1!::: • . . I. — . -1',-:. . t __ . - I •-.K.,... ' ...,-.....-- _ ... , .... . • . I-, - - 0 I I' -101- I • I ... , I I I ' I----;---•. ---'O'--.... _--i-'-_-r.""""'___ ...... -- ~.---~,- .... ,~ - I' o I" .: o 'It -- ---_._--_._- _. __._--_ . .. _------ ; ..... /. .. I i, ' '. I j~ j I· • i I I . I I \ I i ,I ------l I I -102- <5' I· , . 42.. I·. t ___• ~_,_ - ___ -- - ••• • ,'0 _ ,~_~,_ I d (~ g J -.-.------.~- 1 1- -- --.... f -. -'- ---- .._.-_.- l I U \ d ---1- - ~4 , --;1-J-~~I- ,_ _~~-~ _~ __~ __ __ ~ -L-I ______/. ______------____~ · __\ ~ __ 1:: ,f ',:J :; --_.;-_. ---r--,6---...---- ~_._ 1-----0---- . ---+---'~ { i 1 I I '~----41-- ---J.~-""'" -(i 1 - t I I -----t:"-~ -- - . ~ --~- . ------. ---. -.. ------. I I, 6 ~ tt _ l H_'" .... _ ._._ _ ~ ______...... ____ _ .... _ __ ~ .. \.: '. 1··:." . _ _. . L -10'3 - .n,- 0 1------.---1-----.-----.--..---- o I I ' 1 ·-1----- ·--1 ! - ~--. I 0 ·_-_·t --+- 1--- I------,---4-----+-- -J 1- I I · -1'04.------..--.--. I 1------~-+_-II___--_ I--~- - .. ., GO . I 1------.- ._ ._._--- -- I 7 I - ~ --- --.--._- I I I ... - -- - -_ .._- --- _ ... _ ------+ I ._."_ .. ------.----~-~- i .----- ..- -~ -. . - -- "- ! • I f. . I. '. / I I' " . ~~---. ---~1---i----~--...... "....~~...,--· ~----- 1 f.---:...------lr-----+----+ I I 1·<' I • - 01 ·--1--31--'----1--1 I, [' ., I" . '-10f5- i . I' . t ....: I:.. ··: lI !_ I I I . ., I I I I [ I: ' I I I I' I I \ .1 I" I I I [ ' I I' :· '. [. . I:" ' -106- I' - --_.. - r····: r \. ' \'. ' i .. ' i ":': .: \. :... j'" ~- ',' I. t . I." . l:· ,.' I : . j : . 1- . '. . . ] .' . I.... :·: '. 1. .. < . I . j '. f·· :· . ~. ' . , I . , 1-- 1'.. .: f '.: I. ··· • I . I'[."'' ,.', I ;:;: .... f:"~ L ' .. • I ~ .. ' .' , 1 • I • I .... 'I. I i 1 -107- :'.: ' I' "', " f f, :' ~ I', I" I, , I" I :, ~ : . 1 " t': ' I ' I · I ; I ' I I ~ I', , lr-- '. ' I I;." ["' I, -1 08- .1 I I I ' ~ , '1! r-> I J / 1/ 1'1 I- 1- t-I- I- I- -. lL ~ ' ~ J. - I-I- l:±. l- !L v / 11:. 17 I/l< I/ t"\ IA< lL I po: ~ II'Il 1;1'" ' I ~ "" II 1'"- I j V , \ I / 'I' I':t\ / l- V . /"'\ l."- "'- I o I',. I - I / I' I , / - CLli l"- I IV' 11'1'\ [, r 1-1- , I I l - It r-I-t- tII'- I- / , I,.-n; IJ L / I-r-I- I-' f ..t:~ " I r ~ I t ...,., :j I- I I .;. 1/ ~ I-I- I' II· I 1-- I" I":, J' ~c. 1/ 1- 1G : :J 1m .1 - 109- I , •• , I 1 I_I , , , i ' ,- , ~ , , .... no l , , . I - t- ' -- , , I\.- ' I ; I I I . I I ; 7 I I 1\ I I I ~ 1 \.1 . . I I A IC . - I I , {'lTTI --, 'r- . ~I- 'I i : ~ : i-: . J : t _ .~ . .j.. ~. f-1-L -t$~ Y E!= ~; h 1-!4-I. " /-I , /J I rl 1 iii II II / I 11 v II J J : I! ! .1 I II V 1 -~ 1 H- I- - -r- - 1-1- -,.- 'Ii.{ I l~ 1 f -I- j:i ;t ", 1 \llT Ii i, -~l+ t -L". ,rv I ~ I ~ g V l ~ j\ /, I I 1-1-, '/ I . \ :.c I.-L - - ·tt I+itt- \ .1- 1/ 1 III I __ !... II I ! ..Q ! 1/ I ~hl I -L!+I- I o I--t ~H- 11' -r I I Ilr ci: P , 1-+ , -L,-~ I I !L 1/ 1 I( I 7 t- I , 1/1 I m~l-L. . '7 ' ,I I" s: .~~ • 1-1-1- L...+- j-L.. .1:1.-. , ' ,1 o I ~~ ~±il - E:. 1:l-r::t= I 1 'u; .j ! .; f..-..-+:A -t-:r"1 J 1;i,:=tJ::L 1#1-'- -i-1-1-i-- I=f-F-il m~ I:;E ~ H !.~ 1-1r trl- V.L .. _,.- -l-.-1-- Q I I , I o ~ 1- I I / 1/ ;; f-tT+ y;~ V 1 :l £ !I I C' ri~-r- W I ~ 1-·/ I.· .. I 1 , ~ I 1./ o 0 ~ #r , , co o R=!-r-.-~ v , '- ~: 14 R- E~ j IJ I ,."', I , I - I: +-f-!i ··1 ' I e -t-' I....:' . II- I! I I I ·. J" I- ,.. 1 I-I-f- VI_ ~~~ R : ~[fl I- I- t,.! 1-i-I-I-1-1-i- I-r- .T ~ H-H- I I -I-H-- I-!±. III' I I 1' . ' Vl 1-1-I-I-l- , 1-1 -FFi- I -IJ~ i X [} r- t-t-r- l I- I- t-I-\:-yZ ~ ~.'--- I , I , ! I f-l- ~.': ' .. t- f-j---+ I F 1 l"- P'I~ 1-- -!- . H:; t r-H- :-- ~ t- +- n- J1 :> 1-1-1-, ~'p-. - '-i i)4 J I 11= I .. . ( i,:' .. ' -. ~ r-t- J. . - 1-- p :C v 1'1 'j a: I , , -I- 1- -. -I- -~;f I I -{ -.1- L -t:.. _ 1- 1 ~ "'1 , -J. i-- -I-1-1- I- l- 1--1--f-+- III I -~ -,-J14 I I ' -I- 1+ H- C ... ;,,; .- .-'-,-\,;;;.. f -=f~~~' _t ,- r:. rl " i=1- ..r: .:q:~( ~>;r Vr= 1=t:: t": =t::= =i=: Q. I-+- t; b1:E =.1: P--~Ii .!= .f' .-J t t- IV (!l HI- FF=!= + . . ++-.L~ ~ l ~- 1-:~j 1 ~{# H--7- w.~ 1 I I I I I I l- ; I 11 T I I- i- I I ITI l{fit 0 1·-0- ,...1-+- h, . I}\ lL ' T _I w,~f)~ f( Ir-' , : (Jdc.~j .0 .. Vt-- ~-j- I-r--f-, ~ I! P' ~ II I 11 1 I I V II I) I I IVi -, V I :/ .... I-,-q i / 1- I- j-I V! I I 1 It I' I I, I I I : ~j I ri- VI I I I I I- ' -H- ..... " t-.j-f- I- ~1= .'-. - ~ 1- =r f-r-- ,....,...f-+ R=~~ :::r 41~ 1~ 1-1- 1-1-'";H 0\ '~I l iH~l: f - Vf+tt v· I -f+t~~ \ 0\ Xl r ~ 1* -!-4-+ ~ ...., 1--+-1- 1-I-t-I-i • • - f ; -110- 1 I I I • I I .- ~-i.::i=I- I ::;" j - T ~ 1-1- . r):I'-!=- ~'-: I L. . ·t-,: i-!- r :-j-'f=.; r " +-t- I -~i -l- f-f- !-:J/ f=P:: ~ '-IZ..... j., t= -1 H-t- H -'-:t- i 1 Ir ..A 1'1. , I I I : 1-1-1- -.l I _ : ; "",1 _ _ ! i J If r 1-1- .t-J- . 1-1- -- L f-.\ -H-++++++-V 1-4-+-H<-1- H-+'++-+-H-l-H--l-..H-'-H--+-H-4-++-+,I}- 1-H--f-+.· I-\·-r-1-f-H--H f- .. j C- if-c- r-. - j 1/ ,-1- -. 1'-1-1-1- - 1- H--I-I--+--1--+-: L " . I , 1 " t-t-t-r- - . H- f -f- 1- f... 1- 0 ++-- f- f:: 1- ; Ln r- i-<-+- H i ~}~ T i !~D'fEt- - t-H-I-\1 Il\ i-t- 1 I-t- ' ~l1i j- if -- I- 11 . I-t- t- 1-1-1- ...: Q) a: tR=t- ;:; ~ IA t Ll: i' Hft- - I -~/ I- ;-f- .I 'f-t i ,-,- 1 II ~~rl- I -l .- L [:i , I I H- H- 1- A-l--!-H-+-+-+-l,,~ TV-I,-I--H--I-r-H-+-+++-H 1-++1 ++-H -i-H-i-I--t- ~ j ,-j .lpt +-+I-H-f-H--t--t- r:I rr- I I I I ~ 1-1 I i--- i/ F I-+- I t-f- '-H--H-+H-11\~~++-!I- I -H-++++-!I-+I- I -J.!-I 0 ~ II' '1- .. j • r I-I... Y If-I- .c: t::r ; • i-E V J- C- ["-1 I ;. V. it. ~+rf-t lD : · I i I-r+ . ll! :• I -I-~I- 10 ~ +-+..,-;...+++ f-l- H -+1-1+..... 1 1-'-H-1-+-+-+ 1-; -t-4- !- -I-,-!-I- I I -- - I I ------...... ::...... ------ I ( . I 1 I _111- I II' 1, I' ! I I f\ t _ . l : ,;_:! I -:! I , [1,tf,I I f-- 1 I ~ iM:,_l+-+-T+-i++!+ J.V-+-+_I-+-+-+---H~'H-I-:f--+--+-'+-1+-1/1HI I II: I I: 1- • • . • ..,... 1-;- t L j! J-!:I-H- . ; /I I ) 1'\ • ~ i.. -,-, ! • I ...' j • J • 'i ,t ' _:-I=t .... f-- ~ L1 !{ 1 I I ~ · 1-' 1 I-L-:-;-l H-!... --L. ~ ,1 , " I.:....., J I 'ilil I II :0 !, I- ." • 4- -j-i-I-.- •£=.-1- _ ~~ _ -l- "V, ' :=f-r-:l t)/ 'I tl-,_++ .... '[/ 1, l- .tI f-' t 't-I' H r- I-r-I- ' I '" '''If.'-t-+--H-++-H-.,4-j..-H-+-+-+--j-+-f-fLH-+-l-.,''-\-7 ---'-++-+-+-HH-I-+-J-h 0 r r- r r t -1+ . 1-1-1 -I I ~ U r.... -l-J.-hl'-l-'i--l--H--H--lI+,-f-L-H--H-+++h '+ -l-+-wl / T"'"'+I -+-+- I -++--I-!--~-h' Q. )C 1/1 C 0 'iii '> (5 0 ;; a::J w 0 co 70 0 c:> ~ ...,.,. II) II) .... 0 ~ a:~ ~ ." Q I J:. ~ ~ ·C,,-.k ri} 1 ,!":'-j; ~:!OO!. ~ . j f: t'1\1~ !-/ '!=t:i l ~ j(J.J 6j IA-i-:- \;Dn,tJ I H-r-h .. Ih¥ ;'L' t '-t , .. 1 -'01'7" ~r'" ' I'-I-t-I-;- ,....r~t I-+-t-t-t-t-H-H-H-H- I -LLLJ_ " , . \, r rT r r ,- .'. t 1 -=- f - -1-+' ,- ' j ' H-"-+-++--I-!--+-i- t-'-; -t-;-t--'-l-I -++I)T"""-.-.--I-+-+-+-t-t-t-tl I/') I I I =1 I --I • "I -----..::.------41 1 1 - -1 [. ' -11?- J ' 1 I.' I ' I " , I' I' , I' I' I I 1 v' , I 1 I I l/ 1 ) • .....-L t LII I-Lj-L ~ I - H-t c-L ;.. ..' !-- ~!-l- I I~. X ~'-I-"'" - 1 • I +±i-t ~. "'l' ~. ' f ....i ,ti:!±. ! +~y 1-1-1- t-!-t- I : : rti~ ~. ... :Lr. , I VV I.... 1'- I'· I 1/ vV- II II I - I r l~ r: k tr ',I I ~ ,/:"-il v,v Y !'- ,Itt ! I l..i : I ! I ' ) V I I I? I\.. 1'1 i I I I I ! V ~~.. V I -J+- I ~ I I 1 ' 1 1 1/ ) / I I I t,..- v f-I tGJ 1 1 j 4+"- r ri-, ~r,' r. r'S I I i! v V V 1 If v v I , ~~l" Rtl ~ ±t±i t ~~ k j I-M j' fq I 1-1- !L I ~ I · , I I-, .....; {./ 1-1- I-L-1-1- / -1-, I~±t:r 1..1-1- h-- v I ,- I ~~ l- V -+-1- · . f- .1.,. _ ~ I I-:p=n: -+---.,...... H-r-r:;/ r-r-:- . 1.:1:: ~ I ~"Tf-1 I=:!t_oo I .L.. .3 1-,- I.....l-;±..t:: I -':~ ~/~ If ! 1.-:-- ,." t-..,.-1-1- f-l--T- i= ~:? 1-"-1-- -t--i-!- I 1- I 1 ./ ' I I I ~ I j ~ i - i- H- -t+ Hr :Pt-rt l±rJ-l 1 ~L H- L 1 T v II "1 ~ I I .. ! , ! H- f- IT Uvr -fl-l- _ .. t/.... 1 / ;f .J I 1/ ~/ I 11/ ~ I I ,. I , , I ... /+l I I 01 kfrj(5 1+ f+ f,. Al Yl- fj' I I..1:--;} .r I-':j:.. """rt I ~ +; (71 - # t1: We =~~ K- ~ At I h-t-I o ~P):/I 1 I Y1:F{; I:fr.". ?'~. ~--'~(l I ~cP f ,-1- 'T-i- !-L1 ,?, I ~. , I r~ f~ I l-t F- .. ;....,r'f .J? I - VI ~ tt"' ''''' , o /~,t I - t-,-H- I-!-t-I- ';'''' . ;:r~;. "'1,' 1 - ;( )\ , l-+-i-...... 1- -I-W-I- -~ -~ ; , L-t! .~ -.:::±V' ~~ :~ ~-:Jj I t I 1 : [-! I !-t+:S-;. Ij- ,/' -t-t ,..-- r-B±!-' , II"'; 1-1-1-1-1- ; :; / ~~ . .. I ~g l....i.... ~ .... ~-r'l ' H1 ~. ~ I"' F±f I I I f¥. I , I T FF 1-1-1-\+ H- I-f- (V I ~ ~ F~i-'- .. I- t I I -1- I, -~U~iL ~-l ~ 1-1- ll2 r~ I 1 1 H-+- J-L -;- :."'-! !:'r:-~ -- ,.- .. r 1-:-' ~ §§L-t: I • r:r -h-l-i- ;::-- r . f)?T i"E :. ~ ~i :tti -' 1~=f -'-t- i- i-=R=: v . . ' 1-'-I-,....J- L H:- 1-1- l--r+-- I -t-'-!- I-i- l- I- .±r-I- I-H-1-1- 1-1-;::1= 1-/ ~~~7 ~ ~PA I-f-;"" (1\ ' ·T~ 1 ',/ 1- -+-l ~=-r1t· I- - " , .! r-rl 1-1-1- I I 1-1- ' I r-j-Fl- 1·- I-I- '."t1J Iltrt i -\ ~ I -I- 1-- I-f-f-I- f+ , I y~ -I-1-1- !- I- I j til Ii ! I f-TI 1- , - 1-)+-, , lWt 111 1 I I I I I I I I i -tt-! I l- I (Vi I U I! rI -Hf+ t+ R· I I I I I i 1-:- ! , I III ,-; . I l- I 1 I 1;"'1 : I I I NI I. : ' t ., , -r I I_I- -r j:... 1-1- .. , t 1 r , IJ ~ iT- ~:rf-;t tt.: 1..1- FFt f.-!-+t- I . I" -r I I too • t ' • , 1m . ...,-1-'- I ~- ,-t-l ;.=fi · I "1 Hffi I I I 1m' J :: o· (J- () ~ ! I I I _ 11~- ) 0 -6 f\ -:qI .. -=r ~-I- - ~ _J I ~ / 'r-- r in vi--"' 11' 1'\ 1/ 1-1- v ~~ i-"" Jt- f-I-f- .- I/ i>'I ~ ~ f- - .- i-'" ~f\ • I\, t... ,- o ~ '" l ..- I. ... v ,,- ., ''' ) ;; f- ... r- f '* i-'" L.I r-- ,.... ~ ... [J 1, 1:><: [) :.,.... vf\.L/ r;,;; Ii>t - V v' ,... 1I,' 1.1 1,\ ... ,~ ,. rJl;" ° 1- ~ i f-I-l,;. () \II l '[' (Df; - ,"- I- -'" .' . " .; , ~I'- "" I- '-I- ,.;;. I..... <' I';' .. ,h 1::-- ..... ~ ...... ~ N[ 20 - o [ f- - , [ -I--l- i I- -~ \hI L .... I /0 0' :"":--. - [ ~ - :.- H- V i/ °1 .,1--v'" ( ~ 'I I :t ",I I 1 [ I c» N ' ! 1 1 - . - I o· o· a· o· 1 -I I I-· 1 I I \ I \ I 1 I I I· I I. - 'J 14- t ;-..... ,-,' 1 ,. II" 1_1-1- v 1/ f- ~ 1/ , 7 / ,,- ,,-v""" oJ ""V ! - o - r II'~ V ~~ I(DV r' D yo :,),) .~ V- or V ~ V V I ,,~ V V ' 71'1 L.... Il[..l 1...- ~ 1/ V 1;" .- I,,)~ I,,-r-., ::> Il '} . [I V " ,- ,<' , J I-I- ~ .. rj ' .rJ ,. V ;- 1U 1.... 1'" 1"- 1-1- - 11")- 1/ ,0 ~ I- ~ I" 10 :> ],00,1 ... , I " 1 - It/' r\ ~ ~ :r ,J ,. (:71- , ~ ~ so 1- . 1'1 fT I- II t t) , I' ~ 1- "'lI o P Il V II j:;ii -j I- L/ I~ ,- .- ~ , r- r- l=r- - I (~~ r- !\.;T . -I- .- o () I- ,.. I ~' F 1-1- 1 , - I ~ I,-.~I- 1-1- ~[, T I c"( I- 0ft l ./ D., .).1 -I -1115- /1' -,.- - r - , - ' ~ ? f> I",, ~ ,~ IA' !'" !'-v j..- - 1-,,,,- 1,.-' 1\ I ~ I- - I ',., I p I ri v ' ~ I'" V ! 101" I l.<'" I I l- I- I ~ II I 80;> II - - I .. ' ", II I .... ",.. 1'- ~ I - I - ~ I I 70: '" u l -~r L.-- I"" ~ ." IJII' r ... v .. I- I- "'" 60; I .. I- .., t-' ( h ,. .,J() L'" '- ~ r- I-I- I\, 1- I- 1-1-1-- - I.... - ,- l- I,.- ~-I-I-p.r r- ~: F F---- - , 1--1-- 1- .... ~' - 1 I-p ", 1 ~ I , L<.O II v r ,.... - I-" 1'0 ... !-" .... 'r- _ l,.;- k I" I"" M I","" po 10:' 1: '-' l h - j:...1- I- 1:t:.1= r- - 1*'1 . V ,,1 j- L" ?(v - ' HI I- 1·- , ( ,!- I- ! ~ 1/1 ! I-- l' I I r-t- I cb 1·- 1-1- f- I '(~ 1--1-- i i , l- I - <0 ., ~. i, I : i·:. to ,; I' , I ' .,' I I I I I ~ " , l " I I I '\ ~ -117-0$) I I r-----:----~----_r_--____._---~--- , ---___. o· I------+----+------+------+----f~·- -_. 7 f 7 r.OF f= ()- .~ - 0-' ) o · 80 7 ~ 77 -II I I I . I I . I References P. illingsley 1.961 8tatisttoal Inferencefor' Market Processes . Univeraity of, Chicago. Press go er 1961 ,Statistical Methods in Marlow .Chaina Anm.Math. State. 32 012-40 C.B. Edwards and 1961 A 'oleos of distribution Z. Gurland applicable to accidents 4•A,S•A.' 56„p503.517 Ch rnoff 1954 On the distribution of the , likelihood ratio Amn.blath.Statei 25 P573-378 A.W. Kemp and. ,1965 Some properties of the Hermite C.D. Kemp distribution Biometrika 52 p381-394 A.W. Kemp and 1966 An alternative derivation of • C.D. Kemp the Hermit, distribution Biometrika 53 p627,428 G.P. Patil and' • 1968 A dictionary and 'bibliography S.W. Joshi of dioorete distributions Oliver and, Boys . - M.J.D. Powell 1964 An efficient met4od,for finding the minimum of lifuncition of -several variables without calcu- lating,deriVativeo CoMputer Tournal Vol.? io.2 p155-165 -1Iq— A. Wald 1949 A note oil the consi tenoy of the method of maximum likelihood Amn Math.Statai 20 p595-601 A.M. Waller 1961 large sample estimation of parameters for moving average models Biometrika 48 043-357 8.S. Wilke 1962 Mathematical Statistics wile