DOCSLIB.ORG

Stat 8112 Lecture Notes the Wilks, Wald, and Rao Tests Charles J

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Stat 8112 Lecture Notes the Wilks, Wald, and Rao Tests Charles J Stat 8112 Lecture Notes The Wilks, Wald, and Rao Tests Charles J. Geyer September 26, 2020 1 Introduction One of the most familiar of results about maximum likelihood is that the likelihood ratio test statistic has an asymptotic chi-square distribution. Those who like eponyms call this the Wilks theorem and the hypothesis test 1 using this test statistic the Wilks test. Let θ^n be the MLE for a model and ∗ θn the MLE for a smooth submodel. The Wilks test statistic is ^ ∗ Tn = 2 ln(θn) − ln(θn) (1) and the Wilks theorem says w Tn −! ChiSq(p − d); where p is the dimension of the parameter space of the model and d is the dimension of the parameter space of the submodel. ^ ∗ It is sometimes easy to calculate one of θn and θn and difficult or impos- sible to calculate the other. This motivates two other procedures that are asymptotically equivalent to the Wilks test. The Rao test statistic is ∗ T ∗ −1 ∗ Rn = rln(θn) Jn(θn) rln(θn); (2) where 2 Jn(θ) = −∇ ln(θ) is the observed Fisher information matrix for sample size n. Under the conditions for the Wilks theorem, the Rao test statistic is asymptotically equivalent to the Wilks test statistic Rn − Tn = op(1): (3) The test using the statistic Rn called the Rao test or the score test or the Lagrange multiplier test. An important point about the Rao test statistic is 1The name is Wilks with an \s" so if one wants a possessive form it would be Wilks's theorem 1 that, unlike the likelihood ratio test statistic, it only depends on the MLE ∗ for the null hypothesis θn. The Wald test statistic is T −1 T −1 Wn = g(θ^n) rg(θ^n)Jn(θ^n) rg(θ^n) g(θ^n); (4) where g is a vector-to-vector constraint function such that the submodel is the set of θ such that g(θ) = 0. Under the conditions for the Wilks theorem, the Wald test statistic is asymptotically equivalent to the Wilks test statistic Wn − Tn = op(1): (5) An important point about the Wald test statistic is that, unlike the like- lihood ratio test statistic, it only depends on the MLE for the alternative hypothesis θ^n. 2 Setup We work under the setup in Geyer (2013). Write −1 qn(δ) = ln(θ0 + τn δ) − ln(θ0); (6) where θ0 is the true unknown parameter value and τn is the rate of conver- gence of the MLE's, that is, we assume τn(θ^n − θ0) = Op(1) ∗ τn(θn − θ0) = Op(1) (our (6) is (14) in Geyer (2013) except that article allows the true unknown parameter to be different for each n). Geyer (2013) says: in \no-n" asymp- totics τn plays no role and we can take τn = 1, so all (6) does is shift the origin to the true unknown parameter value, so the null hypothesis is δ = 0 in the new parameterization. In the \usual asymptotics" of maximum likeli- 1=2 hood (appendix C of that article) we have τn = n , where n is the sample size. In this handout we need τn ! 1 unless the submodel is flat. We assume the random function qn and the maximum likelihood estima- ^ ∗ tors θn and θn satisfy the conditions of Theorem 3.3 in Geyer (2013), which we call the LAMN conditions. These are implied by the \usual regularity conditions" for maximum likelihood Appendix C of that article. 2 3 Model and Submodel 3.1 Explicit Submodel Parameterization There are two ways of specifying a submodel. The first is to specify a one-to-one vector-to-vector function h whose range is the parameter space of the submodel. Then we can write θ = h(') (7) and consider ' the parameter for the submodel, so the log likelihood for the parameter ' and for the submodel is ∗ ln(') = ln(h(')): As always, the reference distribution used for calculating P -values or α- levels assumes the truth is in the null hypothesis (submodel), that is, the true unknown parameter value for θ is θ0 = h('0); (8) where '0 is the true unknown parameter value for '. The corresponding \q" function for the submodel is ∗ ∗ −1 ∗ qn(") = ln('0 + τn ") − ln('0): (9) ∗ ∗ Maximizing qn gives the MLE "n for the parameter ". Then ∗ −1 ∗ 'n = '0 + τn "n is the MLE for the parameter ', and ∗ ∗ θn = h('n) is the MLE in the submodel for the original parameter θ. 3.2 Lagrange Multipliers Alternatively, the submodel can be given (as in our discussion of the Wald test) by a constraint function g, in which case the submodel param- eter space is the set of θ such that g(θ) = 0. The submodel MLE can be determined without explicit parameterization of the submodel using the method of Lagrange multipliers. The function T Lλ(θ) = ln(θ) − λ g(θ) (10) 3 is called the Lagrangian function for the problem and λ is called the vector of Lagrange multipliers for the problem. The method of Lagrange multipliers seeks to maximize (10) while simultaneously satisfying the constraint, that is, we attempt to solve the simultaneous equations T rln(θ) − λ rg(θ) = 0 (11a) g(θ) = 0 (11b) for the unknowns θ and λ. The dimension of the model is p and the dimen- sion of the submodel is d, so the dimension of θ is p and the dimension of λ is p − d, and (11a) and (11b) constitute 2p − d nonlinear equations in 2p − d ∗ ∗ unknowns. If they have a solution, denote the solution (θn; λn). ∗ If θn is a local maximizer of the Lagrangian, that is, if there is a neigh- ∗ borhood W of θn such that ∗ ∗ ∗ ∗ ln(θ) − λng(θ) ≤ ln(θn) − λng(θn); θ 2 W; ∗ then clearly θn is a local maximizer for the constrained problem, that is, ∗ ln(θ) ≤ ln(θn); θ 2 W and g(θ) = 0 (and the same holds with \local" replaced by \global" in both places if p W = R ). Although either works, the method of Section 3.1 is more straightforward than the method of this section. Hence we will hereafter concentrate on that. Since none of the test statistics we deal with depend on the method ∗ by which θn is found (only on its value), all of our results apply to either method (assuming the method of Lagrange multipliers does manage to find the correct root of the equations (11a) and (11b)). Our main reason for introducing the method of Lagrange multipliers (other than just to briefly cover them) is to make the name \Lagrange multiplier test" for the Rao test make sense. Obviously, from (11a) we have ∗ ∗ T ∗ rln(θn) = (λn) rg(θn) so the left side, which appears in the Rao test statistic (2) can be computed using the Lagrange multipliers. The name \score test" comes from the name \score function" given to θ 7! rln(θ) by R. A. Fisher. 4 4 The Wilks Test Theorem 1. Under the assumptions of Theorem 3.3 of Geyer (2013) and the setup of the preceding section, assume the function h defining the sub- model parameterization is twice continuously differentiable and H = rh('0) is a full rank matrix, and define ∗ d q (") = q(H");" 2 R ; where, from equation (16) in Geyer (2013), T 1 T p q(δ) = δ Z + 2 δ Kδ; δ 2 R satisfies the LAMN conditions (Z is a random vector, K is a random matrix, K is almost surely positive definite, and the conditional distribution of Z given K is Normal(0;K)). Define δ^n = τn(θ^n − θ0) (12a) ∗ ∗ "n = τn('n − '0) (12b) ^ ∗ where θn is the MLE for the model and 'n is the MLE for the submodel (both produced by one-step-Newton or infinite-step-Newton updates of τn- consistent estimators). Assume τn ! 1 unless the function h is affine (h(') = a + H', where a is a constant vector), in which case τn can be any sequence. Then 0 1 0 1 qn q ∗ ∗ BqnC w B q C B^ C −! B −1 C @δnA @ K Z A ∗ T −1 T "n (H KH) H Z Proof. The asymptotics of qn and δ^n come directly from Theorem 3.3 in Geyer (2013). By the Skorohod theorem there exist random elementsq ~n 2 p andq ~ of C (R ) such thatq ~n and qn have the same laws andq ~ and q have a.s. 2 p the same laws andq ~n −! q~ in C (R ). Clearlyq ~ can have the form T 1 T p q~(δ) = δ Ze + 2 δ Kδ;e δ 2 R ; where the pair (Z;e Ke) has the same law as the pair (Z; K). 5 The definition (6) can be rewritten ln(θ) = ln(θ0) + qn(τn(θ − θ0)) (13) from which we get ∗ −1 qn(") = ln(h('0 + τn ")) − ln(θ0) −1 = qn(τn(h('0 + τn ") − θ0)) ∗ Hence if we defineq ~n by ∗ −1 q~n(") =q ~n(τn(h('0 + τn ") − θ0)) ∗ ∗ thenq ~n will have the same law as qn, and ∗ −1 T −1 rq~n(") = rq~n(τn(h('0 + τn ") − θ0)) rh('0 + τn ") and 2 ∗ r q~n(") = −1 T 2 −1 −1 rh('0 + τn ") r q~n(τn(h('0 + τn ") − θ0)) rh('0 + τn ") −1 −1 2 −1 + τn rq~n(τn(h('0 + τn ") − θ0))r h('0 + τn "): Suppose "n ! ". If we assume τn ! 1, then −1 τn(h('0 + τn "n) − θ0) ! H": (14) If instead of assuming τn ! 1 we assume h is an affine function, then we have equality for all n in (14) and the limit holds trivially.
Load more

Recommended publications
  • Three Statistical Testing Procedures in Logistic Regression: Their Performance in Differential Item Functioning (DIF) Investigation
    Research Report Three Statistical Testing Procedures in Logistic Regression: Their Performance in Differential Item Functioning (DIF) Investigation Insu Paek December 2009 ETS RR-09-35 Listening. Learning. Leading.® Three Statistical Testing Procedures in Logistic Regression: Their Performance in Differential Item Functioning (DIF) Investigation Insu Paek ETS, Princeton, New Jersey December 2009 As part of its nonprofit mission, ETS conducts and disseminates the results of research to advance quality and equity in education and assessment for the benefit of ETS’s constituents and the field. To obtain a PDF or a print copy of a report, please visit: http://www.ets.org/research/contact.html Copyright © 2009 by Educational Testing Service. All rights reserved. ETS, the ETS logo, GRE, and LISTENING. LEARNING. LEADING. are registered trademarks of Educational Testing Service (ETS). SAT is a registered trademark of the College Board. Abstract Three statistical testing procedures well-known in the maximum likelihood approach are the Wald, likelihood ratio (LR), and score tests. Although well-known, the application of these three testing procedures in the logistic regression method to investigate differential item function (DIF) has not been rigorously made yet. Employing a variety of simulation conditions, this research (a) assessed the three tests’ performance for DIF detection and (b) compared DIF detection in different DIF testing modes (targeted vs. general DIF testing). Simulation results showed small differences between the three tests and different testing modes. However, targeted DIF testing consistently performed better than general DIF testing; the three tests differed more in performance in general DIF testing and nonuniform DIF conditions than in targeted DIF testing and uniform DIF conditions; and the LR and score tests consistently performed better than the Wald test.
    [Show full text]
  • Testing for INAR Effects
    Communications in Statistics - Simulation and Computation ISSN: 0361-0918 (Print) 1532-4141 (Online) Journal homepage: https://www.tandfonline.com/loi/lssp20 Testing for INAR effects Rolf Larsson To cite this article: Rolf Larsson (2020) Testing for INAR effects, Communications in Statistics - Simulation and Computation, 49:10, 2745-2764, DOI: 10.1080/03610918.2018.1530784 To link to this article: https://doi.org/10.1080/03610918.2018.1530784 © 2019 The Author(s). Published with license by Taylor & Francis Group, LLC Published online: 21 Jan 2019. Submit your article to this journal Article views: 294 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=lssp20 COMMUNICATIONS IN STATISTICS - SIMULATION AND COMPUTATIONVR 2020, VOL. 49, NO. 10, 2745–2764 https://doi.org/10.1080/03610918.2018.1530784 Testing for INAR effects Rolf Larsson Department of Mathematics, Uppsala University, Uppsala, Sweden ABSTRACT ARTICLE HISTORY In this article, we focus on the integer valued autoregressive model, Received 10 April 2018 INAR (1), with Poisson innovations. We test the null of serial independ- Accepted 25 September 2018 ence, where the INAR parameter is zero, versus the alternative of a KEYWORDS positive INAR parameter. To this end, we propose different explicit INAR model; Likelihood approximations of the likelihood ratio (LR) statistic. We derive the limit- ratio test ing distributions of our statistics under the null. In a simulation study, we compare size and power of our tests with the score test, proposed by Sun and McCabe [2013. Score statistics for testing serial depend- ence in count data.
    [Show full text]
  • Wald (And Score) Tests
    Wald (and Score) Tests 1 / 18 Vector of MLEs is Asymptotically Normal That is, Multivariate Normal This yields I Confidence intervals I Z-tests of H0 : θj = θ0 I Wald tests I Score Tests I Indirectly, the Likelihood Ratio tests 2 / 18 Under Regularity Conditions (Thank you, Mr. Wald) a.s. I θbn → θ √ d −1 I n(θbn − θ) → T ∼ Nk 0, I(θ) 1 −1 I So we say that θbn is asymptotically Nk θ, n I(θ) . I I(θ) is the Fisher Information in one observation. I A k × k matrix ∂2 I(θ) = E[− log f(Y ; θ)] ∂θi∂θj I The Fisher Information in the whole sample is nI(θ) 3 / 18 H0 : Cθ = h Suppose θ = (θ1, . θ7), and the null hypothesis is I θ1 = θ2 I θ6 = θ7 1 1 I 3 (θ1 + θ2 + θ3) = 3 (θ4 + θ5 + θ6) We can write null hypothesis in matrix form as θ1 θ2 1 −1 0 0 0 0 0 θ3 0 0 0 0 0 0 1 −1 θ4 = 0 1 1 1 −1 −1 −1 0 θ5 0 θ6 θ7 4 / 18 p Suppose H0 : Cθ = h is True, and Id(θ)n → I(θ) By Slutsky 6a (Continuous mapping), √ √ d −1 0 n(Cθbn − Cθ) = n(Cθbn − h) → CT ∼ Nk 0, CI(θ) C and −1 p −1 Id(θ)n → I(θ) . Then by Slutsky’s (6c) Stack Theorem, √ ! n(Cθbn − h) d CT → . −1 I(θ)−1 Id(θ)n Finally, by Slutsky 6a again, −1 0 0 −1 Wn = n(Cθb − h) (CId(θ)n C ) (Cθb − h) →d W = (CT − 0)0(CI(θ)−1C0)−1(CT − 0) ∼ χ2(r) 5 / 18 The Wald Test Statistic −1 0 0 −1 Wn = n(Cθbn − h) (CId(θ)n C ) (Cθbn − h) I Again, null hypothesis is H0 : Cθ = h I Matrix C is r × k, r ≤ k, rank r I All we need is a consistent estimator of I(θ) I I(θb) would do I But it’s inconvenient I Need to compute partial derivatives and expected values in ∂2 I(θ) = E[− log f(Y ; θ)] ∂θi∂θj 6 / 18 Observed Fisher Information I To find θbn, minimize the minus log likelihood.
    [Show full text]
  • Comparison of Wald, Score, and Likelihood Ratio Tests for Response Adaptive Designs
    Journal of Statistical Theory and Applications Volume 10, Number 4, 2011, pp. 553-569 ISSN 1538-7887 Comparison of Wald, Score, and Likelihood Ratio Tests for Response Adaptive Designs Yanqing Yi1∗and Xikui Wang2 1 Division of Community Health and Humanities, Faculty of Medicine, Memorial University of Newfoundland, St. Johns, Newfoundland, Canada A1B 3V6 2 Department of Statistics, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2 Abstract Data collected from response adaptive designs are dependent. Traditional statistical methods need to be justified for the use in response adaptive designs. This paper gener- alizes the Rao's score test to response adaptive designs and introduces a generalized score statistic. Simulation is conducted to compare the statistical powers of the Wald, the score, the generalized score and the likelihood ratio statistics. The overall statistical power of the Wald statistic is better than the score, the generalized score and the likelihood ratio statistics for small to medium sample sizes. The score statistic does not show good sample properties for adaptive designs and the generalized score statistic is better than the score statistic under the adaptive designs considered. When the sample size becomes large, the statistical power is similar for the Wald, the sore, the generalized score and the likelihood ratio test statistics. MSC: 62L05, 62F03 Keywords and Phrases: Response adaptive design, likelihood ratio test, maximum likelihood estimation, Rao's score test, statistical power, the Wald test ∗Corresponding author. Fax: 1-709-777-7382. E-mail addresses: [email protected] (Yanqing Yi), xikui [email protected] (Xikui Wang) Y. Yi and X.
    [Show full text]
  • Robust Score and Portmanteau Tests of Volatility Spillover Mike Aguilar, Jonathan B
    Journal of Econometrics 184 (2015) 37–61 Contents lists available at ScienceDirect Journal of Econometrics journal homepage: www.elsevier.com/locate/jeconom Robust score and portmanteau tests of volatility spillover Mike Aguilar, Jonathan B. Hill ∗ Department of Economics, University of North Carolina at Chapel Hill, United States article info a b s t r a c t Article history: This paper presents a variety of tests of volatility spillover that are robust to heavy tails generated by Received 27 March 2012 large errors or GARCH-type feedback. The tests are couched in a general conditional heteroskedasticity Received in revised form framework with idiosyncratic shocks that are only required to have a finite variance if they are 1 September 2014 independent. We negligibly trim test equations, or components of the equations, and construct heavy Accepted 1 September 2014 tail robust score and portmanteau statistics. Trimming is either simple based on an indicator function, or Available online 16 September 2014 smoothed. In particular, we develop the tail-trimmed sample correlation coefficient for robust inference, and prove that its Gaussian limit under the null hypothesis of no spillover has the same standardization JEL classification: C13 irrespective of tail thickness. Further, if spillover occurs within a specified horizon, our test statistics C20 obtain power of one asymptotically. We discuss the choice of trimming portion, including a smoothed C22 p-value over a window of extreme observations. A Monte Carlo study shows our tests provide significant improvements over extant GARCH-based tests of spillover, and we apply the tests to financial returns data. Keywords: Finally, based on ideas in Patton (2011) we construct a heavy tail robust forecast improvement statistic, Volatility spillover which allows us to demonstrate that our spillover test can be used as a model specification pre-test to Heavy tails Tail trimming improve volatility forecasting.
    [Show full text]
  • Statistical Asymptotics Part II: First-Order Theory
    First-Order Asymptotic Theory Statistical Asymptotics Part II: First-Order Theory Andrew Wood School of Mathematical Sciences University of Nottingham APTS, April 15-19 2013 Andrew Wood Statistical Asymptotics Part II: First-Order Theory First-Order Asymptotic Theory Structure of the Chapter This chapter covers asymptotic normality and related results. Topics: MLEs, log-likelihood ratio statistics and their asymptotic distributions; M-estimators and their first-order asymptotic theory. Initially we focus on the case of the MLE of a scalar parameter θ. Then we study the case of the MLE of a vector θ, first without and then with nuisance parameters. Finally, we consider the more general setting of M-estimators. Andrew Wood Statistical Asymptotics Part II: First-Order Theory First-Order Asymptotic Theory Motivation Statistical inference typically requires approximations because exact answers are usually not available. Asymptotic theory provides useful approximations to densities or distribution functions. These approximations are based on results from probability theory. The theory underlying these approximation techniques is valid as some quantity, typically the sample size n [or more generally some measure of information], goes to infinity, but the approximations obtained are often accurate even for small sample sizes. Andrew Wood Statistical Asymptotics Part II: First-Order Theory First-Order Asymptotic Theory Test statistics Consider testing the null hypothesis H0 : θ = θ0, where θ0 is an arbitrary specified point in Ωθ. If desired, we may
    [Show full text]
  • Econometrics-I-11.Pdf
    Econometrics I Professor William Greene Stern School of Business Department of Economics 11-1/78 Part 11: Hypothesis Testing - 2 Econometrics I Part 11 – Hypothesis Testing 11-2/78 Part 11: Hypothesis Testing - 2 Classical Hypothesis Testing We are interested in using the linear regression to support or cast doubt on the validity of a theory about the real world counterpart to our statistical model. The model is used to test hypotheses about the underlying data generating process. 11-3/78 Part 11: Hypothesis Testing - 2 Types of Tests Nested Models: Restriction on the parameters of a particular model y = 1 + 2x + 3T + , 3 = 0 (The “treatment” works; 3 0 .) Nonnested models: E.g., different RHS variables yt = 1 + 2xt + 3xt-1 + t yt = 1 + 2xt + 3yt-1 + wt (Lagged effects occur immediately or spread over time.) Specification tests: ~ N[0,2] vs. some other distribution (The “null” spec. is true or some other spec. is true.) 11-4/78 Part 11: Hypothesis Testing - 2 Hypothesis Testing Nested vs. nonnested specifications y=b1x+e vs. y=b1x+b2z+e: Nested y=bx+e vs. y=cz+u: Not nested y=bx+e vs. logy=clogx: Not nested y=bx+e; e ~ Normal vs. e ~ t[.]: Not nested Fixed vs. random effects: Not nested Logit vs. probit: Not nested x is (not) endogenous: Maybe nested. We’ll see … Parametric restrictions Linear: R-q = 0, R is JxK, J < K, full row rank General: r(,q) = 0, r = a vector of J functions, R(,q) = r(,q)/’. Use r(,q)=0 for linear and nonlinear cases 11-5/78 Part 11: Hypothesis Testing - 2 Broad Approaches Bayesian: Does not reach a firm conclusion.
    [Show full text]
  • Power Analysis for the Wald, LR, Score, and Gradient Tests in a Marginal Maximum Likelihood Framework: Applications in IRT
    Power analysis for the Wald, LR, score, and gradient tests in a marginal maximum likelihood framework: Applications in IRT Felix Zimmer1, Clemens Draxler2, and Rudolf Debelak1 1University of Zurich 2The Health and Life Sciences University March 9, 2021 Abstract The Wald, likelihood ratio, score and the recently proposed gradient statistics can be used to assess a broad range of hypotheses in item re- sponse theory models, for instance, to check the overall model fit or to detect differential item functioning. We introduce new methods for power analysis and sample size planning that can be applied when marginal maximum likelihood estimation is used. This avails the application to a variety of IRT models, which are increasingly used in practice, e.g., in large-scale educational assessments. An analytical method utilizes the asymptotic distributions of the statistics under alternative hypotheses. For a larger number of items, we also provide a sampling-based method, which is necessary due to an exponentially increasing computational load of the analytical approach. We performed extensive simulation studies in two practically relevant settings, i.e., testing a Rasch model against a 2PL model and testing for differential item functioning. The observed distribu- tions of the test statistics and the power of the tests agreed well with the predictions by the proposed methods. We provide an openly accessible R package that implements the methods for user-supplied hypotheses. Keywords: marginal maximum likelihood, item response theory, power analysis, Wald test, score test, likelihood ratio, gradient test We wish to thank Carolin Strobl for helpful comments in the course of this research.
    [Show full text]
  • Package 'Lmtest'
    Package ‘lmtest’ September 9, 2020 Title Testing Linear Regression Models Version 0.9-38 Date 2020-09-09 Description A collection of tests, data sets, and examples for diagnostic checking in linear regression models. Furthermore, some generic tools for inference in parametric models are provided. LazyData yes Depends R (>= 3.0.0), stats, zoo Suggests car, strucchange, sandwich, dynlm, stats4, survival, AER Imports graphics License GPL-2 | GPL-3 NeedsCompilation yes Author Torsten Hothorn [aut] (<https://orcid.org/0000-0001-8301-0471>), Achim Zeileis [aut, cre] (<https://orcid.org/0000-0003-0918-3766>), Richard W. Farebrother [aut] (pan.f), Clint Cummins [aut] (pan.f), Giovanni Millo [ctb], David Mitchell [ctb] Maintainer Achim Zeileis <[email protected]> Repository CRAN Date/Publication 2020-09-09 05:30:11 UTC R topics documented: bgtest . .2 bondyield . .4 bptest . .6 ChickEgg . .8 coeftest . .9 coxtest . 11 currencysubstitution . 13 1 2 bgtest dwtest . 14 encomptest . 16 ftemp . 18 gqtest . 19 grangertest . 21 growthofmoney . 22 harvtest . 23 hmctest . 25 jocci . 26 jtest . 28 lrtest . 29 Mandible . 31 moneydemand . 31 petest . 33 raintest . 35 resettest . 36 unemployment . 38 USDistLag . 39 valueofstocks . 40 wages ............................................ 41 waldtest . 42 Index 46 bgtest Breusch-Godfrey Test Description bgtest performs the Breusch-Godfrey test for higher-order serial correlation. Usage bgtest(formula, order = 1, order.by = NULL, type = c("Chisq", "F"), data = list(), fill = 0) Arguments formula a symbolic description for the model to be tested (or a fitted "lm" object). order integer. maximal order of serial correlation to be tested. order.by Either a vector z or a formula with a single explanatory variable like ~ z.
    [Show full text]
  • Wald (And Score) Tests
    Wald (and score) tests • MLEs have an approximate multivariate normal sampling distribution for large samples (Thanks Mr. Wald.) • Approximate mean vector = vector of true parameter values for large samples • Asymptotic variance-covariance matrix is easy to estimate • H0: Cθ = h (Linear hypothesis) H0 : Cθ = h Cθ h is multivariate normal as n − → ∞ Leads! to a straightforward chisquare test • Called a Wald test • Based on the full model (unrestricted by any null hypothesis) • Asymptotically equivalent to the LR test • Not as good as LR for smaller samples • Very convenient sometimes Example of H0: Cθ=h Suppose θ = (θ1, . θ7), with 1 1 H : θ = θ , θ = θ , (θ + θ + θ ) = (θ + θ + θ ) 0 1 2 6 7 3 1 2 3 3 4 5 6 θ1 θ2 1 1 0 0 0 0 0 θ 0 − 3 0 0 0 0 0 1 1 θ = 0 − 4 1 1 1 1 1 1 0 θ 0 − − − 5 θ6 θ 7 Multivariate Normal Univariate Normal • 1 1 (x µ)2 – f(x) = exp − σ√2π − 2 σ2 (x µ)2 ! " – −σ2 is the squared Euclidian distance between x and µ, in a space that is stretched by σ2. 2 (X µ) 2 – − χ (1) σ2 ∼ Multivariate Normal • 1 1 1 – f(x) = 1 k exp 2 (x µ)#Σ− (x µ) Σ 2 (2π) 2 − − − | | 1 # $ – (x µ)#Σ− (x µ) is the squared Euclidian distance between x and µ, −in a space that− is warped and stretched by Σ. 1 2 – (X µ)#Σ− (X µ) χ (k) − − ∼ Distance: Suppose Σ = I2 2 1 d = (X µ)!Σ− (X µ) − − 1 0 x1 µ1 = x1 µ1, x2 µ2 − − − 0 1 x2 µ2 # $ # − $ ! " x1 µ1 = x1 µ1, x2 µ2 − − − x2 µ2 # − $ = !(x µ )2 + (x µ ")2 1 − 1 2 − 2 2 2 d = (x1 µ1) + (x2 µ2) − − (x1, x2) % x µ 2 − 2 (µ1, µ2) x µ 1 − 1 86 APPENDIX A.
    [Show full text]
  • Rao's Score Test in Econometrics
    UNIVERSITY OF ILLINOIS LIBRARY AT URBANA CHAMPAIGN EOOKSTACKS Digitized by the Internet Archive in 2012 with funding from University of Illinois Urbana-Champaign http://www.archive.org/details/raosscoretestine91132bera Faculty Working Paper 91-0132 330 STX B385 1991: 132 COPY 2 Rao's Score Test in Econometrics Anil K. Bern Aman Ullah Department of Economics Department of Economics University of Illinois University of California Indiana University Riverside o' *9 •the UWatK %2& Bureau of Economic and Business Research College of Commerce and Business Administration University of Illinois at Urbana-Champaign BEBR FACULTY WORKING PAPER NO. 91-0132 College of Commerce and Business Administration University of Illinois at Urbana-Champaign April 1991 Rao's Score Test in Econometrics Anil K. Bera Department of Economics University of Illinois and Indiana University Aman Ullah Department of Economics University of California-Riverside We are grateful to Rick Vinod, Vicki Zinde- Walsh and Mann Yoon for comments on an earlier draft of the paper. Financial support from the Research Board and the Bureau of Economics and Business Research of the University of Illinois is gratefully acknowledged. ABSTRACT Rao's work is always inspired by some practical problems. It is not surprising that many of the techniques he developed found their way to various applications in econometrics. In this review paper, we try to evaluate the impact of his pioneering 1948 score test on the current practice of econometric model specification tests. In so doing, we relate Rao's score principle to some of the earlier popular tests in econometrics. We also discuss the recent developments on this topic and problems for further research.
    [Show full text]
  • Week 4: Simple Linear Regression II
    Week 4: Simple Linear Regression II Marcelo Coca Perraillon University of Colorado Anschutz Medical Campus Health Services Research Methods I HSMP 7607 2019 These slides are part of a forthcoming book to be published by Cambridge University Press. For more information, go to perraillon.com/PLH. c This material is copyrighted. Please see the entire copyright notice on the book's website. Updated notes are here: https://clas.ucdenver.edu/marcelo-perraillon/ teaching/health-services-research-methods-i-hsmp-7607 1 Outline Algebraic properties of OLS Reminder on hypothesis testing The Wald test Examples Another way at looking at causal inference 2 Big picture We used the method of least squares to find the line that minimizes the sum of square errors (SSE) We made NO assumptions about the distribution of or Y We saw that the mean of the predicted values is the same as the mean of the observed values and that implies that predictions regress towards the mean Today, we will assume that distributes N(0; σ2) and are independent (iid). We need that assumption for inference (not to find the best βj ) 3 Algebraic properties of OLS Pn 1) The sum of the residuals is zero: i=1 ^i = 0. One implication of this is that if the residuals add up to zero, then we will always get the mean of Y right. Confusion alert: is the error term; ^ is the residual (more on this later) Recall the first order conditions: @SSE = Pn (y − β^ − β^ x ) = 0 @β0 i=1 i 0 1 i @SSE = Pn x (y − β^ − β^ x ) = 0 @β1 i=1 i i 0 1 i From the first one, it's obvious that we choose β^0; β^1 that satisfy this Pn ^ ^ Pn ¯ property: i=1(yi − β0 − β1xi ) = i=1 ^i = 0, so ^ = 0 In words: On average, the residuals or predicted errors are zero, so on average the predicted outcomey ^ is the same as the observed y.
    [Show full text]