DOCSLIB.ORG

Biostatistics

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Biostatistics Last Lecture . Biostatistics 602 - Statistical Inference Lecture 15 • Can Cramer-Rao bound be used to find the best unbiased estimator for any distribution? If not, in which cases? Bayes Estimator . • When Cramer-Rao bound is attainable, can Cramer-Rao bound be used for find best unbiased estimator for any τ(θ)? If not, what is the Hyun Min Kang restriction on τ(θ)? • What is another way to find the best unbiased estimator? • Describe two strategies to obtain the best unbiased estimators for March 12th, 2013 τ(θ), using complete sufficient statistics. Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 1 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 2 / 1 Recap - The power of complete sufficient statistics Finding UMUVE - Method 1 . .Use Cramer-Rao bound to find the best unbiased estimator for τ(θ). 1. If ”regularity conditions” are satisfied, then we have a Cramer-Rao . bound for unbiased estimators of τ(θ). Theorem 7.3.23 . • It helps to confirm an estimator is the best unbiased estimator of τ(θ) Let T be a complete sufficient statistic for parameter θ. Let ϕ(T) be any if it happens to attain the CR-bound. estimator based on T. Then ϕ(T) is the unique best unbiased estimator of • If an unbiased estimator of τ(θ) has variance greater than the .its expected value. CR-bound, it does NOT mean that it is not the best unbiased estimator. [τ (θ)]2 2. When ”regularity conditions” are not satisfied, ′ is no longer a In(θ) valid lower bound. • There may be unbiased estimators of τ(θ) that have variance smaller 2 than [τ ′(θ)] . In(θ) Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 3 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 4 / 1 Finding UMVUE - Method 2 Frequentists vs. Bayesians A biased view in favor of Bayesians at http://xkcd.com/1132/ . Use complete sufficient statistic to find the best unbiased estimator for .τ(θ). 1. Find complete sufficient statistic T for θ. 2. Obtain ϕ(T), an unbiased estimator of τ(θ) using either of the following two ways • Guess a function ϕ(T) such that E[ϕ(T)] = τ(θ). • Guess an unbiased estimator h(X) of τ(θ). Construct ϕ(T) = E[h(X) T], then E[ϕ(T)] = E[h(X)] = τ(θ). | Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 5 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 6 / 1 Bayesian Statistic Bayesian Framework • Prior distribution of θ : θ π(θ). ∼ Frequentist’s Framework • Sample distribution of X given θ. X θ f(x θ) | ∼ | = X f (x θ), θ Ω . P { ∼ X | ∈ } . • Joint distribution X and θ Bayesian Statistic . f(x, θ) = π(θ)f(x θ) • Parameter θ is considered as a random quantity | • Distribution of θ can be described by probability distribution, referred • Marginal distribution of X. to as prior distribution m(x) = f(x, θ)dθ = f(x θ)π(θ)dθ • A sample is taken from a population indexed by θ, and the prior | ∫θ Ω ∫θ Ω distribution is updated using information from the sample to get ∈ ∈ posterior distribution of θ given the sample. • Posterior distribution of θ (conditional distribution of θ given X) . f(x, θ) f(x θ)π(θ) π(θ x) = = | (Bayes’ Rule) | m(x) m(x) Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 7 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 8 / 1 Example Inference Under Bayesian’s Framework . Leveraging Prior Information . Burglary (θ) Pr(Alarm|Burglary) = Pr(X = 1 θ) Suppose that we know that the chance of Burglary per household per | 7 True (θ = 1) 0.95 night is 10− . False 0.01 Pr(θ = 1) (θ = 0) Pr(θ = 1 X = 1) = Pr(X = 1 θ = 1) (Bayes’ rule) | | Pr(X = 1) Suppose that Burglary is an unobserved parameter (θ 0, 1 ), and Alarm Pr(θ = 1) ∈ { } is an observed outcome (X = 0, 1 ). = Pr(X = 1 θ = 1) { } | Pr(θ = 1, X = 1) + Pr(θ = 0, X = 1) • Under Frequentist’s Framework, Pr(X = 1 θ = 1) Pr(θ = 1) = | • If there was no burglary, there is 1% of chance of alarm ringing. Pr(X = 1 θ = 1) Pr(θ = 1) + Pr(X = 1 θ = 0) Pr(θ = 0) • If there was a burglary, there is % of chance of alarm ringing. 95 | 7 | • One can come up with an estimator on θ, such as MLE 0.95 10− 6 = 7 × 7 9.5 10− • However, given that alarm already rang, one cannot calculate the . 0.95 10− + 0.01 (1 10− ) ≈ × probability of burglary. × × − So, even if alarm rang, one can conclude that the burglary is unlikely to happen. Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 9 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 10 / 1 What if the prior information is misleading? Advantages and Drawbacks of Bayesian Inference . Over-fitting to Prior Information Advantages over Frequentist’s Framework . Suppose that, in fact, a thief found a security breach in my place and • Allows making inference on the distribution of θ given data. planning to break-in either tonight or tomorrow night for sure (with the • Available information about θ can be utilized. same probability). Then the correct prior Pr(θ = 1) = 0.5. • Uncertainty and information can be quantified probabilistically. Pr(θ = 1 X = 1) . | Pr(X = 1 θ = 1) Pr(θ = 1) = | . Pr(X = 1 θ = 1) Pr(θ = 1) + Pr(X = 1 θ = 0) Pr(θ = 0) Drawbacks of Bayesian Inference | | . 0.95 0.5 • Misleading prior can result in misleading inference. = × 0.99 0.95 0.5 + 0.01 (1 0.5) ≈ • Bayesian inference is often (but not always) prone to be ”subjective” × × − • See : Larry Wasserman ”Frequentist Bayes is Objective” (2006) However, if we relied on the inference based on the incorrect prior, we may Bayesian Analysis 3:451-456. end up concluding that there are > 99.9% chance that this is a false alarm, and ignore it, resulting an exchange of one night of good sleep with • Bayesian inference could be sometimes unnecessarily complicated to interpret, compared to Frequentist’s inference. .quite a bit of fortune. Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 11 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 12 / 1 Bayes Estimator Solution (1/4) Prior distribution of p is . Γ(α + β) α 1 β 1 Definition π(p) = p − (1 p) − . Γ(α)Γ(β) − Bayes Estimator of θ is defined as the posterior mean of θ. Sampling distribution of X given p is E(θ x) = θπ(θ x)dθ . | θ Ω | ∫ ∈ n xi 1 xi f (x p) = p (1 p) − . X | − Example Problem i=1 . ∏ { } i.i.d. X , , Xn Bernoulli(p) where 0 p 1. Assume that the prior Joint distribution of X and p is 1 ··· ∼ ≤ ≤ distribution of p is Beta(α, β). Find the posterior distribution of p and the .Bayes estimator of p, assuming α and β are known. fX(x, p) = fX(x p)π(p) n | xi 1 xi Γ(α + β) α 1 β 1 = p (1 p) − p − (1 p) − − Γ(α)Γ(β) − i=1 ∏ { } Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 13 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 14 / 1 Solution (2/4) Solution (3/4) The marginal distribution of X is 1 n n Γ(α + β) xi+α 1 n xi+β 1 The posterior distribution of θ x : m(x) = f(x, p)dp = p i=1 − (1 p) − i=1 − dp | 0 Γ(α)Γ(β) ∑ − ∑ ∫ 1 ∫ f(x, p) Γ(α + β) Γ( xi + α)Γ(n xi + β) π(θ x) = = − | m(x) 0 Γ(α)Γ(β) Γ(α + β + n) ∫ ∑ ∑ Γ(α + β) x n x xi+α 1 n xi+β 1 Γ( i + α + i + β) xi+α 1 n xi+β 1 p − (1 p) − − − p − (1 p) − − dp Γ(α)Γ(β) ∑ − ∑ ×Γ( xi + α)Γ(n xi + β) ∑ − ∑ = [ ] ∑ n − ∑ n Γ(α + β) Γ( xi + α)Γ(n xi + β) Γ(α + β) Γ( xi + α)Γ(n xi + β) − = ∑ i=1 ∑ − i=1 Γ(α)Γ(β) Γ(α + β + n) Γ(α)Γ(β) Γ(α + β + n) [ ∑ ∑ ] ∑ ∑ Γ(α + β + n) 1 = p xi+α 1(1 p)n xi+β 1 x n x − − − fBeta( xi+α,n xi+β)(p)dp Γ( i + α)Γ( i + β) ∑ − ∑ × 0 − − ∫ ∑ n ∑ n Γ(α + β) Γ( xi + α)Γ(n xi + β) ∑ ∑ = i=1 − i=1 Γ(α)Γ(β) Γ(α + β + n) ∑ ∑ Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 15 / 1 Hyun Min Kang Biostatistics 602 - Lecture 15 March 12th, 2013 16 / 1 Solution (4/4) Is the Bayes estimator unbiased? The Bayes estimator of p is n n i=1 xi + α i=1 xi + α pˆ = n n = n x + α + n x + β α + β + n i=1 +α np + α i=1 i ∑ i=1 i ∑ E = = p n − α + β + n α + β + n ̸ xi n α α + β = ∑i=1 +∑ [ ∑ ] n α + β + n α + β α + β + n α ∑ Unless α+β = p. = [Guess about p from data] weight · 1 + [Guess about p from prior] weight np + α α (α + β)p · 2 Bias = p = − α + β + n − α + β + n As n increase, weight = n = 1 becomes bigger and bigger and 1 α+β+n α+β +1 n As n increases, the bias approaches to zero.
Load more

Recommended publications
  • On the Meaning and Use of Kurtosis
    Psychological Methods Copyright 1997 by the American Psychological Association, Inc. 1997, Vol. 2, No. 3,292-307 1082-989X/97/$3.00 On the Meaning and Use of Kurtosis Lawrence T. DeCarlo Fordham University For symmetric unimodal distributions, positive kurtosis indicates heavy tails and peakedness relative to the normal distribution, whereas negative kurtosis indicates light tails and flatness. Many textbooks, however, describe or illustrate kurtosis incompletely or incorrectly. In this article, kurtosis is illustrated with well-known distributions, and aspects of its interpretation and misinterpretation are discussed. The role of kurtosis in testing univariate and multivariate normality; as a measure of departures from normality; in issues of robustness, outliers, and bimodality; in generalized tests and estimators, as well as limitations of and alternatives to the kurtosis measure [32, are discussed. It is typically noted in introductory statistics standard deviation. The normal distribution has a kur- courses that distributions can be characterized in tosis of 3, and 132 - 3 is often used so that the refer- terms of central tendency, variability, and shape. With ence normal distribution has a kurtosis of zero (132 - respect to shape, virtually every textbook defines and 3 is sometimes denoted as Y2)- A sample counterpart illustrates skewness. On the other hand, another as- to 132 can be obtained by replacing the population pect of shape, which is kurtosis, is either not discussed moments with the sample moments, which gives or, worse yet, is often described or illustrated incor- rectly. Kurtosis is also frequently not reported in re- ~(X i -- S)4/n search articles, in spite of the fact that virtually every b2 (•(X i - ~')2/n)2' statistical package provides a measure of kurtosis.
    [Show full text]
  • The Probability Lifesaver: Order Statistics and the Median Theorem
    The Probability Lifesaver: Order Statistics and the Median Theorem Steven J. Miller December 30, 2015 Contents 1 Order Statistics and the Median Theorem 3 1.1 Definition of the Median 5 1.2 Order Statistics 10 1.3 Examples of Order Statistics 15 1.4 TheSampleDistributionoftheMedian 17 1.5 TechnicalboundsforproofofMedianTheorem 20 1.6 TheMedianofNormalRandomVariables 22 2 • Greetings again! In this supplemental chapter we develop the theory of order statistics in order to prove The Median Theorem. This is a beautiful result in its own, but also extremely important as a substitute for the Central Limit Theorem, and allows us to say non- trivial things when the CLT is unavailable. Chapter 1 Order Statistics and the Median Theorem The Central Limit Theorem is one of the gems of probability. It’s easy to use and its hypotheses are satisfied in a wealth of problems. Many courses build towards a proof of this beautiful and powerful result, as it truly is ‘central’ to the entire subject. Not to detract from the majesty of this wonderful result, however, what happens in those instances where it’s unavailable? For example, one of the key assumptions that must be met is that our random variables need to have finite higher moments, or at the very least a finite variance. What if we were to consider sums of Cauchy random variables? Is there anything we can say? This is not just a question of theoretical interest, of mathematicians generalizing for the sake of generalization. The following example from economics highlights why this chapter is more than just of theoretical interest.
    [Show full text]
  • The Bayesian Approach to Statistics
    THE BAYESIAN APPROACH TO STATISTICS ANTHONY O’HAGAN INTRODUCTION the true nature of scientific reasoning. The fi- nal section addresses various features of modern By far the most widely taught and used statisti- Bayesian methods that provide some explanation for the rapid increase in their adoption since the cal methods in practice are those of the frequen- 1980s. tist school. The ideas of frequentist inference, as set out in Chapter 5 of this book, rest on the frequency definition of probability (Chapter 2), BAYESIAN INFERENCE and were developed in the first half of the 20th century. This chapter concerns a radically differ- We first present the basic procedures of Bayesian ent approach to statistics, the Bayesian approach, inference. which depends instead on the subjective defini- tion of probability (Chapter 3). In some respects, Bayesian methods are older than frequentist ones, Bayes’s Theorem and the Nature of Learning having been the basis of very early statistical rea- Bayesian inference is a process of learning soning as far back as the 18th century. Bayesian from data. To give substance to this statement, statistics as it is now understood, however, dates we need to identify who is doing the learning and back to the 1950s, with subsequent development what they are learning about. in the second half of the 20th century. Over that time, the Bayesian approach has steadily gained Terms and Notation ground, and is now recognized as a legitimate al- ternative to the frequentist approach. The person doing the learning is an individual This chapter is organized into three sections.
    [Show full text]
  • Approximated Bayes and Empirical Bayes Confidence Intervals—
    Ann. Inst. Statist. Math. Vol. 40, No. 4, 747-767 (1988) APPROXIMATED BAYES AND EMPIRICAL BAYES CONFIDENCE INTERVALSmTHE KNOWN VARIANCE CASE* A. J. VAN DER MERWE, P. C. N. GROENEWALD AND C. A. VAN DER MERWE Department of Mathematical Statistics, University of the Orange Free State, PO Box 339, Bloemfontein, Republic of South Africa (Received June 11, 1986; revised September 29, 1987) Abstract. In this paper hierarchical Bayes and empirical Bayes results are used to obtain confidence intervals of the population means in the case of real problems. This is achieved by approximating the posterior distribution with a Pearson distribution. In the first example hierarchical Bayes confidence intervals for the Efron and Morris (1975, J. Amer. Statist. Assoc., 70, 311-319) baseball data are obtained. The same methods are used in the second example to obtain confidence intervals of treatment effects as well as the difference between treatment effects in an analysis of variance experiment. In the third example hierarchical Bayes intervals of treatment effects are obtained and compared with normal approximations in the unequal variance case. Key words and phrases: Hierarchical Bayes, empirical Bayes estimation, Stein estimator, multivariate normal mean, Pearson curves, confidence intervals, posterior distribution, unequal variance case, normal approxima- tions. 1. Introduction In the Bayesian approach to inference, a posterior distribution of unknown parameters is produced as the normalized product of the like- lihood and a prior distribution. Inferences about the unknown parameters are then based on the entire posterior distribution resulting from the one specific data set which has actually occurred. In most hierarchical and empirical Bayes cases these posterior distributions are difficult to derive and cannot be obtained in closed form.
    [Show full text]
  • Theoretical Statistics. Lecture 5. Peter Bartlett
    Theoretical Statistics. Lecture 5. Peter Bartlett 1. U-statistics. 1 Outline of today’s lecture We’ll look at U-statistics, a family of estimators that includes many interesting examples. We’ll study their properties: unbiased, lower variance, concentration (via an application of the bounded differences inequality), asymptotic variance, asymptotic distribution. (See Chapter 12 of van der Vaart.) First, we’ll consider the standard unbiased estimate of variance—a special case of a U-statistic. 2 Variance estimates n 1 s2 = (X − X¯ )2 n n − 1 i n Xi=1 n n 1 = (X − X¯ )2 +(X − X¯ )2 2n(n − 1) i n j n Xi=1 Xj=1 n n 1 2 = (X − X¯ ) − (X − X¯ ) 2n(n − 1) i n j n Xi=1 Xj=1 n n 1 1 = (X − X )2 n(n − 1) 2 i j Xi=1 Xj=1 1 1 = (X − X )2 . n 2 i j 2 Xi<j 3 Variance estimates This is unbiased for i.i.d. data: 1 Es2 = E (X − X )2 n 2 1 2 1 = E ((X − EX ) − (X − EX ))2 2 1 1 2 2 1 2 2 = E (X1 − EX1) +(X2 − EX2) 2 2 = E (X1 − EX1) . 4 U-statistics Definition: A U-statistic of order r with kernel h is 1 U = n h(Xi1 ,...,Xir ), r iX⊆[n] where h is symmetric in its arguments. [If h is not symmetric in its arguments, we can also average over permutations.] “U” for “unbiased.” Introduced by Wassily Hoeffding in the 1940s. 5 U-statistics Theorem: [Halmos] θ (parameter, i.e., function defined on a family of distributions) admits an unbiased estimator (ie: for all sufficiently large n, some function of the i.i.d.
    [Show full text]
  • Scalable and Robust Bayesian Inference Via the Median Posterior
    Scalable and Robust Bayesian Inference via the Median Posterior CS 584: Big Data Analytics Material adapted from David Dunson’s talk (http://bayesian.org/sites/default/files/Dunson.pdf) & Lizhen Lin’s ICML talk (http://techtalks.tv/talks/scalable-and-robust-bayesian-inference-via-the-median-posterior/61140/) Big Data Analytics • Large (big N) and complex (big P with interactions) data are collected routinely • Both speed & generality of data analysis methods are important • Bayesian approaches offer an attractive general approach for modeling the complexity of big data • Computational intractability of posterior sampling is a major impediment to application of flexible Bayesian methods CS 584 [Spring 2016] - Ho Existing Frequentist Approaches: The Positives • Optimization-based approaches, such as ADMM or glmnet, are currently most popular for analyzing big data • General and computationally efficient • Used orders of magnitude more than Bayes methods • Can exploit distributed & cloud computing platforms • Can borrow some advantages of Bayes methods through penalties and regularization CS 584 [Spring 2016] - Ho Existing Frequentist Approaches: The Drawbacks • Such optimization-based methods do not provide measure of uncertainty • Uncertainty quantification is crucial for most applications • Scalable penalization methods focus primarily on convex optimization — greatly limits scope and puts ceiling on performance • For non-convex problems and data with complex structure, existing optimization algorithms can fail badly CS 584 [Spring 2016] -
    [Show full text]
  • 1 Estimation and Beyond in the Bayes Universe
    ISyE8843A, Brani Vidakovic Handout 7 1 Estimation and Beyond in the Bayes Universe. 1.1 Estimation No Bayes estimate can be unbiased but Bayesians are not upset! No Bayes estimate with respect to the squared error loss can be unbiased, except in a trivial case when its Bayes’ risk is 0. Suppose that for a proper prior ¼ the Bayes estimator ±¼(X) is unbiased, Xjθ (8θ)E ±¼(X) = θ: This implies that the Bayes risk is 0. The Bayes risk of ±¼(X) can be calculated as repeated expectation in two ways, θ Xjθ 2 X θjX 2 r(¼; ±¼) = E E (θ ¡ ±¼(X)) = E E (θ ¡ ±¼(X)) : Thus, conveniently choosing either EθEXjθ or EX EθjX and using the properties of conditional expectation we have, θ Xjθ 2 θ Xjθ X θjX X θjX 2 r(¼; ±¼) = E E θ ¡ E E θ±¼(X) ¡ E E θ±¼(X) + E E ±¼(X) θ Xjθ 2 θ Xjθ X θjX X θjX 2 = E E θ ¡ E θ[E ±¼(X)] ¡ E ±¼(X)E θ + E E ±¼(X) θ Xjθ 2 θ X X θjX 2 = E E θ ¡ E θ ¢ θ ¡ E ±¼(X)±¼(X) + E E ±¼(X) = 0: Bayesians are not upset. To check for its unbiasedness, the Bayes estimator is averaged with respect to the model measure (Xjθ), and one of the Bayesian commandments is: Thou shall not average with respect to sample space, unless you have Bayesian design in mind. Even frequentist agree that insisting on unbiasedness can lead to bad estimators, and that in their quest to minimize the risk by trading off between variance and bias-squared a small dosage of bias can help.
    [Show full text]
  • Download Article (PDF)
    Journal of Statistical Theory and Applications, Vol. 17, No. 2 (June 2018) 359–374 ___________________________________________________________________________________________________________ BAYESIAN APPROACH IN ESTIMATION OF SHAPE PARAMETER OF THE EXPONENTIATED MOMENT EXPONENTIAL DISTRIBUTION Kawsar Fatima Department of Statistics, University of Kashmir, Srinagar, India [email protected] S.P Ahmad* Department of Statistics, University of Kashmir, Srinagar, India [email protected] Received 1 November 2016 Accepted 19 June 2017 Abstract In this paper, Bayes estimators of the unknown shape parameter of the exponentiated moment exponential distribution (EMED)have been derived by using two informative (gamma and chi-square) priors and two non- informative (Jeffrey’s and uniform) priors under different loss functions, namely, Squared Error Loss function, Entropy loss function and precautionary Loss function. The Maximum likelihood estimator (MLE) is obtained. Also, we used two real life data sets to illustrate the result derived. Keywords: Exponentiated Moment Exponential distribution; Maximum Likelihood Estimator; Bayesian estimation; Priors; Loss functions. 2000 Mathematics Subject Classification: 22E46, 53C35, 57S20 1. Introduction The exponentiated exponential distribution is a specific family of the exponentiated Weibull distribution. In analyzing several life time data situations, it has been observed that the dual parameter exponentiated exponential distribution can be more effectively used as compared to both dual parameters of gamma or Weibull distribution. When we consider the shape parameter of exponentiated exponential, gamma and Weibull is one, then these distributions becomes one parameter exponential distribution. Hence, these three distributions are the off shoots of the exponential distribution. Moment distributions have a vital role in mathematics and statistics, in particular probability theory, in the viewpoint research related to ecology, reliability, biomedical field, econometrics, survey sampling and in life-testing.
    [Show full text]
  • Lecture 14 Testing for Kurtosis
    9/8/2016 CHE384, From Data to Decisions: Measurement, Kurtosis Uncertainty, Analysis, and Modeling • For any distribution, the kurtosis (sometimes Lecture 14 called the excess kurtosis) is defined as Testing for Kurtosis 3 (old notation = ) • For a unimodal, symmetric distribution, Chris A. Mack – a positive kurtosis means “heavy tails” and a more Adjunct Associate Professor peaked center compared to a normal distribution – a negative kurtosis means “light tails” and a more spread center compared to a normal distribution http://www.lithoguru.com/scientist/statistics/ © Chris Mack, 2016Data to Decisions 1 © Chris Mack, 2016Data to Decisions 2 Kurtosis Examples One Impact of Excess Kurtosis • For the Student’s t • For a normal distribution, the sample distribution, the variance will have an expected value of s2, excess kurtosis is and a variance of 6 2 4 1 for DF > 4 ( for DF ≤ 4 the kurtosis is infinite) • For a distribution with excess kurtosis • For a uniform 2 1 1 distribution, 1 2 © Chris Mack, 2016Data to Decisions 3 © Chris Mack, 2016Data to Decisions 4 Sample Kurtosis Sample Kurtosis • For a sample of size n, the sample kurtosis is • An unbiased estimator of the sample excess 1 kurtosis is ∑ ̅ 1 3 3 1 6 1 2 3 ∑ ̅ Standard Error: • For large n, the sampling distribution of 1 24 2 1 approaches Normal with mean 0 and variance 2 1 of 24/n 3 5 • For small samples, this estimator is biased D. N. Joanes and C. A. Gill, “Comparing Measures of Sample Skewness and Kurtosis”, The Statistician, 47(1),183–189 (1998).
    [Show full text]
  • Covariances of Two Sample Rank Sum Statistics
    JOURNAL OF RESEARCH of the National Bureou of Standards - B. Mathematical Sciences Volume 76B, Nos. 1 and 2, January-June 1972 Covariances of Two Sample Rank Sum Statistics Peter V. Tryon Institute for Basic Standards, National Bureau of Standards, Boulder, Colorado 80302 (November 26, 1971) This note presents an elementary derivation of the covariances of the e(e - 1)/2 two-sample rank sum statistics computed among aU pairs of samples from e populations. Key words: e Sample proble m; covariances, Mann-Whitney-Wilcoxon statistics; rank sum statistics; statistics. Mann-Whitney or Wilcoxon rank sum statistics, computed for some or all of the c(c - 1)/2 pairs of samples from c populations, have been used in testing the null hypothesis of homogeneity of dis­ tribution against a variety of alternatives [1, 3,4,5).1 This note presents an elementary derivation of the covariances of such statistics under the null hypothesis_ The usual approach to such an analysis is the rank sum viewpoint of the Wilcoxon form of the statistic. Using this approach, Steel [3] presents a lengthy derivation of the covariances. In this note it is shown that thinking in terms of the Mann-Whitney form of the statistic leads to an elementary derivation. For comparison and completeness the rank sum derivation of Kruskal and Wallis [2] is repeated in obtaining the means and variances. Let x{, r= 1,2, ... , ni, i= 1,2, ... , c, be the rth item in the sample of size ni from the ith of c populations. Let Mij be the Mann-Whitney statistic between the ith andjth samples defined by n· n· Mij= ~ t z[J (1) s=1 1' = 1 where l,xJ~>xr Zr~= { I) O,xj";;;x[ } Thus Mij is the number of times items in the jth sample exceed items in the ith sample.
    [Show full text]
  • Analysis of Covariance (ANCOVA) with Two Groups
    NCSS Statistical Software NCSS.com Chapter 226 Analysis of Covariance (ANCOVA) with Two Groups Introduction This procedure performs analysis of covariance (ANCOVA) for a grouping variable with 2 groups and one covariate variable. This procedure uses multiple regression techniques to estimate model parameters and compute least squares means. This procedure also provides standard error estimates for least squares means and their differences, and computes the T-test for the difference between group means adjusted for the covariate. The procedure also provides response vs covariate by group scatter plots and residuals for checking model assumptions. This procedure will output results for a simple two-sample equal-variance T-test if no covariate is entered and simple linear regression if no group variable is entered. This allows you to complete the ANCOVA analysis if either the group variable or covariate is determined to be non-significant. For additional options related to the T- test and simple linear regression analyses, we suggest you use the corresponding procedures in NCSS. The group variable in this procedure is restricted to two groups. If you want to perform ANCOVA with a group variable that has three or more groups, use the One-Way Analysis of Covariance (ANCOVA) procedure. This procedure cannot be used to analyze models that include more than one covariate variable or more than one group variable. If the model you want to analyze includes more than one covariate variable and/or more than one group variable, use the General Linear Models (GLM) for Fixed Factors procedure instead. Kinds of Research Questions A large amount of research consists of studying the influence of a set of independent variables on a response (dependent) variable.
    [Show full text]
  • Introduction to Bayesian Inference and Modeling Edps 590BAY
    Introduction to Bayesian Inference and Modeling Edps 590BAY Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Fall 2019 Introduction What Why Probability Steps Example History Practice Overview ◮ What is Bayes theorem ◮ Why Bayesian analysis ◮ What is probability? ◮ Basic Steps ◮ An little example ◮ History (not all of the 705+ people that influenced development of Bayesian approach) ◮ In class work with probabilities Depending on the book that you select for this course, read either Gelman et al. Chapter 1 or Kruschke Chapters 1 & 2. C.J. Anderson (Illinois) Introduction Fall 2019 2.2/ 29 Introduction What Why Probability Steps Example History Practice Main References for Course Throughout the coures, I will take material from ◮ Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., & Rubin, D.B. (20114). Bayesian Data Analysis, 3rd Edition. Boco Raton, FL, CRC/Taylor & Francis.** ◮ Hoff, P.D., (2009). A First Course in Bayesian Statistical Methods. NY: Sringer.** ◮ McElreath, R.M. (2016). Statistical Rethinking: A Bayesian Course with Examples in R and Stan. Boco Raton, FL, CRC/Taylor & Francis. ◮ Kruschke, J.K. (2015). Doing Bayesian Data Analysis: A Tutorial with JAGS and Stan. NY: Academic Press.** ** There are e-versions these of from the UofI library. There is a verson of McElreath, but I couldn’t get if from UofI e-collection. C.J. Anderson (Illinois) Introduction Fall 2019 3.3/ 29 Introduction What Why Probability Steps Example History Practice Bayes Theorem A whole semester on this? p(y|θ)p(θ) p(θ|y)= p(y) where ◮ y is data, sample from some population.
    [Show full text]