Unit 4: Measures of Center
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Image Segmentation Based on Histogram Analysis Utilizing the Cloud Model
Computers and Mathematics with Applications 62 (2011) 2824–2833 Contents lists available at SciVerse ScienceDirect Computers and Mathematics with Applications journal homepage: www.elsevier.com/locate/camwa Image segmentation based on histogram analysis utilizing the cloud model Kun Qin a,∗, Kai Xu a, Feilong Liu b, Deyi Li c a School of Remote Sensing Information Engineering, Wuhan University, Wuhan, 430079, China b Bahee International, Pleasant Hill, CA 94523, USA c Beijing Institute of Electronic System Engineering, Beijing, 100039, China article info a b s t r a c t Keywords: Both the cloud model and type-2 fuzzy sets deal with the uncertainty of membership Image segmentation which traditional type-1 fuzzy sets do not consider. Type-2 fuzzy sets consider the Histogram analysis fuzziness of the membership degrees. The cloud model considers fuzziness, randomness, Cloud model Type-2 fuzzy sets and the association between them. Based on the cloud model, the paper proposes an Probability to possibility transformations image segmentation approach which considers the fuzziness and randomness in histogram analysis. For the proposed method, first, the image histogram is generated. Second, the histogram is transformed into discrete concepts expressed by cloud models. Finally, the image is segmented into corresponding regions based on these cloud models. Segmentation experiments by images with bimodal and multimodal histograms are used to compare the proposed method with some related segmentation methods, including Otsu threshold, type-2 fuzzy threshold, fuzzy C-means clustering, and Gaussian mixture models. The comparison experiments validate the proposed method. ' 2011 Elsevier Ltd. All rights reserved. 1. Introduction In order to deal with the uncertainty of image segmentation, fuzzy sets were introduced into the field of image segmentation, and some methods were proposed in the literature. -
On the Meaning and Use of Kurtosis
Psychological Methods Copyright 1997 by the American Psychological Association, Inc. 1997, Vol. 2, No. 3,292-307 1082-989X/97/$3.00 On the Meaning and Use of Kurtosis Lawrence T. DeCarlo Fordham University For symmetric unimodal distributions, positive kurtosis indicates heavy tails and peakedness relative to the normal distribution, whereas negative kurtosis indicates light tails and flatness. Many textbooks, however, describe or illustrate kurtosis incompletely or incorrectly. In this article, kurtosis is illustrated with well-known distributions, and aspects of its interpretation and misinterpretation are discussed. The role of kurtosis in testing univariate and multivariate normality; as a measure of departures from normality; in issues of robustness, outliers, and bimodality; in generalized tests and estimators, as well as limitations of and alternatives to the kurtosis measure [32, are discussed. It is typically noted in introductory statistics standard deviation. The normal distribution has a kur- courses that distributions can be characterized in tosis of 3, and 132 - 3 is often used so that the refer- terms of central tendency, variability, and shape. With ence normal distribution has a kurtosis of zero (132 - respect to shape, virtually every textbook defines and 3 is sometimes denoted as Y2)- A sample counterpart illustrates skewness. On the other hand, another as- to 132 can be obtained by replacing the population pect of shape, which is kurtosis, is either not discussed moments with the sample moments, which gives or, worse yet, is often described or illustrated incor- rectly. Kurtosis is also frequently not reported in re- ~(X i -- S)4/n search articles, in spite of the fact that virtually every b2 (•(X i - ~')2/n)2' statistical package provides a measure of kurtosis. -
The Probability Lifesaver: Order Statistics and the Median Theorem
The Probability Lifesaver: Order Statistics and the Median Theorem Steven J. Miller December 30, 2015 Contents 1 Order Statistics and the Median Theorem 3 1.1 Definition of the Median 5 1.2 Order Statistics 10 1.3 Examples of Order Statistics 15 1.4 TheSampleDistributionoftheMedian 17 1.5 TechnicalboundsforproofofMedianTheorem 20 1.6 TheMedianofNormalRandomVariables 22 2 • Greetings again! In this supplemental chapter we develop the theory of order statistics in order to prove The Median Theorem. This is a beautiful result in its own, but also extremely important as a substitute for the Central Limit Theorem, and allows us to say non- trivial things when the CLT is unavailable. Chapter 1 Order Statistics and the Median Theorem The Central Limit Theorem is one of the gems of probability. It’s easy to use and its hypotheses are satisfied in a wealth of problems. Many courses build towards a proof of this beautiful and powerful result, as it truly is ‘central’ to the entire subject. Not to detract from the majesty of this wonderful result, however, what happens in those instances where it’s unavailable? For example, one of the key assumptions that must be met is that our random variables need to have finite higher moments, or at the very least a finite variance. What if we were to consider sums of Cauchy random variables? Is there anything we can say? This is not just a question of theoretical interest, of mathematicians generalizing for the sake of generalization. The following example from economics highlights why this chapter is more than just of theoretical interest. -
Permutation Tests
Permutation tests Ken Rice Thomas Lumley UW Biostatistics Seattle, June 2008 Overview • Permutation tests • A mean • Smallest p-value across multiple models • Cautionary notes Testing In testing a null hypothesis we need a test statistic that will have different values under the null hypothesis and the alternatives we care about (eg a relative risk of diabetes) We then need to compute the sampling distribution of the test statistic when the null hypothesis is true. For some test statistics and some null hypotheses this can be done analytically. The p- value for the is the probability that the test statistic would be at least as extreme as we observed, if the null hypothesis is true. A permutation test gives a simple way to compute the sampling distribution for any test statistic, under the strong null hypothesis that a set of genetic variants has absolutely no effect on the outcome. Permutations To estimate the sampling distribution of the test statistic we need many samples generated under the strong null hypothesis. If the null hypothesis is true, changing the exposure would have no effect on the outcome. By randomly shuffling the exposures we can make up as many data sets as we like. If the null hypothesis is true the shuffled data sets should look like the real data, otherwise they should look different from the real data. The ranking of the real test statistic among the shuffled test statistics gives a p-value Example: null is true Data Shuffling outcomes Shuffling outcomes (ordered) gender outcome gender outcome gender outcome Example: null is false Data Shuffling outcomes Shuffling outcomes (ordered) gender outcome gender outcome gender outcome Means Our first example is a difference in mean outcome in a dominant model for a single SNP ## make up some `true' data carrier<-rep(c(0,1), c(100,200)) null.y<-rnorm(300) alt.y<-rnorm(300, mean=carrier/2) In this case we know from theory the distribution of a difference in means and we could just do a t-test. -
Theoretical Statistics. Lecture 5. Peter Bartlett
Theoretical Statistics. Lecture 5. Peter Bartlett 1. U-statistics. 1 Outline of today’s lecture We’ll look at U-statistics, a family of estimators that includes many interesting examples. We’ll study their properties: unbiased, lower variance, concentration (via an application of the bounded differences inequality), asymptotic variance, asymptotic distribution. (See Chapter 12 of van der Vaart.) First, we’ll consider the standard unbiased estimate of variance—a special case of a U-statistic. 2 Variance estimates n 1 s2 = (X − X¯ )2 n n − 1 i n Xi=1 n n 1 = (X − X¯ )2 +(X − X¯ )2 2n(n − 1) i n j n Xi=1 Xj=1 n n 1 2 = (X − X¯ ) − (X − X¯ ) 2n(n − 1) i n j n Xi=1 Xj=1 n n 1 1 = (X − X )2 n(n − 1) 2 i j Xi=1 Xj=1 1 1 = (X − X )2 . n 2 i j 2 Xi<j 3 Variance estimates This is unbiased for i.i.d. data: 1 Es2 = E (X − X )2 n 2 1 2 1 = E ((X − EX ) − (X − EX ))2 2 1 1 2 2 1 2 2 = E (X1 − EX1) +(X2 − EX2) 2 2 = E (X1 − EX1) . 4 U-statistics Definition: A U-statistic of order r with kernel h is 1 U = n h(Xi1 ,...,Xir ), r iX⊆[n] where h is symmetric in its arguments. [If h is not symmetric in its arguments, we can also average over permutations.] “U” for “unbiased.” Introduced by Wassily Hoeffding in the 1940s. 5 U-statistics Theorem: [Halmos] θ (parameter, i.e., function defined on a family of distributions) admits an unbiased estimator (ie: for all sufficiently large n, some function of the i.i.d. -
Lecture 14 Testing for Kurtosis
9/8/2016 CHE384, From Data to Decisions: Measurement, Kurtosis Uncertainty, Analysis, and Modeling • For any distribution, the kurtosis (sometimes Lecture 14 called the excess kurtosis) is defined as Testing for Kurtosis 3 (old notation = ) • For a unimodal, symmetric distribution, Chris A. Mack – a positive kurtosis means “heavy tails” and a more Adjunct Associate Professor peaked center compared to a normal distribution – a negative kurtosis means “light tails” and a more spread center compared to a normal distribution http://www.lithoguru.com/scientist/statistics/ © Chris Mack, 2016Data to Decisions 1 © Chris Mack, 2016Data to Decisions 2 Kurtosis Examples One Impact of Excess Kurtosis • For the Student’s t • For a normal distribution, the sample distribution, the variance will have an expected value of s2, excess kurtosis is and a variance of 6 2 4 1 for DF > 4 ( for DF ≤ 4 the kurtosis is infinite) • For a distribution with excess kurtosis • For a uniform 2 1 1 distribution, 1 2 © Chris Mack, 2016Data to Decisions 3 © Chris Mack, 2016Data to Decisions 4 Sample Kurtosis Sample Kurtosis • For a sample of size n, the sample kurtosis is • An unbiased estimator of the sample excess 1 kurtosis is ∑ ̅ 1 3 3 1 6 1 2 3 ∑ ̅ Standard Error: • For large n, the sampling distribution of 1 24 2 1 approaches Normal with mean 0 and variance 2 1 of 24/n 3 5 • For small samples, this estimator is biased D. N. Joanes and C. A. Gill, “Comparing Measures of Sample Skewness and Kurtosis”, The Statistician, 47(1),183–189 (1998). -
Covariances of Two Sample Rank Sum Statistics
JOURNAL OF RESEARCH of the National Bureou of Standards - B. Mathematical Sciences Volume 76B, Nos. 1 and 2, January-June 1972 Covariances of Two Sample Rank Sum Statistics Peter V. Tryon Institute for Basic Standards, National Bureau of Standards, Boulder, Colorado 80302 (November 26, 1971) This note presents an elementary derivation of the covariances of the e(e - 1)/2 two-sample rank sum statistics computed among aU pairs of samples from e populations. Key words: e Sample proble m; covariances, Mann-Whitney-Wilcoxon statistics; rank sum statistics; statistics. Mann-Whitney or Wilcoxon rank sum statistics, computed for some or all of the c(c - 1)/2 pairs of samples from c populations, have been used in testing the null hypothesis of homogeneity of dis tribution against a variety of alternatives [1, 3,4,5).1 This note presents an elementary derivation of the covariances of such statistics under the null hypothesis_ The usual approach to such an analysis is the rank sum viewpoint of the Wilcoxon form of the statistic. Using this approach, Steel [3] presents a lengthy derivation of the covariances. In this note it is shown that thinking in terms of the Mann-Whitney form of the statistic leads to an elementary derivation. For comparison and completeness the rank sum derivation of Kruskal and Wallis [2] is repeated in obtaining the means and variances. Let x{, r= 1,2, ... , ni, i= 1,2, ... , c, be the rth item in the sample of size ni from the ith of c populations. Let Mij be the Mann-Whitney statistic between the ith andjth samples defined by n· n· Mij= ~ t z[J (1) s=1 1' = 1 where l,xJ~>xr Zr~= { I) O,xj";;;x[ } Thus Mij is the number of times items in the jth sample exceed items in the ith sample. -
Analysis of Covariance (ANCOVA) with Two Groups
NCSS Statistical Software NCSS.com Chapter 226 Analysis of Covariance (ANCOVA) with Two Groups Introduction This procedure performs analysis of covariance (ANCOVA) for a grouping variable with 2 groups and one covariate variable. This procedure uses multiple regression techniques to estimate model parameters and compute least squares means. This procedure also provides standard error estimates for least squares means and their differences, and computes the T-test for the difference between group means adjusted for the covariate. The procedure also provides response vs covariate by group scatter plots and residuals for checking model assumptions. This procedure will output results for a simple two-sample equal-variance T-test if no covariate is entered and simple linear regression if no group variable is entered. This allows you to complete the ANCOVA analysis if either the group variable or covariate is determined to be non-significant. For additional options related to the T- test and simple linear regression analyses, we suggest you use the corresponding procedures in NCSS. The group variable in this procedure is restricted to two groups. If you want to perform ANCOVA with a group variable that has three or more groups, use the One-Way Analysis of Covariance (ANCOVA) procedure. This procedure cannot be used to analyze models that include more than one covariate variable or more than one group variable. If the model you want to analyze includes more than one covariate variable and/or more than one group variable, use the General Linear Models (GLM) for Fixed Factors procedure instead. Kinds of Research Questions A large amount of research consists of studying the influence of a set of independent variables on a response (dependent) variable. -
What Is Statistic?
What is Statistic? OPRE 6301 In today’s world. ...we are constantly being bombarded with statistics and statistical information. For example: Customer Surveys Medical News Demographics Political Polls Economic Predictions Marketing Information Sales Forecasts Stock Market Projections Consumer Price Index Sports Statistics How can we make sense out of all this data? How do we differentiate valid from flawed claims? 1 What is Statistics?! “Statistics is a way to get information from data.” Statistics Data Information Data: Facts, especially Information: Knowledge numerical facts, collected communicated concerning together for reference or some particular fact. information. Statistics is a tool for creating an understanding from a set of numbers. Humorous Definitions: The Science of drawing a precise line between an unwar- ranted assumption and a forgone conclusion. The Science of stating precisely what you don’t know. 2 An Example: Stats Anxiety. A business school student is anxious about their statistics course, since they’ve heard the course is difficult. The professor provides last term’s final exam marks to the student. What can be discerned from this list of numbers? Statistics Data Information List of last term’s marks. New information about the statistics class. 95 89 70 E.g. Class average, 65 Proportion of class receiving A’s 78 Most frequent mark, 57 Marks distribution, etc. : 3 Key Statistical Concepts. Population — a population is the group of all items of interest to a statistics practitioner. — frequently very large; sometimes infinite. E.g. All 5 million Florida voters (per Example 12.5). Sample — A sample is a set of data drawn from the population. -
Shinyitemanalysis for Psychometric Training and to Enforce Routine Analysis of Educational Tests
ShinyItemAnalysis for Psychometric Training and to Enforce Routine Analysis of Educational Tests Patrícia Martinková Dept. of Statistical Modelling, Institute of Computer Science, Czech Academy of Sciences College of Education, Charles University in Prague R meetup Warsaw, May 24, 2018 R meetup Warsaw, 2018 1/35 Introduction ShinyItemAnalysis Teaching psychometrics Routine analysis of tests Discussion Announcement 1: Save the date for Psychoco 2019! International Workshop on Psychometric Computing Psychoco 2019 February 21 - 22, 2019 Charles University & Czech Academy of Sciences, Prague www.psychoco.org Since 2008, the international Psychoco workshops aim at bringing together researchers working on modern techniques for the analysis of data from psychology and the social sciences (especially in R). Patrícia Martinková ShinyItemAnalysis for Psychometric Training and Test Validation R meetup Warsaw, 2018 2/35 Introduction ShinyItemAnalysis Teaching psychometrics Routine analysis of tests Discussion Announcement 2: Job offers Job offers at Institute of Computer Science: CAS-ICS Postdoctoral position (deadline: August 30) ICS Doctoral position (deadline: June 30) ICS Fellowship for junior researchers (deadline: June 30) ... further possibilities to participate on grants E-mail at [email protected] if interested in position in the area of Computational psychometrics Interdisciplinary statistics Other related disciplines Patrícia Martinková ShinyItemAnalysis for Psychometric Training and Test Validation R meetup Warsaw, 2018 3/35 Outline 1. Introduction 2. ShinyItemAnalysis 3. Teaching psychometrics 4. Routine analysis of tests 5. Discussion Introduction ShinyItemAnalysis Teaching psychometrics Routine analysis of tests Discussion Motivation To teach psychometric concepts and methods Graduate courses "IRT models", "Selected topics in psychometrics" Workshops for admission test developers Active learning approach w/ hands-on examples To enforce routine analyses of educational tests Admission tests to Czech Universities Physiology concept inventories .. -
Lecture 8 Sample Mean and Variance, Histogram, Empirical Distribution Function Sample Mean and Variance
Lecture 8 Sample Mean and Variance, Histogram, Empirical Distribution Function Sample Mean and Variance Consider a random sample , where is the = , , … sample size (number of elements in ). Then, the sample mean is given by 1 = . Sample Mean and Variance The sample mean is an unbiased estimator of the expected value of a random variable the sample is generated from: . The sample mean is the sample statistic, and is itself a random variable. Sample Mean and Variance In many practical situations, the true variance of a population is not known a priori and must be computed somehow. When dealing with extremely large populations, it is not possible to count every object in the population, so the computation must be performed on a sample of the population. Sample Mean and Variance Sample variance can also be applied to the estimation of the variance of a continuous distribution from a sample of that distribution. Sample Mean and Variance Consider a random sample of size . Then, = , , … we can define the sample variance as 1 = − , where is the sample mean. Sample Mean and Variance However, gives an estimate of the population variance that is biased by a factor of . For this reason, is referred to as the biased sample variance . Correcting for this bias yields the unbiased sample variance : 1 = = − . − 1 − 1 Sample Mean and Variance While is an unbiased estimator for the variance, is still a biased estimator for the standard deviation, though markedly less biased than the uncorrected sample standard deviation . This estimator is commonly used and generally known simply as the "sample standard deviation". -
Unit 23: Control Charts
Unit 23: Control Charts Prerequisites Unit 22, Sampling Distributions, is a prerequisite for this unit. Students need to have an understanding of the sampling distribution of the sample mean. Students should be familiar with normal distributions and the 68-95-99.7% Rule (Unit 7: Normal Curves, and Unit 8: Normal Calculations). They should know how to calculate sample means (Unit 4: Measures of Center). Additional Topic Coverage Additional coverage of this topic can be found in The Basic Practice of Statistics, Chapter 27, Statistical Process Control. Activity Description This activity should be used at the end of the unit and could serve as an assessment of x charts. For this activity students will use the Control Chart tool from the Interactive Tools menu. Students can either work individually or in pairs. Materials Access to the Control Chart tool. Graph paper (optional). For the Control Chart tool, students select a mean and standard deviation for the process (from when the process is in control), and then decide on a sample size. After students have determined and entered correct values for the upper and lower control limits, the Control Chart tool will draw the reference lines on the control chart. (Remind students to enter the values for the upper and lower control limits to four decimals.) At that point, students can use the Control Chart tool to generate sample data, compute the sample mean, and then plot the mean Unit 23: Control Charts | Faculty Guide | Page 1 against the sample number. After each sample mean has been plotted, students must decide either that the process is in control and thus should be allowed to continue or that the process is out of control and should be stopped.