DOCSLIB.ORG

A COMPARISON of RANKING METHODS for NORMALIZING SCORES by SHIRA R. SOLOMON DISSERTATION Submitted to the Graduate School Of

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A COMPARISON of RANKING METHODS for NORMALIZING SCORES by SHIRA R. SOLOMON DISSERTATION Submitted to the Graduate School Of A COMPARISON OF RANKING METHODS FOR NORMALIZING SCORES by SHIRA R. SOLOMON DISSERTATION Submitted to the Graduate School of Wayne State University, Detroit, Michigan in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY 2008 MAJOR: EVALUATION AND RESEARCH Approved by: ______________________________ Advisor Date ______________________________ ______________________________ ______________________________ UMI Number: 3303509 Copyright 2008 by Solomon, Shira R. All rights reserved. UMI Microform 3303509 Copyright 2008 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, MI 48106-1346 © COPYRIGHT BY SHIRA R. SOLOMON 2008 All Rights Reserved DEDICATION To my maternal grandmother, Mary Karabenick Brooks, whose love of art, literature, and music has gone hand in hand with her concern for social welfare. To my paternal grandmother, Frances Hechtman Solomon, who played the cards she was dealt with style and wit. ii ACKNOWLEDGEMENTS A dissertation is largely a solitary project, yet it builds on the contributions of many. I have been standing on many shoulders. First, I would like to thank my major advisor, Professor Shlomo Sawilowsky, whose grand passion for argument made him someone I could relate to, and made statistics seem worth doing. Dr. Sawilowsky has been generous with his time, technical help, and the spirited exegeses that put this discipline in its true human context. Professors Gail Fahoome, Judith Abrams, and Leonard Kaplan have brought a great deal to this dissertation and to my graduate experience. Dr. Fahoome has been an excellent teacher, consistently insightful and reassuringly low-key. I lucked into meeting Dr. Abrams through my research assistantship with the medical school. Her assistance and advice have been invaluable. Dr. Kaplan paid me the extraordinary compliment of joining my committee on the brink of his retirement. I am indebted to each of these professors for their intellectual integrity and their simple kindness. I regret the untimely passing of Professor Donald Marcotte, who would have been proud to see this dissertation completed. Dr. Marcotte provided a wonderful initiation into the world of statistics, with his perennial admonition that the faster you can solve problems, the more time you have to enjoy life. When it came time to apply for this doctoral program, I reached out to the professors who knew me best. I did not find them, in the end, in the ideological combat zone of my master’s program or in the artful arena of my literary studies. I iii found them within the seminary walls, among the rabbis and professors who taught me Talmud. Studying Talmud helped me to stop thinking so much and just learn. For accomplishing this ingenious feat, and for supporting all my educational adventures, I would like to thank Professor David Kraemer, Rabbi Leonard Levy, and Professor Mayer Rabinowitz. To Bruce Chapman, the teacher who forced inspiration to the forefront, where it belongs: Here’s to you, Captain. To my great friends, Regina DiNunzio, Tom Kilroe, Katy Potter, and Deborah Mougoue, who keep me on my toes. My parents, Carole and Elliot Solomon, have been the staunchest advocates of this reckless leap. Their unrelenting curiosity and unvarnished pleasure in my pursuits has given me strength. And Mark Sawasky, my constant friend and fan and love, becomes a bigger mensch every day. iv TABLE OF CONTENTS DEDICATION ............................................................................................................ ii ACKNOWLEDGEMENTS ........................................................................................ iii LIST OF TABLES.................................................................................................... vii LIST OF FIGURES................................................................................................... ix CHAPTERS CHAPTER 1 – Introduction...................................................................... …...1 Research problem................................................................................5 Importance of the problem...................................................................6 Assumptions and limitations ................................................................7 Definitions ............................................................................................8 CHAPTER 2 – Literature review...................................................................10 Mental testing and the normal distribution .........................................10 Norm-referencing and the T score .....................................................11 Nonnormality observed ......................................................................13 Statistical considerations ...................................................................14 Standardizing transformations ...........................................................21 Approaches to creating normal scores ..............................................28 CHAPTER 3 – Methodology.........................................................................32 Programming specifications...............................................................33 Sample sizes......................................................................................33 Number of Monte Carlo repetitions....................................................33 Achievement and psychometric distributions.....................................33 v Presentation of results .......................................................................34 CHAPTER 4 – Results .................................................................................43 CHAPTER 5 – Conclusion............................................................................89 Discussion..........................................................................................92 Moment 1—mean ..............................................................................92 Moment 2—standard deviation..........................................................92 Moment 3—skewness........................................................................95 Moment 4—kurtosis...........................................................................95 Recommendations.............................................................................96 REFERENCES........................................................................................................98 ABSTRACT ...........................................................................................................110 AUTOBIOGRAPHICAL STATEMENT...................................................................112 vi LIST OF TABLES Table 1. Differences among Ranking Methods in Attaining Target Moments .........25 Table 2. Smooth Symmetric—Accuracy of T Scores on Means..............................45 Table 3. Smooth Symmetric—Accuracy of T Scores on Standard Deviations ........46 Table 4. Smooth Symmetric—Accuracy of T Scores on Skewness ........................47 Table 5. Smooth Symmetric—Accuracy of T Scores on Kurtosis ...........................48 Table 6. Discrete Mass at Zero—Accuracy of T Scores on Means.........................49 Table 7. Discrete Mass at Zero—Accuracy of T Scores on Standard Deviations ...50 Table 8. Discrete Mass at Zero—Accuracy of T Scores on Skewness ...................51 Table 9. Discrete Mass at Zero—Accuracy of T Scores on Kurtosis.......................52 Table 10. Extreme Asymmetric, Growth—Accuracy of T Scores on Means ...........53 Table 11. Extreme Asymmetric, Growth—Accuracy of T Scores on Standard Deviations................................................................................................................54 Table 12. Extreme Asymmetric, Growth—Accuracy of T Scores on Skewness......55 Table 13. Extreme Asymmetric, Growth—Accuracy of T Scores on Kurtosis .........56 Table 14. Digit Preference—Accuracy of T Scores on Means….............................57 Table 15. Digit Preference—Accuracy of T Scores on Standard Deviations...........58 Table 16. Digit Preference—Accuracy of T Scores on Skewness...........................59 Table 17. Digit Preference—Accuracy of T Scores on Kurtosis ..............................60 Table 18. Multimodal Lumpy—Accuracy of T Scores on Means.............................61 Table 19. Multimodal Lumpy—Accuracy of T Scores on Standard Deviations .......62 Table 20. Multimodal Lumpy—Accuracy of T Scores on Skewness .......................63 Table 21. Multimodal Lumpy—Accuracy of T Scores on Kurtosis...........................64 vii Table 22. Mass at Zero with Gap—Accuracy of T Scores on Means......................65 Table 23. Mass at Zero with Gap—Accuracy of T Scores on Standard Deviations................................................................................................................66 Table 24. Mass at Zero with Gap—Accuracy of T Scores on Skewness ................67 Table 25. Mass at Zero with Gap—Accuracy of T Scores on Kurtosis....................68 Table 26. Extreme Asymmetric, Decay—Accuracy of T Scores on Means.............69 Table 27. Extreme Asymmetric, Decay—Accuracy of T Scores on Standard Deviations................................................................................................................70 Table 28. Extreme Asymmetric, Decay—Accuracy of T Scores
Load more

Recommended publications
  • Multiple Linear Regression
    Multiple Linear Regression The MULTIPLE LINEAR REGRESSION command performs simple multiple regression using least squares. Linear regression attempts to model the linear relationship between variables by fitting a linear equation to observed data. One variable is considered to be an explanatory variable (RESPONSE), and the others are considered to be dependent variables (PREDICTORS). How To Run: STATISTICS->REGRESSION->MULTIPLE LINEAR REGRESSION... Select DEPENDENT (RESPONSE) variable and INDEPENDENT variables (PREDICTORS). To force the regression line to pass through the origin use the CONSTANT (INTERCEPT) IS ZERO option from the ADVANCED OPTIONS. Optionally, you can add following charts to the report: o Residuals versus predicted values plot (use the PLOT RESIDUALS VS. FITTED option); o Residuals versus order of observation plot (use the PLOT RESIDUALS VS. ORDER option); o Independent variables versus the residuals (use the PLOT RESIDUALS VS. PREDICTORS option). For the univariate model, the chart for the predicted values versus the observed values (LINE FIT PLOT) can be added to the report. Use THE EMULATE EXCEL ATP FOR STANDARD RESIDUALS option to get the same standard residuals as produced by Excel Analysis Toolpak. Results Regression statistics, analysis of variance table, coefficients table and residuals report are produced. Regression Statistics 2 R (COEFFICIENT OF DETERMINATION, R-SQUARED) is the square of the sample correlation coefficient between 2 the PREDICTORS (independent variables) and RESPONSE (dependent variable). In general, R is a percentage of response variable variation that is explained by its relationship with one or more predictor variables. In simple words, the R2 indicates the accuracy of the prediction. The larger the R2 is, the more the total 2 variation of RESPONSE is explained by predictors or factors in the model.
    [Show full text]
  • HMH ASSESSMENTS Glossary of Testing, Measurement, and Statistical Terms
    hmhco.com HMH ASSESSMENTS Glossary of Testing, Measurement, and Statistical Terms Resource: Joint Committee on the Standards for Educational and Psychological Testing of the AERA, APA, and NCME. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association. Glossary of Testing, Measurement, and Statistical Terms Adequate yearly progress (AYP) – A requirement of the No Child Left Behind Act (NCLB, 2001). This requirement states that all students in each state must meet or exceed the state-defined proficiency level by 2014 on state Ability – A characteristic indicating the level of an individual on a particular trait or competence in a particular area. assessments. Each year, the minimum level of improvement that states, school districts, and schools must achieve Often this term is used interchangeably with aptitude, although aptitude actually refers to one’s potential to learn or is defined. to develop a proficiency in a particular area. For comparison see Aptitude. Age-Based Norms – Developed for the purpose of comparing a student’s score with the scores obtained by other Ability/Achievement Discrepancy – Ability/Achievement discrepancy models are procedures for comparing an students at the same age on the same test. How much a student knows is determined by the student’s standing individual’s current academic performance to others of the same age or grade with the same ability score. The ability or rank within the age reference group. For example, a norms table for 12 year-olds would provide information score could be based on predicted achievement, the general intellectual ability score, IQ score, or other ability score.
    [Show full text]
  • Measures of Center
    Statistics is the science of conducting studies to collect, organize, summarize, analyze, and draw conclusions from data. Descriptive statistics consists of the collection, organization, summarization, and presentation of data. Inferential statistics consist of generalizing form samples to populations, performing hypothesis tests, determining relationships among variables, and making predictions. Measures of Center: A measure of center is a value at the center or middle of a data set. The arithmetic mean of a set of values is the number obtained by adding the values and dividing the total by the number of values. (Commonly referred to as the mean) x x x = ∑ (sample mean) µ = ∑ (population mean) n N The median of a data set is the middle value when the original data values are arranged in order of increasing magnitude. Find the center of the list. If there are an odd number of data values, the median will fall at the center of the list. If there is an even number of data values, find the mean of the middle two values in the list. This will be the median of this data set. The symbol for the median is ~x . The mode of a data set is the value that occurs with the most frequency. When two values occur with the same greatest frequency, each one is a mode and the data set is bimodal. Use M to represent mode. Measures of Variation: Variation refers to the amount that values vary among themselves or the spread of the data. The range of a data set is the difference between the highest value and the lowest value.
    [Show full text]
  • The Normal Distribution Introduction
    The Normal Distribution Introduction 6-1 Properties of the Normal Distribution and the Standard Normal Distribution. 6-2 Applications of the Normal Distribution. 6-3 The Central Limit Theorem The Normal Distribution (b) Negatively skewed (c) Positively skewed (a) Normal A normal distribution : is a continuous ,symmetric , bell shaped distribution of a variable. The mathematical equation for the normal distribution: 2 (x)2 2 f (x) e 2 where e ≈ 2 718 π ≈ 3 14 µ ≈ population mean σ ≈ population standard deviation A normal distribution curve depend on two parameters . µ Position parameter σ shape parameter Shapes of Normal Distribution 1 (1) Different means but same standard deviations. Normal curves with μ1 = μ2 and 2 σ1<σ2 (2) Same means but different standard deviations . (3) Different 3 means and different standard deviations . Properties of the Normal Distribution The normal distribution curve is bell-shaped. The mean, median, and mode are equal and located at the center of the distribution. The normal distribution curve is unimodal (single mode). The curve is symmetrical about the mean. The curve is continuous. The curve never touches the x-axis. The total area under the normal distribution curve is equal to 1 (or 100%). The area under the normal curve that lies within Empirical Rule one standard deviation of the mean is approximately 0.68 (68%). two standard deviations of the mean is approximately 0.95 (95%). three standard deviations of the mean is approximately 0.997 (99.7%). ** Suppose that the scores on a history exam have a mean of 70. If these scores are normally distributed and approximately 95% of the scores fall in (64,76), then the standard deviation is ….
    [Show full text]
  • Understanding Test Scores a Primer for Parents
    Understanding Test Scores A primer for parents... Norm-Referenced Tests Norm-referenced tests compare an individual child's performance to that of his or her classmates or some other, larger group. Such a test will tell you how your child compares to similar children on a given set of skills and knowledge, but it does not provide information about what the child does and does not know. Scores on norm-referenced tests indicate the student's ranking relative to that group. Typical scores used with norm-referenced tests include: Percentiles. Percentiles are probably the most commonly used test score in education. A percentile is a score that indicates the rank of the student compared to others (same age or same grade), using a hypothetical group of 100 students. A percentile of 25, for example, indicates that the student's test performance equals or exceeds 25 out of 100 students on the same measure; a percentile of 87 indicates that the student equals or surpasses 87 out of 100 (or 87% of) students. Note that this is not the same as a "percent"-a percentile of 87 does not mean that the student answered 87% of the questions correctly! Percentiles are derived from raw scores using the norms obtained from testing a large population when the test was first developed. Standard scores. A standard score is derived from raw scores using the norming information gathered when the test was developed. Instead of reflecting a student's rank compared to others, standard scores indicate how far above or below the average (the "mean") an individual score falls, using a common scale, such as one with an "average" of 100.
    [Show full text]
  • Ch6: the Normal Distribution
    Ch6: The Normal Distribution Introduction Review: A continuous random variable can assume any value between two endpoints. Many continuous random variables have an approximately normal distribution, which means we can use the distribution of a normal random variable to make statistical inference about the variable. CH6: The Normal Distribution Santorico - Page 175 Example: Heights of randomly selected women CH6: The Normal Distribution Santorico - Page 176 Section 6-1: Properties of a Normal Distribution A normal distribution is a continuous, symmetric, bell-shaped distribution of a variable. The theoretical shape of a normal distribution is given by the mathematical formula (x)2 e 2 2 y , 2 where and are the mean and standard deviations of the probability distribution, respectively. Review: The and are parameters and hence describe the population . CH6: The Normal Distribution Santorico - Page 177 The shape of a normal distribution is fully characterized by its mean and standard deviation . specifies the location of the distribution. specifies the spread/shape of the distribution. CH6: The Normal Distribution Santorico - Page 178 CH6: The Normal Distribution Santorico - Page 179 Properties of the Theoretical Normal Distribution A normal distribution curve is bell-shaped. The mean, median, and mode are equal and are located at the center of the distribution. A normal distribution curve is unimodal. The curve is symmetric about the mean. The curve is continuous (no gaps or holes). The curve never touches the x-axis (just approaches it). The total area under a normal distribution curve is equal to 1.0 or 100%. Review: The Empirical Rule applies (68%-95%-99.7%) CH6: The Normal Distribution Santorico - Page 180 CH6: The Normal Distribution Santorico - Page 181 The Standard Normal Distribution Since each normally distributed variable has its own mean and standard deviation, the shape and location of these curves will vary.
    [Show full text]
  • 4. Descriptive Statistics
    4. Descriptive statistics Any time that you get a new data set to look at one of the first tasks that you have to do is find ways of summarising the data in a compact, easily understood fashion. This is what descriptive statistics (as opposed to inferential statistics) is all about. In fact, to many people the term “statistics” is synonymous with descriptive statistics. It is this topic that we’ll consider in this chapter, but before going into any details, let’s take a moment to get a sense of why we need descriptive statistics. To do this, let’s open the aflsmall_margins file and see what variables are stored in the file. In fact, there is just one variable here, afl.margins. We’ll focus a bit on this variable in this chapter, so I’d better tell you what it is. Unlike most of the data sets in this book, this is actually real data, relating to the Australian Football League (AFL).1 The afl.margins variable contains the winning margin (number of points) for all 176 home and away games played during the 2010 season. This output doesn’t make it easy to get a sense of what the data are actually saying. Just “looking at the data” isn’t a terribly effective way of understanding data. In order to get some idea about what the data are actually saying we need to calculate some descriptive statistics (this chapter) and draw some nice pictures (Chapter 5). Since the descriptive statistics are the easier of the two topics I’ll start with those, but nevertheless I’ll show you a histogram of the afl.margins data since it should help you get a sense of what the data we’re trying to describe actually look like, see Figure 4.2.
    [Show full text]
  • Norms and Scores Sharon Cermak
    Chapter 5 Norms and Scores Sharon Cermak There are three kinds of lies: lies, damned lies, and statistics. - Disraeli INTRODUCTION Suppose that on the Bayley Scales of Infant Development,' Ken- dra, age 14 months, received a score of 97 on the Mental Scale and 44 on the Motor Scale. Does this mean that her mental abilities are in the average range? Does it mean that she is motorically retarded? Does it mean that her mental abilities are twice as good as her motor abilities? The numbers reported are raw scores and reflect the number of items that Kendra passed. Since there are many more items on the Mental Scale than on the Motor Scale, it is expected that her raw score on this scale would be higher. These numbers are called raw scores and cannot be interpreted. It is not known what their relation- ship is to each other, or what they signify relative to the average child of comparable age. Raw scores can be interpreted only in terms of a clearly defined and uniform frame of reference. The frame of reference in interpreting a child's scores is based upon the scoring system which is used on the test. Scores on psychological, educational, developmental, and per- Sharon Cerrnak, EdD, OTR, is Associate Professor of Occupational Therapy at Sargent College of Allied Health Professions, Boston University, University Road, Boston, MA 02215. O 1989 by The Haworth Press, Inc. All rights reserved. 91 92 DEVELOPING NORM-REFERENCED STANDARDIZED TESTS ceptual tests are generally interpreted by reference to nom. Norms represent the test performance of the standardization sample.
    [Show full text]
  • Rao's Score Test in Econometrics
    UNIVERSITY OF ILLINOIS LIBRARY AT URBANA CHAMPAIGN EOOKSTACKS Digitized by the Internet Archive in 2012 with funding from University of Illinois Urbana-Champaign http://www.archive.org/details/raosscoretestine91132bera Faculty Working Paper 91-0132 330 STX B385 1991: 132 COPY 2 Rao's Score Test in Econometrics Anil K. Bern Aman Ullah Department of Economics Department of Economics University of Illinois University of California Indiana University Riverside o' *9 •the UWatK %2& Bureau of Economic and Business Research College of Commerce and Business Administration University of Illinois at Urbana-Champaign BEBR FACULTY WORKING PAPER NO. 91-0132 College of Commerce and Business Administration University of Illinois at Urbana-Champaign April 1991 Rao's Score Test in Econometrics Anil K. Bera Department of Economics University of Illinois and Indiana University Aman Ullah Department of Economics University of California-Riverside We are grateful to Rick Vinod, Vicki Zinde- Walsh and Mann Yoon for comments on an earlier draft of the paper. Financial support from the Research Board and the Bureau of Economics and Business Research of the University of Illinois is gratefully acknowledged. ABSTRACT Rao's work is always inspired by some practical problems. It is not surprising that many of the techniques he developed found their way to various applications in econometrics. In this review paper, we try to evaluate the impact of his pioneering 1948 score test on the current practice of econometric model specification tests. In so doing, we relate Rao's score principle to some of the earlier popular tests in econometrics. We also discuss the recent developments on this topic and problems for further research.
    [Show full text]
  • Why Do We Convert Values to Standard Scores?
    Why do we convert values to standard scores? Before we answer the question above, let’s look at the example as follows: Example 1 Julie got a mark of 80 from a math class with a mean of 85 and a standard deviation of 5 and her friend Andrea obtained 65 from a chemistry class with a mean of 55 and a standard deviation of 10. Can you tell who got a “better” mark? At first glance, the mark 80 seems like a “higher/better” mark. But, on second thought, we might realize that it is not true because whether it is a “better” mark or not depends on the other students’ marks in the class as well. It might seem weird to say that 65 is a better mark than 80, but it is true because 65 is above the average mark 55 and meanwhile 80 is below the average mark 85. Therefore, when we compare the two values, we need to consider the means. Beside means, we need to take standard deviation into account as well because standard deviation shows how much variation there is from the average (mean, or expected value). A low standard deviation indicates that the data points tend to be very close to the mean, whereas high standard deviation indicates that the data are spread out over a large range of values. Here is another example with same values and means but different standard deviations as follows: Example 2 Jason got 80 from a history class with a mean of 75 and a standard deviation of 2.5 and his friend Mike obtained 80 from an English class with a mean of 75 and a standard deviation of 5.
    [Show full text]
  • Part Two Page 1 Basic Statistics Review
    Basic Statistics Review – Part Two Page 1 Basic Statistics Review – Part Two Sampling Distribution of the Mean; Standard Error (See Zar 4th ed. pages 65-76; or Zar 5th ed. pages 66-72; 87-91) In our discussion of frequency distributions in Part One, we were discussing the frequency distribution of values of data. However, we are also concerned about frequency distributions of the values of statistics. [Note: The distinction between data and statistics is very important here.] For example, suppose every person in this class went out to the quad, stopped 30 random students, and asked how tall they were. Now, if there are 24 people taking this class, and they each now had a sample of n=30 data points, that’s 24x30 = 720 data points. We could prepare a frequency distribution of the values of the 720 data points. It would have to be frequencies of data intervals, i.e. how many students were between 5’0” and 5’2”; how many between 5’2” and 5’4”; and so on – but it’s still a frequency distribution of the data. But we could also have each person in the class calculate the mean of their data points. If there are 24 people in class, we now have 24 means. We could prepare a frequency distribution of the values of the mean, i.e. how often did different intervals of the mean occur. Since the mean is a statistic, this would be a frequency distribution of the values of a sample statistic. Since “frequency distribution of the values of a sample statistic” is a lot of words, we have a shorter term: sampling distribution.
    [Show full text]
  • Notes 2 Normal Distribution and Standard Scores
    Notes 2: Normal Distribution and Standard Scores 1. Normal Distributions bell shaped: symmetrical, unimodal, and smooth (meaning that there is an unlimited number of observations so a polygon or histogram will show a smooth distribution) central tendency: mean, median, and mode are identical (note that a normal distribution may have any mean and any variance, e.g., the population distribution of IQs supposedly has = 100 and = 15, whereas the population of verbal SAT scores supposedly has = 500 and = 100 and both IQ and SAT could be normally distributed) standard normal, unit normal curve, z-distribution: a normal distribution with = 0.00 and = 2 = 1.00; any normally distributed data set with scores transformed to z scores will have a standard normal distribution; similarly, any non-normally distributed data with scores transformed to z scores will retain their non-normal shape (z scores do not change shape of distribution) area: the area under the normal distribution can be found in a table of z scores, such as the one provided on the next page 2. Relative Position and Standard Scores relative position: relative position or relative standing refers to the position of a particular score in the distribution relative to the other scores in the distribution, e.g., percentile ranks, PR; relative position is similar to the concept of norm-referenced interpretations found with many common tests standard scores: a standard score in one in which the mean and standard deviation of the distribution will have pre-defined values, e.g.,
    [Show full text]