Statistical Manipulation
Total Page：16
File Type：pdf, Size：1020Kb
1 STATISTICAL MANIPULATION: AN EXAMINATION OF VARIOUS SUBJECTIVE METHODS USED IN DATA COLLECTION AND PRESENTATION by Joshua S. Rempfer A Master’s Project submitted to the faculty of The University of Utah In partial fulfillment of the requirements for the degree of Masters of Science Degree for Secondary School Teachers Mathematics (Teaching) College of Science The University of Utah April 2013 2 THE UNIVERSITY OF UTAH GRADUATE SCHOOL SUPERVISORY COMMITTEE APPROVAL of a project submitted by Joshua S. Rempfer This project has been read by each member of the following supervisory committee and by majority vote has been found to be satisfactory. Date Committee Chair: Hugo Rossi Date Committee Member: Amanda Cangelosi Date Committee Member: Davar Khoshnevisan 3 Table of Contents Acknowledgements………………………………………………………………………… 4 Abstract………...……………………………………………………………………………. 5 Part I: Literature Review Introduction………………………………………………………………………………………..7 Strategy I: Consider the Source…………………………………..………………………………8 Strategy II: Identifying Bias……………………………………………………………………..11 Strategy III: Accurate Definitions……………………………………………………………….13 Strategy IV: Misleading Comparisons…………………………………………………………..15 Strategy V: Misleading Presentations……………………………………………………...........22 Conclusion: ……………………………………………………………………………………..26 Part II: Scientific Research Introduction: …………………………………………………………………………………….29 Background Information:………………………………………………………………………...29 Question, Hypothesis, Method of Research:……………..………….…………………………...31 Defining “Clutch” Performance:…………………………………………………………………31 Gathering Data:…………………………………………………………………………………..32 Glossary of Terms:…………………………………………………………………………….....32 Making Accurate Comparisons:………………………………………………………………....34 Results:…………………………………………………………………………………………...35 Analysis of Results:……………………………………………………………………………...40 Additional Commentary:………………………………………………………………………...47 Conclusion:………………………………………………………………………………………49 Bibliography:…………………………………………………………………………………….50 Appendix A:……………………………………………………………………………………...51 4 Acknowledgements: I would like to thank the professors who taught each class throughout this cohort. I learned a great deal and felt that my understanding and appreciation for math grew each semester. I would like to thank Davar Khoshnevisan for getting me started in the right direction and for the advice he offered in finding books that would aide in my research. I would like to thank Hugo Rossi and Amanda Cangelosi for the hours they spent reading my work, the sound advice they offered, and for taking the time to meet with me on numerous occasions. I would also like to thank my family, especially my wife, Kory. She rearranged schedules for 2 years, allowed me to spend countless evenings and weekends doing homework and research, and encouraged me every step of the way. 5 ABSTRACT STATISTICAL MANIPULATION: An examination of various subjective methods used in data collection and presentation by Joshua Rempfer, Master of Science University of Utah, 2013 Major Professor: Dr. Hugo Rossi Degree: Master of Science for Secondary School Teachers in Mathematics Methods of data collection in statistical research can often be flawed. As a result, many statistics presented to the general public are unsound. Unfortunately, people willingly believe errant or misleading statistics without doing any analysis. My goal in this project was really three-fold. I wanted to see if I could find commonalities among factors contributing to unsound statistics. I wanted to provide simple questions, or strategies, that could aid in the process of analyzing statistics. I also wanted to do some research of my own to find out if various commonly used baseball statistics accurately measured a player’s performance. In addition, it is my hope that this reading and research can help the statistics unit I teach in my 7th grade classes be more meaningful. 6 Project Outline: Part 1: I completed a literature review examining methods of data collection and presentation. Questionable figures (and their publication) can be credited to a number of factors. I addressed five major contributing factors: innumeracy, bias, poor definitions/measurements, poor comparisons, and unsound presentation. I provided a series of examples illustrating how these factors can be problematic. In addition, I provided a series of questions that an individual can ask when they encounter statistics that appear to be questionable. Part 2: I researched and analyzed a particular data set (the batting average of Major League Baseball players) that I felt was misleading. My findings are presented and I made note of the accurate methods that were used as I gathered and analyzed all of the data. Part 3: Although not included in this project, lesson plans that implement the strategies from this study will be taught in my 7th grade classes. The goal of these lessons will be to provide my students with the preliminary tools needed for critical analysis. 7 Part I: Literature Review Introduction “All statistics, even the most authoritative, are created by people. This does not mean that they are all inevitably flawed or wrong, but it does mean that we ought to ask ourselves just how the statistics we encounter were created (Best, 22).” “Misinforming people by the use of statistical material might be called statistical manipulation (Huff, 100).” These two quotes help to illustrate why views concerning statistics can be quite polarized. I observed two basic opinions concerning misleading statistics. 1) The belief that unsound information is presented with the intent to mislead. 2) People who mislead with statistics do so unintentionally because they have no idea what they are doing. In other words, the method by which data is gathered and presented is filled with errors. Prior to any research and reading, an individual might assume that misleading statistics are presented intentionally. However, in my research I observed enough errors in statistical application to become convinced that the second opinion is closer to the truth and the reason misleading statistics take shape. As Huff (2004) stated, data gathered by people has the potential to be misinterpreted and misleading. This observation led me to ask two questions. 1) Why do so many people believe unsound statistical information? 2) What can I do to help people (specifically my students) become more critical when it comes to recognizing unsound statistics? In following pages, I will seek to thoroughly answer both of these questions. 8 Defining Innumeracy: People, says Paulos (1988), can be very poor judges of the numbers they encounter. His words serve as a simple definition for innumeracy. I am not saying that our population lacks intelligence. Rather, a good portion of the population lacks the critical numeracy skills necessary to spot misleading statistics. Examples of innumeracy often go unnoticed: the cashier who does not understand why you give $5.05 when your bill is $4.05, the 7th grade student who asks which side of the ruler measures inches, people who are excited to get their tax return, as if receiving their own money is some kind of a gift. The general public seems to have a real hesitation or fear of anything having to do with mathematics (or even just numbers). In parent teacher conferences, I have had more than a handful of parents tell me they cannot help their children with the 7th grade work that is sent home. Parents who simply want their child to “pass” math only help to promote innumeracy. Many students like to think that most mathematics is not readily useful and cannot wait to be finished with their math requirements. The problem that results from innumeracy is an inability to accurately analyze various forms of numerical information. In my opinion, innumeracy is the main reason unsound statistics “sound” believable. My goal is to help combat this innumeracy. It is not my goal to turn people into skeptics, questioning every statistic they encounter. I simply want to provide some questions (I’ll call them strategies) that can be asked when an individual encounters numbers or statistics. Huff (1954) called this the ability to “talk back” to a statistic. 9 Strategy 1: Consider the Source The general public is bombarded by statistics. Often times these “statistics” are simply numbers attached to claims. However, just attaching a number does not mean research or proper analysis occurred. One of the more memorable examples of attaching a number to make a claim sound believable can be credited to Joseph McCarthy. In 1950, McCarthy made a claim about communists in the state department. Seife (2010) was quick to point out that it was not McCarthy’s first statement that people believed. “In my opinion the State Department, which is one of the most important government departments, is thoroughly infested with communists.” It was his second. “I have a list in my hand of 205 known communists.” Of course the number had no foundation and since no audio recording remains, people still argue whether it was 205 or 207. By simply attaching a number, McCarthy’s claim seemed to have validity and as a result people believed it. This is a rather dramatic example, but simpler ones can be found every day. In a recent email, a group of people (myself included) was informed that a child of the sender had been named the Class 1A Volleyball MVP. Then a claim was made, “we are so proud that our daughter is one of the top 5 players in the state.” There are 5 classes in Utah High School Volleyball. Each class named an MVP. The assumption was