The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing
Total Page:16
File Type:pdf, Size:1020Kb
The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing by Sharon L. Nichols Assistant Professor University of Texas at San Antonio and David C. Berliner Regents’ Professor Arizona State University Education Policy Research Unit (EPRU) Education Policy Studies Laboratory College of Education Division of Educational Leadership and Policy Studies Box 872411 Arizona State University Tempe, AZ 85287-2411 March 2005 EDUCATION POLICY STUDIES LABORATORY EPSL | Education Policy Research Unit EPSL-0503-101-EPRU http://edpolicylab.org Education Policy Studies Laboratory Division of Educational Leadership and Policy Studies College of Education, Arizona State University P.O. Box 872411, Tempe, AZ 85287-2411 Telephone: (480) 965-1886 Fax: (480) 965-0303 E-mail: [email protected] http://edpolicylab.org This research was made possible by a grant from the Great Lakes Center for Education Research and Practice. Table of Contents Executive Summary…………………………………………………………………. i Introduction………………………………………………………………………….. 1 Criticisms of Testing………………………………………………………………… 1 Corrupting the Indicators and the People in the World Outside of Education……… 6 Corrupting the Indicators and the People in Education.…………………………….. 20 Methodology………………………………………………………………………… 20 Administrator and Teacher Cheating...……………………………………………… 23 Student Cheating and the Inevitability of Cheating When the Stakes are High ……. 53 Excluding Students from the Test..………………………………………………...... 63 Misrepresentation of Dropout Data…………………………………………………. 83 Teaching to the Test..…….…………………………………………………….……. 89 Narrowing the Curriculum....………………………………………………………... 101 Conflicting Accountability Ratings.……………………..………………….………. 110 The Changing Meaning of Proficiency.....………………………………………..…. 118 The Morale of School Personnel...………………………………………………….. 129 Errors of Scoring and Reporting…………………………………………………….. 143 Conclusion……………..…………………………….……………………………… 163 Notes & References…………………………………………………………………. 171 The Inevitable Corruption of Indicators and Educators Through High-Stakes Testing Sharon L. Nichols University of Texas at San Antonio and David C. Berliner Arizona State University Executive Summary This research provides lengthy proof of a principle of social science known as Campbell’s law: “The more any quantitative social indicator is used for social decision- making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”1 Applying this principle, this study finds that the over-reliance on high-stakes testing has serious negative repercussions that are present at every level of the public school system. Standardized-test scores and other variables used for judging the performance of school districts have become corruptible indicators because of the high stakes attached to them. These include future employability of teachers and administrators, bonus pay for school personnel, promotion/non-promotion of a student to a higher grade, achievement/non-achievement of a high school degree, reconstitution of a school, and losses or gains in federal and state funding received by a school or school district. Evidence of Campbell’s law at work was found in hundreds of news stories across America, and almost all were written in the last few years. The stories were gathered using LexisNexis, Inbox Robot, Google News Alerts, The New York Times, and Ed Week Online. In addition to news stories, traditional research studies, and stories told by educators about the effects of high-stakes testing are also part of the data. The data fell into 10 categories. Taken together these data reveal a striking picture of the corrupting effects of high-stakes testing: 1. Administrator and Teacher Cheating: In Texas, an administrator gave students who performed poorly on past standardized tests incorrect ID numbers to ensure their scores would not count toward the district average. 2. Student Cheating: Nearly half of 2,000 students in an online Gallop poll admitted they have cheated at least once on an exam or test. Some students said they were surprised that the percentage was not higher. 3. Exclusion of Low-Performance Students From Testing: In Tampa, a student who had a low GPA and failed portions of the state’s standardized exam received a letter from the school encouraging him to drop out even though he was eligible to stay, take more courses to bring up his GPA, and retake the standardized exam. 4. Misrepresentation of Student Dropouts: In New York, thousands of students were counseled to leave high school and to try their hand at high school equivalency programs. Students who enrolled in equivalency programs did not count as dropouts and did not have to pass the Regents’ exams necessary for a high-school diploma. ii 5. Teaching to the Test: Teachers are forced to cut creative elements of their curriculum like art, creative writing, and hands-on activities to prepare students for the standardized tests. In some cases, when standardized tests focus on math and reading skills, teachers abandon traditional subjects like social studies and science to drill students on test-taking skills. 6. Narrowing the Curriculum: In Florida, a fourth-grade teacher showed her students how to navigate through a 45-minute essay portion of the state’s standardized exam. The lesson was helpful for the test, but detrimental to emerging writers because it diluted their creativity and forced them to write in a rigid format. 7. Conflicting Accountability Ratings: In North Carolina, 32 schools rated excellent by the state failed to make federally mandated progress. 8. Questions about the Meaning of Proficiency: After raising achievement benchmarks, Maine considered lowering them over concerns that higher standards will hurt the state when it comes to No Child Left Behind. 9. Declining Teacher Morale: A South Carolina sixth-grade teacher felt the pressure of standardized tests because she said her career was in the hands of 12-year-old students. 10. Score Reporting Errors: Harcourt Educational Measurement was hit with a $1.1 million fine for incorrectly grading 440,000 tests in California, accounting for more than 10 percent of the tests taken in the state that year. iii High-stakes tests cannot be trusted – they are corrupted and distorted. To avoid exhaustive investigations into these tests that turn educators into police, this research supports building a new indicator system that is not subject to the distortions of high- stakes testing. iv Introduction The United States faces a severe crisis. Because the dangers do not seem imminent, the few individuals and organizations alerting politicians and federal agencies to the crisis are generally unheeded. This crisis concerns the corruption of what is arguably America’s greatest invention—its public schools. This research joins with others in documenting the damage to education caused by overreliance on high-stakes testing. Our documentation suggests that the incidence of negative events associated with high-stakes testing is so great, corruption is inevitable and widespread. As will be made clear, below, public education is presenting serious and harmful symptoms. Unlike other critics of the high-stakes testing movement, however, we demonstrate that a powerful social science law explains the etiology of the problems we document. Ignorance of this law endangers the health of our schools. Criticisms of Testing Concerns about the negative effects associated with testing are certainly not new, as demonstrated by comments from the Department of Education of the state of New York: It is an evil for a well-taught and well-trained student to fail in an examination. It is an evil for an unqualified student, through some inefficiency of the test, to obtain credit in an examination. It is a great and more serious evil, by too frequent and too numerous examinations, so to magnify their importance that students come to regard them not as a means in education but as the final purpose, the ultimate goal. It is a very great and more serious evil to sacrifice systematic instruction and a comprehensive view of the subject for the scrappy and unrelated knowledge gained by students who are persistently drilled in the mere answering of questions issued by the Education Department or other governing bodies. This Department of Education raises issues about the reliability and validity of its tests, as every testing agency should. But they also are concerned about over-testing and about how testing programs can mislead students (and by implication—parents and politicians) into thinking test scores are indicators of a sound education. This Department of Education also expresses its worry about how testing can distort the educational system by narrowing the curriculum. The enlightened bureaucrats who wrote this report to the legislature were warning the state’s politicians that it is possible, with the best of intents, for testing programs to corrupt the educational process. The archaic language in their report is better understood if the date of the report is known. It was written in 1906.2 Another warning about corruption and distortion from high-stakes tests surfaced when a plan to pay teachers on the basis of their students’ scores was offered, making indicators of student achievement very high-stakes for teachers. A schoolmaster noted that under these conditions … a teacher knows that his whole professional status depends on the results he produces and he is really turned into a machine for