Redesigning Accountability Systems for Education SUSAN H

Critical Issues in Educational Leadership Series Joseph Murphy, Series Editor Redesigning Accountability Systems for Education SUSAN H. FUHRMAN AND RICHARD F. E LMORE, EDS. Taking Account of Charter Schools: What’s Happened and What’s Next? KATRINA E. BULKLEY AND PRISCILLA WOHLSTETTER, EDS. Learning Together, Leading Together: Changing Schools through Professional Learning Communities SHIRLEY M. HORD, ED. Who Governs Our Schools? Changing Roles and Responsibilities DAVID T. C ONLEY School Districts and Instructional Renewal AMY M. HIGHTOWER, MICHAEL S. KNAPP, JULIE A. MARSH, AND MILBREY W. MCLAUGHLIN, EDS. Effort and Excellence in Urban Classrooms: Expecting—and Getting—Success with All Students DICK CORBETT, BRUCE WILSON, AND BELINDA WILLIAMS Developing Educational Leaders: A Working Model: The Learning Community in Action CYNTHIA J. NORRIS, BRUCE G. BARNETT, MARGARET R. BASOM, AND DIANE M. YERKES Understanding and Assessing the Charter School Movement JOSEPH MURPHY AND CATHERINE DUNN SHIFFMAN School Choice in Urban America: Magnet Schools and the Pursuit of Equity CLAIRE SMREKAR AND ELLEN GOLDRING Lessons from High-Performing Hispanic Schools: Creating Learning Communities PEDRO REYES, JAY D. SCRIBNER, AND ALICIA PAREDES SCRIBNER, EDS. Schools for Sale: Why Free Market Policies Won’t Improve America’s Schools, and What Will ERNEST R. HOUSE Reclaiming Educational Administration as a Caring Profession LYNN G. BECK Redesigning Accountability Systems for Education EDITED BY SUSAN H. FUHRMAN RICHARD F. ELMORE Teachers College, Columbia University New York and London Published by Teachers College Press, 1234 Amsterdam Avenue, New York, NY 10027 Copyright 2004 by Teachers College, Columbia University All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, or any information storage and retrieval system, without permission from the publisher. Chapter 2 is a reprint from the Harvard Educational Review. Chapter 9 copyright Martin Carnoy and Susanna Loeb. Library of Congress Cataloging-in-Publication Data Redesigning accountability systems for education / edited by Susan H. Fuhrman, Richard F. Elmore. p. cm. — (Critical issues in educational leadership series) Includes bibliographical references and index. ISBN 0-8077-4425-5 (cloth : alk. paper) 1. Educational accountability—United States. 2. Competency based education—United States. 3. School improvement programs—United States. I. Fuhrman, Susan. II. Elmore, Richard F. III. Series. LB2086.22.R433 2004 379.1′58—dc22 2003059537 ISBN 0-8077-4425-5 (cloth) Printed on acid-free paper Manufactured in the United States of America 1110090807060504 87654321 Contents Acknowledgments vii PART I INTRODUCTION 1. Introduction 3 Susan H. Fuhrman 2. Complexity, Accountability, and School Improvement 15 Jennifer A. O’Day PART II ISSUES IN DESIGNING ACCOUNTABILITY SYSTEMS 3. Validity Issues for Accountability Systems 47 Eva L. Baker and Robert L. Linn 4. Accountability Models 73 Robert L. Linn 5. Benchmarking and Alignment of State Standards and Assessments 96 Robert Rothman 6. Biting the Bullet: Including Special-Needs Students in Accountability Systems 115 Martha L. Thurlow PART III EFFECTS OF ACCOUNTABILITY SYSTEMS 7. The Effects of Testing on Instruction 141 Joan L. Herman 8. The Challenge of the High Schools 167 Leslie Santee Siskin 9. Does External Accountability Affect Student Outcomes? A Cross-State Analysis 189 Martin Carnoy and Susanna Loeb v vi Contents 10. High-Stakes Testing in a Changing Environment: Disparate Impact, Opportunity to Learn, and Current Legal Protections 220 Jay P. Heubert PART IV MOVING FORWARD: REFINING ACCOUNTABILITY SYSTEMS 11. “Slow Down, You Move Too Fast”: The Politics of Making Changes in High-Stakes Accountability Policies for Students 245 Susan H. Fuhrman, Margaret E. Goertz, and Mark C. Duffy 12. Conclusion: The Problem of Stakes in Performance-Based Accountability Systems 274 Richard F. Elmore About the Contributors 297 Index 303 Acknowledgments Redesigning Accountability Systems for Education is the result of a collab- orative effort between the Consortium for Policy Research in Education (CPRE) and the Center for Research on Evaluation, Standards, and Student Testing (CRESST). Both CPRE and CRESST receive key funding from the U.S. Department of Education’s Institute of Education Sciences (Grants No. R308A960003 and R305B960002). CPRE unites five of the nation’s leading research institutions to improve elementary and secondary education through research on policy, finance, school reform, and school governance. CPRE studies alternative approaches to education reform in order to determine how state and local policies can promote student learning. Members of CPRE are the University of Pennsyl- vania, Harvard University, Stanford University, the University of Michigan, and the University of Wisconsin–Madison. CRESST’s mission focuses on the assessment of educational quality, addressing persistent problems in the design and use of assessment systems to serve multiple purposes. CRESST is a partnership of the University of Cali- fornia–Los Angeles, the University of Colorado, Stanford University, RAND, the University of Pittsburgh, the University of Southern California, the Edu- cational Testing Service, and the University of Cambridge–United Kingdom. In February 2001, CPRE and CRESST co-sponsored a meeting on accountability issues and thereafter commissioned a series of papers. This volume is the result of those efforts. We gratefully acknowledge Deborah Nelson, who compiled the volume and patiently worked with the authors, the editors, and Teachers College Press. We also would like to thank Robb Sewell who copyedited and proof- read each chapter, and Kelly Stanton who handled the logistics for the meeting. We appreciate the support and patience of Brian Ellerbeck and Cheryl DeJong-Lambert of Teachers College Press. Finally, we wish to thank Eliza- beth Demarest, formerly of the Office of Educational Research and Improve- ment, who provided helpful comments on the papers. Susan H. Fuhrman Richard F. Elmore vii PART I INTRODUCTION Chapter 1 Introduction Susan H. Fuhrman This book is about understanding and redesigning accountability systems. The accountability systems we write about are those established over the past 5 to 10 years, mostly at the state level, although a number of districts have similar systems. These systems are distinguished by their attention to school-level performance and by their inclusion of consequences for that performance. They are quite different from earlier approaches to accountability that attended primarily to district compliance with state regulations. The new systems grow out of a climate that makes strong parallels between education and business; they intend to focus schools on the bottom line. They also reflect an attempt, strong in rhetoric if not reality, by states to back off from detailed regulations about the process of education. The new systems have an explicit theory of action about improving student achievement that stresses the motivation of teachers, students, and adminis- trators. In this book we address that theory of action, looking at how these new systems are framed, their technical specifications, their implementa- tion, and their effects. Our purpose is to illustrate the technical, political, and educational challenges these systems pose. It is our hope that understanding these challenges will assist policy makers in their efforts to make mid-course corrections to these systems in ways that address some of the problems we write about. Redesigning Accountability Systems for Education. Copyright 2003 by Teachers College, Columbia University. All rights reserved. ISBN 0-8077-4425-5 (cloth). Prior to photocopying items for classroom use, please contact the Copyright Clearance Center, Customer Service, 222 Rosewood Drive, Danvers, MA, 01923, USA, telephone (978) 750-8400. 3 4 Introduction NEW ACCOUNTABILITY SYSTEMS All states currently have accountability systems of the type we examine in this book. The systems vary considerably from state to state. For example, 33 states set performance goals for schools, but they differ in the level of performance they set as a goal (e.g., basic or proficient), in the percentage of students required to meet the goal, and in the length of time schools are given to meet the goal. Time periods set by states range from 5 to 20 years (Goertz & Duffy, 2001). Despite the differences, the systems share a number of characteristics that distinguish them from earlier efforts of states to keep tabs on local districts and schools. The new systems are centered around student performance on state- wide assessments that are specifically developed for states or purchased commercially. Forty-eight states use a state assessment to measure performance; Iowa and Nebraska require districts to test in certain grades but don’t specify the assessment (Goertz & Duffy, 2001). Schools are held ac- countable either for achieving a certain level of performance (e.g., a percentage of students achieving at a “proficient” level) or for increasing performance from assessment period to assessment period. Some states use both status and gain measures. In most states, the assessments are said to be aligned with state standards for student learning, although as a number of authors in this volume point out, alignment is frequently partial and insuf- ficient. Performance measures frequently are combined

Load more