Schools on Probation in the States of Maryland and Kentucky. Technical Report
Total Page:16
File Type:pdf, Size:1020Kb
DOCUMENT RESUME ED 468 185 EA 031 872 AUTHOR Mintrop, Heinrich; Cibulka, James; Lindle, Jane Clark TITLE Schools on Probation in the States of Maryland and Kentucky. Technical Report. Volumes I-IV [and] Appendix. SPONS AGENCY Office of Educational Research and Improvement (ED), Washington, DC. PUB DATE 2001-00-00 NOTE 960p.; Submitted by Center for Education Policy and Leadership, University of Maryland, College Park. Produced with Masako Nishio, Daria Buese, Kim Curtis, Betty King, Ann MacLellan, Margaret Quintero, Lea Plut-Pregelj, and Robert Pettit. CONTRACT R308F970035-97 AVAILABLE FROM Center for Education Policy and Leadership, Room 2110, Benjamin Building, University of Maryland, College Park, Maryland 20742. Tel: 301-405-3574; Fax: 301-405-3573; Web site: http://education.umd.edu/Depts/EDPA/research.htm. For full text: http://www.gseis.ucla.edu/faculty/ mintrop/Schools-on-probation/table_of_contents-schools-on- probation.htm. PUB TYPE Reports Research (143) Tests/Questionnaires (160) EDRS PRICE EDRS Price MF06/PC39 Plus Postage. DESCRIPTORS *Accountability; *Educational Policy; *Effective Schools Research; Elementary Secondary Education; Federal Regulation; Government Publications; Instructional Effectiveness; Instructional Improvement; Sanctions; *School Effectiveness; Teacher Attitudes; Teacher Behavior; Teacher Motivation IDENTIFIERS *Kentucky; *Maryland ABSTRACT This study on accountability designs for underperforming schools focuses on probation as a tool for policy. Findings are based on the analysis of state performance data, the reading of approximately 100 school improvement plans, and the study of 11 focal schools in Maryland and Kentucky. Test-score data from Maryland show that probation is associated with an initial reversal of decline in the worst performing schools that fades out in subsequent years. Teachers in the 11 schools fundamentally do not buy into the accountability system. Low job commitment diminishes the threat of low-performance penalties. Many teachers were skeptical about the goals, felt unfairly judged by the system, and did not judge themselves according to the standards of the system. The accountability system also does not provide meaningful tools for self-evaluation. To a large degree, mild pressures and directives molded the teachers' responses to probation, with the principal being the system's conduit. The result was that probation, by itself, did not trigger much self-directed action on the part of the great majority of teachers. The four volumes and appendix contain the following parts:(1) analytical framework and study design;(2) the accountability systems of Maryland and Kentucky;(3) probation and performance motivation; (4) organization responses to probation;(5) planning for school improvement; (6) probation and instructional change;(7) summary of findings and discussion;(8) case studies;(9) the politics of accountability for school improvement in Kentucky and Maryland;,(10) case study research guide; and Reproductions supplied by EDRS are the best that can be made from the original document. (11) interview protocols. (RT) Reproductions supplied by EDRS are the best that can be made from the original document. Schools on Probation in the States of Maryland and Kentucky. Technical Report. Volumes IIV [and] Appendix. Heinrich Mintrop James Cibulka Jane Clark Lind le U.S. DEPARTMENT OF EDUCATION Office of Educational Research and Improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) This document has been reproduced as received from the person or organization originating it. Minor changes have been made to improve reproduction quality Points of view or opinions stated in this document do not necessarily represent official OERI position or policy. BEST COPYAVAILABLE CENTER FOR EDUCATION POLICY AND LEADERSHIP UNIVERSITY OF MARYLAND, COLLEGE PARK TECHNICAL REPORT SUBMITTED TO THE OFFICE OF EDUCATIONAL RESEARCH AND IMPROVEMENT U.S. DEPARTMENT OF EDUCATION FIELD INITIATED STUDY: THE EFFECT OF RECONSTITUTION ON SCHOOL IMPROVEMENT PRINCIPAL INVESTIGATORS HEINRICH MINTROP, UCLA JAMES CIBULKA, UNIVERSITY OF MARYLAND JANE CLARK LINDLE, UNIVERSITY OF KENTUCKY This study is supported by the Office of Educational Research and Improvement, U. S. Department of Education, Grant No. R308F970035 -97. The views expressed herein are those of the authors alone. Lf4:70A. TABLE OF CONnisitS, Volume I Executive Summary Part I. Analytical Framework and Study Design 1 1. Introduction 1 2. Analytical Framework 17 3. The Study 55 Part II. The Accountability Systems of Maryland and Kentucky 56 1. Maryland 74 2. Kentucky 84 3.The Kentucky and Maryland Accountability Systems Compared 88 Part III. Probation and Performance Motivation 90 1. Interpretation of the Signal 94 2. Teacher Capacity 110 3. Explanation of Decline 120 4. The Meaning of the Accountability System 136 5. Social Interactions at the Workplace 182 6. Job Commitment 186 7. Levels of Performance Motivation and Commitment 199 8. Part III Summary: Teachers' Responses to Probation 213 Volume II Part IV. Organization Responses to Probation 1 1. The Challenge of Performance in the Seven Maryland Schools 3 2. Change in the Maryland Schools 15 3. School Differences (Maryland) 24 4. The Kentucky Schools 51 5. Differences Between Maryland and Kentucky Schools 71 6. The Management of Probation 74 7.Organization Responses to Probation A Summary 80 Part V. Planning for School Improvement 86 1. Content Analysis of the Maryland School Improvement Plan 90 2. Case Study Data 102 3. Comparison of Accountability Designs 118 4. Discussion 129 Part VL Probation and Instructional Change 131 1. Observed Lessons in the Maryland Schools 133 2. Conversations about Teaching and Accountability 139 3. Instructional Change in Kentucky 179 4. Summary 188 3 Part VII. Summary of Findings and Discussion 191 1. External Pressures 194 2. Educational Meanings 199 3. Individual and Organizational Capacities 203 4. Differences between Kentucky and Maryland Schools 208 5. The Persistent Gap between Performance and Classroom Reality 210 6. Mismatch of Priorities 214 7. Generalized Policy Effect or Varied Local Capacities 216 8. Improving Probation Policies 219 References 223 Volume III - Case. Studies Volume IV Part IX. The Politics of Accountability for School Improvement in Kentucky and 1 Maryland 1. Introduction 1 2. Research Methods 3 3. Conceptual Framework 4 4. Overview of Reforms in Both States 6 5. A Summary of Policy Developments in Kentucky 10 6. Maryland: The Dominance of One Political Coalition 21 7. Conclusion 40 References 44 Appendix Roadmap for Case Studies 2 Case Study Research Guide 6 School Background Data for Case Study 12 Interview Protocol I 14 (Focus: School background, personal background, reaction to probation) Interview Protocol II 21 (Focus: Improvement strategies, group accountability) Qualitative Analysis: Codes for NUDIST 21 Research Instrument: Instructional Capacity 35 (Lesson Observation Guide, Debriefing Interview Guide) Interview Guide State Monitors, Support Staff 53 Meeting Observation Guide 57 Case Report Guide 60 Survey Questionnaire 65 Content Analysis Codebook 77 AUTHORSHIP Volume I' Part I Heinrich Mintrop Part II Heinrich Mintrop Masako Nishio, conducted the statistical analysis Part II Heinrich Mintrop Masako Nishio, conducted the statistical analysis Volume if Part IV Heinrich Mintrop Kim Curtis Lea Plut-Pregelj Margaret Quintero with assistance from Betty King Part V Heinrich Mintrop Ann MacLellan, conducted the content analysis Margaret Quintero, assisted in the Kentucky content analysis Part VI Heinrich Mintrop Dana Buese Part VII Heinrich Mintrop Appendix Heinrich Mintrop Votun:ie III Part VIII Kim Curtis Heinrich Mintrop Lea Plut-Pregelj Margaret Quintero Robert Pettit Betty King voinne Part IX James Cibulka Jane Clark Lindle ACKNOWLEDGMENTS We would like to thank the U.S. Department of Education for their financial support and Elizabeth Demerest for her stewardship of this project. Stephanie Goodwin, the business manager at the Educational Policy and Leadership Department atthe University of Maryland, helped us immensely. Blanca Rivera patiently tended to our technological needs. Jane Clark Lind le oversaw the Kentucky portion of the project. We also want to thank Jennifer O'Day for her advice on the study design. She and her assistant Matthew Kelemen also contributed data from San Francisco. Kim Curtis and Dana Buese edited the entire four volumes of this report. Thank you also to Betty Ma len for stimulating thoughts. 6 EXECUTIVE SUMMARY Probation as a Tool for Policy Across the United States, incentive policies that reward schools for good performance and penalize them for failing to maintain acceptable performance levels have proliferated. The sanction of probation works on schools in several ways: it may attach a stigma, make a threat, heighten scrutiny, issue a warning signal, or symbolize public concernand commitment. We define probation as a performance status, institutionalized in various nomenclatures, that publicly identifies a school as underperforming according to stipulated criteria of its accountability agency. This report describes what schools do when they are put on probation. Once a state department of education or a local district has put a school on probation, four main policy levers define its relationship with the school: sanctions, performance standards and assessments, process controls, and resource provisions. Sanctions, standards, controls, and resources may exert pressure, activate goals, constrain