Miami-Dade County Public Schools Office of Program Evaluation 1500 Biscayne Boulevard Miami, Florida 33132 Evaluation of the School Improvement Zone 2006-07 January 2009 Principal Evaluator/Author: Steven M. Urdegar, Ph.D. THE SCHOOL BOARD OF MIAMI-DADE COUNTY, FLORIDA Dr. Solomon C. Stinson, Chair Dr. Marta Pérez, Vice Chair Mr. Agustin J. Barrera Dr. Lawrence S. Feldman Ms. Perla Tabares Hantman Mr. Renier Diaz de la Portilla Dr. Martin Karp Dr. Wilbert “Tee” Holloway Ms. Ana Rivas Logan Ms. Angelique Gayle, Student Advisor Mr. Alberto M. Carvalho Superintendent of Schools Jerome L. Levitt, Ph.D., Executive Director Office of Program Evaluation ACKNOWLEDGMENTS This is to recognize the contribution of Mr. Fred Reed, Regional Coordinator of the William C. Golden School Leadership Program, who with his team of retired administrators Ms. Thelma Davis, Mr. Jose Enriquez, Ms. Darleen Mooney, Dr. Fred Morley, Dr. Robert Russell, Ms. Sue- Ann Wendover, and Ms. Willa Young availed us of more than two centuries of combined expertise by conducting classroom observations and school reviews that enabled us to further the knowledge base and scope of this evaluation. Special thanks are owed to Dr. Beverly Karrenbauer who served as both a team member and a mentor principal for the Zone for also helping to develop the practices instrument, and to Ms. Barbara Bell for helping us adapt the principals’ walkthrough. Finally, I would like to thank Ms. Lucia Herrera for allowing the team to participate in this effort, and in so doing, serving as an exemplar for interdepartmental collaboration. i TABLE OF CONTENTS LIST OF TABLES.................................................................................................................................... iv LIST OF FIGURES ................................................................................................................................. vi EXECUTIVE SUMMARY ................................................................................................... vii DESCRIPTION OF THE PROGRAM ................................................................................... 1 REVIEW OF THE LITERATURE......................................................................................... 7 Educational Practices......................................................................................................... 7 Implementation ................................................................................................................ 10 Change and Diffusion ...................................................................................................... 13 Leadership ........................................................................................................................ 14 Contextual Factors ........................................................................................................... 15 Value Added Modeling.................................................................................................... 17 METHODOLOGY ................................................................................................................ 19 Implementation ................................................................................................................ 19 Learning Environment ..................................................................................................... 25 Student Achievement....................................................................................................... 28 Student Performance on the FCAT-NRT ................................................................. 29 Student Performance on the FCAT-SSS................................................................... 31 FCAT Writing............................................................................................................ 31 School Accountability Grades................................................................................... 32 Other Indicators Associated with Student Achievement.......................................... 33 Home-School and School-Community Relations........................................................... 34 Program Effectiveness ..................................................................................................... 34 RESULTS .............................................................................................................................. 35 Implementation ................................................................................................................ 35 Return Rates............................................................................................................... 35 Resources ................................................................................................................... 37 Adherence .................................................................................................................. 38 Capacity ..................................................................................................................... 43 Differentiation............................................................................................................ 55 Leadership.................................................................................................................. 59 Discrepant Events...................................................................................................... 64 Learning Environment ..................................................................................................... 64 Student Achievement....................................................................................................... 68 Student Performance on the FCAT-NRT ................................................................. 68 Reading ................................................................................................................ 68 Mathematics......................................................................................................... 73 Student Performance on the FCAT/SSS................................................................... 78 Reading ................................................................................................................ 79 Mathematics......................................................................................................... 83 ii TABLE OF CONTENTS, (CONTINUED) Science ................................................................................................................. 88 Student Performance on the FCAT Writing Assessment......................................... 90 School Accountability Grades................................................................................... 92 Other Indicators Associated with Student Achievement.......................................... 93 School Characteristics ......................................................................................... 93 Teacher Movement.............................................................................................. 94 Pupil Progression................................................................................................. 95 Transitions ........................................................................................................... 96 Home, School, and Community Relations...................................................................... 97 Program Effectiveness ..................................................................................................... 98 DISCUSSION ...................................................................................................................... 101 Summary ........................................................................................................................ 101 Conclusions .................................................................................................................... 107 Recommendations.......................................................................................................... 108 REFERENCES..................................................................................................................... 109 APPENDIX A: Interview Protocols................................................................................... 115 APPENDIX B: Observation Instruments........................................................................... 119 APPENDIX C: Zone Stakeholder Surveys ........................................................................ 123 APPENDIX D: Reading Instruction and Leadership Survey ............................................ 131 APPENDIX E: School Climate Survey.............................................................................. 139 APPENDIX F: Hierarchical Linear Modeling (HLM) Analysis....................................... 147 iii LIST OF TABLES Table 1: Demographic Characteristics of the School Improvement Zone, 2006-07 ......................2 Table 2: School Improvement Zone Schools by Feeder Pattern, 2006-07 .....................................3 Table 3: Schools in the Control Groups for the Analysis of Instruction and Leadership.............23 Table 4: Schools in the Control Groups for the Analysis of Learning Environment ...................27 Table 5: Schools in the Control Groups for the Analysis of Achievement on the FCAT-NRT.......................................................................................................................30 Table 6: Return