PLANNING AND EVALUATION SERVICE
EARLY IMPLEMENTATION OF THE COMPREHENSIVE SCHOOL REFORM DEMONSTRATION (CSRD) PROGRAM
Summary Report 2000
U.S. DEPARTMENT OF EDUCATION ~ OFFICE OF THE UNDER SECRETARY
Early Implementation of the Comprehensive School Reform Demonstration (CSRD) Program
Summary Report
2000
Prepared by:
Kathryn M. Doherty Planning and Evaluation Service Office of the Undersecretary U.S. Department of Education
September 2000
U.S. Department of Education Richard Riley Secretary
Office of the Under Secretary Judith A. Winston Under Secretary (A)
Planning and Evaluation Service Alan Ginsburg Director
Elementary and Secondary Education Division Ricky Takai Director
September 2000
This report is in the public domain. Authorization to reproduce it in whole or in part is granted. While permission to reprint this publication is not necessary, the citation should be: U.S. Department of Education, Office of the Under Secretary, Planning and Evaluation Service, Elementary and Secondary Division, Early Implementation of the Comprehensive School Reform Demonstration (CSRD) Program, Washington D.C., 2000.
To order copies of this report, write:
ED Pubs Editorial Publications Center U.S. Department of Education P.O. Box 1398 Jessup, MD 20794-1398
Via fax, dial (301) 470-1244
Or via electronic mail, send your request to: [email protected]
You may also call toll free: 1-877-433-7827 (1-877-4ED-PUBS). If 877 service is not yet available in your area, call 1-800-872-5327 (1-800-USA-LEARN). Those who use a telecommunications device for the deaf (TDD) or a teletypewriter (TTY), should call 1-800-437-0833.
To order online, point your Internet browser to: www.ed.gov/pubs/edpubs.html
On request, this publication is available in alternate forms, such as Braille, larger print, audiotape, or computer diskette. For more information, contact the Department’s Alternate Format Center (202) 260- 9895 or (202) 205-8113.
ii CONTENTS
Executive Summary ………………………………………………………………….….…..…vii
I. Introduction .…………………………………………………………………..………..1 Approach of This Report .…...……………………………...…………………………….1 Why Comprehensive School Reform? …………..………………………………………. 3 CSRD Program Overview ………………………………………………………...………6 Incentive Grants for Comprehensive Reform……………………………………..6 Nine Components of CSRD……………………………………………………….7 CSRD Funding…………………………………………………………………….8
II. Profile of Participating Schools………....…………………………………………..…10 Summary of Information from the National Database of CSRD Schools………...……..10 Access to CSRD in Rural and Urban Schools………………………..………………….12 Summary…………………………………………………………………………………13
III. Federal Implementation of CSRD and Support for Research and Development on Comprehensive School Reform ……………………………………………………….14 CSRD Program Administration………………………………………………...………..15 Implementation Guidance…………………………………………………..……15 Peer Reviews of State CSRD Applications………………………………...……15 CSRD Coordinators Conferences…………………………………………...…...16 Electronic Communication…………………………………………………...….17 Products to Support Comprehensive School Reform ……………………………17 Technical Assistance and Research Activities …………...…………………………...…18 Regional Education Laboratory Technical Assistance and Research ...………....18 Comprehensive Regional Assistance Center Activities…………………....…….20 National Clearinghouse for Comprehensive School Reform…………………….21 Middle and High School Model Development…………………………………..22 Model Developer Capacity Grants……………………………………………….23 National Evaluation of CSRD…………………………………………………...……….24 Collecting Baseline Information on All CSRD Schools…………………………25 Large Scale Longitudinal Data Collection on Implementation in CSRD Schools………………………………………………...…………………25 Field Focused Case Studies of CSRD……………………………………………25 Monitoring School Reform Research …………………………………………...26 CSRD Reports from the Field…………………………………………………....26 Summary…………………………………………………………………………………28
IV. State and District Implementation of CSRD ………………………………………....29 Findings from New American Schools CSRD Implementation Project ……..……….....29 State Applications and Grant Competitions…...……………………………...….30 State CSRD Review Process ……………………………………………………30 CSRD-Like Grants and Other State Support Activities………………………….31 State Efforts to Help Schools Make Informed Choices………………………….33
iii Early Lessons from NCREL on State Implementation of CSRD………………………..34 Reports from the Field: State and District Considerations……………………………....35 Helping Schools Develop and Choose Reform Programs……………………….36 Addressing Alignment of Models with State and District Goals……………...…36 Helping Schools Make Time for Reform Efforts………………………………..37 Coordinating Support Roles…………………………………………...…………38 Developing “Buy-In”…………………………………………………………….38 Cultivating Leadership………………………………………………..…………39 Summary…………………………………………………………………………………40
V. School Implementation of CSRD ………………………………………………...……41 Importance of Implementation……………………………………………...……………41 Findings from the National Longitudinal Survey of Schools (NLSS)…………….……..43 CSRD School and Student Characteristics…………………...………………….44 Characteristics of Teachers in CSRD Schools……………………...……………46 How CSRD Schools Selected Models……………………………………...……47 Status of Model Implementation ………………………...………………………48 External Assistance to CSRD Schools…………………………………….……..49 Title I Services in CSRD Schools………………………………………………..51 Professional Development…………………………………………………...…..54 Parent Involvement………………………………………………………………56 Technology………………………………………………………………………57 CSRD Schools in Need of Improvement………….……………………………..58 Student Expectations ………..…………………………………………………..60 Reports from the Field: Observations on School Implementation of CSRD…………….61 Using Data Effectively …………………………………………………………..61 Integrating Models With Reform Schoolwide……………………………...……62 Challenge of Implementing Multiple Models ……………………………………62 Addressing the Needs of Special Populations…………………………...………63 Negotiating with Technical Assistance Providers……………………………….64 Building Schoolwide Support for Reform……………………………………….64 Developing Benchmarks…………………………………………………………65 Involving Parents in Comprehensive School Reform……………………………66 Highlights from Selected State Evaluations of CSRD………………………………...…67 Missouri………………………………………………………………………….67 Wisconsin………………………………………………………………………...68 Summary…………………………………………………………………………………70
VI. Emerging Research on Comprehensive School Reform and Models…………..……72 Analysis of Existing Research on Models……………………………………………….72 Research Study Profiles……………………………………………………………...…..74 New American Schools…………………………………………………………..74 Study of Instructional Improvement…………………………………………...... 75 Scaling Up………………………………………………………………………..76 Longitudinal Research on Whole School Improvement…………………………77 FEPSI (Formative Evaluation Process for School Improvement)……………….78 Summary…………………………………………………………………………………78
iv VII. Conclusion ……….……………………………………………………………………..79 Early Implementation Findings and Challenges…………………………………………79 Implications of Findings for Administration of CSRD…………………………………..80 Looking Ahead………………………………………………………………...…………81
Appendix A: CSRD Legislation Appendix B: CSRD Funding Appendix C: CSRD Program Performance Plan Appendix D: American Institutes for Research Rubric for Classifying Types of Models Appendix E: Continuum of Evidence of Effectiveness Appendix F: CSRD Continuum for Assessing the Comprehensiveness of School Reform Plans Appendix G: CSRD Research Tracking Appendix H: Resources to Support Comprehensive School Reform
v LIST OF TABLES
Table Table Title Page
1 Frequency of Schools Adopting Models Cited in CSRD Legislation 11
2 Distribution of First-Year CSRD Sample, by State 43
3 Comparison of CSRD and Title I Schools by Selected Characteristics 45
4 Reasons for Adopting Model-Based Reform 47
5 Factors Influencing Model Choice 48
6 Implementation 49
7 Assistance on Implementation of Models 50
8 School Activities Promoted to “A Great Extent” by Changes in Title I Legislation 51
9 Title I Services Provided in CSRD and Title I Schools 52
10 Challenges in Coordinating Federal Resources with Other School Funds 53
11 Principal Reports about Teacher Professional Development 55
12 Daily Parent Involvement Activities 57
13 Major Barriers for Using Technology for Instruction al Purposes 58
14 CSRD Schools in Need of Improvement 59
15 Additional Strategies Implemented by Schools as a Result of Being Identified as in Need of Improvement 60
vi
I. Introduction
Authorized in Public Law 105-78, the FY 1998 Department of Education Appropriations Act, the Comprehensive School Reform Demonstration (CSRD) Program aims to raise student achievement by helping public schools across the United States successfully implement comprehensive school reform strategies. The legislation promotes the adoption of reform models based on reliable research and effective practices, with an emphasis on coordinated, aligned school reform programs.
The legislation authorizing CSRD also mandates national evaluation activities to “assess results achieved by the implementation of comprehensive school reform in Title I schools.” According to the conference language, the initial evaluation activities will include “collection of baseline data and assessment of the first-year implementation activities.” This report, prepared by the Planning and Evaluation Service of the U.S. Department of Education, is a description of the early federal, state, and school-level implementation of CSRD.
Approach of This Report
What follows aims to be a unique evaluation report. This paper draws on numerous sources of data–some evaluation activities funded directly by the Planning and Evaluation Service and other activities sponsored by the Department funded Regional Educational Laboratories and Comprehensive Regional Assistance Centers, and organizations involved in research and evaluation of comprehensive school reform. The report takes advantage of information from the Southwest Educational Development Laboratory (SEDL) database of CSRD schools, the National Longitudinal Survey of Schools (NLSS), Department visits to CSRD schools during early implementation, Regional Educational Laboratory research activities, special analyses by New American Schools (NAS) and the American Institutes for Research (AIR), and early state evaluation findings.
However:
• This report does not include an analysis of student achievement outcomes. While the three- year evaluation mandate for CSRD requires an assessment of the “results achieved by the implementation of comprehensive school reform in Title I schools,” it is too early in the implementation process to draw conclusions on achievement outcomes. In the meantime, the Department is collecting student achievement data for every CSRD school, will track progress, and will report outcomes in the third year evaluation report.
• This report is not intended to assess the effectiveness of particular research-based models. In part, this is because the federal program is focused on comprehensive school reform, of which research-based models are only a part. While research has documented the strength of comprehensive rather than piecemeal approaches to reform, more rigorous research is needed to evaluate the effects of particular reform designs being implemented with CSRD funds.
1
This report is written with an eye towards being useful to a variety of audiences. The report examines CSRD implementation from the federal level to the school level. It includes not only observations and descriptive information, but also advice, resources, and lessons learned during the early implementation process.
• For Congress: This report is meant to inform lawmakers about the early implementation of the CSRD program.
• For federal and state administrators: This report is meant to help those with program responsibilities take stock of the CSRD program at an opportune time. In fiscal year 2000, an increase in CSRD funds has been made available so that additional schools can obtain CSRD awards to implement research-based models and comprehensive school reform programs.
• For school districts and schools participating in CSRD: This report provides feedback on what we are learning from CSRD evaluation and comprehensive school reform research in general and shares the many resources that have been developed to support this program.
• For technical assistance providers: This report contains feedback from schools about their satisfaction with external assistance as well as information about professional development needs and the challenges CSRD schools face.
• For parents and community partners: This report provides an overview of the rationale for comprehensive reform, as well as insights and findings on the issues school stakeholders face in implementing comprehensive school reform.
The report is divided into six main sections. The introduction or section I reviews the research and the rationale behind the CSRD program. Section II presents a current profile of participants in the CSRD program, including descriptive information on the approximately 1,800 current CSRD grantee schools. Sections III and IV of the report turn to descriptions of the early implementation of CSRD at the federal and state levels. These sections of the report draw on information from the CSRD program office, the Office of Educational Research and Improvement (OERI), and an analysis of state implementation by New American Schools.
Section V of the report focuses on the school-level implementation of CSRD. The report presents first-year results from the National Longitudinal Survey of Schools (NLSS), a survey of a nationally representative sample of Title I schools that was also administered to approximately 300 CSRD schools that received awards early in the 1998-99 school year. The survey explores issues related to model implementation, external assistance, professional development, standards-based reform, and parent involvement.
Section VI of the report highlights a number of current research and evaluation projects related to comprehensive school reform that will help inform future reform efforts. Finally, Section VII of the report summarizes major findings from this early implementation report and looks ahead to future evaluation reports.
2
Why Comprehensive School Reform?
The design of the Comprehensive School Reform Demonstration (CSRD) program builds upon much of what we have learned from research about how children learn and how schools improve. Decades of research on effective schools point to the importance of rigorous curriculum and high standards for all students, an atmosphere of collaboration and mutual respect among school staff, ongoing and high-quality staff development, efficient school management, and meaningful parent involvement. Research also shows that piecemeal, fragmented approaches to school reform rarely add up to a coherent, sustainable whole.
As an alternative, comprehensive school Excerpt from Comprehensive School Reform: improvement strategies aim to help Five Lessons From the Field transform entire schools through a consistent and coordinated program. Comprehensive school reform is not just another Research supports the need for school improvement strategy–it is a significant leap schoolwide approaches for improving forward in reforming today’s public schools. student achievement, particularly for Comprehensive school reform addresses all students, ll students in high-poverty schools. academic subjects and all teachers. When done well, a Emerging research suggests that school is overhauled from top to bottom. Adding one externally developed research-based program on top of another is thrown out in favor of the much more difficult work of reorganizing schools, reform approaches can help schools targeting professional development for teachers and implement whole school reform. principals, changing curriculum and making tough budget decisions. When findings from education research in the 1960s and 1970s suggested that In short, comprehensive school reform transforms the school effects on student achievement way a school functions to accomplish one goal: are minimal compared to parent improved student achievement for all students. education and social background, some Comprehensive school reform is a breakthrough that concluded that schools do not make a allows schools, districts and states to move beyond difference. However, during the last 25 finger pointing and blame to real improvements in years, a considerable body of evidence student learning. Implementing this reform strategy is not easy, however. There is nothing tougher than has accumulated showing that, although spending money differently, sticking with an approach ability and family background are major long enough to see results, and overcoming turf battles influences on student achievement levels, along the way. some schools serving disadvantaged students promote high levels of -Education Commission of the States, 1999 achievement, as measured by test scores and such outcomes as improved attendance, attitude, and behavior.
While there is no single factor that accounts for school effectiveness, research has identified particular factors that are present in or associated with effective schools.1 The characteristics include:
1 Edmonds, R. 1979. “Effective Schools for the Urban Poor.” Educational Leadership 37(1) October; Edmonds, R. 1981. “Making Public Schools Effective.” Social Policy 12(2) September.
3
• clearly stated goals and objectives,
• high expectations for all students,
• strong leadership that has a vision for the school and the expertise to make the vision a reality,
• an emphasis on academics,
• systems for monitoring and assessing pupil progress tied to instructional objectives,
• maximizing the amount of classroom time that is devoted to active learning,
• the use of a variety of instructional methods and groupings,
• a safe and orderly school climate, and
• high levels of parental involvement.
More recent studies of high-performing, high-poverty schools–such as Hope for Urban Education and Dispelling the Myth–update and emphasize again the findings of the effective schools literature.2
In a set of case studies of high-performing, high poverty schools, Hope for Urban Education found that leaders and staff in these schools (1) set attainable and ambitious goals; (2) focused on students rather than adult issues in the school building; (3) fostered a sense of collective responsibility for behavior and learning; (4) used data effectively; (5) focused on instructional activities; (5) increased instructional time; (6) aligned instruction with standards; and (7) created collaborative time for teachers. That study recommended that the federal government “infuse the tenets of comprehensive school reform into federal education programs” with emphasis on the common components and principles of effective schools and schoolwide programs.
Given the attributes of effective schools described above, the challenge for educators and policymakers has been to develop strategies to help schools develop and institutionalize such effective practices. When, in 1993, the U.S. Department of Education reported to Congress on the effectiveness of Chapter 1–the largest K-12 federal education program designed to assist low-performing students and students in high-poverty schools–serious questions were raised about the extent to which Chapter 1 was helping create effective schools that could raise the achievement of low-performing students. The report also documented the important effect of school poverty on student achievement for all students in high-poverty schools. Findings from the report, Reinventing Chapter 1, include the following:
2 U.S. Department of Education. 1999. Hope for Urban Education (Washington D.C: Government Printing Office); The Education Trust. 1999. Dispelling the Myth (Washington D.C.: Author).
4
• Students in high-poverty schools were held to lower expectations than students in low- poverty schools. When measured on a common test, an “A” student in a high-poverty school would be about a “C” student in a low-poverty school;
• Poverty affected the whole school, not just poor students; and
• Chapter 1 services to low-performing students were mostly piecemeal, in the form of pullouts from regular classrooms, adding only 10 minutes of instruction time a day, and often uncoordinated with the rest of the educational experience of the student.3
What these findings suggested was the need to focus energy on creating schoolwide approaches to increasing student achievement in high-poverty schools where the majority of students might need extra help to succeed. Such approaches needed to raise expectations for all and coordinate school activities and programs to address the needs of all students.
With increasing numbers of schoolwide programs in the United States adding great flexibility to how schools address the needs of all of their students, attention is becoming more focused on identifying the particular kinds of whole school strategies that will most raise student performance.
In 1995, the Department of Education’s study Special Strategies looked more specifically at the effectiveness of schoolwide programs in a small number of in-depth cases. 4 The study asked the question–are there specific programs or designs that enhance learning for disadvantaged students? The evaluation examined high-poverty schools and compared programs that had targeted interventions with whole-school restructuring efforts. It also compared programs that were externally developed (developed outside of the school setting) with school-designed efforts. Special Strategies found that:
• Students in schools implementing externally developed school reforms tended to achieve greater gains than did students in schools using locally-developed programs, and
• Students in schools working with whole school reform tended to achieve greater gains than did students in schools attempting various targeted pullout programs.
These findings indicating positive results for schools assisted by externally developed designs were corroborated in a pre-CSRD study by Steven Ross on 25 elementary schools in Memphis, Tennessee, using externally developed, research-based models for school improvement. The 1998 study indicated that student achievement gains after two years were higher in those schools with comprehensive restructuring programs than in demographically similar but non-reforming elementary schools in the district. 5
3 U.S. Department of Education. 1993. Reinventing Chapter 1 (Washington D.C.: Government Printing Office) 4 U.S. Department of Education. 1997. Special Strategies for Educating Disadvantaged Children (Washington D.C.: Government Printing Office)
5 Ross, Steven. 1998. The Memphis Restructuring Initiative (Memphis: University of Memphis).
5
In 1999 the American Association of School Administrators, the National Association of Elementary School Principals, the National Association of Secondary School Principals, the American Federation of Teachers, and the National Education Association collaborated to publish An Educator’s Guide to Schoolwide Reform, which reviewed more than 100 studies of 24 externally developed school reform models. The review found three models–Success for All, Direct Instruction, and High Schools That Work–that had documented strong evidence of positive effects on student achievement. Several other models were rated as having promising evidence of positive effects. 6
Research has documented the strengths of comprehensive, rather than piecemeal, approaches to school improvement. Today, the focus of schoolwide reform efforts is on the identification and implementation of effective, proven, research-based practices for improving student achievement. While research on the effectiveness of specific, externally developed models is still at an early stage, CSRD is explicitly designed to help schools undertake coherent and research-driven reforms.
CSRD Program Overview
The CSRD program was created in fiscal year 1998 to provide financial incentives for schools to develop comprehensive school reforms based on reliable research and effective practices and including an emphasis on basic academics and parental involvement, so that all children can meet challenging state content and performance goals.
Incentive Grants for Comprehensive Reform
Congress authorized the program in order to encourage more schools to make more effective use of Title I funds by adopting effective, research-based models for school improvement. CSRD grants are not expected to fully fund comprehensive school reform efforts. Rather, CSRD provides schools with start-up or incentive grants to begin the reform process, with schools potentially drawing on all the resources available to them in order to make improvements in teaching and learning. The bulk of the CSRD funds are targeted toward Title I schools. It has been the hope in Congress that focusing the incentive funding on schools eligible for Title I will help leverage systemic improvements in the use of Title I funds for research-based strategies and raise student achievement throughout the $8 billion Title I program which serves more than 47,000 schools, including more than 19,000 schoolwide programs. 7
The expectation of the program has been to provide three-year grants to schools to help cover the initial costs of adopting an externally developed model or a comparable set of research-based strategies, as part of a school's comprehensive school reform plan. The goal of CSRD is to serve a mix of schools–urban and rural schools, elementary, middle and secondary schools as well as significantly target schools with low levels of student achievement, and schools identified as in need of improvement by state and Title I accountability systems. The legislation also encouraged state education agencies (SEAs) to support awards to local education agencies (LEAs) that demonstrate a commitment to assisting schools with budget reallocation strategies
6 American Institutes for Research. 1999. An Educator's Guide to Schoolwide Reform (Arlington, VA). 7 See Appendix A for text of the CSRD authorizing legislation.
6
necessary to ensure that comprehensive school reforms are properly implemented and sustained beyond the CSRD incentive grant period.
The CSRD program provides formula grants to SEAs for competitive grant awards to LEAs on behalf of individual schools. In awarding competitive grants to LEAs, the conferees directed SEAs to make awards of sufficient size and scope to support the initial start-up costs for particular comprehensive reform plans selected or designed by schools, but to provide no less than $50,000 per school. The legislation provided for CSRD grants to be renewable for two additional years after the initial award, for a total of three years of support.
Nine Components of CSRD
The CSRD legislation calls for schools to coherently address nine key components of school reform. These nine components build on what we know from research on effective schools. All participating schools must use program funds to adopt or develop research-based comprehensive school reform approaches that: The Comprehensive School • employ innovative strategies and proven Reform Program focuses on… methods for student learning, teaching, and school management that are based on reliable • Whole school rather than piecemeal research and effective practices, and have reform efforts, been replicated successfully in schools with • Research-based strategies for diverse characteristics; school improvement, • External partnerships and support • have a comprehensive design for effective for improvement process, • Extensive use of data for assessing school functioning, including instruction, needs and school improvement, assessment, classroom management, and • Flexible use of resources to support professional development; the design should reform priorities. align the school's curriculum, technology, professional development into a schoolwide reform plan designed to enable all students to meet challenging state content and performance standards and addresses needs identified through a school needs assessment;
• provide high-quality and continuous teacher and staff professional development and training;
• have measurable goals for student performance and benchmarks for meeting those goals;
• are supported by school faculty, administrators, and staff;
• provide for the meaningful involvement of parents and the local community in planning and implementing school improvement activities;
• utilize high-quality external support and assistance from a comprehensive school reform entity (which may be a university) with experience or expertise in schoolwide reform and improvement;
7
• include a plan for the evaluation of the implementation of school reforms and the student results achieved;
• identify how other resources (federal/state/local/private) available to the school will be utilized to coordinate services to support and sustain the school reform.
The legislation encourages schools to examine successful, externally developed comprehensive school reform approaches that can be adapted to their own communities. The legislation included a list of seventeen research-based school reform models that might be included as part of a comprehensive reform program meeting these criteria. At the same time, Congress was explicit about not restricting schools to using only these identified approaches. Schools are able to use other models or develop their own schoolwide reform programs that are based on rigorous research and meet the criteria described above. Excerpt from Joint Explanatory Statement of the Committee of Conference CSRD Funding (H.R. No. 390, 105th Congress)
CSRD funds first became available While no single school improvement plan can be to states and their grantees in July best for every school, the conferees believe that 1998. Funding for CSRD has been more schools should be encouraged to examine provided from two sources–Title I successful, externally developed comprehensive and the Fund for the Improvement of school reform approaches that can be adapted in Education (FIE). Funds their own communities... However, schools are not appropriated under Title I have been restricted to using only those approaches… [and] are allocated to states on the basis of free to develop their own school-wide reform programs that are based on rigorous research…. each state's share of prior year Title I Basic Grants. In turn, states make competitive subgrants to schools eligible for Title I funds.
Funding provided through the Fund for the Improvement of Education is for additional state allocations based on each state's share of school-aged children. SEAs may subgrant their FIE funds to any public school in the state. The Department was authorized to reserve up to 1 percent of CSRD funds for the Outlying Areas (the Virgin Islands, American Samoa, Guam, and the Northern Marianas) and the Bureau of Indian Affairs, and up to 1 percent for national evaluation of the program.
SEAs may reserve up to 5 percent of CSRD funds for administrative, evaluation, and technical assistance expenses, including expenses necessary to inform LEAs and schools about research- based comprehensive school reform approaches.
For fiscal year 1998, Congress appropriated $120 million to support comprehensive reforms in schools eligible for Title I funds. An additional $25 million was available to all public schools through FIE. As of June 1999, all 50 states, the District of Columbia, and Puerto Rico had been awarded $145 million in first-year funding. In July 1999, $145 million in second-year CSRD funding was awarded. Third-year CSRD funding–increased to $220 million ($170 million for Title I and $50 million through FIE) for FY2000–was made available on July 1, 2000. This
8
increase will support the third year of most current CSRD grants plus support the implementation of CSRD in up to 1,000 additional schools across the country.
See Appendix B for a summary of CSRD awards to states for fiscal years 1998-2000.
The fiscal year 1998 appropriation for CSRD provided an additional $5 million for dissemination and technical assistance (not included in the $145 million described above). Included were $4 million for the Department's Regional Educational Laboratories to assist schools in selecting, evaluating, and implementing comprehensive school reforms, and $1 million for Department activities to identify and disseminate information, in consultation with outside experts, about effective research-based approaches to comprehensive school reforms. Funds for these activities have been sustained and expanded for fiscal years 1999 and 2000.
9
II. CSRD: A Profile of Participating Schools
In allocating CSRD funds, Congress encouraged states to award grants to LEAs that will use these funds to support schools in need of improvement under Title I of the Elementary and Secondary Education Act (ESEA); support schools and districts in different parts of the state, including urban and rural communities; and support districts proposing to serve schools at different grade levels (elementary/middle/high school). This section of the report presents data on the distribution of CSRD grants to schools across the nation.
Summary of Information from the National Database of CSRD Schools
In order to track the allocation of CSRD awards and evaluate the CSRD program, the U.S. Department of Education required that each state submit basic demographic and award information on each CSRD school grantee. The Department arranged that this information be submitted by the states to the Southwest Educational Development Laboratory Database of CSRD Schools (SEDL) and be made available in a publicly accessible database. The SEDL Database, which provides basic information on all CSRD grantees is accessible Each state was asked to provide the to the public at: following data on each CSRD grantee: school name, contact information, award http:/ / www.sedl.org/csrd/awards.html date, award amount, model(s) implemented, whether the school is a Title I school, whether the school is a schoolwide program or targeted assistance program, school poverty rate (usually measured by percent of students receiving free and reduced price lunch), and whether the school has been identified as low- performing or in need of improvement under Title I and/or other state or local accountability systems.
The following summary profile of schools receiving CSRD grants, as of June 1, 2000, indicates that CSRD has substantially met the program goal of awarding CSRD grants to a high-need schools in diverse settings. CSRD is substantially targeted to high-poverty and low-performing schools.
• To date, CSRD grants have been awarded to districts on behalf of 1,790 schools.
• Eighty-five percent of CSRD schools receive Title I funds and 65 percent of CSRD schools are Title I schoolwide programs. (Overall, 40 percent of Title I schools are schoolwide programs).
• Seventy percent of CSRD schools are elementary schools, and 30 percent are middle or high schools.
• Approximately half of CSRD schools are located in large or midsize cities; 20 percent are located on the urban fringe of a large or midsize city or in a large town; and 30 percent are located in a small town or rural area.
10
• The average poverty rate in CSRD schools is 70 percent. The poverty rates of CSRD schools ranges from a low of 21 percent to a high of 100 percent. In most cases, the poverty rate of schools is based on the percentage of students in schools receiving free or reduced price lunch.
• The average amount of CSRD awards is almost $69,000. CSRD awards range from $50,000 to three small rural schools in a consortium of schools implementing Success for All, to a $600,000 award to a large California high school implementing a locally developed program called "Project STEPS."
• A high percentage of CSRD schools have been identified as in need of improvement under Title I. Compared to Title I in general–where recent state performance report data indicate that approximately 20 percent of Title I schools are identified as in need of improvement–42 percent of CSRD schools have been identified as in need of improvement under Title I. Of the 745 CSRD schools identified for improvement under Title I, 411 of those schools have also been identified as low-performing or in need of improvement according to state or local accountability systems. An additional 100 schools have been identified as low-performing only by their states or districts and not through Title I.
• Forty-eight percent of CSRD schools have adopted one of the seventeen models cited in the legislation as part of their comprehensive school reform program. The remaining schools have adopted other nationally available models, as well as local university or district- developed models, and some school developed models, as part of their comprehensive school reform program.
Table 1: Frequency of Schools Adopting Models Cited in CSRD Legislation
Model Number of CSRD Sites Accelerated Schools 123 America's Choice 60 ATLAS 19 Audrey Cohen College 8 Coalition of Essential Schools 54 Comer School Development Program 32 Community for Learning 24 Co-NECT 49 Direct Instruction 60 Expeditionary Learning 30 High Schools That Work 49 Modern Red Schoolhouse 26 Paideia 3 Roots and Wings 58 Success for All 255 Talent Development High School 11 Urban Learning Center 6
11
• Although CSRD schools have named over 200 models in use in their schools, over 70 percent of CSRD schools have adopted one of just 25 models. The following are the top 25 chosen models under CSRD:
1) Success for All 14) Ventures Initiative and Focus System 2) Accelerated Schools 15) Expeditionary Learning Outward Bound 3) Lightspan 16) AVID 4) Direct Instruction 17) Different Ways of Knowing 5) America's Choice 18) Modern Red Schoolhouse 6) Roots and Wings 19) Reading Recovery 7) Coalition of Essential Schools 20) Community for Learning 8) High Schools That Work 21) DePaul University Center for Urban 9) Co-NECT Education 10) Core Knowledge 22) Reading Renaissance 11) HOSTS 23) Early Literacy Learning Initiative 12) Coalition of Effective Schools 24) Middle Start 13) Comer School Development Program 25) ATLAS
• The most frequently selected model is Success for All, adopted by 14 percent of all CSRD schools. Twenty-three percent of schools are using a New American Schools (NAS) design.
• Based on data reported to the Southwest Educational Development Laboratory database, of the schools implementing one of the top 25 chosen models, 16 percent are implementing more than one model. The models most often implemented in conjunction with other models are Success for All, Roots and Wings, Reading Recovery, Early Learning Literacy Initiative, and Lightspan.
• Approximately 369 schools, or 21 percent of CSRD schools, are using a model rated strong by the American Institutes for Research in An Educator’s Guide to Schoolwide Reform study of school reform models. Approximately 531 schools, or 30 percent of CSRD schools, are using a model rated either strong or promising by the American Institutes for Research.
For a state by state summary of CSRD awards, funding amounts, and school descriptions, see Appendix B.
Access to CSRD in Rural and Urban Schools
At the outset of the CSRD program, concerns were raised about rural access to CSRD, particularly as it relates to access to external technical assistance providers and model developers. In particular, the isolated location of some schools might make it more difficult to secure model providers, and rural schools might be underfunded in the CSRD program. Concerns also were raised about whether the currently available research-based models were tailored more for urban school needs rather than for rural needs. Given these concerns, some of
12
the Department's Regional Educational Laboratories have begun to examine the issue of rural access to CSRD grants.
Using the Southwest Educational Development Laboratory’s database of CSRD schools as a source of information, a recent report by the Rural Laboratory Network Program examined the distribution of CSRD funds to rural schools.8 The report concluded that:
• Rural high-poverty schools (where at least 50 percent of students receive free or reduced price lunch) are funded by CSRD at a rate that would be expected and is appropriate given the percentage of high-poverty rural schools nationwide.
• When comparing all schools (not only high-poverty schools), rural schools appear to be funded by CSRD at a slightly lower rate than would be expected given their numbers in the overall population of schools. Urban schools appear to be participating at a slightly higher rate than would be expected given their numbers in the overall population of schools.
• CSRD grants to urban schools tend to be larger than to rural schools. This may be accounted for by the fact that urban schools are more likely to have a larger number of students. Rural schools funded by CSRD are more likely to have fewer students than urban schools but more funding per student to implement reforms.
The Northwest Regional Education Lab examined the access to external technical assistance of the 20 rural CSRD schools in their region (Alaska, Idaho, Montana, Oregon and Washington). Of 11 rural CSRD schools in that region that responded, 10 reported that they had been able to contract with the model developer of their choice.9
While these analyses are limited and preliminary, both suggest that there seems to be a relative balance in CSRD funding and that rural schools and urban schools have had adequate access to CSRD incentive grants and access to model providers. However, more research is needed to determine the effectiveness of models in urban versus rural settings.
Summary
Overall, the distribution of CSRD funds has been well targeted and distributed to schools in accordance with the purposes of the legislation. A significant portion of the funding is serving high poverty schools and schools that have been identified as in need of improvement or as low- performing. While the situation may vary from state to state, nationwide, about half of CSRD funds are serving urban schools, and rural schools do not appear to be at a disadvantage for acquiring CSRD funds. From the basic demographic data the Department collected on each CSRD school, we can see that CSRD grantees are implementing a variety of models. About half are implementing one of the 17 models cited as examples in the CSRD legislation, and almost three-quarters of CSRD schools are implementing one of 25 models.
8 Dewees, Sarah. 2000. "Participation of Rural Schools in CSRD: What Do We Know?” Paper presented at the American Education Research Association meeting (April). 9 Zuckerman, David, Rex Hagans, Steve Nelson. 1999. Rural CSRD Schools' Access to Technical Assistance.
13
III. Federal Implementation of CSRD and Support for Research and Development on Comprehensive School Reform
This section of the report describes the CSRD implementation efforts on the federal level. Beginning in 1997, shortly after the passage of the legislation, the U.S. Department of Education began preparing for the implementation of CSRD. First, the Office of Elementary and Secondary Education prepared non-regulatory guidance and state applications for CSRD. This was followed by extensive peer reviews of state applications, discussions with states, and follow- up revisions and improvements on state plans.
With the rollout of CSRD the Department has established extensive communication links, with a Web site and frequent bulletins of information to all state CSRD coordinators through an e-mail listserv. The CSRD program has sponsored three summer institutes on schoolwide programs and comprehensive school reform and has hosted numerous regional and national meetings of state CSRD coordinators. The CSRD program office also has worked with the Regional Educational Laboratories network to develop, among other products, a guide for schools working with model providers to assist schools and districts in making the most of partnerships with external technical assistance providers.
The Office of Educational Research and Improvement (OERI) has also played a significant role in federal support for CSRD by providing funding to help develop school reform models that address the needs of middle and high school students, increase the capacity of model developers to serve the growing number of schools interested in research-based reform, and develop a clearinghouse of information for policy makers and practitioners on comprehensive school reform.
Technical assistance and capacity building have been the theme for other federal support for CSRD. Through the Regional Educational Laboratories, the Department has supported technical assistance to schools in the process of researching models, assessing needs, and preparing applications for CSRD. The Labs are involved in ongoing technical assistance and are helping coordinate CSRD evaluations across the states. The Department's Comprehensive Regional Assistance Centers have contributed to efforts to network schools involved in the comprehensive school reform process.
Finally, the Department has used CSRD as a pilot effort to change the way the agency evaluates programs and provides technical assistance for implementation. In addition to more traditional evaluation activities, since 1998, the CSRD program office has partnered with the Planning and Evaluation Service to conduct CSRD Report from the Field, with participation from other offices in the Department. Department staff has visited more than 30 schools in 10 states to gather information and early lessons on comprehensive school reform. This information has helped the program develop guidance, address issues and problems early in the implementation process, and has helped shape a research and evaluation agenda around CSRD.
14
CSRD Program Administration
Through communication with states, site visits, products, and efforts to coordinate with other comprehensive school reform support strategies promoted by the Department, the CSRD program office, in conjunction with a number of other partners, has played a very active role in guiding the early implementation of CSRD.
Implementation Guidance
In March 1998, the Department issued nonregulatory guidance on the CSRD program, which was distributed to all states along with state applications for CSRD funds. States and organizations were given an opportunity to comment on the guidance. The guidance, updated in fall 1999, includes descriptions of and information on: program purposes, the nine components of CSRD programs, the award system, use of local funds, technical assistance requirements, and evaluation requirements. As part of the guidance, the Department also developed a "continuum" of effectiveness to help states and districts think about the kinds of evidence needed to demonstrate that models are research-based and effective.10 This continuum is a helpful tool for schools and communities to use for evaluating the research evidence on school reform strategies. The purpose of the guidance has been to help states, districts, and schools understand what is involved in a fully comprehensive school reform effort.
In a 1998 survey to federal program administrators, state coordinators reported that Department guidance on CSRD was very helpful. Overall, 60 percent of CSRD state program coordinators said that written information such as guidance or mailings from the U.S. Department of Education was "very helpful." Another 38 percent of program coordinators described the guidance materials as "helpful."
-U.S. Department of Education, Follow Up Study of State Implementation (1998)
Peer Review of State Applications for Funding
An important feature of the funding process for CSRD was a peer review of each state’s application to participate in the program. In this process, which was conducted on a rolling basis beginning in the spring of 1998, the Department convened small panels of Copies of state applications for CSRD are available on the web at: knowledgeable educators and program administrators to review and comment on http://www.temple.edu/LSS/csr_rfp.htm each state’s submission. Reviewers included representatives of state and local education agencies, business and professional organizations, and federal representatives. Using review guidance that was previously circulated to all SEAs, panel members were asked to identify strengths of each state’s application, as well as areas for clarification, if any, that should
10 See Appendix E for continuum of evidence on effectiveness of models.
15
be addressed in order for the panel to recommend that the state receive its first-year program funding. In addition, an important function of the panel was to act as a critical friend and suggest improvements that could strengthen the state's plan for implementing CSRD. The process was designed to ensure that each application met legislative requirements, and had the greatest possible chance of accomplishing state goals and fulfilling the purposes of the program.
Following a review of each state’s application and a discussion among the reviewers, each panel conducted a conference call with state officials to share feedback in each of these four areas and to clarify any questions or concerns that had arisen from the review. In most cases, reviewers recommended that applications be funded with only minor clarifications or The CSRD Web site contains legislation, funding improvements. In some cases, more information, guidance (updated in October 1999), significant areas were identified that resources, contact information, and links to other states were asked to address prior to final relevant sites. The CSRD home page can be approval. Ultimately, applications from accessed at: all 50 states, the District of Columbia, www.ed.gov/programs/compreform/
Puerto Rico, and the Bureau of Indian Affairs were reviewed and approved for funding.
In addition to the assistance provided directly by each panel to the states under review, there were other benefits. Panel members from around the country reported that they benefited from learning about the approaches to comprehensive school reform being used in other states and local school districts, which reviewers were then able to use in constructing and strengthening their own reform initiatives. Further, CSRD program staff was able to share impressions of strengths and weaknesses common to state applications generally. These were compiled and shared with SEA staff across the United States.
CSRD Coordinator Conferences
Technical assistance has also been provided by the CSRD program office–in collaboration with the Regional Educational Laboratories, the Comprehensive Regional Assistance Centers, and the National Clearinghouse for Comprehensive School Reform–through a series of conferences for state CSRD coordinators. The purpose of these conferences has been to assist states in their efforts to support schools and districts undertaking comprehensive reform. Summer Institutes were held in July 1998, 1999 and 2000 to discuss comprehensive school reform, schoolwide programs, and turning around low-performing schools.
In addition, sessions are held at the Improving America's Schools conferences each fall for state coordinators, as well as a day-long session for district and school grantees. In February 2000 nearly all of the State CSRD coordinators met in Washington, D.C. to discuss lessons learned from the first two years of the program and ways to strengthen the program as states planned for a new round of grant competitions. Many states incorporated ideas shared at that meeting into the plans they submitted for use of FY 2000 funds. CSRD program staff have also participated in and presented at many state-level technical assistance sessions and national conferences. Electronic Communication
16
The CSRD program office has developed several mechanisms for providing information on the program. Two listservs are used to send information directly to interested parties. One listserv supplies frequent updates to State CSRD coordinators and others with official responsibility for managing CSRD programs, as well as the Regional Educational Laboratories and Comprehensive Regional Assistance Centers. Topics such as new resources, program guidance and best practices are covered. The other listserv provides periodic news related to CSRD to partner organizations with an interest in comprehensive school reform, including professional associations and model developers. The program office also maintains an extensive Web site with program information and links to resources and partners.
Early in the program, state CSRD coordinators were asked how helpful they found program conferences, workshops, on-line services and telephone contacts. Overall, 49 percent of state CSRD program coordinators reported that such contacts were "very helpful" and 38 percent reported that these contacts were "helpful."
-U.S. Department of Education, Follow Up Study of State Implementation (1998)
Products to Support Comprehensive School Reform
The CSRD program office has also worked with various partners to develop useful products to support comprehensive reform efforts. For example, two recent products developed with support from the CSRD program are Comprehensive School Reform: Research-Based Strategies to Achieve High Standards and Guide to Working with Model Providers.
Research-Based Strategies is Products to Support Comprehensive School Reform a tool kit that was developed for the CSRD program office A Web-based version of the Comprehensive School Reform by the Region XI Northern guide is available at: California Comprehensive http://www.wested.org/csrd/guidebook Regional Assistance Center. The tool kit, which consists of The guide to working with model developers is available at: a guidebook and two videos, offers a framework that www.ed.gov/programs/compreform/ schools can use to plan their own school-wide improvement efforts, as well as profiles of schools undertaking comprehensive reform.
The Guide to Working with Model Providers, which was prepared to address needs identified through site visits to early-implementing CSRD schools and then published by the Regional Educational Laboratories, offers advice to schools and districts preparing to work with an external model provider. The guide follows the process through initial stages, contract negotiations, and creating an ongoing partnership. Both of these products have been widely disseminated. Throughout the text and in Appendix H are featured additional products developed by Regional Educational Laboratories and others to support comprehensive school reform. These resources
17
may be helpful to schools, districts, and communities as they begin to explore and implement comprehensive school reform.
Technical Assistance and Research Activities
One of the key components of CSRD that makes it a unique program is the prominence of external technical assistance providers. Because the role of external partners is so critical to CSRD, Congress has helped to fund efforts to raise the capacity of external providers to serve more schools, improve their service delivery, and develop new and innovative models for comprehensive school reform.
Regional Educational Laboratory Technical Assistance and Research
In FY 1998 and 1999, approximately thirteen million dollars was awarded to the Regional Educational Laboratories (see www.ed.gov/programs/compreform/labs.html) to provide assistance to states and districts supporting CSRD. The Labs have assisted states as they designed their CSRD competitions and are directing their efforts toward understanding of comprehensive school reform among staff in districts and schools. The following are some examples of Lab activities in support of CSRD:
• In 1998 Regional Educational Laboratories hosted more than 25 major school reform model showcases and 100 technical assistance workshops attended by 6,000 people from more than 1,000 schools and districts.
• The Southwest Educational Development Laboratory (SEDL) maintains the CSRD school database for the Department. In 1999 the Web site received 15,248 "hits." The database can be accessed on the Web at www.sedl.org/csrd/awards.html.
• The Northwest Regional Education Laboratory (NWREL) developed The Catalog of School Reform Models, which provides information on 64 models, including 33 entire-school reform models and 31 skill- and content-based models (reading, math, science, and other areas). Criteria for selecting models included evidence of effectiveness in improving student academic achievement, extent of replication, implementation assistance provided to schools, and comprehensiveness. This guide has been widely distributed to districts and schools across the nation and can be obtained from the Web at www.nwrel.org/scpd/natspec/catalog.
• The North Central Regional Education Laboratory's (NCREL) Making Good Choices was developed to help schools conduct thorough needs assessments and make good decisions about models with a three-step strategy for making choices about comprehensive school reform. This guide can be obtained from the Web at www.ncrel.org.
• WestEd has designed, in a cross-lab effort, a Web site at www.csrdweb.net that contains, to date, 75 profiles of CSRD schools in 26 states. The purpose of this Web site is to help network and link schools across the nation as an interactive learning community. • Pacific Resources for Education and Learning (PREL) has adapted CSRD materials and workshops for groups for whom English is not their primary language.
18
• AEL (formerly Appalachian Educational Laboratory) has created an academy for CSRD facilitators serving 60 state and district CSRD facilitators/coordinators each year. AEL also hosts an annual meeting to network researchers engaged in comprehensive school reform research and evaluation.
• The Laboratory for Student Success (LSS) has posted approved state applications for CSRD on the Web so that they could be shared among the states in one convenient location. The lab is also conducting a regional evaluation of CSRD in the five states in its region.
The Regional Educational Laboratories are also undertaking significant research and evaluation activities that will increase our knowledge about comprehensive school reform in general as well as the implementation of CSRD specifically. The common themes in the research efforts include rural sites, homegrown models vs. externally developed models, leadership, school readiness, and infrastructure and support for reform. For example:
• The LAB at Brown (LAB) is conducting the CSRD state evaluations in Maine, Rhode Island and Vermont and, in coordination with RMC Research, conducting New Hampshire's evaluation of CSRD so that these four states have a common framework through which to assess implementation and outcomes for CSRD. Initial data are expected in 2000.
• The Mid-Continent Regional Education Laboratory (McREL) is leading a cross-lab effort to align and synthesize findings from studies conducted across the laboratories. In addition to analyzing the implementation of the program in the central U.S. region, the lab is undertaking two focused studies. One study will look at the relative success of literacy- oriented vs. nonliterary focused comprehensive school reform models. The second study, conducted in cooperation with WestEd and SEDL, will examine comprehensive school reform in schools serving Native American communities. McREL has distributed more than 5,000 copies of Evaluating for Success, a guide designed by the Lab to help schools and districts evaluate CSRD.
• The Regional Educational Laboratory at SERVE is conducting a cross-state research study of approximately 40 CSRD schools in the southeast region (Alabama, Florida, Georgia, Mississippi, North Carolina, and South Carolina). The instruments SERVE will use for this study–including a school climate inventory; interviews; surveys of teachers, parents, and principals; and classroom observations–were developed by AEL and will also be administered as part of state evaluations in Tennessee and Kentucky.
• AEL (formerly Appalachian Educational Laboratory) has developed the Formative Evaluation Package for School Improvement (FEPSI) in a joint effort with the University of Memphis. The evaluation design is being used by six states for state evaluations of CSRD. In addition, AEL is using the design to study a set of 32 CSRD schools from states in its region. The study includes an examination of student outcomes using an independently administered assessment. The first data from this study are expected in 2000. • The Northwest Regional Educational Laboratory (NWREL) is conducting case studies of CSRD implementation in rural Oregon and Washington schools.
19
• The North Central Regional Educational Laboratory (NCREL) has completed a preliminary study of state implementation of CSRD (cited below in section on state implementation) and is conducting a comparative study of high-performing, high-poverty schools and CSRD schools in Wisconsin.
• The Southwest Regional Educational Laboratory (SEDL) is conducting three implementation surveys in CSRD schools in their region over a two-year period (through fall 2000) to chart the progress of schools as they implement CSRD. The study will include five case studies (one in each of the region's states–Arkansas, Louisiana, New Mexico, Oklahoma, and Texas). SEDL is also conducting case studies of rural CSRD sites and examining state CSRD competitions.
Comprehensive Regional Assistance Center Activities
Through an arrangement with the Office of Educational Research and Improvement (OERI), the fifteen Comprehensive Regional Assistance Centers received funding to support a variety of CSRD-related activities in Fiscal Years 1998 and 1999. In addition to ongoing work in support of schoolwide improvement, each center received a supplement averaging $50,000 (according to the general formula for distributing funds to the Centers).
The Centers’ activities supported through the CSRD supplement include: (1) holding networking sessions to pass on promising practices and to share lessons learned in the comprehensive school reform process; (2) providing direct assistance to CSRD school sites; (3) helping promote the use of data to initiate and carry through comprehensive improvement activity; (4) producing useful publications to enhance the reform process; and (5) helping schools devise strategies to support special needs populations, especially disabled students and English language learners. For example:
• The Region I Comprehensive Center in New England established a principals’ information and leadership network.
• Region II Comprehensive Center in New York developed and maintains a CSRD Web site and developed a tracking system that documents services and contacts of all individuals across the organizations that provide services to CSRD schools. At the request of New York City Board of Education’s Office of Funded Programs, the center provided technical assistance to 25 schools who were not successful in the competition for CSRD funds. Further, the Center convened a panel to consider parent involvement in leading reform models. The Center also created a "think tank" to assist the New York State Education Department, local school districts, and schools to continuously apply the best knowledge of quality practices in teaching and learning in order to ensure that all students achieve challenging content and performance standards.
• The Region III Comprehensive Center serving the mid-Atlantic states compiled a directory of existing and emerging resources on formative evaluation techniques of CSRD schools. The directory is aimed primarily at state, district, and local educational practitioners
20
responsible for tracking comprehensive school reform implementation. The Region III Center is also sponsoring a review of the parent involvement features of leading reform models.
• CSRD ROADMAP is an effort of the Region VI Comprehensive Center to create a network of CSRD schools in Iowa, Michigan, Minnesota, North Dakota, South Dakota, and Wisconsin. The site is located at:www.wcer.wisc.edu/ccvi/csrdroadmap/.
• The Region VII Center has provided peer review training. Region VIII Comprehensive Center in Texas developed and administered a survey of the effectiveness of models used with special needs populations, especially English language learners. The Center produced a guide on models that have been used to serve English language learners.
• Regions VIII and XIII Centers have provided one-on-one implementation consultation to local CSRD schools in southern California, and Alaska.
• Regions XI and XII Comprehensive Centers have partnered with state educational agencies, regional labs, and state technical assistance centers to provide intensive training to district personnel about comprehensive reform.
• The Region XIV Comprehensive Center, serving Florida, Puerto Rico, and the Virgin Islands, developed a model consumer's guide to allow schools to make better choices about models at the school level.
National Clearinghouse on Comprehensive School Reform
The creation of the CSRD program has highlighted the need for sources of information for schools and policy makers on research-based practices. The Office of Educational Research and Improvement (OERI) has awarded George Washington University, in collaboration National Clearinghouse for with the Institute for Educational Comprehensive School Reform Leadership and the Council for Basic Education, a contract to create a new The National Clearinghouse for National Clearinghouse for Comprehensive Comprehensive School Reform has two School Reform (NCCSR) to provide regular publications -- Benchmark and educators and the public with the most Bookmark – one electronic and the other recent research and evaluation of reform printed. For more information and to subscribe to these Clearinghouse designs, models, and strategies. In addition, publications, go to: the Clearinghouse will assist customers in promoting the use and application of http://www.goodschools.gwu.edu/ research and development to upgrade the capacity of schools so they can prepare all students to achieve to high standards. The Clearinghouse, which began operating during fall 1999, was created to improve the national dissemination of information on comprehensive school reform. The Clearinghouse collects, analyzes, and disseminates information on comprehensive school reform in education institutions serving the K-12 population, including public schools, public school districts, state education
21
agencies, and private schools in the United States. Among other services, the Clearinghouse Web site includes a feature on "Comprehensive School Reform Step-by-Step" that helps viewers learn about the process of implementing reform with links to tools for each step along the way.
Middle and High School Model Development
A great number of the reform models developed and available to schools today are focused on elementary school reform. In order to build a larger body of knowledge around effective practices for middle and high school reform, in FY 1999, OERI awarded $12.7 million in funds to support the design and implementation of new research-based models focused on these grade spans. These awards included research and evaluation requirements for developers to assess the models' promise for raising student achievement for middle and high school students.
The following model developers were awarded contracts to develop, implement, and evaluate school reform models that addressed the needs of middle and high school students:
• Education Development Center, Inc. (EDC) will develop, implement, and replicate a comprehensive school reform design for middle schools that brings together the best features of the elementary school model ATLAS Communities and EDC's decade-long experience with middle-grades reform.
• The Galef Institute will build a middle school reform program drawing on its extensive experience with Different Ways of Knowing (K-6) in more than 500 schools. The program will use drama, dance, music, art, and media as tools to promote in-depth, creative thinking.
• The Talent Development Model builds on recent work conducted at Johns Hopkins University's Center for Research on the Education of Students Placed at Risk and at the University of California-Berkeley's Career Academy Support Network. The Talent Development Model is designed to organize middle and high schools into small learning communities. It emphasizes reform of curriculum and instruction and stresses reform of parent and community involvement through the use of multiple partnerships.
• First-Things-First is a model for district and whole school reform. It is currently being phased into all comprehensive high schools and middle schools in Kansas City, Kansas. The model includes a research-based set of seven critical features aimed at building strong, mutually accountable relationships among adults and adolescent students; an initial focus on literacy; and realigning organizational supports to achieve these two goals.
• The National Center on Education and the Economy, in association with the Consortium for Policy Research in Education, aims to strengthen the middle school and high school components of the America's Choice Design. The Design will tie together a curriculum that extends between middle and high school. The Design calls for an extensive system of safety nets intended to make sure that all students reach the standards, no matter where they start. These include intensive courses for entering students, an after-hours tutoring program, and a dropout recovery program. All students will participate in “houses” of 200 to 400 students, and core teachers will follow their students through the program.
22
• The Southern Regional Education Board will design, develop, implement, and refine a comprehensive middle and high school model by bringing together its High Schools That Work and emerging middle-grades efforts into an integrated whole-school improvement initiative for clusters of rural schools.
• The Success for All Foundation will develop, evaluate, and disseminate a middle school program. This program will be designed to accommodate the developmental needs and extraordinary capabilities of young adolescents, especially those who are placed at risk due to poverty, limited English proficiency, minority status, or location in inner cities or isolated rural areas. The Success for All Middle School will be organized to create close connections between young adolescents and their teachers to help students succeed in rigorous coursework—both by improving the quality of curriculum and instruction and by providing a variety of support services.
Model Developer Capacity Grants
With approximately 1,800 schools currently administering CSRD grants and new schools receiving grants as part of the FY 2000 increase in CSRD funds, OERI is providing $8.7 million in funds to model providers to raise their capacity as demand rises for external technical assistance. The funds will help developers scale up, hire technical assistance providers, improve training and strengthen quality. The capacity-building grants have been awarded as follows:
• The Success for All Foundation will support approximately 250 additional schools and hire trainers and pursue strategies that will update and improve materials and provide additional technology-based technical assistance to Success for All schools.
• Co-NECT Schools, Inc. will serve approximately 100 new schools with this funding. New site directors will be hired to conduct training sessions and workshops for school faculty and staff. Funding will also be used to train local facilitators, school employees who serve as day- to-day consultants to teachers, staff, and students on implementing the Co-NECT model.
• Temple University/Center for Research in Human Development and Education will serve an additional 27 schools. Implementation specialists will provide organizational and professional support at the school and classroom levels. These personnel assist in the development of a comprehensive needs assessment and in implementation planning.
• Expeditionary Learning Outward Bound is a design for comprehensive school improvement that challenges students to meet rigorous academic and character standards. The funding will help serve an additional 35 schools. Field directors will be recruited from among teachers and principals in current Expeditionary Learning schools. These Field directors will ensure the quality of implementation and service delivery through activities that include arranging summer institutes and other professional development opportunities.
• Different Ways of Knowing will serve approximately 155 schools. The developers will hold a National Coaches Institute for developing 40 expert coaches skilled in coaching for
23
breakthroughs in teaching and learning. Refined partnership-building and introductory materials will be developed to help new and veteran coaches guide new teachers and administrators through introductory activities, follow-up debriefing discussions, and collegial planning within the first few months of work.
• Northwest Regional Educational Laboratory's (NWREL) Onward To Excellence (OTE) will serve 62 schools. NWREL will use this funding to establish five Regional Service Centers. New OTE trainers will be selected for these regions and NWREL will hold OTE Trainer Development Institutes for all new trainers. NWREL will refine training materials for the school leadership teams, school staff and the external study team and will also refine selected supplemental resource materials.
• Education Development Center, Inc. (EDC) will serve approximately 30 additional schools with the ATLAS model. Funding will help provide training to new trainers and site developers. EDC also will expand its technology support system through the use of online workshops and video conferencing.
• The Talent Development Middle School Program will serve an additional 28 schools with these funds. Three additional instructional facilitators will be hired, one each for mathematics, science and U.S. history. Teachers will be provided with the materials and lessons they need to incorporate the use of primary sources (documents, photographs, diaries, etc.) in their instruction. A Web-based learning community will be developed allowing for web-based professional development.
• The Center for Social Organization of Schools will serve approximately 30 schools with its career academies approach. Organizational facilitators will be hired to assist schools in instituting Ninth Grade Success Academies and Career Academies. Instructional facilitators will be trained in a national conference in order to work with English teachers in implementing strategic reading, student team literature, and student team writing.
National Evaluation of CSRD
The national evaluation of CSRD is designed to gather baseline information on early implementation and decision making, while setting a foundation for longitudinal work and field studies. The evaluation plan includes efforts to assess the implementation of CSRD at the state, district, and school level; evaluate the effectiveness of technical assistance provided to schools by external partners; add to the research base on effective models; and track student achievement outcomes for students in CSRD schools. Activities include:
Collecting Baseline Information on All CSRD Schools
Through an arrangement with the Southwest Educational Development Laboratory (SEDL), the Department is collecting baseline information from states on all CSRD grantees as awards are made. This database is providing early information on the implementation of CSRD, including
24
the extent to which states and districts are targeting CSRD funds to high-poverty and/or low- performing schools, the distribution of CSRD funds to schools of different characteristics, and the reform models that schools are choosing. In addition to providing this baseline information, the database is being used as the sampling frame for drawing the nationally representative sample of CSRD schools for the large-scale longitudinal survey as well as other national and regional evaluation efforts. See pages 10-12.
Large-Scale Longitudinal Data Collection on Implementation in CSRD schools
One of the primary components of the national evaluation of CSRD is a large-scale nationally representative survey of CSRD schools and teachers, focusing on the implementation and impact of standards-based school reform. The survey uses the CSRD database as a sampling frame and the Department’s National Longitudinal Survey of Schools (NLSS) as the survey vehicle. The NLSS emphasizes the extent to which provisions in Title I designed to support school improvement, including the use of research-based models, have been implemented and how such efforts are contributing to improved teaching and learning.
Approximately 900 CSRD schools are being sampled for the NLSS, about 300 schools that received awards before February 1999 and 600 schools that received awards after February 1999 (and reported to SEDL by November 1999). The NLSS is simultaneously being administered to a nationally representative sample of Title I schools to track the progress of standards-based reform. The sample of 1,000 Title I schools provides appropriate comparison groups for the Title I schools participating in CSRD. For findings see section on school implementation of CSRD beginning on page 42.
Longitudinal Achievement Data Collection
As part of the National Longitudinal Survey of Schools (NLSS) and as part of the program office’s state CSRD performance report, state assessment data will be collected for all CSRD schools beginning in December 2000. This will allow the Department to track student achievement over time against previous performance and relative to schools that are comparable on the basis of school conditions, student demographics, and reform programs. The data collection will continue annually; analysis will particularly focus on states that can provide trend data for the period prior to and throughout the implementation of CSRD. In depth analysis of student achievement data will be conducted by RAND on state assessment results for CSRD schools from a select number of states with high-quality, consistent data and, where available, on student-level, longitudinally linked data.
Field Focused Case Studies of CSRD
The aim of the CSRD field-focused studies is to examine comprehensive school reform with particular attention to the quality of the implementation of comprehensive reform activities in CSRD schools. The field focused case studies will look in greater depth than the large scale sample survey allows at the process of program implementation, the context of reform, and how comprehensive reform efforts are linked to improvements in the quality of classroom instruction. The studies will take place at approximately 20 CSRD sites. Data collection begins in fall 2000 with the first report in summer 2001.
25
Monitoring School Reform Research
There are many researchers engaged in studies of comprehensive school reform, some CSRD funded and many others sponsored by foundations or other funding sources. Planned CSRD Evaluation Study: Much of this research addresses basic Longitudinal Evaluation of the questions of interest to the Department Effectiveness of School Interventions about school reform, research-based models, and their implementation and The purpose of this study will be to examine the impacts. effects of instructional interventions in high- poverty elementary schools. The study will The purpose of this aspect of the examine schoolwide reforms focused on instructional improvement, such as those evaluation is to identify the most models and programs supported by Title I and promising and rigorous research underway CSRD. The three-year longitudinal study will across the nation on comprehensive focus on schoolwide programs and CSRD reform, to monitor the progress of these schools and will collect data on comprehensive studies, and to annually convene those planning, teacher quality, professional involved to share their research plans and development, technical assistance, curriculum, results. A key focus of this effort will be instruction, parent involvement, and the to track promising third-party, rigorous, effectiveness of models and interventions on outcome-focused evaluations of models. student performance. The evaluation will This will help to identify common themes include an independent assessment of student and gaps in the broad research agenda and performance, tests of teacher knowledge, and in-depth classroom observations. The award for stimulate collaboration and initiation of this study is expected by September 2000. Data new research studies. These studies can collection will begin in the 2001-02 school also help inform policymakers and year. practitioners about what works. For initial tracking efforts, see Section VI of this report.
CSRD Reports from the Field
In the fall of 1998, the U.S. Department of Education began piloting an initiative to gain early information on the new CSRD program, by conducting visits to schools and districts in the first stages of implementing school reform. This effort represents a new approach to early program implementation in the Department–collecting early information, offering technical assistance, and sharing information with other grantees–that is serving as a model for other federal program offices. The purposes of the project include:
• Timely program information–Because CSRD is being implemented in stages–some states made grants to districts in time to begin implementation early in the 1998-99 school year, while most others made grants later, and with funding increases for new awards in 2000– what is learned from the first implementers can help those implementing later to anticipate challenges and avoid problems.
26
• Helping the Department become a better partner–CSRD Reports from the Field has provided opportunities for the Department to offer early technical assistance to schools and districts, particularly in the use of federal education funds to support comprehensive reform efforts, and to help districts and schools become aware of the resources available to them.
• Guiding policy decisions–Often the Department gets only limited information on implementation during the early stages of a new program. These field visits are helping to inform decisions about necessary adjustments in policy and providing additional support to improve program operations and the chances for success.
• Informing reauthorization–As the Department works with Congress to move forward on the reauthorization of the Elementary and Secondary Act, CSRD Reports from the Field is providing information about the implementation of federal programs and improvements that can be made in the law to better support comprehensive school reform efforts.
CSRD Reports from the Field Highlights
Highlights from the report are included in Sections IV and V.
To date 35 schools in 26 local school districts in 10 states have been visited. The schools visited include all grade spans and are implementing a variety of models, some locally designed by the districts, local universities or schools and others that are externally developed and nationally available such as Accelerated Schools, ATLAS, America's Choice, Roots and Wings, Marva Collins, Literacy Learning, Coalition of Essential Schools, Direct Instruction, and Modern Red Schoolhouse.
Site visit and response teams from across the Department, including the Office of Elementary and Secondary Education, the Office of Educational Research and Improvement, and the Planning and Evaluation Service, participated in the visits.
Overall, the Department's visits revealed great enthusiasm and hope for the CSRD program. The districts and schools recognized the need for and were committed to change and improvement. Visitors observed significant variety in the role districts are playing in helping schools choose models and supporting comprehensive reform in CSRD schools. The visits also revealed some of the common challenges districts are facing as they implement comprehensive reform strategies. Lessons learned on these visits are shared throughout this report in Sections IV and V on state, district, and school implementation of CSRD.
Summary
The federal implementation of CSRD has proceeded on many fronts and marks an unprecedented degree of collaboration among multiple program offices and Department-supported technical assistance providers. In addition to application review, guidance, and communication efforts by the CSRD program office, CSRD implementation has been supported by the Regional
27
Educational Laboratories, Comprehensive Regional Assistance Centers, and the Office of Educational Research and Improvement. The pilot initiative of CSRD Reports from the Field has allowed the Department to collect rich, detailed, and timely data on program implementation.
28
IV. State and District Implementation of CSRD
We now turn to the states, which have major responsibility for administering this competitive grant program, to examine their early CSRD implementation efforts. While comprehensive reform ultimately involves fundamental changes on the school level, research shows that both state and district support for school reform are essential. School change requires states and districts to provide leadership, resources, and support. Experience shows that essential ingredients for success include district support in helping schools to choose reform models that best fit their needs; arrange budgets and professional development activities to support comprehensive reform; and engage the community in the process of change.
“The response to this demonstration program has been enthusiastic from the local and state levels." -Brooke Fitchett Consultant, Colorado Department of Education
One of the early observations to be made about CSRD has been the innovative and interesting ways that states are administering the program through award competitions, technical assistance in the application process, and how states are using CSRD to shape the way that other state and federal program funds, particularly Title I funds, are used for school reform efforts. At the state level, CSRD seems to be helping to catalyze changes in how many states support school improvement efforts.
This section of the report draws on Early State Profiles several sources of information to describe CSRD implementation efforts The CSRD program office Web site features on the state and district levels. These selected profiles of early CSRD implementation data sources include New American efforts in Delaware, Illinois, Kentucky, Schools, the North Central Regional Maryland, Ohio and Wisconsin. See: Educational Laboratory, and the / Department's CSRD Reports from the www.ed.gov/programs/compreform profiles.html Field effort.
Findings from New American Schools CSRD Implementation Project
New American Schools (NAS) is a nonprofit organization dedicated to helping schools implement design-based comprehensive school reforms. As part of the organization’s efforts to track design-based reform through the CSRD program, NAS began the CSRD Implementation Project. The project shares implementation information among CSRD stakeholders, including state education agencies, districts, schools, funders, reform-model providers, and others interested in improving schools and districts. The project posts state-by-state CSRD profiles and comprehensive school reform resources on the Web at www.naschools.org/csrd. 11
11 Matthew Hornbeck, the Director of the NAS CSRD Implementation Project, prepared the information included in this section of the report. New American Schools will publish the full report.
29
In 1998, Congress charged the states with competitively distributing CSRD grants to local school districts applying on behalf of schools. This section presents an analysis of self-reported state- level data on 43 states.12 Taken together, responses from states show the effort that went into “rolling out” CSRD on the state level. The protocol used to interview and survey state staff was developed by New American Schools and NAS Design Teams, the Council of Chief State School Officers, the Education Commission of the States, and selected state staff. Additional data on state implementation was gathered from the U.S. Department of Education and the federally funded Regional Educational Laboratories.
State Applications and Grant Competitions
States applied to the U.S. Department of Education for CSRD funding beginning in the late spring of 1998. The application process involved extensive review and sometimes required states to improve their proposals for implementing CSRD. Although the “The current 11 CSRD schools are making great Department anticipated a quick rollout of strides and serving as important role models for subgrant competitions by the states, Maine’s secondary education reform initiative many states took much of the 1998-99 Promising Futures: A Call to Improve Learning school year to make their CSRD awards. for Maine’s Secondary Students." Several states ran their competitions early and made CSRD award -Susan Johnson, CSRD Program Coordinator announcements on the same day they Maine Department of Education received federal approval of the state application. Most states took about three months to announce at least a first round of awards.
• By June 30, 1999, the U. S. Department of Education had approved all 50 state applications for CSRD funds (plus Puerto Rico and the District of Columbia).
• The longest period of time between receipt of federal funds and the state announcement of a first round of CSRD awards occurred in Oregon, where it took nearly nine months. Oregon staff spent this time working intensively, on site, with a relatively small group of finalists to adequately prepare school staff for comprehensive school reform. All finalists received CSRD awards.
State CSRD Review Process
States used application-scoring rubrics, previously peer reviewed by experts for the U.S. Department of Education, to score and rank CSRD applications. Among other criteria, the competitive process implemented by states included targeting CSRD resources to particular schools and building a team of expert reviewers. For example:
12 At the time of this publication, the remaining seven states, Puerto Rico and the District of Columbia had not affirmed the accuracy of the data gathered. The remaining data sets will be included in an expanded report on state implementation of the CSRD program to be produced by the New American Schools CSRD Implementation Project in late summer 2000.
30
• All reporting states had expert review teams of varied composition to make CSRD awards. Team members were most often representatives from higher education, school board members, principals, consultants, state school support staff, and teachers. Forty-three states reported assembling a total of about 860 individuals to review CSRD applications.
• Most states provided training for team members specific to the CSRD application process. CSRD applications were each read or reviewed an average of 3.2 times by review team members.
• Several states, including West Virginia and Oklahoma, made it an absolute priority to fund low-performing schools, as identified by the state. Oregon limited its pre-application competition to the 129 lowest-performing schools in the state. Oklahoma limited CSRD eligibility to the 95 schools in the state identified as in need of improvement.
• Maine received a waiver from the U.S. Department of Education to limit its CSRD competition. Maine decided to focus its CSRD funds on reforming the state’s high schools through the state's "Promising Futures" initiative.
Many states viewed the CSRD application process, in itself, as a rigorous professional development activity that provided intensive assistance to school administrators and teachers as they considered the needs of their students in choosing reform models that matched their needs. The CSRD implementation process has been a means for states to improve their own capacity to provide assistance to low-performing schools.
• The 43 states that responded to the New American Schools' project logged 3,685 CSRD applications from 1,313 local districts. Seven states screened or accepted “pre-applications” from schools.
• To check on the willingness of school staff to participate in comprehensive school reform, 17 of 43 states conducted site visits to schools applying for CSRD funds. And 34 of 43 states required each applicant to demonstrate how the proposed reform program would help students meet state standards.
• States varied on the assistance offered to unsuccessful applicants. For example, Kansas contacted unsuccessful applicants and offered to help them reallocate resources to implement comprehensive school reform. Maryland provided written comments on the strengths and weaknesses of applications, identified where to go for technical assistance, and encouraged schools to reapply in later rounds.
CSRD-Like Grants and Other State Support Activities
The CSRD program is providing states, districts, and schools with a common framework for coordinating comprehensive school reform and for organizing their use of Title I school improvement funds. A number of states have leveraged additional resources to increase the number of schools implementing comprehensive school improvements and complement the CSRD program, as described below:
31
• To support CSRD, states are leveraging other funds to support comprehensive school reform. Four out of 43 states reported leveraging almost a million dollars in “carryover” Title I resources; 6 states looked to Goals 2000 resources, leveraging a total of more than $8 million; and 6 states identified other federal, state, and private resources totaling $86 million to support comprehensive school reform.
• Beginning with the 1999-2000 school year, the Maryland School Accountability "CSRD has served as a model for an Funding for Excellence (SAFE) Program intensive, in-depth school improvement provided about $60 million ($12 million planning process. Oregon is electing to per year for five years) in new state use this same model to strengthen the Title I schoolwide program planning process money to support effective programs for throughout the state, and to provide a low-performing students. In Maryland, vehicle for change in schools that are in 180 schools currently identified as low Title I school improvement status." performing by the state, and a number of schools locally identified as in need of -Chris Rhines, improvement, will have adopted a Oregon State Department of Education comprehensive school reform model by 2003.
• Through its Immediate Intervention/Underperforming Schools Initiative, California is adding $82 million over five years to its CSRD funding to reach many more schools. In September 1999, California awarded $16.5 million in state money for comprehensive school reform planning grants ($50,000 per site) to develop school action plans with the purpose of improving student achievement. In 2000, additional state-funded implementation grants of $50,000 or up to $200 per pupil will be awarded to schools with approved action plans.
“Montana is not the sort of place that usually comes to mind in connection with schoolwide restructuring. It has a lot of rural, one-school districts, a lot of places where there are more members on the school board than students. The state has low- performing schools most of them on or near Indian reservations. Many of these schools face not only the usual problems associated with poverty but also those associated with isolation. Bringing members of the community in to see what their school is doing had tremendous positive impact. It’s developed school-based leadership and made people in the community feel they have a stake in the plan. Schools have given teachers more planning time and forged new relationships with tribal colleges, other higher education institutions and the state education agency. Within the state agency, there is more collaboration among program offices, and there is a greater understanding of school programs at the state level as a result of CSRD.”
-Ron Lukenbill, Title I Specialist Montana Department of Education
32
• Colorado leveraged $3 million in Goals 2000 resources to support CSRD in two ways. First, Goals money was used to offset administration and technical assistance costs associated with CSRD implementation. Secondly, Colorado established a program called Advancing Student Academic Achievement that awards grants to schools most in need of improvement. Eligible schools apply for funds (on average $50,000) to "jumpstart" reform in their schools.
• Nevada’s legislature appropriated $9.6 million (over three years) for low-performing schools to adopt research-based programs. Nevada also published a catalog of programs the state has identified as effective.
• Wisconsin used Goals 2000 and Improving America's Schools Act administrative funds to partially support showcases where model providers were brought together with school and district personnel to provide information on research-based models.
• Altogether, Hawaii, North Carolina, and West Virginia integrated more than $4.8 million in Goals 2000 resources with the CSRD program. In the case of North Carolina, these Goals resources doubled the number of grants made to schools to implement comprehensive school reform.
• Hawaii used $200,000 in Title I carryover resources to make additional grants to schools.
• A number of states, including Oregon, Tennessee, and West Virginia, are using CSRD as a framework for how they use federal School Improvement Fund resources. States are making CSRD-like grants and supporting comprehensive planning and needs assessments in low- performing schools. “In the past two years, the CSRD State Efforts to Help Schools Make Informed program has helped eighty-seven schools Choices in thirty-nine Ohio school districts to improve the quality of their educational States have worked hard to disseminate programming. This important resource information on the CSRD program by sharing has not only enabled school buildings to research information on various models, implement professional practices to hosting grant-writing seminars and sponsoring address individual building needs, but model "fairs" where school and district leaders also strengthened the connection between can meet with model developers. Below is single buildings and districts in an effort to maximize the impact of their reform more specific information on these efforts: efforts. We hope to use future CSRD funds to strengthen the foundation we • Twenty-four of 43 states made it easier for have built, and better serve even larger schools to access information by posting numbers of students and schools.” CSRD information on the state Web site. Seventeen of the reporting states made -Frank Schiraldi, Associate Director, CSRD school applications available online. Comprehensive School Improvement Ohio Department of Education • Thirty-nine of 43 states sponsored CSRD workshops for schools to “walk them through” the application process. Thirty states reported cosponsoring CSRD workshops with their regional educational laboratory.
33
• Nearly all profiled states disseminated CSRD information to every school district in the state and held CSRD grant-writing workshops.
• Almost every state provided CSRD Web site addresses to prospective applicants. Most frequently cited were the U.S. Department of Education’s CSRD program Web site and the Northwest Regional Educational Laboratory's Catalog of School Reform Models.
• Thirty-one of 43 states sponsored a “showcase” of models or model "fairs" to enable school staff to meet with providers. States wanted to help schools become savvy consumers before choosing reform models to meets their needs. A number of states, including Wisconsin, decided that for future competitions, the showcase would not be held until school staff had conducted a thorough needs assessment.
• Written materials on models were made available to schools. Most widely distributed was The Catalog of School Reform Models developed by the Northwest Regional Educational Laboratory (NWREL); 36 of 43 states distributed it. Sixteen of 43 states distributed the list of 17 models provided as examples in the CSRD authorizing legislation (all 17 are included in the NWREL Catalog). Thirteen of 43 states distributed An Educator’s Guide to Schoolwide Reform, by the American Institutes for Research (AIR). Sixteen of 43 states distributed other catalogs or provider-specific information; and four of 43 states reported that they produced their own catalog or list of reform providers.
There is much that can be learned from the initial state implementation of this demonstration program. Substantial progress has been made in moving design-based and research-based practices into the nation’s schools. States will make thousands of new CSRD awards in the coming school years. Based on the New American Schools Implementation Project findings to date, there is evidence that:
• States are leveraging substantial state dollars to fund comprehensive school reform in ways that complement CSRD. CSRD is influencing how states think about their efforts to support school improvement.
• State application scoring rubrics are an important part of the competitive process. Many states are rethinking scoring rubrics and considering weighting the rubrics to "Some SEAs have indicated that the CSRD emphasize the research evidence and program has had a significant impact on how comprehensiveness of reform models. they think about the nature of statewide competitions and the processes that are necessary to provide technical assistance, Early Lessons from NCREL on monitor program implementation, and evaluate State Implementation of CSRD program outcomes."
Other organizations are also examining the -Lawrence Friedman and Matthew Hanson state implementation of CSRD. For Launching Comprehensive School Reform: example, in 1998, the North Central Early Lessons for State and Federal Regional Education Laboratory (NCREL) Policymakers began investigating how CSRD was being implemented in the region (Illinois, Indiana, Michigan, Minnesota, Ohio, and Wisconsin) at the
34
state level. The following are some of NCREL's findings based on interviews with state administrators:13
• CSRD has complemented existing state and federal initiatives.
• CSRD is being launched in states without major disruptions to existing structures and processes for providing technical assistance to schools.
• CSRD has had a significant impact on how state administrators think about statewide competitions and the processes they use to provide technical assistance, monitor implementation, and assess program outcomes.
The NCREL report included recommendations for state policy makers, which included a suggestion that states consider funding alternative support systems and technical assistance providers that can act as design consultants or external facilitators to schools.
State Efforts to Strengthen Support for CSRD in Fiscal Year 2000
"Kentucky will focus on increasing district level assistance to schools during the implementation period. It was determined that district level assistance was very strong during the application process but less evident in a majority of schools during the first year of implementation. To strengthen district assistance for the new competition, more specific information will be provided during the technical assistance sessions, a detailed plan for district support must be submitted with the application, and greater point value will be given to this section on the reviewer's rubric. In addition, Kentucky will train a cadre of external facilitators to provide on site assistance."
-Joseph Clark, Division of Program Resources, Kentucky Department of Education
Reports from the Field: State and District Considerations
As mentioned earlier, in an effort to gather early CSRD implementation data, U.S. Department of Education staff visited 35 CSRD schools in 26 districts over the past year and a half. These visits revealed numerous issues for further consideration. Several of these issues were particularly pertinent for state and district staff, and the visits suggested some strategies states and districts have developed to deal with these issues. Along with a discussion of issues such as helping schools plan for reform, clarifying roles, developing buy-in and cultivating leadership, this section also includes a list of resources that states, districts, and schools can use to confront the challenges involved in successfully implementing comprehensive school reform.
13 Friedman, Lawrence and Matthew Hanson. 2000. "Launching Comprehensive School Reform: Early Lessons for State and Federal Policymakers," Policy Issues (NCREL).
35
Helping Schools Develop and Choose Reform Programs
• The Department's early visits to CSRD sites and districts revealed some concerns about how the time frames established by some states for CSRD competitions may have hindered the process of matching schools and reform models.
Some of the CSRD sites visited indicated that a short application period made it difficult to conduct a full needs assessment and effectively research, match, and build support for reform models. Some schools described having only a matter of weeks to pull together their CSRD applications. This finding suggests that states and districts need to consider how to build into the CSRD application process adequate planning and preparation time. CSRD grants themselves are to be used for implementation–not planning. One district visited dealt with this challenge by offering planning grants to schools, thereby encouraging them to prepare adequately for a comprehensive reform effort prior to beginning the CSRD application and grant process.
• States, technical assistance providers, and districts should be clear with schools that there are no models "approved" for CSRD funding. Therefore states and districts need to encourage schools to use multiple sources of information on models and designs, particularly for examining actual data supporting the effectiveness of various models.
The Northwest Regional Educational Laboratory (NWREL) catalog and other resources on models and designs are simply tools for examining the elements and effectiveness of various models. In preparing and reviewing applications, states and districts should Resources on the State and District Role in pay careful attention to whether schools Reform demonstrate that the needs of the school are reflected in the design of the Consortium for Policy Research on Education comprehensive reform program, (CPRE) including the models chosen. States and Districts and Comprehensive School Reform Addressing Alignment of Models with http://www.upenn.edu/gse/cpre/docs/pubs/rb24.pdf
State and District Goals New American Schools
How to Create and Manage a Decentralized • Department visits to CSRD schools Education System highlight the need for states, districts, www.naschools.org/resource/howto/oddec.pdf and technical assistance providers to provide further assistance in assessing the alignment among school reform models/programs and state and district school improvement plans, standards, and assessments.
As important as the fit between schools and their chosen reform models is the fit between schools' plans for reform and districts' and states' priorities. Some models come with their own “standards” and it is important to consider how those standards fit with state content and student performance standards, particularly as schools are likely to be held accountable for student performance on state assessments.
36
Two of the districts visited as part of “CSRD Reports from the Field” provide school-reform facilitators for CSRD schools who help address such issues. These educators act as liaisons between the schools and the district and are seen as a valuable resource in making sure that school and district efforts are aligned.
In one school, the external partner–a local university–assists all teachers in developing weekly instructional plans that address state and local standards. A part of the professional development for implementing its chosen model includes expert staff from the university working with teachers in the classroom on successful teaching strategies designed to teach according to standards.
Helping Schools Make Time for Reform
• Visits to CSRD sites indicate that time is a major challenge for virtually all schools. States and districts need to support schools in planning the implementation of proposed reform efforts, particularly in terms of staffing and in arranging time and other support for professional development.
Changes in uses of time are a major feature of reform in a number of CSRD schools. However, staff in most schools visited as part of CSRD Reports from the Field felt pressed for time and found lack of time to be a barrier to reform. As noted above, several states are dealing with part of this challenge by offering planning grants to schools, encouraging them to take time to adequately prepare for the effort.
While a number of schools visited are meeting the challenge of finding time for professional development by restructuring the school day, redesigning faculty meetings, and offering summer courses, there are things that states and districts can do to support such efforts.
For example, many districts and schools use Resources on District Support for substitute teachers to make time for teachers Comprehensive Reform to participate in professional development.
The concern with this approach is that days RAND with substitute teachers can be lost days for Lessons from New American Schools' Scale-Up students–especially where the model being Phase rand.org/publications/MR/MR942/MR942.pdf/ implemented requires specific skills and instructional techniques. One district is Northwest Regional Education Laboratory addressing this concern by assigning three Implementing School Reform Models: The Clover specific substitute teachers for a CSRD Park Experience school. These substitute teachers are http://www.nwrel.org/csrdp/clover.html participating with regular staff in professional development activities on the reform model so that they are better able to step into classrooms at the school when they are needed without disrupting student learning.
37
Coordinating Support Roles
• To be successful, comprehensive school reform involves the coordination of numerous stakeholders from within and outside the school community. CSRD requires that schools have external technical assistance providers, but states and districts also have a key role in reform implementation. With assistance coming from several sources, it is important for various actors to work together to support school reform.
States, districts and other external assistance providers all have important support roles to play in comprehensive school reform. The goal is to have the efforts of these various actors to work together to improve student achievement. Districts need to take the lead in CSRD to develop a clear mutual understanding with schools about the role each district will play in supporting a school’s reform efforts, in conjunction with outside technical assistance providers such as model developers. Districts and states can also be instrumental in helping schools become informed consumers of school reform models.
Many district offices are playing a large role in providing support for implementing comprehensive reform in the schools visited. For example, two districts visited provide facilitators to work in the schools implementing comprehensive reform. These facilitators act both as resources and leaders at the school level, as well as liaisons to the district. One district visited has created an area superintendent position to support all the schools in the district (CSRD funded or not) implementing one particular reform model. The area superintendent’s role is to make sure that the schools have the resources and authority to make and implement necessary decisions, and the time to demonstrate expected improvements.
There is little doubt that the role of the district in supporting school reform is critical. However, the design of the CSRD program includes an expectation that model developers and external technical assistance providers will be key sources of external support for CSRD schools. Because CSRD involves the participation of other external technical assistance providers, it is important for roles to be clearly defined and efforts coordinated.
Developing "Buy-In"
• Districts have an important role to play in helping build school support for change and helping teachers and staff “buy in” to reform efforts–both in selecting models and in sustaining reform.
For example, in one site visited, the district and a local foundation partnered to help teachers, particularly those most skeptical that change could happen, visit other schools implementing a similar comprehensive reform effort. When these teachers came back enthusiastic about what they had seen, other teachers became more committed to change. Another district we visited provided funds for a week-long summer institute sponsored by a model developer in order to give school faculty an opportunity to understand and support the reform effort.
Department staff found such efforts on the state level as well. For example, one state visited provides ongoing technical assistance to all CSRD schools and their districts on the evaluation, selection, and development of CSRD programs. A liaison from the state department of education
38
is assigned to each school implementing CSRD and is specifically trained in the different models used by those schools. The liaisons ensure communication with the state, broker resources, provide support, and help keep the reform process on track at the school level.
Cultivating Leadership
• Leadership is an essential ingredient to school reform. In CSRD schools, the principal’s understanding of the model and how it fits in with a broader vision for school change is crucial. Sustaining that vision and helping implement the necessary changes takes skill.
States and districts can support the cultivation of good leaders. For example, in one district where CSRD has been integrated into a broader effort to turn around low-performing schools, the district provides schools with a business manager to help allocate and monitor funds and generally oversee the administrative issues in the schools. This resource is available primarily to low-performing schools and is intended to free school principals’ time and provide them with support so they can focus on being instructional leaders in their schools.
Stability of leadership is an ongoing challenge, both at the school and district level. One district we visited has had nine superintendents in the past eight years. This makes maintaining any reform efforts extraordinarily difficult. Yet, in another district, the school board and the principal agreed to a five-year contract for the principal at the outset of the school reform effort, assuring that the school’s leadership will have the time necessary to fully implement and evaluate the comprehensive reform model.
Clover Park School District Lakewood, Washington
The Clover Park school district is near the city of Tacoma, Washington, and serves the city of Lakewood as well as two major military installations. Clover Park has 19 elementary schools, four middle schools, two high schools, and four special schools. The student population is diverse. Half of the students come from military families, 24 percent are African American, 11 percent are Asian, 8 percent are Hispanic, and 2 percent are Native American. A majority of the students are eligible for free- and reduced-price lunch. Eighteen of the district's schools operate Title I schoolwide programs.
In 1995, the district began data-based, districtwide reform efforts designed to connect research- based reform to needs identified in individual schools. Schools were given full control over their budgets and district staff adjusted their roles to better support school-based reform. Today, seventeen elementary schools, three middle schools, both high schools and the alternative high school use national models as part of their whole school reform efforts. Among the most frequently used models are Accelerated Schools, Success for All, Coalition of Essential Schools, and Padeiea. Three of the schools using Success for All received CSRD grants in 1998 to add Success for All's Math Wings program.
While the Clover Park reform remains a work in progress, it provides an important example of how a school district can organize itself to challenge and assist local schools to undertake comprehensive reform.
39
Summary
We know from research and practice that the states' and districts' support roles are central to successful school reform. Preliminary findings on state implementation of CSRD indicate that the program is leveraging some changes in how states look at and support school improvement efforts. States have played a very active role in providing information to districts and schools about the program and have provided technical assistance in helping schools "match" models to their needs and prepare to take on reform. In a number of states, CSRD is helping administrators rethink how they use their own state funds in conjunction with federal funds such as Title I and how they structure state education programs.
At the same time, early implementation of CSRD reveals some challenges that states and districts need to face as the CSRD program matures. States and districts are key in ensuring alignment between school-based reforms, state standards and assessments, and coordinating the role of technical assistance providers at the school level. States need to play an active role in helping districts and schools become prepared for comprehensive reform–through making connections among schools, districts, and model providers; giving schools and districts adequate planning time and resources; and cultivating strong leadership at the school level.
40
V. School Implementation of CSRD
Ultimately, CSRD is implemented at the school level. For this reason, we now turn to the CSRD schools and examine their early implementation status, issues, expectations, and concerns. Readers are reminded that these data were collected at an early stage in the implementation of CSRD. Schools began receiving CSRD awards in the late summer and fall 1998. The data below were collected in winter and spring 1999. However, early data suggest that CSRD schools are attending to the requirements of the legislation and are engaging in many of the strategies that are associated with successful schools and that are encouraged by Title I. While our preliminary data are self-reported accounts of CSRD implementation, follow-up evaluation work will examine implementation in-depth.
Importance of Implementation
The research-to-date on comprehensive school reform indicates clearly the importance and challenge of effective implementation. Research conducted by RAND on the demonstration and scale-up phases of New American Schools (NAS) models particularly highlights the important implementation issues related to comprehensive school reform and the use of externally developed models.
In the recent study Lessons from New American Schools' Scale-Up Phase, of 40 pre-CSRD schools implementing NAS designs, RAND found that half were operating below adequate levels of implementation. Problems included rushed selection, variance in design team capabilities, and unstable leadership. Stronger reform progress was present in elementary schools compared with slower, more difficult progress in high schools. In its evaluation, RAND expressed concern with a lack of “deepening” implementation over time in schools and the great variance in implementation despite the comprehensive nature of some designs.
In a recent review of more than 100 research studies (most pre-CSRD) on 24 different models, the American Institutes for Research found that school-level implementation was a key issue across models:14
• Reforms that were chosen by schools and selected based on schools' needs were more likely to be implemented successfully. Buy-in was important as was strong, visible, supportive school-level leadership.
• Reforms selected because of prior success were more likely to be successfully implemented.
• Teacher and program support (i.e. training, resources, planning time, money, organizational plans) were associated with successful reform implementation. Schools that fared less well in implementing reforms lacked awareness of the need for change and had a poor understanding of the reform effort.
14 American Institutes for Research, What We Know About Comprehensive School Reform Models (August 2000). This report is being prepared for the U.S. Department of Education, Planning and Evaluation Service.
41
• District leadership and provision of technical assistance or material assistance supported implementation. Lack of support, turnover in district leadership, and limited school-based management were factors that hindered reform.
• Technical assistance by developers, consultants, and partner universities aided the implementation process. On-site facilitators were beneficial to implementation.
• Inadequate professional development and limited planning time hindered implementation. Professional development that was beneficial to implementation involved summer retreats and conferences, training through partnerships with universities, and discussions and sharing among teachers. 15
Omaha Nation Public School Macy, Nebraska
At this high-poverty, rural K-12 school serving a predominantly Native American population, students in grades 8-12 conduct statistical studies of regional plant life through observation and the use of computer spreadsheets, in preparation for a journey to retrace the Buffalo Trail. The school, which operates a Title I schoolwide program, is implementing an entire-school reform effort that includes Expeditionary Learning Outward Bound, a reform model that features interdisciplinary curricula, cooperative learning, ongoing assessment of student work, and team teaching. In the elementary grades, teachers are also implementing Success for All, a program focused on improving reading instruction. Through these efforts, teachers and school staff in grades K-12 are engaged in ongoing professional development and collaboration to improve teaching and learning throughout the school. Students in the elementary grades, most of whom were reading below grade level, have shown significant gains in reading achievement, and increasingly, families are reading with their children at home. Furthermore, while the school is still in the early stages of its reform effort, there has been a marked improvement in school climate, especially in the upper grades.
This research highlights the need for intense attention to the issue of implementation. If research-based models are not well implemented and if schools do not attend to improvement efforts in a coherent and coordinated way, positive results are unlikely. With this background information and important implementation issues in mind, we turn to the implementation of the CSRD program at the school level.
15 American Institutes for Research, What We Know About Comprehensive School Reform (draft, June 2000).
42
Findings from the National Longitudinal Survey of Schools (NLSS)
To understand the implementation of CSRD at the school level, as well as the impact of the CSRD program on schools, teachers, and students, the national evaluation of CSRD includes a national longitudinal data collection on CSRD schools. The NLSS is being administered not only to a nationally representative sample of approximately 1,000 Title I schools, but also to approximately 900 CSRD schools (85 percent of which are also Title I schools). When Congress created CSRD, part of the expressed motive was for CSRD to help leverage the way schools use their Title I funds. For this reason, the NLSS allows important comparisons between how CSRD and Title I schools think about and spend allocated Title I funds. Because of the timing of the administration of the first year of the NLSS, and the rolling basis of CSRD awards, the first-year survey was administered to approximately 300 CSRD schools. As Table 2 illustrates, this early implementation data on CSRD was collected from schools in just 20 states. These states had submitted CSRD award information to the Department by February 1999.
Table 2: Distribution of First-Year CSRD Sample, by State State Sample Respondents AL 33 26 AZ 18 17 HI 7 7 IL 82 68 KS 10 9 MA 25 23 MD 12 8 MO 5 5 MS 29 27 ND 2 2 NE 2 2 NV 7 7 NY 25 18 PA 2 2 TN 3 3 UT 7 6 VA 24 23 WA 9 8 WI 13 12 WV 3 2 Total 318 275
The NLSS is collecting information on school-level implementation of standards-based reforms and Title I supports for reform—especially in schools serving significant proportions of migrant, limited English proficient (LEP), and Native American students, and schools that have been identified as in need of improvement. The first year of the three-year data collection for the NLSS began in the 1998-1999 school year. The principal and up to six teachers in each of the
43
selected Title I schools are surveyed using computer-assisted telephone interviewing (CATI). Following is information on the sample of CSRD awards made between July 1998 and mid- February 1999. 16
The response rate for CSRD schools in this survey was high—about 82 percent. The principal and teacher surveys address awareness and understanding of standards, standards-driven planning, reporting and feedback for improvement, selection and implementation of externally developed comprehensive school reform models, Title I services, parent involvement, and professional development. For schools identified as in need of improvement, the surveys also include questions regarding activities aimed at school improvement and changes in the school as a result of being identified as in need of improvement. In addition to the surveys, two other data collection efforts are planned, including collection and analyses of school-level documents and student assessment data.
The following discussion highlights findings on the sample of CSRD schools in terms of where schools are in implementing sets of activities promoted by CSRD and standards-based reforms.
Because one of the goals of the CSRD program is to help facilitate the adoption and implementation of research-based models in Title I schools, the data below include numerous comparisons between CSRD and Title I schools in general or, where appropriate, CSRD schools and Title I schoolwide programs. According to the NLSS, in 1998-99, about 31 percent of Title I schools overall reported that they have adopted research-based models. This baseline figure will be tracked by the NLSS over the next three years to examine the extent that CSRD may be a catalyst for reform in Title I schools overall.
The comparisons between CSRD and Title I schools reveal some promising data on how the requirements in CSRD may be helping schools focus on research-based strategies, external assistance, professional development, and teacher buy-in.
CSRD School and Student Characteristics
First, this section of the report provides an overview of CSRD schools in the sample in terms of school type, size, poverty status (percent of students receiving free/subsidized lunch), special populations, and urbanicity. It examines Title I-related characteristics such as whether schools were identified as in need of improvement under Title I, and whether the school is a schoolwide program school or a targeted assistance school.
About 78 percent of CSRD schools surveyed are operating schoolwide programs under Title I; 13 percent are Title I targeted-assistance schools; and another 9 percent of CSRD schools were not Title I schools. About 42 percent of CSRD schools in the sample were identified as in need of
16 For a full description of the NLSS sample see the forthcoming RAND report, Comprehensive School Reform Demonstration (CSRD) Schools: Early Findings on Implementation. The sample universe for the first-year sample of CSRD schools was defined as all schools receiving CSRD funds from July 1998 through February 17, 1999—a total of 423 schools in 20 states. A total of 318 schools were selected with equal probability from these 423 identified CSRD schools (a sampling rate of three-quarters). This was a stratified, simple random sample, using the following school characteristics: State; Title I status (schoolwide, targeted assistance, non Title I); school type (elementary, middle, high); and urbanicity.
44
improvement under Title I. Overall, CSRD schools are comparable to Title I schools generally as to the grade levels served and size. However, CSRD appears to be serving particularly high poverty schools with larger minority populations. CSRD serves a mix of urban (50 percent), suburban (15 percent) and rural (35 percent) schools, but are more likely than Title I schools to be located in urban areas.
• CSRD is highly focused on turning around low-performing schools. CSRD schools (42 percent) are more likely than Title I schools to be identified as in need of improvement (10 percent). In general, CSRD schools in the sample had been identified as in need of improvement longer than Title I schools similarly identified.
Table 3: Comparison of CSRD and Title I Schools by Selected Characteristics
CSRD Schools Title I Schools Selected Characteristics Percent School Type Elementary School 75.8 77.3 Middle School 10.9 12.5 High School 13.3 10.2 Enrollment 1-200 8.0 18.0 201-400 24.3 28.0 401-600 32.0 31.6 601-800 19.0 12.3 801 and over 16.7 10.0 Percent of students eligible for free/reduced lunch 0-34.9 7.0 28.6 35-49.9 7.5 18.9 50-74.9 30.6 33.1 75-100 55.0 19.4 Percent minority students 0-24.9 17.8 50.2 25-49.9 11.7 18.8 50-74.9 13.9 11.1 75-100 56.6 19.9 Urbanicity Urban 52.9 24.8 Suburban/Large Town 15.4 30.0 Rural/Small Town 31.7 45.3
Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, and Common Core of Data, 1997-98
45
• States are targeting CSRD funds on high-poverty Title I schools. About 86 percent of CSRD schools are high-poverty schools with 50 percent or more of their students on free- or reduced-price lunch and over one-half (55 percent) of the schools are in the highest-poverty category, with 75 percent or more of their students on free- or reduced-price lunch. For context, about 53 percent of Title I schools overall are high-poverty schools, and about one- fifth fall into the highest-poverty category.
• CSRD serves schools with a high concentration of minority students. In the sample, 70 percent of CSRD schools have a high concentration of minority students (50 percent or more minority). In a majority of the CSRD schools in the sample, more than 75 percent of the students are minority. In 19 percent of the CSRD schools in the sample, at least a quarter of students are Hispanic. In half of the CSRD schools, at least 50 percent of the student population is African American.
• CSRD schools are serving substantial numbers of special education students. Virtually all CSRD schools in the sample have special education students with Individual Education Plans (IEPs). About a third of the schools have less than 10 percent of students with IEPs and about half the schools have between 10-20 percent of their students with IEPs. About 18 percent of schools have 20 percent and higher representation of special education students with IEPs.
• CSRD schools serve some migrant and Native American students. Three percent of the CSRD schools in the sample are classified as "high" migrant and 6 percent as "high" Native American schools, defined as schools with 50 or more migrant students and with 10 percent or more Native American students, respectively. These figures may change as the second year sample of CSRD schools includes schools from across all 50 states.
• Student mobility in CSRD schools is higher than in Title I schools in general. The mean transfer rates of students into and out of the school during the school year averages approximately 19 percent in CSRD schools. This is somewhat higher than the 14 percent reported by all Title schools but comparable to that of high-poverty Title I schools.
Characteristics of Teachers in CSRD Schools
The longitudinal survey allows us to describe some basic demographic characteristics of teachers in CSRD schools. This section examines selected characteristics of teachers in CSRD schools including race/ethnicity, educational attainment, and some indicators of “teacher quality.” The data indicate that CSRD schools have high proportions of minority teachers with an average of about 15 years teaching experience. Principals in CSRD schools largely indicate that teachers in their schools teach to high standards.
• The percentage of minority teachers in CSRD schools is comparable to Title I schools. Mirroring what national data have shown about high-poverty schools, we find that CSRD schools have high proportions of minority teachers—an average of about 30 percent. About 5 percent of teachers identified themselves as being of Hispanic origin.
46
• Teachers in CSRD schools have been teaching in their particular school for an average of 9- 10 years. CSRD school principals had served as principals for about 8 years, similar to principals in high-poverty Title I schools. About half the teachers in CSRD schools have a master’s degree or a degree beyond a bachelor’s degree.
How CSRD Schools Selected Models
At the heart of comprehensive school reform, for most schools, is the model that a school adopts to reform school curriculum, processes, and activities. Research on the implementation of comprehensive school reform highlights the importance of the process schools use to choose reform models. Schools need to choose reform models that "match" schools' particular needs. Buy-in from school faculty is an important determinant of implementation success. Models such as Success for All, for example, require faculty to vote before implementing the model. School reform is more likely to fail if it is forced upon schools or if teachers feel that they have no say in the changes happening in their schools.
For these reasons, it is important to understand the basis upon which CSRD schools are choosing their reform programs. The NLSS asked school principals about the impetus for model-based reform at their schools and how and what models were selected. Comparative findings with Title I schools implementing models suggests that the explicit attention to these issues in the legislation may be helping schools focus on implementing the components of comprehensive school reform.
• CSRD schools report that they are matching models with school needs. About 90 percent of CSRD schools reported that they became involved in the model-based reform because there was a fit between the school’s needs assessment and the research base on the model.
Table 4: Reasons For Adopting Model-based Reform CSRD CSRD All CSRD Title I What were the reasons your school became Elementary Secondary Schools Schools with involved with model? Schools Schools models Percent Was it a result of your school being identified 69.3 64.0 68.1 42.6 as in need of improvement under Title I? Was it at the direction of the district? 28.2 36.2 30.1 29.0 Was there a fit with your school’s needs 90.7 93.9 91.5 88.9 assessment and research? Was it at the direction of the community? 10.4 21.4 13.1 19.8 Source: NLSS Principal Survey, CSRD and NLSS Samples, Spring 1999, Section C, Q. PC6
• CSRD schools appear slightly more focused on research evidence than Title I schools with models. Ninety-five percent of CSRD school principals, compared to 88 percent of principals in Title I schools with models report that the research evidence was an important factor that influenced their choice of models. Ninety-five percent of CSRD school principals compared to 85 percent of principals of Title I schools with models report that student performance in similar schools was an important factor that influenced their choice of models.
47
• CSRD schools are more likely than Title I schools with models to have had teachers vote on the adoption of models. In 80 percent of CSRD schools, compared to 53 percent of Title I schools with models, teachers voted on the adoption of the model.
• Some CSRD schools adopted models at the direction of their districts. Secondary CSRD schools were more likely than elementary schools to adopt model-based reform at the direction of their districts. In general, CSRD schools identified as in need of improvement under Title I were more likely than Title I schools in general to report that they adopted models at the direction of the district—42 percent compared with 22 percent.
• CSRD school principals highlight a number of reasons for choosing models. Factors such as a comprehensive approach, a professional development component, research evidence, and improved student performance in other schools with similar populations ranked high with all schools as reasons for choosing the models being implemented. Surprisingly, pragmatic factors such as ease of implementation (71 percent) and affordability (75 percent) were somewhat lower down on the list of reasons schools chose models.
• CSRD principals reported that a number of individuals were involved with the decision to adopt a model for the school–primarily principals, teachers, and school staff. Over 75 percent of CSRD schools also involved parents and community members in the decision.
Table 5: Factors Influencing Model Choice
Elementary Secondary All CSRD Title I Schools
Schools Schools Schools with Models Important Factors Percent A comprehensive approach in addressing the 96.5 97.3 96.7 96.0 schools needs? A professional development component? 97.0 91.3 95.6 89.6 The research evidence? 98.0 86.7 95.2 88.3 Improved student performance in a school with 95.7 92.8 95.0 85.3 populations similar to your school? Compatibility with other activities you were 92.6 97.3 93.7 89.9 trying to implement in the school? Its focus on a content area of need? 89.1 79.2 86.7 84.7 A curriculum component? 86.1 81.8 85.1 86.5 Affordability? 75.8 71.5 74.8 78.2 Ease of implementation? 71.9 66.9 70.6 67.1 Source: NLSS Principal Survey, CSRD and NLSS Samples, Spring 1999, Section C, Q. PC10
Status of Model Implementation
Careful implementation and implementation that is faithful to model designs are important concerns for CSRD. The extent to which CSRD schools implement models as prescribed is important to assessing whether research-based models will be effective in improving student achievement. In the survey, principals responded to questions regarding the implementation process including the model-adoption process and stage of implementation.
48
The NLSS data show that at the time of data collection, most schools were at the initial training stage or said the model was partially implemented in their schools. Comparisons to Title I schools with models suggest that CSRD schools may be more faithfully implementing model designs. Still, a majority of CSRD schools reported that they made small adaptations to the models when they implemented them in their schools.
• In terms of stages of implementation, a little more than half–56 percent of the first year CSRD sample–reported that the model was partially or mostly implemented in their schools. For context, it is important to note that model developers often report that full implementation of models takes about 3 years. While this self-reported data does not allow us to directly evaluate the extent to which implementation is taking place, the responses would indicate that schools are making progress implementing models.
• CSRD schools are more likely than Title I schools adopting models to report that they are strictly implementing their chosen models. Most of the CSRD schools reported that they had chosen to strictly adopt the model or make small adaptations (38 and 52 percent, respectively). Elementary schools (42 percent) were more likely than secondary schools (26 percent) to strictly adopt the model, and secondary schools were more likely than elementary schools to adopt just parts of the model. According to the NLSS, only 8 percent of CSRD schools report adopting just parts of models while 22 percent of Title I schools with models report adopting just parts of models.
Table 6: Implementation
Elementary Secondary All CSRD All Title I Schools Schools In implementing a model in your school, did you or do you plan to….. Strictly adopt the model without making any 41.6 25.9 37.8 26.7 adaptations? Make small adaptations? 50.9 53.2 51.5 47.1 Adopt just parts of the model? 4.5 19.4 8.1 22.0 Where is your school in the process of implementing the model? Would you say…. Initial selection and planning? 12.9 15.2 13.5 6.7 Initial staff training and development is 28.3 34.9 29.9 11.2 underway? The model is partially implemented? 21.8 34.6 24.9 20.0 The model is implemented in most or all 36.5 15.3 31.3 61.5 aspects? Source: NLSS Principal Survey, CSRD and NLSS Samples, Spring 1999, Section C, Q. PC4, PC4A
External Assistance to CSRD Schools
The requirement that CSRD schools obtain external assistance providers is a key component of CSRD that sets it apart from Title I. Not surprising given the explicit requirement in CSRD that schools obtain external assistance, 96 percent of CSRD school principals compared to 82 percent of principals in Title I schools implementing models report that their staff have received professional development or assistance implementing their chosen model.
49
• Almost all CSRD principals report that their schools are receiving professional development on their chosen models. Over 90 percent of principals reported that professional development or assistance on model implementation is being provided through the entire planning and adoption process–during planning, initial start up, and on a continuing basis during implementation. CSRD schools were more likely than Title I schools to receive assistance on-site or through alternative methods, such as telephone, e-mail, or video.
• CSRD schools are more likely than Title I schools implementing models to report receiving assistance from model developers. In 80 percent of the CSRD schools, compared to 52 percent of Title I schools with models, external assistance is being provided by the model developer.
• CSRD principals are generally satisfied with assistance. Seventy percent of CSRD principals reported being satisfied to a “great extent” with the professional development or assistance provided by external partners.
Table 7: Assistance on Implementation of Models
CSRD CSRD All CSRD All Title I Elementary Secondary Schools Schools
Has your school staff received or is your 96.5 95.5 96.3 82.2 school staff receiving any professional development or assistance in implementing model? Who is providing the professional development or assistance? Is it… District staff? 33.4 41.3 35.3 55.1 State staff? 22.1 23.9 22.5 25.3 A comprehensive regional assistance 32.4 36.6 33.4 34.9 center? The model developer? 82.5 82.4 82.5 52.0 Teachers from another school? 25.5 46.3 30.6 40.5 University consultants? 35.3 52.6 39.5 41.8 Independent consultants? 30.7 41.4 33.3 42.9 Was or is this professional development or assistance provided… As you planned to implement the model? 94.9 92.0 94.2 93.7 During the initial start up of the model in 94.9 97.2 95.4 95.7 your school? On a continuing basis as you implement 97.9 98.4 98.1 97.9 the model? On-site? 93.8 98.4 95.0 85.8 Off-site? 86.1 77.7 84.1 88.6 Through alternative methods, such as 71.4 63.3 69.4 59.3 telephone, e-mail, or video? Percent reporting “great extent” To what extent are you satisfied with the 73.3 60.5 70.2 66.1 professional development or assistance? Source: NLSS Principal Survey, CSRD and NLSS Samples, Spring 1999, Section C, Q. PC11, PC12, PC12a
50
Title I Services in CSRD Schools
This section provides an overview of the Title I services provided–as reported by principals or teachers in the CSRD schools–and focuses on the following issues: the extent to which principals report that changes in Title I legislation have helped them promote school improvement activities; activities that comprise the school’s program; and the degree to which federal funds are combined with other funding sources and the challenges faced in such coordination.
The results indicate that CSRD schools are more positive than Title I schoolwides about how Title I affects their schools' ability to plan for schoolwide needs assessment and comprehensive reform. Furthermore, CSRD schools appear to be using Title I funds in ways that support the priorities of the Elementary and Secondary Education Act (ESEA) and are consistent with practices in successful schools. For example, in the early going, CSRD schools more often report having extended time programs and more activity in fostering parent involvement. CSRD schools are less likely than Title I schools in general to report using pullout services or teacher aides for instruction. In line with the priorities of the CSRD legislation, CSRD schoolwide programs are less likely than Title I schoolwide programs to report barriers to coordinating funds.
Table 8: School Activities Promoted to “A Great Extent” by Changes in Title I Legislation
Schoolwide CSRD Schools Title I Schools Activities (n=256) (n=634) Percent responding “to a great extent” Apply content standards to all students? a 57.6 54.4 Assess student performance against high standards? 48.8 44.9 Use student performance results for school 60.9 55.2 accountability and continuous improvement? Have more flexibility in identifying students for 59.7 61.1 services? Plan for schoolwide needs assessment and 70.7 53.8 comprehensive reform? Minimize pullout programs? 60.9 58.2 Extend learning time? 50.7 41.7 Use your federal resources to support overall school 70.3 65.5 improvement goals? Coordinate Title I with other federal education 53.1 45.4 programs? Use a parent involvement policy? 51.2 46.4 Use school-parent compacts? 45.4 53.2 Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, Section D, Q. PD1 aAsked only of principals reporting use of content standards at their schools (n=255 for CSRD & 633 for Title I).
51
• CSRD schools report positive effects of changes in Title I legislation. Compared with Title I schools, principals in CSRD schools were more likely to report that changes in Title I legislation had had very positive effects on their schools. About 70 percent of CSRD principals reported that these changes had helped their schools to plan for schoolwide needs assessment and comprehensive reform and to use their federal resources to support overall school improvement goals, compared with 54 percent of Title I schoolwides. About 60 percent reported that changes in Title I legislation had helped their schools apply content standards to all students, use student performance results for school improvement and continuous improvement, minimize pullout programs, and had given them more flexibility in identifying students for services.
• CSRD schools use various sources of information for planning their Title I programs. Principals were asked about the various sources of information about students that were used to plan the Title I program in their schools. The vast majority of CSRD principals reported that performance on standardized tests was used for program planning. Student performance relative to the established content and performance standards of the district or state was also important, with 90 percent of principals in CSRD schools reporting use of such student information. Principals reported using other student information to plan their Title I programs, such as student performance on teacher tests (77 percent), attendance patterns (80 percent), parent surveys (71 percent), and promotion rates (70 percent). About 85 percent of CSRD schools with LEP students also reported using English-language-proficiency assessment results in planning their Title I programs.
Table 9: Title I Services Provided in CSRD and Title I Schools
All CSRD Schoolwide All Title I Schools Program Schools Title I Schools (n=1,081) (n=256) (n=634) Percent A summer or intersession program? 72.3 66.0 68.0 A before- or after-school program? 70.7 53.2 52.4 An extended school year program? 44.9 30.1 31.0 A year-round program? 13.7 12.1 15.5 A weekend program? 13.3 7.2 6.1 College and career awareness and prep activities? 84.0 78.9 82.9
Counseling or pupil services? 78.5 83.5 82.8 Mentoring or tutoring? 81.2 78.3 73.6 Transition activities for children in preschool 44.1 53.7 45.0 programs? School-to-work activities? 25.4 32.4 29.6 Services for out-of-school youth? 21.2 13.2 15.7 Training for parents? 80.1 69.9 61.0 A parent liaison? 71.5 58.4 54.8 A family literacy program? 37.9 31.4 28.0 Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, Section D, Q. PD10
52
• CSRD schools are more likely than Title I schoolwides to support extended learning time. Nearly 70 percent of CSRD schools report having before- and after-school programs, compared with 52 percent of Title I schools and 53 percent of Title I schoolwides. CSRD schools are more likely than Title I schools in general to report having summer school, extended-year, and weekend programs.
• Improving parent involvement is being addressed in CSRD schools. CSRD schools in general were much more likely to report parent services programs supported with Title I than Title I schools. About 80 percent of CSRD principals reported parent training, 72 percent had a parent liaison, and 40 percent had a family literacy program. This was compared to 61, 54, and 29 percent respectively in Title I schools.
• CSRD school principals are less likely than principals of Title I schoolwides to report using pullout services. The percentage of Title I schoolwide elementary schools offering pullout services (57 percent) is higher than that of CSRD elementary schools (45 percent).
• CSRD school principals report use of teacher aides for instruction less often than do Title I school principals generally. Overall, far fewer CSRD school principals reported using teacher aides to provide Title I instructional services in reading (66 percent) compared with schoolwide or all Title I principals (81 percent and 83 percent respectively).
• Fewer CSRD schoolwide principals than Title I schoolwide principals report challenges to coordinating federal resources with other funding sources. Overall, principals in a little more than half of the CSRD schoolwides reported that they combined federal funds with other funding sources to a great extent in their school. In citing barriers, 48 percent of Title I schoolwide principals said they were unsure of what was allowed in combining funds compared to 38 percent of CSRD schoolwide principals.
Table 10: Challenges in Coordinating Federal Resources with Other School Funds
Challenges in coordinating federal resources CSRD Title I with other funds a Schoolwides Schoolwides Percent The district controls the use of funds? 50.7 55.3 You are unsure what is allowed? 38.0 47.7 The state controls the use of funds? 37.1 39.1 There is resistance by school staff? 5.6 10.4 Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, Section D, Q. PD26 a Asked only of principals reporting combining federal funds with funding from other sources.
53
Professional Development
From the start, professional development and staff training needs have been a focus of CSRD implementation. A critical piece of any effort to change schools is professional development and training. Indeed, an important component of CSRD is to focus professional development around the reform programs being implemented in the schools.
The survey results indicate that nearly all teachers in CSRD schools reported that they received professional development in the past 12 months. CSRD teachers report that they are receiving professional development in instruction and content areas and many report that this professional development is sustained, rather than a one-shot workshop or seminar.
However, while almost all principals report that staff is receiving professional development on the model, when asked of teachers directly, a significant percentage indicated that they had not yet received professional development in implementing a model. This may be due to the early timing of data collection, a phasing-in process at the school level, or may be because some models do not affect all grade levels. However, the number of CSRD teachers not receiving professional development in schools' chosen models is McCoy Elementary School substantial and will be tracked Kansas City, Missouri carefully in the coming years. Building on work the school already had underway with the • Most CSRD teachers report Comer School Development Program to strengthen school professional development on governance and family involvement, McCoy is using its instruction and are more CSRD grant to partner with the Institute for Learning, a likely than Title I teachers to nonprofit organization based at the University of Pittsburgh report that professional that supports the use of research-based, innovative literacy approaches. Faculty meetings have been replaced with staff development is sustained study groups addressing instructional issues. Rubrics are rather than a one-time used to assess both teacher practice and student progress. A activity. Overall, 85 percent new enthusiasm for reading and a love of books are evident of teachers in CSRD schools throughout the entire school, from the lunchroom to the report that they have library to the classroom. The percentage of third graders received professional scoring at or above the proficient level on the Missouri development in instructional Assessment Program has tripled, surpassing the district strategies and subject area average. content. While this is comparable to the responses of Title I teachers overall, CSRD teachers are more likely to report (41 percent) than Title I teachers (28 percent) that professional development activities in instruction are part of an ongoing, multi-year activity, as opposed to a one-time activity.
• A majority of teachers and principals in CSRD schools report changes in teaching practices due to professional development. Overall, more than half of CSRD teachers reported that most professional development activities led to changes in their teaching practices. About half of the CSRD principals reported that professional development was helpful to a great extent in becoming part of the teachers’ regular work life.
54
• A substantial percentage of teachers in CSRD schools report that they have not yet received professional development on model implementation. Overall, 43 percent of teachers in CSRD schools implementing externally developed models reported receiving no professional development on the model.
• CSRD teachers indicate lack of professional development in other areas. Between 60-65 percent of the teachers reported receiving professional development in approaches to assessment (65 percent), for using assessment results (61 percent), or for strategies to teach low-achieving students (60 percent). However, less than half of the teachers received professional development for strategies to manage discipline (48 percent), strengthen parent involvement (45 percent), and leadership development (36 percent).
Table 11: Principal Reports about Teacher Professional Development
To what extent does professional development for All CSRD Schools Schoolwide Title teachers at your school do the following? (n=275) I Schools (n=634) Percent reporting “to a great extent” Follow-up with resources that teachers need to make 55.3 46.1 changes in the classroom? Become part of teachers’ regular work? 52.4 43.4 Evaluate evidence of effects on student achievement? 45.8 30.3
Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, Section G, Q. PG5
• CSRD teachers are exposed to other professional development opportunities. CSRD teachers were also asked a series of questions about other professional development opportunities over the past 12 months. The most common professional development activity reported by the sampled NLSS teachers was attending workshops and conferences, with 90 percent reporting attendance overall. Eighty-five percent of the teachers reported engaging in independent professional reading related to teaching or subject matter, and nearly 80 percent reported participating in school or district committees.
• CSRD teachers were more likely than Title I teachers to participate in certain kinds of professional development activities. Teachers (62 percent) in CSRD schools were more likely to examine written narratives or videotapes of classroom events or student work, than teachers in Title I schools (51 percent). Teachers in CSRD schools were more likely to observe other classrooms in their school (60 percent vs. 51 percent) or to observe classrooms in other schools (31 percent vs. 23 percent) than teachers in Title I schools. Both teachers in CSRD schools (46 percent) and teachers in Title I schools (39 percent) participated in mentoring activities; and 53 percent of teachers in CSRD schools and 43 percent of Title I teachers participate in regularly scheduled study groups within their schools.
55
Parental Involvement
The link between parental involvement in a child’s learning, both at home and at school, and the child’s academic achievement has been well documented over the years. One of the explicit components of CSRD is for schools to provide for the meaningful involvement of parents in school reform. This section focuses on the use of strategies to encourage and foster parental involvement in CSRD schools. CSRD schools and Title I schools are generally alike in their efforts to communicate with parents. However, CSRD schools are much more likely to report parent services programs supported with Title I than Title I schools in general.
• Almost all CSRD schools provide parents with school report cards. Approximately 95 percent of CSRD principals reported providing parents a school performance profile or school report card. About 40 percent of CSRD schools reported translating school documents into languages other than English.
• CSRD school principals report that they use their parent-school compacts. Overall, 72 percent of CSRD school principals reported using a school-parent compact. Elementary schools (79 percent) were more likely than secondary schools (48 percent) to have school- parent compacts. CSRD schools were more likely than Title I schools to have asked the parents of all students to participate in the school-parent compact. Less than 10 percent of CSRD schools limited their compact to parents of Title I students only compared with almost a quarter of Title I schools.
• Teachers in CSRD schools are more likely than teachers in Title I schools overall to report sending learning activities home with students. The NLSS asked teachers about at-home activities that engaged parents in their children’s learning. About 40 percent of teachers reported sending home mathematics and reading activities that parents could work on together with students in order to reinforce learning on a daily basis. Elementary school teachers were far more likely to report these activities than were secondary school teachers. Also, teachers (40 percent) at CSRD schools were more likely than their counterparts (27 percent) in all Title I schools to report sending home these activities on a daily basis. Approximately 30 percent of all teachers reported requiring parents to sign off on students’ homework on an almost daily basis. Again, elementary teachers were more likely to report this activity as an involvement strategy.
• CSRD principals report efforts to involve parents in school events. Over 90 percent of CSRD principals reported that they invited parents to assemblies or fairs, hosted social events, used parents as volunteers, and encouraged parents to serve on school or district committees.
• CSRD schools are more likely than Title I schools in general to provide support services for families. Eighty percent of CSRD schools provided social support services, compared with 64 percent of Title I schools. CSRD school principals were also more likely than Title I principals to report that they provided information in culturally or linguistically appropriate formats and provided materials translated into other languages.
56
Table 12: Daily Parent Involvement Activities
All CSRD Schoolwide Title All Title I Parent Involvement Activities Schools I Schools Schools (n=256) (n=1635) (n=2765) Percent reporting “almost daily” Require parents to sign off on students’ homework? 30.9 24.5 26.7
Send home reading activities parents can do with 40.1 28.7 27.1 students that reinforce what students are learning in the classroom?
Send home math activities parents can do with 41.7 30.9 28.2 students that reinforce what students are learning in the classroom? Source: NLSS Teacher Survey, CSRD and NLSS Samples, Spring 1999, Section E, Q. TE10
Technology
Although there is considerable debate about the most effective uses of technology in schools, there is also considerable consensus that open and equal access to technology is needed if we are to prepare students for the information age. Of particular concern is whether high-poverty schools have adequate numbers of computers, connections to the Internet, and teachers trained in technology so that they are not disadvantaged in this respect compared with lower-poverty schools. This section presents data on the availability of computers and barriers to use of technology collected from principals in CSRD schools. The survey found that CSRD schools tend to have technology plans but appear to have fewer resources and staff preparation than other Title I schools in the area of technology
• Almost all CSRD schools have technology plans. Over 90 percent of CSRD schools had a plan that includes a goal for using technology to improve student instruction, and about 70 percent had a computer or technology coordinator.
• Computer availability may be lower in CSRD schools than in Title I schools in general. For CSRD schools the ratio of students per computer was about 10:1. Title I school principals reported an average of 7.3 students per computer at their schools.
• Most CSRD schools have some Internet accessibility. Eighty-five percent of elementary classrooms had at least one computer, and two-thirds of computers in CSRD schools had CD ROM, graphics display, and sound cards. Approximately 40 percent of CSRD classrooms had at least one computer permanently connected to the Internet.
57
Table 13: Major Barriers for Using Technology for Instructional Purposes
All CSRD Schoolwide All Title I Major Barriers Schools Title I Schools Schools (n=141) (n=310) (n=532) Percent Lack of teacher knowledge regarding ways to integrate 80.1 68.2 71.5 technology into the curriculum? Lack of or inadequately trained staff? 69.6 44.5 48.9 Insufficient equipment? 66.1 62.7 54.1 Lack of software that is integrated with the school’s 66.1 56.4 55.7 curriculum? No telephone lines or insufficient telephone lines? 56.9 37.7 40.5 Lack of technical support or advice? 48.3 53.7 46.8 Source: NLSS Principal Surveys, CSRD and NLSS Samples, Spring 1999, Section H, Q. PH16
• CSRD principals were more likely to report barriers to using technology for instructional purposes than principals in Title I schools. For example, 70 percent of CSRD principals reported that lack of staff or inadequately trained staff was a barrier while only 49 percent of all Title I school principals found this a barrier. While about 57 percent of CSRD school principals reported that the lack or insufficiency of telephone lines was a barrier to using technology for instructional purposes, only 40 percent of all Title I school principals reported it a barrier.
CSRD Schools in Need of Improvement
CSRD is meant to be targeted to low-performing schools that could benefit from research-based reform strategies and external technical assistance. In order to collect some baseline data on what is happening in terms of assistance for low-performing schools, the longitudinal survey asked CSRD principals to describe the consequences they have faced as a result of being identified as in need of improvement. The data reveal that CSRD schools have generally been identified as in need of improvement under Title I for longer than other Title I schools, and that CSRD schools are generally receiving extra assistance and are engaging in improvement efforts as a result of being identified as a low-performing school.
• CSRD school principals report being in improvement status for a longer period of time than Title I schools. For example, principals in almost half of the CSRD schools had been identified in need of improvement for three or more years compared with about one-fifth of Title I schools. Ten to 15 percent of CSRD schools appeared not to know how many years they had been identified.
• More than a quarter of CSRD school principals identified as in need of improvement are not aware of the criteria for identification. A substantial proportion of CSRD schools (27 percent) reported that they did not know what their district considered adequate yearly progress or substantial progress. There was little difference between elementary and secondary schools with respect to knowledge of what the district considered adequate progress. Of the 73 percent of schools that were familiar with their district metrics of
58
performance, well over a third (38 percent) seemed to feel that these measures were not adequate to judge their schools' performance.
Table 14: CSRD Schools in Need of Improvement
Selected Characteristics CSRD Schools Title I Schools Do you know what the district considers adequate yearly progress or substantial progress? Yes 73.0 62.4 Of those familiar with district performance measures Do you feel that the measures used to determine adequate yearly progress or substantial progress were adequate to judge your school’s performance? Yes 61.9 61.3 As a result of your school’s identification as in need of improvement, did any outside entities provide you with additional technical assistance or professional development?a Yes 63.5 47.3 As a result of your school’s identification as in need of improvement, have you implemented any additional strategies to address it?a Yes 88.7 76.4
Source: NLSS Principal Survey, CSRD and NLSS Samples, Spring 1999, Section E, Q. PE3, PE8, PE9 Notes: a Numbers will not add to 100 percent because of missing values or “Don’t Know” responses.
• The majority of CSRD school principals report receiving extra technical assistance because they were identified as in need of improvement. About 64 percent of the CSRD schools in the sample reported receiving additional technical assistance or professional development as a result of being identified as in need of improvement. This was somewhat higher than in all Title I schools identified as in need of improvement. The majority of this additional assistance was supplied by the school district, the state department of education, and school support teams. Because the large majority of CSRD schools had adopted externally developed models, close to 60 percent of them reported receiving assistance from model developers as well, compared with a third of Title I schools. Distinguished educators and consultants (either independent or from a university) provided support to about 50 percent of the schools who received additional assistance.
• CSRD schools are implementing school improvement strategies. Almost 90 percent of CSRD schools in need of improvement report that they have implemented additional strategies to address this issue, compared with about three-quarters of Title I schools similarly identified. These strategies included more family and community involvement, revising or developing a school plan, more professional development, and adoption of a research-based model. The survey found that 84 percent of CSRD teachers and 70 percent of Title I teachers participated in more professional development than other schools in their districts as a result of being identified as in need of school improvement.
59
Table 15: Additional Strategies Implemented by Schools as a Result of Being Identified as In Need of Improvement
CSRD Schools Title I Schools Strategies Percent More professional development than other schools 84.3 69.9 District staff spending more time in this school than 39.2 39.3 other schools Closer supervision of school decisions 70.6 50.7 Assistance brought in from outside the district 72.5 42.2 Strong encouragement to adopt a new, comprehensive 74.5 58.2 model program Revising or developing a new school plan 86.3 86.5 More family and community involvement 92.2 91.6 Source: NLSS Principal Survey, Spring 1999, Section E, Q. PE9A
The Delaware Academy, Syracuse, New York
The Delaware Academy, a high-poverty public elementary school, has a diverse student population, including a large percentage of limited-English proficient Latino students. The school’s restructuring effort aims to strengthen student learning, teaching and school management. The district has provided an on-site facilitator to assist the school in implementing Success for All, a program that emphasizes early reading, intensive professional development in reading instruction, and family involvement in reading at home. The Comprehensive School Reform Demonstration grant acted as a catalyst for the school to coordinate funds and resources, including Title I schoolwide program funds, state and local funds, and considerable community support, to make schoolwide improvements in curriculum and instruction. Due to these efforts, more students, including English Language Learners, are meeting and exceeding state standards.
Student Expectations
The comparative data between Title I and CSRD schools raise some concern in the area of expectations of students. Some of these differences may be due to the significantly more targeted use of CSRD funds in high-poverty and low-performing schools. Recall that CSRD schools are more likely to be identified as in need of improvement under Title I than Title I schools in general (42 percent compared with 10 percent) and significantly higher poverty (86 percent high-poverty CSRD schools compared to 53 percent high-poverty Title I schools).
• Although the percentages are fairly small, CSRD school principals are more likely than Title I schoolwide principals or Title I principals in general to report that standards are too rigorous for most of their students (14 percent compared with 7 percent). Twenty-two percent of teachers in CSRD schools report that standards and assessments are too hard for most of their students.
60
Reports from the Field: Observations on School Level Implementation of CSRD
The Department's CSRD Reports from the Field visits to more than 30 schools over the past year have provided further and more in-depth information on early CSRD implementation efforts at the school level. The visits revealed several challenges as well as some promising and innovative solutions on the part of schools facing the challenges of implementing comprehensive school reform. Some of these issues and strategies are shared here to help guide schools as they begin and continue comprehensive school reform efforts. Additional resources are Resources on Effective Use of Data included in appendix H. North Central Regional Education Laboratory Using Data Effectively Comprehensive School Reform: Making Good Choices: A Guide for Schools and Districts ncrel.org/csri/tools/makegood.pdf • The effective use of data is key to making
good decisions about matching reform Northwest Regional Educational Lab models with schools and developing Comprehensive Center programs to meet the needs of students. Evaluating Whole-School Reform Efforts: A Guide The CSRD Reports from the Field visits for District and School Staff suggest that many schools need to move http://www.nwrac.org/pub/whole-school.html beyond the level of identifying broad problem areas, such as the need to improve Mid-Continent Regional Educational Lab achievement in reading or math, to a more Evaluating for Success: An Evaluation Guide for detailed analysis of these issues. Schools and Districts http://www.mcrel.org/products/csrd-eval.asp
While some of the schools visited made Maryland Department of Education connections between the general academic School Improvement in Maryland needs of students, the needs of the school http://www.mdk12.org/index.html community, and the models they chose, others did not seem to be making extensive use of Lab for Student Success detailed school and student-level data to make Achieving Student Success: An Interactive Online decisions to guide the change process. Tool Based on a Handbook of Widely Implemented Research-Based Education Reform Models A number of schools are effectively using data http://www.reformhandbook-LSS.org/ to guide practice. In one school visited, teachers are administering weekly assessments Ohio Department of Education Reference Guide to Continuous Improvement designed for each grade level to monitor Planning for Ohio School Districts student mastery of state and local standards http://schoolimprovement.ode.state.oh.gov and identify what is working and how daily instruction can be improved. Tests are graded MPR Associates quickly to give teachers immediate feedback. At Your Fingertips: Using Everyday Data to Teachers and the principal at the school use the Improve Schools information to discuss progress and plan future .www.mprinc.com/html/resources/ lessons. ayf_brochure_main.htm
61
Another school is using data to analyze whether the model chosen is adequately meeting the needs of all students. Early data on the model is showing improvements for the school as a whole, but closer analysis of the data reveals that the progress of the lower performing students in the school is lagging. The principal and teachers concluded that the lower achieving students needed more attention in basic skills than the model provides. As a result, the school will continue implementing the current model, but is incorporating instructional features at the early grades that are more specifically focused on the needs of low-performing students.
The data schools need to analyze in order to improve teaching and learning are not limited to student assessment information. In one school visited, a “school implementation group” has developed a self-assessment for teachers. The results of the assessment are used to help teachers learn what about their instruction is going well, what needs work, and how to develop professional development activities to better meet their classroom needs.
Integrating Models with Reform Schoolwide
• The CSRD legislation sets out nine components of comprehensive reform. But few models, if any, fully address all nine of these aspects of school operations. In some of the schools visited, it was clear that staff see their selected model as part of an overall effort–a piece of a larger reform vision. Some of the schools visited are using CSRD as an opportunity to organize their reform efforts into a comprehensive, coherent effort.
But changes in some other schools are primarily confined to implementing the models rather than addressing the entire operation of the schools. While this may be due, in part, to the early timing of site visits, it is important for schools to understand the implementation of models as just a part of a coherent and broader reform plan.
Comprehensive school reform also requires schools to be attentive to including all students as part of reform efforts. In one high school implementing career academies, for example, only about a quarter of students participate in the academies; the curriculum and instruction remain very much the same for other students as before the reform effort. Although the school plans to expand the career academies, it does not seem likely that the effort will expand to include all students and teachers. Comprehensive reform efforts cannot just focus on one grade or one particular subject area. The concept behind comprehensive school reform is that the reforms are for all students.
Challenge of Implementing Multiple Models
• As part of CSRD, some schools are implementing more than one new model from the start of their comprehensive reform efforts; in other cases schools are significantly expanding their reform efforts by implementing new models in addition to existing models or programs in use. While implementing multiple models may be an acceptable approach for some schools, a great deal of attention must be paid to the coherence and coordination among the models within the schools. Schools must be certain that the models complement their overall vision for reform. The point of CSRD is to help schools move away from piecemeal approaches to school reform that mix and match different programs and models together without an overarching vision and purpose.
62
During this past year a number of schools have struggled with the integration of various programs and initiatives within their schools. In some schools, faculty and administrators were attentive to the integration of multiple models. However, in one school visited where teams were created to deal with various aspects of reform, it was not clear if the faculty saw the teams as part of one effort or as separate efforts. In another school implementing a new reading program and a new computer-based program designed to improve literacy, efforts did not appear to be complementary. Unfortunately, prior to adopting the computer-based program, neither the alignment between the computer programs and state standards not coordination between the two literacy programs was considered. The facilitator of the reading program has only a basic awareness of the computer-based program and has not worked with teachers coordinate the two programs.
A common thread among schools successfully integrating different models is the involvement of the school principal and the awareness of the program facilitator of the strengths of each model component. For example, the Department team visited a school well into the implementation stage of reform efforts that was successfully integrating several parts of different reform models to create a comprehensive reform effort. The facilitator was aware of the different reform efforts and was able to work with the vision and mission of the school as developed by the faculty and staff. The facilitator encouraged collaboration among teachers as well as provided feedback to teachers based on observations. A cohesive program appears to have been implemented; it is difficult to tell where one model starts and another stops.
Addressing the Needs of Special Populations
• States, districts and schools should ensure that reform efforts meet the needs of all students, including special education students and English language learners.
The concept behind comprehensive school Resources on Comprehensive School reform is that improvements should benefit Reform and Special Populations every student in the school. It is critical that the needs of special populations, such as Region IX Southwest Comprehensive special education students and English Center Comprehensive School Reform Models language learners, are addressed by the Addressing the Needs of English Language school’s reform program. Many schools Learners struggle with integrating special populations cesdp.nmhu.edu/CSRD-Guide/csrd-title.htm into their reform efforts. Others, however, are taking significant steps in this area.
At one rural school, special education students are full participants in the school’s reform program. This school, which has a high percentage of students with individualized education plans, operates an inclusion program. Because all special education students spend at least part of their day in general education classrooms, they are involved in all aspects of the school’s comprehensive reform program, including the model that guides the school’s work overall. The school has one set of challenging expectations for all students and includes special education students in standardized assessments whenever appropriate.
63
Elsewhere, an urban school with a high percentage of limited-English-proficient students selected a reform model that includes a Spanish-language component. The model has a curriculum and materials specifically designed for Spanish speakers and features literature originally written in Spanish rather than simply translated into Spanish. In this dual language program, English language learners receive their primary literacy instruction in Spanish followed by an English-as-Second Language block. Once students achieve English proficiency, their primary literacy instruction is in English, supplemented by a literature block in Spanish later in the day.
Other schools are using variations of this dual-language approach. One school uses a two-way program in which an equal number of Spanish speakers and English speakers learn both languages together. Another school gradually increases the amount of time students are taught in each language. At this school, kindergarten and first grade are taught 90 percent of the time in Spanish and receive 10 percent of instruction in English. In second grade the ratio changes to 80:20. This pattern continues until fifth grade, when instructional time is equally Resources on Arrangements with divided between English and Spanish. Model Developers
Negotiating With Technical Assistance Regional Education Laboratories Providers A Guide to Working with Model Developers http://www.ed.gov/programs/compreform/ • The design of the CSRD program includes an expectation that model developers and external technical Education Commission of the States assistance providers will be key Comprehensive School Reform: Criteria and sources of support for CSRD schools. Questions Selecting School Reform Models http://www.ecs.org/ In general, the schools visited in the CSRD Reports from the Field initiative are feeling well supported by external model developers. Yet some schools express concern that the agreed-upon level of support they are receiving from model developers will not be sufficient, either because not enough ongoing contact has been negotiated or because schools think they may need additional support beyond what the model developer provides.
In a few instances, schools visited expressed difficulty contacting staff representing some models, and schools with upcoming staff development activities hosted by developers did not know what was to take place in these sessions. Thus, states and districts have a role to play in helping schools become good “consumers” of reform models’ services and can play an important role in helping schools and technical assistance providers learn to work together.
Building Schoolwide Support for Reform
• The Department's CSRD Reports from the Field team observed promising school-level efforts to build ongoing support for reform.
One CSRD school is making funds available so that all teachers in the school can visit another school implementing the same model–this has been a major investment. In addition, the school
64
has arranged for teachers from its partner school to visit and provide professional development, including modeling lessons to demonstrate effective instructional strategies.
In another school, a veteran teacher is being trained as the model facilitator. The teacher has leadership skills, is respected by the other teachers, and has an interest in the reform effort. His job is to help teachers stay on task; to provide professional development; to meet with and develop leadership teams; and to make certain that plans are implemented. The facilitator also helps teachers with instruction and scheduling problems.
One of the biggest challenges to Resources on Building and Sustaining sustaining a reform effort is School-Level Support for Reform incorporating and familiarizing faculty who are new to the school and RAND not familiar with the efforts. One Lessons from New American Schools' Scale-Up school is overcoming this obstacle by Phase assigning a mentor to teachers new to http://www.rand.org/publications/MR/MR942/MR9 the school. These teachers are also 42.pdf/ allowed to visit a school further along in the implementation process of the same models so that he or she can gain a better understanding of what is envisioned at the "home" school. Another school schedules a 40- minute planning block each day for teachers new to the school. Each week new teachers spend one of these blocks with administrators and one with the school team.
Developing Benchmarks
• The recent visits to CSRD sites raised some concerns about the extent to which schools are developing benchmarks related to implementation and progress of comprehensive school reform.
The CSRD program requires schools to develop benchmarks related to the implementation of their comprehensive reform programs. These benchmarks can help guide a school through the change process and are useful tools for states, districts, and technical assistance providers to understand a school’s expectations and measure progress.
Some schools are setting benchmarks. One school visited has developed a regular progress plan that helps staff stay on track, and shows faculty how things are going. This regular return to the school’s goals, expectations, and progress is helping to build and reinforce commitment to the reform effort. Another site has developed a tool for principals and teachers to observe whether real change is taking place in classroom practice; it is designed to ensure that reform is not an add-on but a replacement of old practices. The tool includes teacher questionnaires regarding the degree of implementation and classroom observation of changes in instructional approaches.
65
Blackstone Primary School Blackstone, Virginia
Blackstone Primary is an elementary school located in Nottoway County, Virginia, a small rural school district. Blackstone, a Title I schoolwide program, serves approximately 500 students in grades Pre-K to 4. Sixty-three percent of students are eligible to receive free lunch. The school population tends to be stable. The school has recently undergone a major facility renovation.
Blackstone was among the highest achieving schools in the state on the 1999 Virginia Standards of Learning assessments. On the grade three test, over 70 percent of students passed all four tests (English, math, science, and social studies). Based on this level of achievement, Blackstone was one of a small percentage of schools that qualified for full state accreditation. The leadership of the school, however, knows there is still room for improvement. “We want them all” to pass is the school’s goal.
Identified as a school in need of improvement under Title I in the past, Blackstone has been instituting reforms for the last eight years. The staff became involved in finding new programs that would result in increased student achievement. Support has steadily grown. Data-driven decision making and a rigorous focus on literacy are the key themes at Blackstone Primary. The implementation of the Onward to Excellence II reform model, supported by a grant from the Comprehensive School Reform Demonstration program, is assisting the school in these efforts. The whole staff is involved in the data collection and analysis process. Data are collected on achievement, discipline, attendance and teaching experience and are disaggregated by student, teacher, gender, free lunch and race. Priorities and goals for the school, along with strategies to reach them, are based on this information. Individualized strategies are also planned for students not making adequate progress.
The literacy program at Blackstone is based on instilling in children a love of reading and a belief that they can succeed as readers. Students are constantly assessed on their reading level, and every child knows exactly what his or her reading level is. The school also has an incentive system to reward students based on the books they have read.
Fourteen percent of students at Blackstone have individualized education plans to receive special education services. The school operates under an inclusion model. With the exception of one kindergarten class, there are no self-contained special education classes. The philosophy of Blackstone is to have one set of expectations for all students, including special education, and the school is committed to including special education students in testing where appropriate. On the 1999 Standard of Learning test, 70 percent of third-grade special education students were tested.
Involving Parents in Comprehensive School Reform
• Parent involvement in the education of children has long been understood to be an important predictor of academic success. CSRD schools need to play a key role in cultivating this involvement by reaching out to parents and other community partners, involving them in decision making about school reform, and inviting their active participation in their children’s learning at school and in the home.
66
This is especially important when reform models require a new role for parents. For example, in one school visited, the reform model requires significant changes in the kind and amount of homework students are assigned. It became clear that parents would need more information about the purpose of the new work and how they could help their children. In response, the school invited parents to a detailed orientation where they could ask questions and learn about the model.
Another school visited is implementing a model that requires parents to read to their child every night. The school reached out to parents through special programs to explain the model and the time commitment, as well as to provide suggestions on reading with children each Resources on night. Additionally, the school is Family Involvement attempting to make parents feel welcome at the school through a program that U.S. Department of Education encourages parents to walk their students to Compact for Learning class, get a cup of coffee, and talk with the http://www.ed.gov/pubs/Compact/ faculty. Strong Families, Strong Schools At another site, parents were closely http://eric-web.tc.columbia.edu/families/strong involved in the year-long model selection process for the school. Parents studied Partnership for Family Involvement various models during the school year, and http://pfie.ed.gov one parent was able to attend a national National Network of Partnership Schools conference related to school reform. http://www.csos.jhu.edu/p2000 Parents were invited to view videotapes on the models they found most appealing. The tapes helped illustrate the roles parents could play in supporting implementation of the reform model.
Highlights from Selected State Evaluations of CSRD
The CSRD legislation requires states to undertake evaluations of CSRD. In most states, schools are early in the implementation process and state evaluations are just beginning. However, several states have completed preliminary reports on CSRD. Highlights from first-year state evaluation reports, from Wisconsin and Missouri, are included below. The findings in both of these state evaluations reinforce and highlight much of what we are learning from the national evaluation of CSRD. Both state evaluations also provide richer detail of the challenges schools face in pursuing comprehensive school reform.
Missouri
Missouri's evaluation of eight CSRD schools that received grants for the 1998-99 school year examined baseline implementation issues. The Missouri CSRD schools are implementing Success for All, Accelerated Schools, Project Construction, Child Development Project, and Instruction and Learning Profile. The evaluation is finding that:
67
• Models appear to be chosen based on their responsiveness to current needs of the school and students and not generally based on their comprehensiveness. Most schools desired an instructional model that contained "sufficient scaffolding to inform classroom instruction on a daily basis and provide a framework for articulating instruction across grades."
• Regardless of the model implemented, teachers and school administrators uniformly reported that as a result of participating in their chosen model, they had become better observers, more critical thinkers, and spent more time reflecting on good teaching and learning.
• Forty-one percent of CSRD funds to schools in the state were used for professional development provided by external consultants. About a third of the funds were used for materials. Staff salaries were funded with approximately 15 percent of CSRD funds.
• CSRD schools in Missouri received from 6 to 23 days of assistance from external assistance providers during the school year.
• However, despite an early push on professional development activities, most schools did not have long-term professional development plans in place.
• Teachers had mixed feelings about external consultants. In general, teachers were highly supportive of external providers who had classroom teaching experience.
Wisconsin
Wisconsin also prepared a first-year evaluation report on CSRD implementation. Wisconsin's evaluation reflects both enthusiasm as well as some frustration with early CSRD reform efforts and describes CSRD schools in the state as "making strides toward achieving their goals." In Wisconsin:
• As of 1999, 12 of 21 schools were at the pilot stage of implementing models, 7 were implementing, and 2 were planning. Since schools had different goals and timelines, the evaluation judges whether schools were on schedule relative to achieving their goals. The Wisconsin evaluation found that a majority of the CSRD schools are on schedule.
Overall Program Implementation Progress in Wisconsin 14 12 10 8 6 4 2 0 no pl pi im fu t im an lot p lf nin i lem illi p ng ng lem g e nti en n ti g ng
68
• The Wisconsin evaluation found two particular factors associated with implementation progress. One factor is whether the schools believed they were well informed about the reform model prior to adopting it. A second factor important for implementation is the proportion of experienced teachers on the school staff. Only four of 21 CSRD "Teachers and administrators at each of the CSRD schools in the state felt that they were schools expressed a great deal of enthusiasm about not well informed about their models their comprehensive reform program. Some prior to selection. veteran teachers reported feeling more excitement about teaching now than at any other point in their careers. Teachers and administrators also • A majority of schools reported receiving conveyed a great deal of frustration. They report it strong support for implementing models. is a constant challenge of find the time to do all the Sixteen of 21 CSRD schools reported extra work their reform efforts require." receiving high quality technical assistance from model developers. Of -Wisconsin First Year CSRD Evaluation those schools reporting high quality technical assistance, six were at implementing stage and seven were piloting.
• Nineteen of 21 CSRD schools reported receiving "consistent support" from their districts for CSRD. However, schools did express some concerns. Two schools had concerns about how well their own reform program effort aligned with district initiatives. Two other schools reported that lack of substitute teachers in the district impeded professional development efforts.
4th Grade Student Achievement in Wisconsin
40 37
35 32 32 Wisconsin schools 30 28 CSRD 24 25 schools 21 20 20 20 W 15
10 point gain 1997-98 to 1998-99 4 5 3
0 reading enhanced language sumbajtehct science social studies
• Wisconsin also tracked the performance of 4th graders on the statewide assessment. In a year when students collectively scored considerably better than the previous year, CSRD school students made even larger gains. Wisconsin examined student outcomes for CSRD schools compared with Wisconsin schools as a whole and matched comparison schools with similar demographics from across the state. In the state overall, Wisconsin students scored
69
higher in 1998-99 than in 1997-98 in reading, math, and other subject areas; the gains in CSRD schools surpassed those of other Wisconsin schools in almost every subject.
Summary
Preliminary findings from school-level implementation of CSRD indicate both progress and challenges. CSRD schools appear to be attentive to both the needs of students and the match between models and school needs in selecting reforms under CSRD. For the most part, models and reform programs have been chosen by schools themselves, without imposition by districts administrators or others. Teachers have generally played a role in the selection process at the school level.
Most schools participating in the NLSS, as well as schools in state evaluations highlighted here, were at an early stage of CSRD implementation. Most are satisfied with the external technical assistance they are receiving. The data suggest that CSRD may be helping Title I schools to make more effective use of federal resources. CSRD schools are more likely than Title I schools to offer extended time programs, and seem to be taking more active steps to promote family involvement. Fewer CSRD schools than Title I schools are using pullouts and teacher aides for instruction, and fewer report challenges in coordinating federal funds with other funds.
The data reveal some areas of concern as well. According to the NLSS, a substantial number of teachers in CSRD schools have not yet received training (at least at the time of this data collection) in the model. The CSRD Reports from the Field highlight the need for CSRD schools to be more attentive to using data more effectively to develop strategies for addressing student needs. Site visits also reveal some concern with schools' attention to the comprehensiveness of reform efforts. CSRD involves more than the implementation of a particular model. Schools need to be thinking about all facets of school operations and all students in the schools. Building support for reform and sustaining that support is a constant challenge for CSRD schools. As the examples of state evaluations illustrate, staff and district turnover can be disruptive to reform efforts. Because CSRD involves external partners and requires alignment of efforts on several levels, hope can give way to frustration if there is not buy-in and cooperation across stakeholders.
70
Hacienda Heights Elementary School El Paso, Texas
Hacienda Heights is a CSRD elementary school in a high-poverty El Paso neighborhood. The school has 660 students, a high mobility rate, and a free and reduced lunch rate of 90 percent. Thirty-five percent of the students are English language learners. Many students live all or part of the year in nearby homeless shelters. Hacienda Heights operates as a schoolwide program under Title I.
The school was reconstituted after poor performance on the statewide TAAS Test in the early 1990s. Principal Marvyn Luckett came aboard shortly after the reconstitution. The school has now achieved an 85-100 percent pass rate on the 1999 TAAS test at various grade levels. It had a passing rate of nearly 100 percent on the third grade reading test. The challenge now is to move student achievement from the basic level to the mastery or proficiency level, in part, so students can gain access to professional and technological careers.
The school uses the El Paso Collaborative's Literacy Initiative as its basic reform model. The Collaborative is a project of the University of Texas El Paso (UTEP), in collaboration with the mayor, city council, and chamber of commerce, and all three school districts in the El Paso area. The Collaborative began in 1992 and works with Hacienda Heights and about 80 other schools. The Collaborative's approach has proven effective in significantly raising TAAS passing rates. CSRD funds support the school’s implementation of the Literacy Initiative as well as the Reading Renaissance program, which promotes literature-based reading. The principal selected the models and strategies in collaboration with the teachers, facilitator, and a mentor principal.
Hacienda Heights uses data to assess and forecast the academic performance of its students. Reading levels are assessed early in the school year. Students and teachers come to agreement on objectives to increase reading levels, and students are tested regularly to determine if they are achieving these objectives. Three people are primarily responsible for supporting school staff as they carry out these strategies. There is an on-site facilitator who works with teachers every day. There is an El Paso Collaborative professional developer who helps train the facilitator and frequently works with Hacienda Heights personnel on-site. Finally, there is a Reading Renaissance professional developer who also helps train the on-site facilitator and works with school personnel.
The school uses the Dual Language Immersion approach as its basic instructional format. Participants are approximately equal numbers of Spanish and English speakers. As children become proficient in English, the district provides ongoing instruction in Spanish language arts so that they can become fully fluent in both. Spanish speakers who score low on the Spanish TAAS test need more intensive Spanish instruction while they are learning English.
As part of its literacy initiative, the school employs Writer's Workshops, Book-of-the-Month Clubs, and the Literacy Leaders' Program for school-based facilitators. The principal and onsite facilitator ensure that the components of the school’s reform effort work in tandem. For example, all Hacienda Heights students read the same book in a given month. They also read and are tested on several library books that are slightly ahead of their current reading level. Essays, drawings, poems, printed pages, and other materials fill the walls, and the monthly book selection and other books are subjects of learning conversations in all classrooms.
71
VI. Emerging Research on Comprehensive School Reform and Models
As CSRD grows and the scope of comprehensive school reform continues to widen, emerging research is focusing more on the effectiveness of particular externally developed research-based approaches.
Analysis of Existing Research on Models
The American Institutes for Research (AIR), which conducted the analysis of existing research on models in An Educators’ Guide to Schoolwide Reform, has synthesized findings from across numerous studies of school reform models. Looking across the studies, AIR is finding that reform models that focus primarily on curriculum and instruction tend to have stronger effects on student achievement than models with a broader focus (on school philosophy, management, etc.).17
Evidence of Effects on Student Achievement by Reform Emphasis
70%
60% Governance (n=5)
50%
40% Curriculum (n=10)
30% Percent Effective 20% Instruction (n=8) 10% * Some reform models emphasize more than one focus. 0% Strong Promising Marginal Weak
Evidence of Effects
Overall, AIR found that models that emphasize instructional change were implemented at higher levels than other models that address curriculum change or reform of school governance or management. AIR also found that models that are more structured–directing teachers what or how to teach–had higher levels of implementation than philosophy-based models that provide a general approach and ask the school and teachers to operationalize that approach. Together the
17 See Appendix D for American Institutes for Research's rubric for classifying model types. The charts in this section of the report are taken from AIR's draft paper, "What We Know About Comprehensive School Reform" (draft, June 2000)
72
studies show that more than 80 percent of schools using structured reform models implemented their reforms well or adequately, while less than 40 percent of schools using philosophy-based reform models were implementing the models well.
Level of Implementation by Reform Emphasis
60%
50%
40%
Governance 30% phasis Curriculum m
E Instruction 20%
10% Percent of Schools for Each Reform
0% Good Adequate Poor Inadequate
Level of Implementation
Level of Implementation by Type of Reform
50%
45%
40%
35%
30% Structured 25% Philosophy-based 20%
15%
10%
5% Percent of Schools for Each Type Reform
0% Good Adequate Poor Inadequate Level of Implementation
73
Research Profiles
As part of the national evaluation of CSRD, the Department's Planning and Evaluation Service, in partnership with AEL and the National Clearinghouse on Comprehensive School Reform, are tracking promising ongoing research underway on models and comprehensive school reform.
The five research studies discussed below hold promise for enlarging the body of knowledge around comprehensive school reform. These studies are part of a larger group of studies being tracked by the Department. The entire list of studies are described in appendix G. Appendix G includes the following five studies plus formative evaluations on OERI's middle and high school model development grants. The following examples are research projects that will look at multiple reform models:
1) New American Schools 2) Study of Instructional Improvement 3) Scaling Up: Effects of Major National Restructuring Models in Diverse Communities 4) Longitudinal Research on Whole School Improvement 5) Formative Evaluation Process for School Improvement
New American Schools Purpose: To study the seven New American Schools design teams and their activities in different districts.
Researchers: Susan Bodilly and Mark Berends, RAND
Timeline: Assessment began in 1993; final publication due 2001. Model(s): Atlas, Audrey Cohen Purpose Centered Education, Co-NECT, Expeditionary Learning Outward Bound, Modern Red Schoolhouse, America's Choice, Roots and Wings
Research Questions: (1) Have the designs been implemented? (2) Have they affected classroom practice? (3) If not, why not? How can implementation be improved? (4) Have the designs impacted student performance? In what ways?
Methodology: Multiple methods including case studies, longitudinal tracking of outcomes on surveys, and district data.
Sample: Case studies of 40 schools. Schools in the outcome tracking study are located in Ohio, Florida, Kentucky, Tennessee, Pennsylvania, Texas, and Washington. The tracking study includes sample of 106 schools at all grade levels and uses surveys of over 2,000 teachers and 106 principals. The study uses district-mandated tests and other data to track performance gains.
74
Study of Instructional Improvement Purpose: To gain a deeper understanding of the processes of school improvement, to investigate the conditions under which school improvement efforts improve classroom teaching and student learning; and to examine how state and local policies assist or detract from school improvement initiatives. The goal in doing the research is to increase the American educator’s knowledge about school improvement, especially in schools serving America’s disadvantaged students, and thereby to improve educational policy and practice
Researchers: David K. Cohen, Brian Rowan, and Deborah L. Ball, Consortium for Policy Research in Education (CPRE), based at the University of Michigan
Timeline: Six years of data collection beginning in 2000-01 school year. Model(s): America’s Choice, Community for Learning, Success for All, and Accelerated Schools Program Research Questions: (1) How are different instructional interventions designed, and how do they operate in the complete environment of American public education? (2) What types of changes occur in schools that implement different instructional interventions, and how do such changes affect instruction? (3) What types of changes in instruction show particular promise for improving student learning?
Methodology: Multiple strategies including questionnaires, teacher log, observations, interviews, secondary analysis, and independent assessment of outcomes. Sample: 100 elementary schools in 8-10 geographic regions in the country with each school participating in one of four interventions mentioned above. Additionally, 25 elementary schools “matched” to the 100 in the sample but not involved in intervention programs.
75
Scaling Up: Effects of Major National Restructuring Models in Diverse Communities of Students At Risk Purpose: To assess the impact of externally developed school reform models in schools serving multilingual, multicultural populations.
Researchers: Amanda Datnow, Sam Stringfield, Johns Hopkins University; and Steven M. Ross, University of Memphis
Timeline: July 1996-July 2001 Model(s): Audrey Cohen College System of Education. Comer School Development Program, Core Knowledge, Coalition of Essential Schools, Modern Red Schoolhouse, and Success for All.
Research Questions: (1) How effective are various school restructuring models in improving the achievement of students in schools serving large numbers of language minority students in a multicultural context? (2) Are some of the current school restructuring models better suited to multilingual, multicultural contexts than others? Can the various reforms be successfully modified? (3) What actions at the federal, state, district, sub-district, and school levels increase (or decrease) the probability of obtaining full benefits from any or all of the restructuring models?
Methodology: Four-year longitudinal qualitative case studies of 13 schools with externally developed reform models combined with quasi-experimental design comparing achievement in these 13 reform schools with matched control schools. Limited data gathering in replication sites.
Sample: 13 elementary schools in one urban school district in a sunbelt state; 30 stepwise replication sites around the country.
76
Longitudinal Research on Whole School Improvement Through CRSD And Other Reform Efforts: Choice, Implementation, Institutionalization, and Systemic Support Purpose: To examine whole-school improvement through the Comprehensive Reform Demonstration Program (CSRD).
Principal Researchers: Amanda Datnow, Geoffrey Borman, and Sam Stringfield, Johns Hopkins University
Study Timeline: November 1999-November 2004 Research Questions: (1) Why and how do some external reform designs succeed at school improvement and others do not? (2) What is the process by which schools choose, successfully implement, and institutionalize whole-school reform designs? (3) What institutional factors (at the federal, design team, district, and school) levels facilitate or hinder the viability and effectiveness of external reforms? (4) What are the effects of whole- school reform designs on the achievement of students from low-income and racial minority backgrounds? Methodology: Longitudinal qualitative case studies combined with quasi-experimental design comparing achievement in 12 reform schools and demographically matched control schools.
Sample: 12 Title I schools (elementary, middle, and high) in 3 states; locations TBA.
77
FEPSI-Formative Evaluation Process for School Improvement Purpose: To evaluate the effectiveness of models. Researchers: Steven Ross, University of Memphis/AEL
Timeline: Fall 1999-Spring 2002 Model(s): Success for All, Direct Instruction, Roots and Wings, Early Literacy, Basic School, Sun Catchers, Core Knowledge, Community for Learning, Middle Schools that Work, High Schools that Work, Accelerated Schools, Modern Red Schoolhouse, and Co- NECT.
Research Questions: (1)What conditions are necessary at the building, community, and district levels to restructure schools through the implementation of a schoolwide model (climate, resources, priorities, innovative spirit, morale, professional development programs, faculty's commitment to improvement)? (2) What processes of internal review and self-study tools do schools utilize to adopt a particular school-wide model? (3) What changes or evidence result from the successful implementation of each model? (4) How does each model improve student achievement? (5) Which design components are associated with the greatest degree of success with different subgroups of at-risk students?
Methodology: Principal interview, teacher focus group, classroom observations and instruments
Sample: Four state regions; 101 schools
Summary
There is a great deal of research and evaluation activity underway on comprehensive school reform, and CSRD has done much to encourage these activities as well as help to shape the research and evaluation agenda. With support from the Department, AEL and the National Clearinghouse on Comprehensive School Reform hosted a second annual symposium of comprehensive school reform researchers in Summer 2000, featuring many of the researchers mentioned above and in appendix G. In partnership with a foundation, the Planning and Evaluation Service of the Department is convening a group of methodologists to help think through opportunities for rigorous research designs to evaluate model effectiveness and add to the available research base for CSRD. As part of the national evaluation of CSRD, the Planning and Evaluation Service will continue to track promising research and evaluation efforts that can help inform both policymakers and educators involved in the implementation of CSRD.
78
VII. Conclusion
This report has presented a great deal of data on the early implementation of CSRD on the federal, state, and local levels. While the program is still in its early stages, the information we have to date suggests several preliminary observations.
Early Implementation Findings and Challenges
The early implementation data on CSRD includes some promising trends:
• At the federal level, CSRD has pushed the U.S. Department of Education to be more innovative about how it provides information, technical assistance, and coordinates program activities. CSRD has spurred a great deal of activity in research, development, evaluation, and technical assistance on the part of the federal agency, Regional Educational Laboratories, Comprehensive Regional Assistance Centers, states, model developers, foundations, and others. Through these activities the Department is learning to be more responsive to the needs of states, districts, and schools and becoming more innovative in the role of helping schools improve student achievement.
• CSRD seems to be helping to catalyze some changes in how states think about and support school improvement efforts. The implementation of this federal program has been accompanied by a great deal of state activity. States very widely disseminated information about the program and about research-based approaches to school improvement and a majority sponsored workshops and "showcases" where school and district officials were brought together with model developers. A number of states used other federal funds, such as Goals 2000, to support CSRD planning and implementation, and a number more are now using CSRD, with its nine components, as a model for how they use state and Title I school improvement funds.
• State awards to schools appear to be consistent with the goals and purposes of the CSRD legislation to assist low-performing schools. CSRD is reaching a large proportion of schools with high rates of poverty and that have been identified as in need of improvement by Title I or other state or local mechanisms because of low-levels of student achievement.
• CSRD schools appear to be consciously taking the research base of models into account when making decisions about their reform programs. At the school level, the majority of CSRD schools are implementing nationally available models, including, but not limited to, models mentioned in the legislation.
• Comparisons of CSRD and Title I schools in the National Longitudinal Survey of Schools suggest that the explicit attention to faithful implementation of research-based strategies, external assistance, professional development, teacher buy-in and parent involvement in the CSRD legislation may be helping schools focus on these essential components to comprehensive school reform.
79
• Early findings from the National Longitudinal Survey of Schools also indicate that CSRD may be helping to leverage Title I funds to support the priorities of the Elementary and Secondary Education Act and to undertake strategies associated with successful schools. CSRD schools appear to have more sustained professional development experiences, offer more extended- time programs, and seem to be more active in fostering parent involvement.
Some of the initial implementation data on CSRD raise some concerns as well:
• At the time of data collection, a substantial percentage of teachers in the early sample of CSRD schools had not yet received professional development in the model being implemented. While this may be due to the early timing of data collection, or the phasing in of models over time, this is an issue that will need to be tracked carefully.
• Although a majority of schools are using nationally available school reform models, a large number are implementing locally-developed approaches for which we need more evidence on effectiveness.
• Some schools appear to consider the implementation of a model to be the sole purpose of CSRD. However, although CSRD funds are designed to help support the adoption of research-based models, ultimately CSRD schools need to be attentive to their entire school program and all of the components of comprehensive school reform. CSRD is not an "add- on" program. It is meant to help schools begin to rethink their entire school operations with reform models and their providers playing a central role.
• Without coordination, alignment of efforts, and consistency of support, CSRD cannot work at the school level. While most schools report satisfaction with the support they are receiving under CSRD, the early data reveal concerns with several issues including the amount of time schools have to engage in reform; turnover in staff and leadership; changes in district priorities; and the need for ongoing support from model developers.
Implications of Findings for Administration of CSRD
The early findings on CSRD suggest several lessons for the U.S. Department of Education, states, districts, and external assistance providers as each continues its role in implementing CSRD.
• In partnership with other stakeholders, the U.S. Department of Education needs to continue to expand the research-base on school reform models and increase the availability of information on the track records of specific models. In general, there exists a remarkable array of resources available to schools and districts that want to undertake comprehensive school reform. The Department can help direct states, districts, and schools to identify these resources and can help sponsor research and evaluation to improve the research base on reform models.
80
• The Department needs to continue efforts to help develop new models and increase capacity of model developers so that increasing numbers of schools will have access to high-quality designs and technical assistance as they pursue school improvement efforts.
• The Department, states, districts and assistance providers all need to continue to reinforce attention to the components of comprehensive school reform and stress that comprehensive school reform involves more than what any particular model or strategy can address alone.
• Districts and model developers need to make sure that all school staff affected by model implementation receive adequate training and professional development.
• States, models developers and other technical assistance providers need to stay engaged with CSRD schools, help solve problems, and hold schools accountable for active implementation and for results.
• CSRD schools must stay focused on the meaning of comprehensive school reform–research- based schoolwide change involving the coordination of all resources and activities in a coherent manner to improve student achievement for all students.
Looking Ahead
Because CSRD is at an early stage at the time of this report, the data presented here should be thought of as a baseline from which future reports and analyses will occur. The National Longitudinal Survey of Schools, from which much of the school level data in this report are taken, will be administered for three years, and a report will be made available on school level CSRD implementation after each of those years, with the first full report by fall 2000. As part of the national evaluation of CSRD, data will also be available in 2001 on the CSRD Field Focused Study of the implementation of CSRD in approximately 20 sites. The Planning and Evaluation Service is also designing a study to look at the effectiveness of particular reform interventions in CSRD and Title I schoolwide schools. As part of states’ annual reports to the Department on CSRD, the Department will be collecting, starting in December 2000, student achievement data from state assessments for all CSRD schools. Analysis of data trends in achievement will become part of the third-year evaluation report on CSRD. In addition to these activities, data collections, and reports, the Department will continue to track some of the most promising research on comprehensive school reform and evaluation activities by the states, labs, and others on CSRD.
81