Communities In Schools National Evaluation Volume 2: Case Studies Prepared by: ICF International 9300 Lee Highway Fairfax, VA 22031 (703) 934‐3000 www.icfi.com October 2008 Table of Contents 1. Introduction............................................................................................................... 3 1.1 Communities In Schools..................................................................................... 3 1.2 The CIS National Evaluation.............................................................................. 3 1.3 Case Studies........................................................................................................ 4 2. Methodology .............................................................................................................. 4 2.1 Case Study Overview.......................................................................................... 4 2.2 Selection Process................................................................................................ 5 2.3 Audiences Targeted for Interviews and Focus Groups....................................... 9 2.4 Topics Covered by the Interviews and Focus Groups with Key Stakeholders. 10 3. Findings.................................................................................................................... 11 3.1 What is CIS? ..................................................................................................... 11 3.2 What are Common Components of the CIS Model across Affiliates? ............. 11 3.3 What are the Benefits of CIS? .......................................................................... 15 3.4 What are Challenges and Barriers to CIS Success?.......................................... 17 3.5 What Factors Contribute to CIS Success? ........................................................ 18 4. CIS Moving Forward.............................................................................................. 23 4.1 Future Vision.................................................................................................... 23 4.2 Making the CIS Vision a Reality...................................................................... 23 4.3 Advice to Others Implementing CIS................................................................. 24 Appendix A: Protocols and Questionnaires ................................................................ 25 Appendix B: Individual Case Study Reports .............................................................. 76 2 1. Introduction 1.1 Communities In Schools Communities In Schools is the nation’s largest dropout prevention organization. For more than 30 years, CIS has connected needed community resources with schools. By bringing caring adults and a wide variety of services into schools to address children's unmet needs, CIS provides a link between educators and the community. The result: Teachers are free to teach, and students – many in jeopardy of dropping out – have the opportunity to focus on learning. Communities In Schools has worked to ensure that every child has access to the "Five Basics": A one-on-one relationship with a caring adult A safe place to learn and grow A healthy start and a healthy future A marketable skill to use upon graduation A chance to give back to peers and community Each year, more than two million young people in 27 states and the District of Columbia have access to integrated student support services through Communities In Schools. 1.2 The CIS National Evaluation In May 2005, The Atlantic Philanthropies funded the first three years of a comprehensive national evaluation of CIS. The evaluation, designed and conducted by ICF International can be envisioned as a three-level pyramid, as shown in Exhibit 1: Base-level studies provide for the identification of Network-wide findings. These studies will provide CIS National with a more in-depth understanding of the strategies used throughout the Network to help students in need. This Exhibit 1: National Evaluation component also provides for the development of strategies to ensure that CIS remains an organization dedicated to evidence-based practices. Mid-level studies provide for the identification of CIS’s effects at the school- level. These studies will help discover not only how much of an impact CIS is having, but also how and why those impacts are being achieved. 3 Top-level studies provide for the identification of CIS’s effects at the student- level. Two experimental studies are currently underway in Austin, Texas and Jacksonville, Florida. As depicted in Exhibit 1, as we move up the pyramid, fewer sites are involved in each study, but the studies become more rigorous. The power in this design lies in the fact that each level of the pyramid informs the other, and by bringing the results of all studies together, we can gain a comprehensive understanding of how CIS is impacting students, schools, and communities. 1.3 Case Studies This report focuses on results from the final school-level study depicted in the mid-level of the evaluation design—the case studies. While the other school-level studies focused on what impact CIS is having at the school level, the case studies were conducted to gather information about how CIS is changing students’ lives. 1 2. Methodology 2.1 Case Study Overview In May 2008, the evaluation team conducted multi-day site visits to eight CIS affiliates and selected schools served by each affiliate in order to obtain in-depth, descriptive information from key stakeholders to help answer the following questions: How do local CIS programs describe the CIS model? What are the processes and activities that characterize the CIS model as implemented at the local program level? What are best practices at the local program level for supporting effective programming intended to help youth learn, stay in school, and prepare for life? What are the best strategies for a national organization, state office, and local affiliate to promote and support effective local programming? What are important lessons learned (what works, what does not work, and why) from local affiliates and schools that can be shared with the field? How effective have the National Office and state offices been in conducting activities to support local affiliates and local programming? 1 The results of the quasi-experimental, natural variation, and typology studies are presented in the Communities In Schools National Evaluation School Level Report. 4 What improvements are needed to the CIS Network at the national, state, and local levels? Findings from these case studies provide valuable information for CIS to encourage the adoption of common factors and strategies that have been identified to have a positive effect on schools and students. 2.2 Selection Process The case studies involved the study of eight affiliates and up to three schools served by each affiliate. The selection process proceeded according to the following steps and is shown in Exhibit 2: 1. First, CIS high schools included in the quasi-experimental study were targeted because of the focus of CIS as a dropout prevention program. It was important to “get into the high schools” to see how CIS was impacting dropout. Within the CIS high schools, those defined as high implementing sites were selected. According to the typology developed for the quasi- experimental study, high implementers were those sites that reported implementing 70 percent of identified processes that were determined to be central to the CIS model. These processes include: needs assessment, planning, service delivery, and monitoring and adjustment.2 2. Next, high performing high schools from among the high implementers were identified. High performers were defined as those CIS schools that showed improvements in relation to their comparison schools on the following measures: promotion power, graduation rates, and average effect size across multiple outcomes. Promotion power and graduation rates were selected given the focus of CIS on dropout prevention. Additionally, because CIS is a multi-faceted program, it was important to study schools or sites recording success on a number of dimensions (academic and behavioral). 3. Once the high implementing/high performing high schools were identified, we looked for high implementing/high performing middle and/or elementary schools within each affiliate. The same typology was applied to determine high implementers, when data were available. For high performers, emphasis was placed on academics and behavioral outcomes (where available). 4. With the high implementing/high performing schools identified, the next step was to ensure diversity across the sample. Specifically, we examined the urbanicity of the affiliates serving the schools (urban, rural, and suburban), the state’s represented by the affiliates (Texas, Florida, Georgia, North Carolina, Pennsylvania, Michigan, and Washington), and the program focus of the 2 Adherence to the CIS model was gauged using responses to the Critical Processes Survey and the Site Coordinator Survey, and the core processes identified were aligned to CIS’s Total Quality System (TQS) whenever possible. 5 schools (academic, behavioral, and mix of whole-school and case-managed services). 5. Next, the identified affiliates and schools were reviewed by each CIS State Director. A list of possible case study affiliates and identified high implementing/high performing schools (high, middle, and elementary, when available) was shared with each CIS State Director. In some cases,