DOCUMENT RESUME CG 028 150 ED 412 487 Hahn, Karen J.; AUTHOR Mitchell, Stephanie J.; Gabriel, Roy M.; Laws, Katherine E. Chrysalis: Year 2 Evaluation TITLE Portland Public Schools Project Report. and Evaluation Dept.; INSTITUTION Portland Public Schools, OR. Research RMC Research Corp., Portland, OR. PUB DATE 1996-12-00 NOTE 198p. PUB TYPE Reports Evaluative (142) EDRS PRICE MF01/PC08 Plus Postage. High Risk Students; High DESCRIPTORS Adolescents; Child Abuse; *Females; Schools; *Intervention; *Mental Health;Program Effectiveness; Program EValuation; SchoolCounseling; *Sexual Abuse; *Substance Abuse IDENTIFIERS Portland School District OR ABSTRACT In 1994, the Chrysalis Projectin Portland Public Schools received funding to prevent or delaythe onset of substance abuse among a of special target population: high-risk,female adolescents with a history of childhood abuse. Findings from theevaluation of the project's second year providing assistance to these students arereported here. During the 1995-1996 school year, the projectserved 370 young women in grades 9through and 2 alternative 12. The program operates in all10 traditional high schools schools in the district. Theevaluation reported here assesses the effectiveness of the specific programintervention strategies. It also the documents the process of servicedelivery and program implementation at The key schools to help interpret and givecontext to the project outcomes. relationships findings of the outcome evaluationindicate several significant students who among different healthrisk behavior areas (i.e., Chrysalis attended more support groups reportedluvicr rates of marijuana use inthe and past month). The results showrelationships among a history of abuse increased use of alcohol and other drugs,sexual behaviors, violence-related behaviors, and suicide ideation in young women. (RJM) ******************************************************************************** Reproductions supplied by EDRS are thebest that can be made from the original document. ******************************************************************************** U.S. DEPARTMENT OF EDUCATION Office of Educational Research and improvement EDUCATIONAL RESOURCES INFORMATION CENTER (ERIC) 0 This document has been reproduced as received from the person or organization originating it O Minor changes have been made to improve reproduction quality Points of view or opinions stated in this docu ment do not necessarily represent official Poragend Pub[lric Schoco[16 OERI position or policy .501 Monh Dimon PoAlland9 0$ 97227 "PERMISSION TO REPRODUCE THIS MATERIAL HAS BEEN GRANTED BY nA-c-p ANC Res:such Cogporeadn 522 Southuyesa Flah.Auso9StIoRe `W`? TO THE EDUCATIONAL RESOURCES P4CEBTA 9720., INFORMATION CENTER (ERIC)." Portland Public Schools Project Chrysalis Year 2 Evaluation Report Stephanie J. Mitchell Research and Evaluation Portland Public Schools Roy M. Gabriel Karen J. Hahn Katherine E. Laws RMC Research Corporation December 1996 3 Acknowledgements We would like to acknowledge the help of many people in making this evaluation possible.It is no simple task to capture the diversity, excitement, and hard work generated by the individuals involved in Project Chrysalis. We especially appreciate the following individuals' efforts to make this a more accurate and useful evaluation: Project Chrysalis Case Managers and Students: We appreciate the time taken by the Chrysalis case managers and student participants to share with us their experiences, their [earnings, and their ideas about what still needs to be done. Center for Substance Abuse Prevention (CSAP) staff and associates: We want to thank our Project Officer, Patricia Sabry, for providing us with information about CSAP's policies and perspectives on evaluation. We also thank CSAP evaluation contractors and consultants: Elizabeth Harris of EMT Associates, Marian James of the CDM Group, and Faye Bellgrave of George Washington University for offering their comments and suggestions on the program evaluation. CSAP is an agency of the Substance Abuse and Mental Health Services Administration, Department of Health and Human Services. Project Chrysalis Staff: A number of people were involved in various phases of this evaluation. Marilyn Richen 'sensitively led the project management team. Stevie Newcomer and Joanne McClarty worked to ensure an accurate linkage between program activities and the evaluation. Larry Marcy provided us with the backbone of the evaluation, its random assignment procedure. Gayle Waiwaiole provided invaluable support in evaluating the supplementary grant activities. We appreciate the unique talents of Karen Bond and Mike Shadder who assisted with data entry and copy-editing of the report. Questions about the content of the evaluation report can be addressed to the following authors: Stephanie J. Mitchell Roy M. Gabriel, Ph.D. Portland Public Schools Evaluation Department RMC Research Corporation P.O. Box 3107 522 S.W. Fifth Avenue, Suite 1407 Portland, OR 97208-3107 Portland, OR 97208 Page i Table of Contents Page List of Exhibits EXECUTIVE SUMMARY vii CHAPTER 1. OVERVIEW OF THE INTERVENTION 1 Statement of the Problem 1 Brief Description of the Intervention 2 Theoretical Underpinnings Linking the Intervention to Desired Outcomes 2 Hypotheses Relating Intervention Activities to Measured Change 3 CHAPTER 2. IMPLEMENTATION SUMMARY 5 Program Description 5 Supplementary Grant Activities 8 Training for School Staff 11 Linkage to National Cross-Site Evaluation 12 Dosage of the Intervention 12 CHAPTER 3. EVALUATION METHODOLOGY 19 Sampling Design/Approach 19 Research Design 20 Proposed and Actual Sample Size 22 Properties and Characteristics of Operationalized Dependent Measures 23 Methods/Procedures 28 Data Analysis 30 CHAPTER 4. RESULTS 35 Analysis of Initial Comparability of Treatment and Control Groups 35 Attrition/Attrition Analysis 39 Statistical Analysis for Key Dependent Measures 40 Health Risk Behaviors 42 Protective Factors 59 Intercorrelations 66 Student Achievement 71 Process Evaluation 73 Supplementary Research Activities 75 Case Study 83 CHAPTER 5. COST-BENEFIT DATA AND ANALYSIS 91 CHAPTER 6. CONCLUSIONS AND LESSONS LEARNED 95 Project Management 96 Project Evaluation 97 Project Implementation 100 Summary 102 REFERENCES 103 Page iii 5 Table of Contents (continued) APPENDIX A.Support Group Curriculum Outline 107 APPENDIX B.The Validity Study of Self-Reported Drug Use 111 APPENDIX C.Evaluation of HIV/AIDS Education Supplementary Grant 131 APPENDIX D.Evaluation Chrysalis Media Literacy Supplementary Grant 143 APPENDIX E.Evaluation Instruments 157 APPENDIX F.Evaluation Summary of Other Chrysalis Components 191 J Page iv List of Exhibits Page Exhibit 1.1 Chrysalis Logic Model 4 Exhibit 2.1 Comparison of Ideal vs Actual Program Implementation, Year 2 13 Exhibit 2.2 Chrysalis Program Dosage by Component, Year 2 14 Exhibit 2.3 Chrysalis Implementation Across Schools and Program, Year 2 15 Exhibit 2.4 Case Management Activities, Year 2 17 Exhibit 3.1 Evaluation Design: Number of Participants in the Treatment Group by Year. 20 Exhibit 3.2 Case Study Sample, Year 2 22 Exhibit 3.3 Reliability Coefficients of IPFI from National Field Test & Current Application 26 Exhibit 3.4 Summary of Data Collection Procedures, Year 2 29 Exhibit 3.5 Repeated Measures ANOVA Design, Year 2 31 Exhibit 4.1 Age of Female Adolescents, Year 2 35 Exhibit 4.2 Grade Level of Female Adolescents, Year 2 36 Exhibit 4.3 Geocultural Group of Female Adolescents, Year 2 37 Exhibit 4.4 Prevalence of Top 12 Risk Factors, Year 2 38 Exhibit 4.5 Student History by Type of Abuse, Year 2 39 Exhibit 4.6 Needs Assessment Summary, Year 1 and Year 2 41 Exhibit 4.7a Prevalence of Alcohol Use, Year 2 42 Exhibit 4.7b Repeated Measures ANOVA for Alcohol Use 43 Exhibit 4.8a Prevalence of Smoking Cigarettes, Year 2 45 Exhibit 4.8b Repeated Measures ANOVA for Cigarette Use 45 Exhibit 4.9a Prevalence of Marijuana Use, Year 2 47 Exhibit 4.9b Repeated Measures ANOVA for Marijuana Use 47 Exhibit 4.10 Relationship Between Risk Level and Marijuana Use 49 Exhibit 4.11aPrevalence of Sexual Activity, Year 2 50 Exhibit 4.11bRepeated Measures ANOVA for Sexual Activity 51 Exhibit 4.12aConsequences of Sexual Activity, Year 2 53 Exhibit 4.12bRepeated Measures ANOVA for Consequences of Sexual Activity 53 Exhibit 4.13aPrevalence of Suicidal Behavior, Year 2 55 Exhibit 4.13bRepeated Measures ANOVA for Suicidal Behavior 55 Exhibit 4.14aPrevalence of Fighting and Weapon Carrying, Year 2 57 Exhibit 4.14bRepeated Measures ANOVA for Fighting and Weapon Carrying 58 Page v 7 List of Exhibits (continued) Exhibit 4.15 Repeated Measures ANOVA by Grade Level for Personal Competence 60 Exhibit 4.16 Repeated Measures ANOVA by Grade Level for Social Competence 61 Exhibit 4.17 Repeated Measures ANOVA by Grade. Level for Social Bonding 62 Exhibit 4.18 Repeated Measures ANOVA by Grade Level for Support/Guidance 63 Exhibit 4.19 Repeated Measures ANOVA by Risk Level for Personal Competence 64 Exhibit 4.20 Repeated Measures ANOVA by Risk Level for Social Competence 65 Exhibit 4.21 Repeated Measures ANOVA by Risk Level for Social Bonding 65 Exhibit 4.22 Repeated Measures ANOVA by Risk Level for Support/Guidance 66 Exhibit 4.23aIntercorrelations Among Health Risk Behaviors and Program Activities 68 Exhibit 4.23bIntercorrelations Among Health Risk Behaviors and Risk & Dosage Levels 69 Exhibit 4.24 Intercorrelations Among Protective Factors and Program Activities And Among Risk and Dosage Levels 70 Exhibit 4.25 Comparison of Academic