Community Access to Technology Program Evaluation Report—Year 3 March 2008 Study Conducted by MGS Consulting for the Community Access to Technology Program of the Bill & Melinda Gates Foundation For more information, or to hear a presentation on these findings, contact Maureen MacCarthy, Principal Consultant for MGS Consulting, at 206.760.1051 or via email: [email protected]. Executive Summary............................................................................................................................................................ 4 Developmental Assets................................................................................................................................................... 4 Key Findings.............................................................................................................................................................. 4 Technical Fluency .......................................................................................................................................................... 6 Key Findings.............................................................................................................................................................. 6 Outlook for the Future ................................................................................................................................................. 7 Key Findings.............................................................................................................................................................. 7 Characteristics of Effective Programs........................................................................................................................ 8 Introduction......................................................................................................................................................................... 9 The Community Access to Technology (CAT) Program........................................................................................ 9 The Multi-Year Evaluation of the CAT Program .................................................................................................... 9 About this Report .......................................................................................................................................................... 9 I. Developmental Assets..................................................................................................................................................11 Summary........................................................................................................................................................................11 Key Findings............................................................................................................................................................11 Indicators of Developmental Assets.........................................................................................................................12 Results: DAP Score Categories and Averages.........................................................................................................14 Average DAP Scores..............................................................................................................................................14 Increases in DAP Score Category ........................................................................................................................15 Effect of Time in Program and Frequency of Attendance ...................................................................................17 All Participants―Time in Program ......................................................................................................................18 Matched Set—Time in Program ..........................................................................................................................20 All Participants—Hours per Week ......................................................................................................................24 Implications .............................................................................................................................................................25 Effect of Grade Level, Academic Achievement, Age, and Gender.....................................................................25 DAP Scores by Grade in School..........................................................................................................................25 DAP Scores and Academic Achievement...........................................................................................................26 Effect of Age (Teen vs. Preteen)..........................................................................................................................26 Effect of Gender (Male vs. Female) ....................................................................................................................28 Effect of Program Type (High-Touch vs. Low-Touch)...................................................................................29 Implications .............................................................................................................................................................29 II. Technical Fluency........................................................................................................................................................31 Summary........................................................................................................................................................................31 Key Findings............................................................................................................................................................32 Indicators of Technical Fluency ................................................................................................................................33 Results: Tech Fluency by Category ...........................................................................................................................34 Effect of Time in Program.........................................................................................................................................36 Results for All Participants....................................................................................................................................36 Results for Matched Set.........................................................................................................................................38 Implications .............................................................................................................................................................39 Results: Tech Fluency and DAP Scores...................................................................................................................40 Effect of Academic Achievement, Age, Gender, and Program Type.................................................................41 Effect of Academic Achievement........................................................................................................................41 Effect of Age (Teen vs. Preteen)..........................................................................................................................41 Effect of Gender (Male vs. Female) ....................................................................................................................42 Effect of Program Type (High- vs. Low-Touch) ..............................................................................................43 Implications .............................................................................................................................................................44 CAT Program Evaluation 2 MGS Consulting Services www.mgs-us.com III. Outlook for the Future .............................................................................................................................................45 Summary........................................................................................................................................................................45 Key Findings............................................................................................................................................................45 Indicators of Future Outlook ....................................................................................................................................47 Results: School Performance and Participation......................................................................................................48 High-Touch vs. Low-Touch .................................................................................................................................49 Results: Job Skills and Readiness...............................................................................................................................49 Effect of Length of Time in a Program ...................................................................................................................52 Effect of Age (Preteen vs. Teen)...............................................................................................................................53 Effect of Gender (Male vs. Female) .........................................................................................................................53