Final Report Evaluation of the National Guard Youth ChalleNGe / Job ChalleNGe Program May 22, 2020 Jillian Berk, Ariella Kahn-Lang Spitzer, Jillian Stein, and Karen Needels (Mathematica) Christian Geckeler, Anne Paprocki, and Ivette Gutierrez (Social Policy Research Associates) Megan Millenky (MDRC) Submitted to: Submitted by: U.S. Department of Labor Mathematica Chief Evaluation Office 1100 First Street, NE 200 Constitution Ave, NW Suite 1200 Washington, DC 20210 Washington, DC 20002-4221 Project Officer: Jennifer Daley Phone: (202) 484-9220 Contract Number: DOLQ129633249/ Project Director: Jillian Berk DOL-OPS-15-U-00089 Reference Number: 50120.340 This page has been left blank for double-sided copying. Job ChalleNGe Final Report Mathematica Contents EXECUTIVE SUMMARY .............................................................................................................................. xi I. INTRODUCTION ................................................................................................................................... 1 A. The Youth ChalleNGe program model and Job ChalleNGe expansion ........................................ 1 1. Youth ChalleNGe program .................................................................................................... 1 2. Evidence on the effectiveness of Youth ChalleNGe .............................................................. 2 3. Job ChalleNGe grants ........................................................................................................... 3 B. The Job ChalleNGe Evaluation ..................................................................................................... 5 1. Study enrollment .................................................................................................................... 5 2. Data collection ....................................................................................................................... 6 3. Analysis ................................................................................................................................. 8 II. IMPLEMENTATION OF YOUTH CHALLENGE UNDER THE JOB CHALLENGE GRANT ................................................................................................................................................ 11 A. Defining court involvement .......................................................................................................... 12 B. Recruitment and enrollment ........................................................................................................ 13 C. YC services under the Job ChalleNGe grant .............................................................................. 14 D. Snapshot of Youth ChalleNGe participants ................................................................................. 15 E. Youth ChalleNGe program completion ........................................................................................ 19 III. JOB CHALLENGE RECRUITMENT AND PARTICIPANT CHARACTERISTICS .............................. 21 A. Recruitment, application, and selection....................................................................................... 21 1. Recruitment ......................................................................................................................... 21 2. Application ........................................................................................................................... 22 3. Selection .............................................................................................................................. 23 B. Challenges in tracking Job ChalleNGe data ............................................................................... 24 C. Snapshot of Job ChalleNGe participants .................................................................................... 24 IV. JOB CHALLENGE RESIDENTIAL CULTURE AND PROGRAM STAFF ........................................... 29 A. Job ChalleNGe housing............................................................................................................... 29 B. Staffing Job ChalleNGe to meet participants’ needs ................................................................... 31 C. An evolving approach to discipline .............................................................................................. 33 1. Initial tone of “structured independence” ............................................................................. 34 2. Move toward increased discipline or “earned freedom” ...................................................... 34 3. Infractions and punishments ................................................................................................ 35 iii Job ChalleNGe Final Report Mathematica V. JOB CHALLENGE PROGRAM SERVICES........................................................................................ 37 A. Overview of Job ChalleNGe services .......................................................................................... 37 B. Education and occupational skills training................................................................................... 38 C. Employment services .................................................................................................................. 45 D. Non-academic support services .................................................................................................. 47 E. Post-residential services.............................................................................................................. 49 F. Job ChalleNGe completion .......................................................................................................... 50 VI. JOB CHALLENGE PARTICIPANT OUTCOMES ................................................................................ 53 A. Job ChalleNGe participant outcomes .......................................................................................... 53 1. Post-secondary education ................................................................................................... 54 2. Employment ......................................................................................................................... 59 3. Military ................................................................................................................................. 64 4. Any productive outcome ...................................................................................................... 65 5. Case studies ........................................................................................................................ 66 6. Youth perceptions of what they would have done in the absence of Job ChalleNGe ........................................................................................................................... 69 7. Criminal justice outcomes .................................................................................................... 70 B. Comparison to Youth ChalleNGe participant outcomes .............................................................. 72 1. Comparison of Youth ChalleNGe and Job ChalleNGe youth under the Job ChalleNGe grants ................................................................................................................ 72 2. Comparison to prior literature .............................................................................................. 74 VII. LESSONS FROM THE JOB CHALLENGE EVALUATION ................................................................. 79 A. Findings on program implementation .......................................................................................... 79 1. Increasing access to Youth ChalleNGe for court-involved youth ........................................ 79 2. Implementation of Job ChalleNGe program ........................................................................ 80 B. Findings on Job ChalleNGe participant outcomes ...................................................................... 81 C. Considerations for the future ....................................................................................................... 82 REFERENCES ............................................................................................................................................ 83 APPENDIX A: JOB CHALLENGE PROGRAM PROFILES ...................................................................... 85 APPENDIX B: QUANTITATIVE DATA SOURCES .................................................................................. 91 APPENDIX C: ADJUSTMENTS FOR NONCONSENT AND NONRESPONSE .................................... 103 APPENDIX D: REGRESSION ANALYSES OF POST-PROGRAM OUTCOMES ................................. 117 APPENDIX E: GRANTEE-LEVEL RESULTS ......................................................................................... 123 iv Job ChalleNGe Final Report Mathematica APPENDIX F: SURVEY ESTIMATES OF CRIMINAL JUSTICE OUTCOMES ..................................... 133 APPENDIX G: PARTICIPANT OUTCOMES USING GRANTEE-DEFINED MEASURES OF COURT INVOLVEMENT .......................................................................................... 137 v This page has been left blank for double-sided copying. Job ChalleNGe Final Report Mathematica Exhibits I.1 JC grantees ...................................................................................................................................... 3 I.2 YC and JC typical program timeline ................................................................................................