The External Review of Job Corps: An Evidence Scan Report March 2018 Jillian Berk Linda Rosenberg Lindsay Cattell Johanna Lacoe Lindsay Fox Myley Dang Elizabeth Brown Submitted to: U.S. Department of Labor Chief Evaluation Office 200 Constitution Ave., NW Washington, DC 20210 Project Officer: Jessica Lohmann Contract Number: DOLQ129633249/DOL-OPS-16-U-00122 Submitted by: Mathematica Policy Research 1100 1st Street, NE 12th Floor Washington, DC 20002-4221 Telephone: (202) 484-9220 Facsimile: (202) 863-1763 Project Director: Jillian Berk Reference Number: 50290.01.440.478.001 DISCLAIMER: THIS REPORT WAS PREPARED FOR THE U.S. DEPARTMENT OF LABOR (DOL), CHIEF EVALUATION OFFICE, BY MATHEMATICA POLICY RESEARCH, UNDER CONTRACT NUMBER DOLQ129633249/DOL-OPS-16-U- 00122. THE VIEWS EXPRESSED ARE THOSE OF THE AUTHORS AND SHOULD NOT BE ATTRIBUTED TO DOL, NOR DOES MENTION OF TRADE NAMES, COMMERCIAL PRODUCTS, OR ORGANIZATIONS IMPLY ENDORSEMENT OF SAME BY THE U.S. GOVERNMENT. JOB CORPS EXTERNAL REVIEW MATHEMATICA POLICY RESEARCH ACKNOWLEDGMENTS We would like to thank the many people whose cooperation and efforts made the Job Corps External Review possible. In particular, we would like to thank staff at the U.S. Department of Labor who supported this study. Our Project Officer Jessica Lohmann at the Chief Evaluation Office guided this study. Molly Irwin, the Chief Evaluation Officer, was the External Review’s original project officer and provided input throughout the project. The study’s two reports—the Evidence Scan and Future Research reports—benefited from their careful review. We are indebted to staff at the National Office of Job Corps, including Lenita Jacobs-Simmons, Robert Pitulej, Marcus Gray, Erin Fitzgerald, Felicia Charlot, and Edward Benton, who attended in- person meetings, provided helpful comments on draft reports, and participated in interviews. Thanks to the Expert Working Group for their helpful comments and suggestions, which provided the foundation for the Evidence Scan Report. In addition, Monique Miles from the Aspen Institute and Mamadou Ndiaye from Jobs for the Future provided helpful comments on a draft of the Evidence Scan Report. Thank you also to Dan Bloom from MDRC who provided helpful comments on the Future Research Report. We also thank the other members of the Job Corps External Review team. Decision Information Resources, Inc. was an invaluable partner in this project. Thanks to Russell Jackson for helping to facilitate the Expert Working Group meeting and commenting on multiple report drafts and Julie Gloudemans for data analysis. Several other Mathematica staff made important contributions to this effort. Thanks to Peter Schochet, a co-principal investigator on this project, who led the work on the Future Research Report. Our quality assurance reviewers, Annalisa Mastri and Ken Fortson, provided helpful comments on drafts of the Evidence Scan and the Future Research reports. Casey Ellis provided excellent project management. Courtney Powers provided research assistance. William Rafferty helped us track down research sources. We also thank the multiple dedicated staff that provided high-quality editing and production support for both reports. iii This page has been left blank for double-sided copying. JOB CORPS EXTERNAL REVIEW MATHEMATICA POLICY RESEARCH CONTENTS I. INTRODUCTION .............................................................................................................................. 1 A. The Job Corps program ............................................................................................................. 1 1. Administrative or operational structure ............................................................................... 1 2. Participant characteristics ................................................................................................... 4 3. Program components .......................................................................................................... 5 4. Behavior and discipline policy ............................................................................................. 8 5. Performance monitoring ...................................................................................................... 9 B. National Job Corps Study ........................................................................................................ 10 C. The Evidence Scan report ....................................................................................................... 11 PART I: YOUTH SERVICES ....................................................................................................................... 13 II. ENGAGING DISCONNECTED YOUTH ........................................................................................ 15 A. Cultivating positive youth development ................................................................................... 15 1. Theorized elements required for successful PYD programs ............................................ 16 2. Evidence for PYD programmatic elements or strategies .................................................. 16 B. Communicating high expectations for students ....................................................................... 18 1. Theory behind the effect of expectations on student performance ................................... 18 2. Evidence of the effect of expectations on student performance ....................................... 19 C. Fostering growth mindsets ...................................................................................................... 21 1. Theory behind growth mindsets’ effects on student performance .................................... 21 2. Evidence for improving performance through growth mindsets ....................................... 22 D. Creating a trauma-informed environment ................................................................................ 23 1. The effects of trauma on youth ......................................................................................... 24 2. Evidence on trauma-informed therapies and approaches ................................................ 24 E. Summary ................................................................................................................................. 27 III. CREATING A SAFE AND SUPPORTIVE ENVIRONMENT .......................................................... 29 A. Positive behavioral interventions and supports ....................................................................... 30 1. Core elements of PBIS ..................................................................................................... 30 2. Evidence of the effect of PBIS on student safety and behavior ........................................ 32 B. Restorative practices ............................................................................................................... 32 1. Elements of school-based restorative practices ............................................................... 33 2. Evidence of the effect of restorative practices on student safety and behavior................ 33 C. Building self-regulation skills ................................................................................................... 34 v JOB CORPS EXTERNAL REVIEW MATHEMATICA POLICY RESEARCH 1. Self-regulation skills for youth ........................................................................................... 35 2. Evidence of the effect of self-regulation skills on behavior ............................................... 35 D. Summary ................................................................................................................................. 37 IV. PREPARING YOUTH FOR CAREERS ......................................................................................... 39 A. Career pathways programs ..................................................................................................... 40 B. Micro-credentials ..................................................................................................................... 42 C. Work-based learning and work experience ............................................................................. 43 D. Apprenticeship and pre-apprenticeship programs .................................................................. 45 E. Summary ................................................................................................................................. 46 V. ORGANIZING YOUTH SERVICES ............................................................................................... 49 A. Age of participants ................................................................................................................... 50 B. Gender of participants ............................................................................................................. 51 C. Residential environment .......................................................................................................... 52 D. Learning in groups of different sizes ....................................................................................... 53 E. Enrollment practices ................................................................................................................ 55 F. Summary ................................................................................................................................. 58 PART II: PROGRAM OPERATIONS .........................................................................................................