National Guard Youth ChalleNGe Program Progress in 2017–2018 Louay Constant, Jennie W. Wenger, Linda Cottrell, Wing Yi Chan, Kathryn Edwards C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RR2907 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-1-9774-0250-9 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2019 RAND Corporation R® is a registered trademark. Cover: U.S. Air National Guard photo by Master Sgt. Becky Vanshur. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface The National Guard Youth ChalleNGe program is a residential, quasi-military pro- gram for youth ages 16 to 18 who are experiencing difficulty in traditional high school. This report covers the program years 2017–2018 and is the third in a series of annual reports that RAND researchers will issue over the course of a research project spanning September 2016 to June 2020. The first and second National Guard Youth ChalleNGe Annual Reports cover program years 2015–2016 and 2016–2017, respectively, and can be found on the RAND website (Wenger et al., 2017; Wenger, Constant, and Cottrell, 2018). A fourth annual report, which will cover program years 2018–2019, will be released early in fiscal year 2020. Each annual report documents the progress of participants who entered Chal- leNGe during specific program years and then completed the program. A focus of RAND’s ongoing analysis of the ChalleNGe program is collecting data in a consistent manner. Based on these data, each report also includes trend analyses. In the present report, we provide information in support of the National Guard Youth ChalleNGe Program’s required annual report to Congress. In addition to information on par- ticipants who entered the ChalleNGe program and completed it in 2017, we include follow-up information on those who entered the program and completed it in 2016, as well as an update on our work to develop metrics to improve program effectiveness. Finally, we describe other ongoing research efforts to support the ChalleNGe program (these efforts will be detailed in future reports). Methods used in this study include site visits, collection and analysis of quantitative and qualitative data, literature reviews, and development of tools to assist in improving all program metrics—for example, a program logic model. Caveats to be considered include documented inconsistencies in reported data across sites, our focus on those who completed the program and not all participants, and the short-run nature of many of the metrics reported here. This report will be of interest to ChalleNGe program staff, personnel providing oversight for the program, and policymakers and researchers concerned with designing effective youth programs or determining appropriate metrics by which to track prog- ress in youth programs. This research was sponsored by the Office of the Assistant Secretary of Defense for Manpower and Reserve Affairs and conducted within the Forces and Resources iii iv National Guard Youth ChalleNGe: Program Progress in 2017–2018 Policy Center of the RAND National Defense Research Institute, a federally funded research and development center sponsored by the Office of the Secretary of Defense, the Joint Staff, the Unified Combatant Commands, the Navy, the Marine Corps, the defense agencies, and the defense Intelligence Community. For more informa- tion on the RAND Forces and Resources Policy Center, see http://www.rand.org/nsrd/ ndri/centers/frp.html or contact the director (contact information is provided on the webpage). Contents Preface ................................................................................................. iii Figures .................................................................................................vii Tables .................................................................................................. ix Summary .............................................................................................xiii Acknowledgments ................................................................................ xxiii Abbreviations ....................................................................................... xxv CHAPTER ONE Introduction .......................................................................................... 1 The ChalleNGe Model ................................................................................ 3 Focus of This Report and Methodology ............................................................ 6 Organization of This Report ......................................................................... 9 CHAPTER TWO Data and Analyses: 2017 ChalleNGe Classes...................................................11 Cross-Site Metrics for the 2017 Classes ............................................................12 Tests of Adult Basic Education Scores ............................................................. 42 Staffing ................................................................................................ 50 Program Completion .................................................................................51 Placement ..............................................................................................57 Time Trends, 2015–2017 .............................................................................59 Summary .............................................................................................. 60 CHAPTER THREE Assisting ChalleNGe Through Benchmarking Youth Outcomes and Research on Mentoring ....................................................................................63 Logic Model .......................................................................................... 64 Site Visits ...............................................................................................67 Other Analytic Efforts ................................................................................69 Summary ...............................................................................................79 v vi National Guard Youth ChalleNGe: Program Progress in 2017–2018 CHAPTER FOUR Concluding Thoughts ...............................................................................81 APPENDIX Site-Specific Information ..........................................................................83 References ........................................................................................... 147 Figures 2.1. Cadet Scores on TABE Total Battery at Beginning and End of ChalleNGe Residential Phase ........................................................................ 44 2.2. Cadet Scores on TABE Subject Tests at Beginning and End of ChalleNGe Residential Phase ........................................................................ 44 2.3. Cadet Scores on TABE Battery Tests at Beginning and End of ChalleNGe Residential Phase, by Credentials Awarded .......................................... 46 2.4. Admissions Factors ......................................................................47 2.5. Policies to Encourage Positive Behavior .............................................. 48 2.6. Factors Taken into Consideration When Placing Cadets in Platoons ............ 48 2.7. Activities and Ceremonies Provided to Cadets .......................................49 2.8. Job Placement Supports Provided to Cadets......................................... 50 2.9. College Placement Supports Provided to Cadets .....................................51 2.10. Number of Sites with at Least 25 Percent of Staff Hired in the Last 12 Months: Total Staff, Recruiters.....................................................52 2.11. Number of Sites with at Least 25 Percent of Staff Hired in the Last 12 Months: Instructors, Case Managers ..............................................52 2.12. Number of Sites with at Least 25 Percent of Staff Hired in the Last 12 Months: Cadre, Administrators ....................................................53 2.13. Graduation Rates, by Cadet Characteristics ......................................... 54 2.14. Graduation Rates, by Credentials Awarded ..........................................55 2.15. Graduation Rates, by Cadre Turnover ............................................... 56 2.16. Cadet Placements in Months 1, 6, and 12 After Graduation.......................58 2.17. Trends in Cadets, Graduates, and TABE Battery Scores over Time.............. 60 2.18. Trends in Graduation