A Random Assignment Evaluation of a College and Career Coaching Program Sarah Burks Moore University of Arkansas, Fayetteville
Total Page:16
File Type:pdf, Size:1020Kb
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by ScholarWorks@UARK University of Arkansas, Fayetteville ScholarWorks@UARK Theses and Dissertations 7-2015 A Random Assignment Evaluation of a College and Career Coaching Program Sarah Burks Moore University of Arkansas, Fayetteville Follow this and additional works at: http://scholarworks.uark.edu/etd Part of the Education Policy Commons, and the Higher Education Commons Recommended Citation Moore, Sarah Burks, "A Random Assignment Evaluation of a College and Career Coaching Program" (2015). Theses and Dissertations. 1302. http://scholarworks.uark.edu/etd/1302 This Dissertation is brought to you for free and open access by ScholarWorks@UARK. It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of ScholarWorks@UARK. For more information, please contact [email protected]. A Random Assignment Evaluation of a College and Career Coaching Program A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Education Policy by Sarah Burks Moore Duke University Bachelor of Arts, 2009 July 2015 University of Arkansas This dissertation is approved for recommendation to the Graduate Council. _____________________________________ Dr. Gary Ritter Dissertation Director _____________________________________ _____________________________________ Dr. Patrick Wolf Dr. Kristin Higgins Committee Member Committee Member Abstract Razor C.O.A.C.H. (Creating Opportunities for Arkansan’s Career Hopes) is a college and career coaching program for at-risk students in 15 Northwest Arkansas high schools. To perform a random assignment evaluation, at-risk students were targeted to apply to the program, and applicant students were randomly assigned to the program. Academic coaches from the University of Arkansas College of Education and Health Professions utilized a needs-based intervention focusing on pro-academic behaviors, college or technical school preparation, and post-secondary and career exploration. The evaluation included two cohorts of students. Cohort One treatment students received the full intervention throughout the 2012-13 school year, and a follow-up intervention with lower dosage in the 2013-14 school year. Cohort Two treatment students received the full intervention in the 2013-14 school year, meeting with a coach at least once a week in individual or group meetings. To determine the impact of participation in Razor C.O.A.C.H., I examine the impact of the program on students’ academic, short-term non-cognitive, and post-secondary preparation outcomes. The main research questions are: 1. What is the impact of being assigned a coach on high school academic outcomes (high school GPA, core-subject GPA, credits earned, and ACT performance)? 2. What is the impact of being assigned a coach on short-term non-cognitive outcomes (academic self-efficacy, academic responsibility, grit, and future-mindedness) as measured during high school? 3. What is the impact of being assigned a coach on post-secondary preparation outcomes (college and career readiness outcomes, FAFSA and scholarship completion, and future plans)? The results from the evaluation suggest that the program is impacting students’ non- cognitive outcomes, as treatment students display higher levels of self-efficacy and responsibility in school and are preparing for post-secondary life more than the control students. In addition, students feel more accountable to adults at school; however, there is no evidence to suggest that the program is impacting students’ academic outcomes overall. These null results on academic measures are consistent with other evaluations of college and career coaching programs. In the future, it will be important to continue to examine the impact of the program by examining longer-term outcomes, including college attendance and graduation. Acknowledgements In this section, I would like to thank all the individuals who supported me with my dissertation and my graduate work. First, I would like to thank the Razor C.O.A.C.H. program for allowing the Office for Education Policy to evaluate the program. As director, Josh Raney has worked to ensure that Northwest Arkansas’ students received quality treatment through Razor C.O.A.C.H. I would like to thank Josh Raney for allowing me access to the coaches and students, collecting data, and supporting the end-of-year survey. In addition, I would like to thank Dr. Kristin Higgins for being instrumental in starting the program and preparing the coaches. Next, I would like to thank the Department of Education Reform at the University of Arkansas for providing an excellent program in education policy. I have grown tremendously as a scholar and practitioner, and I owe that to the professors and my colleagues in the Department. In particular, I would like to thank Dr. Gary Ritter for mentoring me through this project and many other projects. In addition, I would like to thank Dr. Patrick Wolf for providing a sound example of a high-quality program evaluation, through his courses and his work. Next, I would like to thank my colleagues in the Department of Education Reform for teaching and supporting me on a daily basis. In particular, I would like to thank Michael Crouch and Alexandra Boyd who worked on the Razor C.O.A.C.H. evaluation. Lastly, I would like to thank my family for their support and encouragement. Table of Contents Chapter 1 – Introduction ............................................................................................................. 1 The Issues.................................................................................................................................... 1 Razor C.O.A.C.H. ....................................................................................................................... 4 Random Assignment Evaluation................................................................................................. 6 Research Questions ..................................................................................................................... 7 Paper Organization...................................................................................................................... 7 Chapter 2 – Literature Review .................................................................................................... 9 Barriers to Post-Secondary Education ...................................................................................... 10 Review of research on college access and career preparation programs .................................. 14 Literature Review Process: Definition and Selection Criteria .............................................. 15 Literature Review Process: Application of Selection Criteria .............................................. 18 Literature Review Findings................................................................................................... 19 Chapter 3 – Overview of the Razor C.O.A.C.H. Program...................................................... 28 Background Information ........................................................................................................... 28 Description of the Schools ........................................................................................................ 29 Description of Targeted Students.............................................................................................. 29 Description of the Coaches ....................................................................................................... 31 Description of the Intervention ................................................................................................. 31 Chapter Conclusion ................................................................................................................... 35 Chapter 4 – Methods .................................................................................................................. 36 Research Questions ................................................................................................................... 36 Sample....................................................................................................................................... 37 Random Assignment Process ................................................................................................ 37 Overall Sample...................................................................................................................... 39 Survey Sample ...................................................................................................................... 49 Instrument ................................................................................................................................. 50 Analytic Methods ...................................................................................................................... 54 Research Question #1 ........................................................................................................... 54 Research Questions #2 and #3 .............................................................................................. 57 Chapter Conclusion ................................................................................................................... 58 Chapter 5 – Results ....................................................................................................................