Virtually the same: Using Bayesian methods to investigate the relationships between online course delivery and postsecondary student enrollment, course outcomes, and degree attainment By Benjamin Thomas Skinner Dissertation Submitted to the Faculty of the Graduate School of Vanderbilt University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY in Leadership and Policy Studies 11 August 2017 Nashville, Tennessee Approved: William R. Doyle, Ph.D. Angela Boatman, Ph.D. Carolyn J. Heinrich, Ph.D. Joshua D. Clinton, Ph.D. Copyright © 2017 by Benjamin Thomas Skinner All Rights Reserved ii To Katherine iii ACKNOWLEDGMENTS The work of this dissertation was supported in part by the Bonsal Applied Education Research Award. Some of the analyses were conducted using the resources of the Advanced Computing Center for Research and Education at Vanderbilt University, Nashville, TN. Any and all opinions, findings, conclusions, and recommendations are those of the author and do not necessarily reflect the views of the funder or ACCRE. I first would like to thank Dr. Will Doyle, who, as my advisor, dissertation committee chair, and research collaborator, has been a mentor throughout my doctoral program. I am indebted to his advice and most especially his willingness to give me space to cultivate the skills I needed to complete this dissertation. I also wish to thank the other members of my dissertation committee, Drs. Angela Boatman, Josh Clinton, and Carolyn Heinrich, for their feedback and guidance throughout this project. I am grateful to all the faculty at Vanderbilt from whom I have learned so much over the years. In particular, I want to thank Dr. Michael Rose. From my time as an undergraduate, Michael has been a mentor and tireless advocate on my behalf. I cannot thank him enough. I must also thank my fellow graduate students for their knowledge, assistance, and friendship. You are too many to name, but you have each helped this dissertation in your own way. A special thanks go to members of my cohort: Chris, Dominique, Ngaire, Richard, and Walker. We need not have become close, but I am a better researcher and, more importantly, a better person because we have. Lastly, I want to recognize the love and support of my family. Thank you to my mother, Billie, who shared with me her love of reading, my father, Tim, who taught me to take extra time if it meant doing it right, and my sisters, Sarah and Lyndsey, who have always championed my endeavors, academic or otherwise. Thank you also to two important women, my grandmother, Maxine, and my grandmother-in-law, Pearl. Though Maxine was not able to see the completion of this work, her and Pearl’s strong examples and unwavering support have meant more to me than they could ever know. Finally, I thank my wife, Katherine. She is the reason I had the courage to try something new when applying to my Ph.D. program, the reason I succeeded, and an ongoing example of all the good that hard work and kindness can accomplish. iv TABLE OF CONTENTS Page DEDICATION .............................................. iii ACKNOWLEDGMENTS ......................................... iv LIST OF TABLES ............................................ vii LIST OF FIGURES ............................................viii 1 Introduction ............................................... 1 1.1 Objective ............................................. 1 1.2 Research questions ........................................ 3 1.3 Conceptual framework ...................................... 3 1.4 Methods .............................................. 5 1.5 Results ............................................... 6 1.6 Contributions ........................................... 8 1.7 Structure of the dissertation .................................... 8 2 Estimating the Relationship between Broadband Access and Online Course Enrollments at Open Admissions Public Higher Education Institutions ........................... 9 2.1 Introduction ............................................ 9 2.2 Literature review ......................................... 11 2.2.1 Rise of online education .................................. 11 2.2.2 The digital divide: an overview .............................. 13 2.2.3 The digital divide in higher education ........................... 14 2.3 Theoretical framework ...................................... 15 2.4 Estimation strategy ........................................ 18 2.5 Data ................................................ 21 2.5.1 Institution data ....................................... 21 2.5.2 Broadband data ....................................... 22 2.5.3 Geographic and demographic data ............................. 26 2.6 Results ............................................... 27 2.6.1 Single-level models .................................... 28 2.6.2 Hierarchical models .................................... 31 2.7 Conclusion ............................................ 34 Appendices ............................................... 48 2.A Alternative specifications ..................................... 48 3 Estimating the Effect of Online Courses on Student Persistence and Likelihood of Passing in Geor- gia Public Postsecondary Institutions ................................. 55 3.1 Introduction ............................................ 55 3.2 Literature review ......................................... 56 3.2.1 Non-experimental evidence ................................ 57 v 3.2.2 Experimental evidence ................................... 58 3.2.3 Quasi-experimental evidence ............................... 60 3.3 Conceptual framework ...................................... 63 3.3.1 Online courses and human capital decisions ........................ 66 3.4 Methodology ........................................... 68 3.4.1 Model ........................................... 68 3.4.2 Identification ........................................ 70 3.4.3 Estimation ......................................... 72 3.5 Data ................................................ 75 3.5.1 University System of Georgia ............................... 75 3.5.2 National Broadband Map ................................. 78 3.6 Results ............................................... 80 3.7 Conclusion ............................................ 86 Appendices ............................................... 99 3.A Bivariate probit Stan script .................................... 99 3.B Formulas for treatment effects ..................................102 3.B.1 Average treatment effect (ATE) ..............................102 3.B.2 Treatment on the treated (TOT) ..............................102 3.B.3 Local average treatment effect (LATE) . 102 4 Predicting Degree Outcomes of Online Course Enrollment at the State Level using Bayesian Mul- tilevel Regression and Poststratification ................................104 4.1 Introduction ............................................104 4.2 Bayesian multilevel regression with poststratification . 105 4.2.1 Procedure ..........................................105 4.2.2 Extension ..........................................108 4.3 Model ...............................................110 4.4 Data ................................................113 4.4.1 Beginning Postsecondary Students Longitudinal Study, 2004/2009 . 113 4.4.2 American Community Survey ...............................114 4.4.3 State-level predictors: miscellaneous sources . 114 4.5 Results ...............................................115 4.5.1 Validation .........................................115 4.5.2 Bachelor’s degree attainment within six years . 117 4.5.3 Other degree outcomes ...................................120 4.5.3.1 Any degree attainment within six years . 120 4.5.3.2 Associate and any degree attainment within three years . 121 4.6 Forecasting ............................................122 4.7 Case study: Tennessee ......................................123 4.8 Conclusion ............................................126 5 Conclusion ...............................................139 5.1 Discussion of results .......................................139 5.2 Future research ..........................................142 REFERENCES .............................................144 vi LIST OF TABLES Table Page 2.1 Distance from student’s home (in miles) to NPSAS institution by dependency status . 37 2.2 Descriptive statistics of institution sample ........................... 38 2.3 Descriptive statistics of broadband measures ......................... 39 2.4 Single level Bayesian regressions of log number of students who enrolled in some distance education courses on broadband measures .......................... 40 2.5 Single level Bayesian regressions of log number of students who enrolled in some distance education courses on broadband measures: two year institutions . 41 2.6 Varying intercept Bayesian regressions of log number of students who enrolled in some distance education courses on broadband measures ...................... 42 2.7 Varying intercept Bayesian regressions of log number of students who enrolled in some distance education courses on broadband measures: two year institutions . 43 2.A.1 Single level Bayesian beta regressions of percentage of students who enrolled in some distance education courses on broadband measures ...................... 49 2.A.2 Single level Bayesian beta regressions of percentage of students who enrolled in some distance education courses on broadband measures: two year institutions