Technology-Based Learning (TBL) in Workforce Development and Education: A Review of the Research Literature Contract #GS10F0086K September 2014 Prepared for: US Department of Labor, Employment and Training Administration 200 Constitution Ave, NW Washington, DC 20210 Submitted by: Abt Associates 6130 Executive Boulevard Rockville, MD 20852 MEF Associates 99 Canal Center Plaza Suite 305 Alexandria, VA 22314 This work has been funded, either wholly or in part, with Federal funds from the Department of Labor, Employment & Training Administration under contract number GS10F0086K. The contents of this publication do not necessarily reflect the views or policies of the Department of Labor, nor does mention of trade names, commercial products, or organizations imply endorsement of same by the U.S. Government. Suggested citation: Gan, Katherine N., Glen Schneider, Zachary Epstein, and Alex Silverman. 2014. Technology-Based Learning (TBL) in Workforce Development and Education: A Review of the Research Literature. Prepared for the U.S. Department of Labor. Cambridge, MA: Abt Associates Inc. TABLE OF CONTENTS Table of Contents Acknowledgements ..................................................................................................................................... ii List of Acronyms and Abbreviations ....................................................................................................... iii Executive Summary ................................................................................................................................... iv 1. Introduction .................................................................................................................................... 1 1.1 Organization of the Paper ....................................................................................................... 2 1.2 Methodology .......................................................................................................................... 3 2. Technology-Based Learning: Basic Concepts, Strengths, and Limitations .............................. 5 2.1 Review of TBL Concepts ....................................................................................................... 5 2.1.1 A Continuum of Technology Use ............................................................................. 5 2.1.2 Advanced Forms of TBL .......................................................................................... 7 2.2 Overview of Strengths and Limitations of Using TBL .......................................................... 8 2.2.1 Monitoring and Assessing Learner Engagement Remotely ...................................... 9 2.2.2 Adapting Learning to Individuals’ Needs ................................................................. 9 2.2.3 Promoting Interpersonal Interaction ....................................................................... 10 2.2.4 Saving Costs and Resources ................................................................................... 11 2.2.5 Strengthening Access and Readiness ...................................................................... 12 2.3 Summary .............................................................................................................................. 12 3. Considerations for Using and Providing TBL ........................................................................... 14 3.1 Learner Considerations ........................................................................................................ 14 3.1.1 Basic Skills .............................................................................................................. 14 3.1.2 Technological Access and Readiness ...................................................................... 16 3.1.3 Psychological Factors ............................................................................................. 18 3.1.4 Support for Engagement in TBL ............................................................................. 19 3.2 Provider Considerations ....................................................................................................... 20 3.2.1 Developer Considerations ....................................................................................... 20 3.2.2 Procurement Considerations ................................................................................... 21 3.2.3 Implementer Considerations ................................................................................... 22 3.2.4 Employer Considerations ........................................................................................ 23 3.3 Summary .............................................................................................................................. 24 4. Current TBL Use in Workforce Services, Training, and Education ...................................... 26 4.1 Workforce System Program and Services ............................................................................ 26 4.1.1 ETA Support for TBL ............................................................................................. 27 4.1.2 The Use of TBL Within Public Workforce Programs............................................. 27 Abt Associates and MEF Associates TBL in Workforce Development and Education ▌pg. i TABLE OF CONTENTS 4.2 Career and Technical, Adult, and Developmental Education .............................................. 28 4.2.1 Career and Technical Education ............................................................................. 28 4.2.2 Adult Education ...................................................................................................... 29 4.2.3 Developmental Education ....................................................................................... 30 4.2.4 ED’s TBL Initiatives and Guidance ........................................................................ 30 4.2.5 Prevalence and Use of TBL in CTE and Developmental Education ...................... 32 4.2.6 Prevalence and Use of TBL Within Adult Education ............................................. 37 4.3 Summary .............................................................................................................................. 38 5. Empirical Evidence on the Use of TBL for Improving Workforce Outcomes ....................... 39 5.1 The Effectiveness of Technology-only Interventions for Improving Workforce-related Outcomes ............................................................................................................................. 40 5.1.1 Asynchronous, Technology-only Interventions ...................................................... 41 5.1.2 Design Features of Promising Asynchronous, Technology-only Interventions ...... 42 5.1.3 Synchronous, Technology-only Interventions ........................................................ 44 5.1.4 Design Features of Promising Synchronous, Technology-only Interventions ........ 46 5.2 The Effectiveness of Blended Interventions for Improving Workforce-related Outcomes ............................................................................................................................. 47 5.2.1 Design Features of Promising Synchronous, Blended Interventions ...................... 52 5.3 Summary .............................................................................................................................. 55 6. Recent and Emerging Trends in TBL ........................................................................................ 56 6.1 Improving Access To and Scalability of Learning ............................................................... 56 6.2 Individualization Through New Instructional Approaches and Data Analytics .................. 59 6.3 Individual Engagement Through Entertainment and Active, Exploratory Learning ........... 61 6.4 Facilitating Social Learning ................................................................................................. 63 6.5 Utilizing Data ....................................................................................................................... 66 6.6 Summary .............................................................................................................................. 68 7. Conclusion .................................................................................................................................... 69 7.1 Key Findings ........................................................................................................................ 69 7.2 Design Challenges ................................................................................................................ 71 7.3 Research and Measurement Challenges ............................................................................... 72 7.4 The Next Step ....................................................................................................................... 73 Works Cited ............................................................................................................................................... 75 Appendix A. Documentation of Literature Search Strategy ............................................................. 91 Abt Associates and MEF Associates TBL in Workforce Development and Education ▌pg. ii LIST OF EXHIBITS List of Exhibits Exhibit 1: Continuum of Technology Use .............................................................................................. 5 Exhibit 2: Highest Educational