Teens have the highest crash rate of any group in the United States. Evaluation of Beginner Driver Education Programs Studies in Manitoba and Oregon September 2014 607 14th Street, NW, Suite 201 | Washington, DC 20005 | AAAFoundation.org | 202-638-5944 Title Evaluation of Beginner Driver Education Programs: Studies in Manitoba and Oregon (September 2014) Authors Dan Mayhew, Kyla Marcoux, Katherine Wood, Herb Simpson, and Ward Vanlaar TRAFFIC INJURY RESEARCH FOUNDATION (TIRF) Larry Lonero and Kathryn Clinton NORTHPORT ASSOCIATES (NPA) Acknowledgments This study was made possible by financial support from the AAA Foundation for Traffic Safety, the Centers for Disease Control and Prevention, the National Highway Traffic Safety Administration, and Manitoba Public Insurance. The support of these agencies is gratefully acknowledged. Manitoba Public Insurance (MPI) also provided assistance in numerous other ways in the conduct of the study in Manitoba, including: administrative support for the survey of teens through high schools across the province; the contact information for the mail survey of teens on an Intermediate license and their parents; the Service Centers and office space for setting up and running teens on simulated drive tests; and, the capture and provision of road test results of teen applicants for an Intermediate license. In particular, the authors acknowledge MPI and its management/staff for their support and assistance throughout the course of this investigation. Special thanks are also given to the numerous high school principals and their staff, MPI driver education instructors, and MPI driver examiners for their support and assistance as well as to the teens and parents who participated in this investigation. Both the Transportation Safety Division (TSD) and the Department of Motor Vehicles (DMV) of the Oregon Department of Transportation, provided assistance in numerous ways in the conduct of the study in Oregon: they provided the legal authority and permission for TIRF to conduct the study on their behalf; DMV provided driver licensing and driver record information of teens participating in the study as well as of a larger sample of teens; and, TSD provided driver education information on teens who graduated from their program. In particular, the authors acknowledge TSD and DMV and its management/staff for their support and assistance throughout the course of this investigation. Special thanks are given to the parents for their consent and to the teens who completed the survey in Oregon. Finally, we acknowledge the guidance and assistance of members of our Stakeholder Panel and the group of ad-hoc research advisors who provided independent advice and consultation on deliverables and reports produced over the course of the investigation. The opinions, findings, and conclusions expressed in this report are those of the authors and do not necessarily reflect the views of the funding sponsors or any other agencies or reviewers involved in the report. About the Sponsor AAA Foundation for Traffic Safety 607 14th Street, NW, Suite 201 Washington, DC 20005 202-638-5944 www.aaafoundation.org Founded in 1947, the AAA Foundation in Washington, D.C. is a not-for-profit, publicly supported charitable research and education organization dedicated to saving lives by preventing traffic crashes and reducing injuries when crashes occur. Funding for this report was provided by voluntary contributions from AAA/CAA and their affiliated motor clubs, from individual members, from AAA-affiliated insurance companies, as well as from other organizations or sources. This publication is distributed by the AAA Foundation for Traffic Safety at no charge, as a public service. It may not be resold or used for commercial purposes without the explicit permission of the Foundation. It may, however, be copied in whole or in part and distributed for free via any medium, provided the AAA Foundation is given appropriate credit as the source of the material. The AAA Foundation for Traffic Safety assumes no liability for the use or misuse of any information, opinions, findings, conclusions, or recommendations contained in this report. If trade or manufacturer’s names are mentioned, it is only because they are considered essential to the object of this report and their mention should not be construed as an endorsement. The AAA Foundation for Traffic Safety does not endorse products or manufacturers. ©2014, AAA Foundation for Traffic Safety Table of Contents Executive Summary 4 Driver Education Programs in Manitoba and Oregon 5 Method 5 Key Findings 7 Implications 10 Conclusions for Driver Education 13 Final Note 18 Background 18 Project Purpose and Overview 19 Scope of This Report 19 Manitoba Public Insurance High School Driver Education 21 Program Overview 22 Administration, Setting, and Scope 22 Relation to the Graduated Driver License (GDL) Program 22 Goals and Objectives of Manitoba’s HSDE 23 Content 23 Delivery Methods 23 The New Driver Survey 23 Background 23 Objectives 24 The New Driver Questionnaire 25 Survey Method 27 Data Treatment 29 Data Analysis 30 Results 33 Longitudinal Comparisons 39 Summary 51 Student Outcomes: Survey of Teen Drivers and Their Parents 54 Background 54 Purpose and Design 54 Method 55 Data Treatment 57 Results: Teen Drivers 58 Results: Parents 65 Summary 71 Student Outcomes: Simulated Drive Test 72 Purpose 72 Evaluation Design 73 Simulated Driving Performance 75 Method 75 Data Treatment 80 Results – Error Rate Based on Driver Examiner Scoring 80 Results – Error Rate Based on Computer Scoring 86 Summary 90 Student Outcomes: Road Test Performance 91 Purpose 91 Road Tests 92 Method 93 Results 94 Summary 98 Summary and Discussion 98 New Driver Surveys: Waves 1 and 2 Results 99 Teen Driver and Parent Survey Results 101 Simulated Test Drive Results 102 Road Test Results 104 Oregon Driver Education 105 Program Overview 106 Administration, Setting, and Scope 106 Relation to Graduated Driver License (GDL) Program 107 Goals and Objectives 108 Content 108 The Oregon New Driver Survey 111 Background 111 Survey Method 111 Data Treatment 113 Data Analysis 115 Teen Attributes – Baseline Comparisons 118 Teen Attributes – Factors Associated with Driver Education 124 Wave 1 and Wave 2 Comparisons 127 Summary 142 Safety Performance: Survey Participants 145 Linking Survey Data with Driver Information 147 Final Sample 148 Statistical Analyses 149 License Test Results 150 Collisions 155 Convictions 159 Suspensions 161 Summary 165 Safety Performance: Historical Records 167 Method 167 Results 170 Summary 181 Safety Performance: The Influence of Confounding Factors 182 Summary and Discussion 185 New Driver Surveys: Waves 1 and 2 Results 185 Safety Performance: Survey Participants 187 Safety Performance: Historical Records 188 Safety Performance: The Influence of Confounding Factors 189 Discussion and Implications 190 Discussion 191 Key Findings 192 Implications 197 References 207 Appendices 214 Executive Summary Driver education is designed to teach new drivers the rules of the road and the driving skills to prepare for the road test and obtain a driver’s license. Most jurisdictions, in the United States, Canada, and elsewhere, deliver driver education in high schools and/or commercial driving schools. These programs typically consist of both “theoretical” instruction in the classroom and practical training in the vehicle. Although driver education provides an efficient means to learn how to drive, evaluations have failed to show that such formal programs produce safer drivers. Poor evaluation methods may be part of the reason for this, but even well-designed evaluations have produced findings that raise questions about the safety benefits of driver education. These evaluations have focused primarily on whether driver education reduces crashes and not whether it achieves other important program objectives, such as improving safe driving knowledge, attitudes, motivations, skills, and behaviors. Evaluation, however, is increasingly being viewed not simply as a tool for determining the success of the program in achieving its safety benefits, in terms of fewer crashes, but as a tool for determining the effectiveness of a program in achieving these other safety-related objectives. It also is a means of gathering intelligence about which aspects of programs are effective and which are not. This investigation was designed to apply this more comprehensive approach to the evaluation of driver education programs in the United States (Oregon) and Canada (Manitoba). More specifically, its objectives were to: Generate new knowledge about the safety and operational effectiveness of driver education; Provide new information about how to improve the delivery and content of driver education to enhance its safety impact; Demonstrate the implementation of the Comprehensive Guidelines for evaluating driver education developed by the AAA Foundation for Traffic Safety (AAAFTS); and, Showcase more effective and constructive methods to evaluate driver education. The purpose of the project was to demonstrate a more comprehensive evaluation and highlight how evaluations can better support program development in driver education. The project was not intended to provide a definitive answer to the question of whether driver education, in its current common forms, “works” or if it could “work” in some future form. In striving to achieve these important objectives, this investigation, on balance, produced some results that are favorable to the driver education