eRider™ Field Test Final Report July 3, 2014 Revised October 10, 2014 Revised March 25, 2015 Prepared By Pat Hahn, TEAM OREGON Communications Manager For TEAM OREGON Motorcycle Safety Program TEAM OREGON eRider™ Field Test Final Report Table of Contents 1. Introduction .......................................................................................................................................... 4 2. Objective ............................................................................................................................................... 4 3. Executive Report ................................................................................................................................... 4 4. Background ........................................................................................................................................... 6 A. TEAM OREGON ................................................................................................................................. 6 Mandatory Training .............................................................................................................................. 6 B. TEAM OREGON Basic Rider Training Curriculum 1984-Present ....................................................... 8 5. eRider™ vs. BRT Classroom ................................................................................................................... 9 A. eRider™ Development and Curriculum Overview ............................................................................ 9 B. Curriculum Template ........................................................................................................................ 9 C. Curriculum Development Process ................................................................................................... 10 D. Proof-of-Concept (Chapters 1-2b) Field Test Results ..................................................................... 13 E. Curriculum Overview ...................................................................................................................... 14 Learning Objectives ............................................................................................................................. 14 6. The eRider™ Field Test ........................................................................................................................ 15 A. Design and Modifications ............................................................................................................... 15 B. Marketing ........................................................................................................................................ 19 C. Registration and Troubleshooting .................................................................................................. 19 D. Data Collection and Analysis ........................................................................................................... 21 Statistical Analysis ............................................................................................................................... 21 Curriculum Comparison ...................................................................................................................... 22 E. Survey Results ................................................................................................................................. 22 Student End-of-Course Evaluations .................................................................................................... 22 Student Post-then-Pre-Course Surveys............................................................................................... 26 Survey Results – Conclusions .............................................................................................................. 28 F. Student and Instructor Feedback.................................................................................................... 28 Online Student Comments .................................................................................................................. 28 Instructor Surveys ............................................................................................................................... 30 2 Copyright © 2014 TEAM OREGON Motorcycle Safety Program. Do not reproduce without permission. TEAM OREGON eRider™ Field Test Final Report G. Pass Rates, Knowledge and Skill Test Results ................................................................................. 31 Field Tests 1 and 2 ............................................................................................................................... 31 Demographics ..................................................................................................................................... 32 Pass Rates............................................................................................................................................ 33 BRT Knowledge Test (BRT and ONB) ................................................................................................... 35 Online Quizzes (ONB and ONI) ............................................................................................................ 36 Skill Test (BRT, ONB, IRT, ONI) ............................................................................................................ 38 Pass Rates, Knowledge and Skill Test Results – Conclusions .............................................................. 38 7. Conclusions and Recommendations ................................................................................................... 39 Conclusions ............................................................................................................................................. 39 Recommendations .................................................................................................................................. 41 8. List of Appendices ............................................................................................................................... 41 3 Copyright © 2014 TEAM OREGON Motorcycle Safety Program. Do not reproduce without permission. TEAM OREGON eRider™ Field Test Final Report 1. Introduction The purpose of this report is to provide detail on the background and development of the eRider™ online classroom, its field test design and protocol, the field test results and recommendations. 2. Objective The objective of the field test was to compare the eRider™ online classroom to the Basic Rider Training (BRT) classroom to determine its suitability as an approved classroom option for TEAM OREGON beginning and intermediate rider training students. The field test also measured how well each curriculum addresses the National Highway Traffic Safety Administration (NHTSA) Model National Standards for Entry-Level Motorcycle Rider Training and TEAM OREGON priority areas. 3. Executive Report TEAM OREGON has a strong history of developing effective motorcycle rider training curricula, designing the BRT classroom in 2003 and BRT range in 2004 – both are approved by Oregon Department of Transportation (ODOT) for a motorcycle endorsement testing waiver. TEAM OREGON developed Intermediate Rider Training (IRT) shortly thereafter, which is also a course approved by ODOT for a skill test waiver. TEAM OREGON also developed Advanced Rider Training (ART), Police ART, Police High-Speed and Rider Skills Practice (RSP) to meet the various needs of Oregon motorcycle riders. In 2010-2014, to better meet the needs of Oregon riders and to address the newly published Model National Standards for Entry Level Rider Training, TEAM OREGON developed and field tested the eRider™ online classroom. eRider™ is not a redesign of any existing curricula – it was created brand new from the ground up. The curriculum topics are informed by the National Standards, with extra emphasis placed on TEAM OREGON priority areas. As part of the eRider™ field test, TEAM OREGON conducted an analysis and comparison of eRider™ and BRT curricula to measure how well each addresses the National Standards. In the final tally, eRider™ exceeds BRT – the National Standards are addressed 633 times by eRider™ and 538 times by BRT. When BRT range exercises are taken into account, eRider™ addresses every standard in the National Standards. The eRider™ curriculum focuses heavily on eight curriculum priority areas based on needs of Oregon riders: 4 Copyright © 2014 TEAM OREGON Motorcycle Safety Program. Do not reproduce without permission. TEAM OREGON eRider™ Field Test Final Report • Risk awareness • Riding strategy • Scanning • Cornering and vehicle control skills • Judgment • Impairment • Riding gear • Group riding The comparison of eRider™ and BRT to the National Standards also demonstrated that eRider™ exceeds BRT in addressing TEAM OREGON curriculum priority areas by a margin of 34-50 percent. TEAM OREGON conducted two eRider™ field tests to evaluate effectiveness. The first showed slightly lower pass rates and knowledge test scores, so the curriculum was modified for a second field test. The second field test showed no difference in pass rates and test scores. This report combines the results of both field tests. The field test design focused on student outcomes, comparing the online classroom and range to those of the traditional classroom and range. In all, 1,086 basic and intermediate students participated in the field test – 440 in the beta group and 646 in the control group.