Fatigue and distraction detection A review of commercially available devices to detect fatigue and distraction in drivers R-2020-6 Authors Dr Frouke Hermens Prevent crashes Reduce injuries Save lives Report documentation Report: R-2020-6 Title: Fatigue and distraction detection Subtitle: A review of commercially available devices to detect fatigue and distraction in drivers Author(s): Dr Frouke Hermens Project leader: Dr Ragnhild J. Davidse Project number SWOV: E19.18 Contract ID: CW315832 Contractor: Shell Global Solutions International B.V., BP International Limited, Total S.A., and Chevron Services Company Contents of the project: A substantial portion of work-related deaths are due to road crashes during the course of work or on the way to and from work, and fatigue and distraction are known risk factors for such road crashes. To reduce the risks of fatigue and distraction, devices have therefore been developed to warn drivers before starting their journey or while driving. The present report, commissioned by Shell Global Solutions International B.V., BP International Limited, Total S.A., and Chevron Services Company, provides a detailed comparison of around 100 such technologies and systems, with the overall aim to provide a recommendation on which devices to consider for further testing or use. Number of pages: 161 Photographers: Paul Voorham (omslag) – Peter de Graaff (portret) Publisher: SWOV, The Hague, 2020 This publication contains public information. Reproduction is permitted with due acknowledgement. SWOV – Institute for Road Safety Research Bezuidenhoutseweg 62, 2594 AW Den Haag – PO Box 93113, 2509 AC The Hague +31 70 – 317 33 33 – [email protected] – www.swov.nl @swov / @swov_nl linkedin.com/company/swov Abstract Fatigue and distraction are important risk factors for road crashes, particularly in professions that involve driving long hours (such as drivers of trucks, buses and taxis). To mitigate the risks, systems have been developed to detect driver fatigue and distraction. The recent interest has led to an explosion of the number of devices currently on the market, or start-ups that aim for funding to develop their products and bring these to market. This study presents a review of the potential effectiveness of these systems. Previous reviews of such systems have either (1) focused on the scientific evidence underlying such systems, (2) provided an overview and classification without comparing systems, or (3) compared a relatively small number of systems. In response to a call from Shell Global Solutions International B.V., BP International Limited, Total S.A., and Chevron Services Company, this study extends this work by (1) considering a substantially larger number of devices, (2) comparing these devices on a broad range of safety and acceptability-related criteria, with the overall aim to (3) recommend a list of devices to further explore and compare in a field test. Due to extensive scientific evaluation, low cost, and high acceptability, fitness-for-duty tests (PVT, FIT and/or OSPAT) score best on the original set of criteria and could be recommended. These systems, however, do not monitor fatigue in real time. Therefore, fitness-for-duty tests should be compared to real-time systems, with the most promising candidates being dry EEG (Smart Cap and/or B-alert) and computer vision systems (Guardian, EyeSight, Stonkam, Mobileye, Streamax and Nauto). It is, however, not clear how acceptable these devices would be to drivers. Therefore, a broad field study is recommended, which may also include systems that combine activity tracking and computational modelling (Readiband and/or Cat smartband), Optalert (an established method using eyelid closure) and steering movements (Bosch). If the aim is to monitor fatigue and distraction in real-time, only computer visions that focus on the driver are plausible candidates (Guardian, Eyesight, Stonkam, Streamax and Nauto, from the above list), but drivers may find these devices difficult to accept. Overall, there appears to be no single perfect fatigue and distraction system that meets all requirements, and thus a combination of devices and methods may be needed. Title Fatigue and distraction detection Report R-2020-6 Page 4 Summary A substantial portion of work-related deaths are due to road crashes during the course of work or on the way to and from work, and fatigue and distraction are known risk factors for such road crashes. While fatigue management programmes (e.g., maximum driving and minimum resting hours) are at the core of preventing fatigue related crashes, not all crashes may be prevented with such traditional measures. To reduce the risks of fatigue and distraction, devices have therefore been developed to warn drivers before starting their journey or while driving. The present report, commissioned by Shell Global Solutions International B.V., BP International Limited, Total S.A., and Chevron Services Company, provides a detailed comparison of around 100 such technologies and systems, with the overall aim to provide a recommendation on which devices to consider for further testing or use. The first step in the comparison was the compilation of the list of devices. Devices were added that (1) were brought forward by the commissioners of this review, (2) were described in past reviews (mostly related to fatigue detection), (3) were found in an internet search for devices that specifically target distraction, or (4) were suggested by colleagues or suppliers. In a second step, devices were screened to determine (i) whether sufficient information was available to rate the device, (ii) whether the device was still on the market, and (iii) whether the device exclusively served to detect fatigue and / or distraction (or the consequences of fatigue and distraction) instead of a different general purpose (e.g., eye tracking) or a different purpose (e.g., illegal substance use). In a third and important step, devices were rated on eight criteria: validity, intrusiveness, availability, robustness, sustainability, acceptability, cost, and compatibility with other devices in the vehicle or used by the driver. It was also determined whether the device would be portable, detect fatigue and distraction or fatigue as a ‘stand-alone’ device, in a non-intrusive way, or whether it would involve wearing a sensor, whether it would provide real-time feedback, and what kind of feedback it would provide. To make these judgments various sources were used: (a) the website of the supplier, (b) the scientific literature (Google scholar search, past reviews), (c) online articles (e.g., blogs, online newspapers, news websites), (d) online videos ( from suppliers and users), (e) reviews from users on commercial websites and forums, and (f) direct contact with suppliers. While past reviews tended to classify devices into those that test fatigue before driving (fitness- for-duty tests), systems that monitor the driver, and systems that monitor driving performance, a more fine-grained distinction proved more appropriate in the present context. The present review therefore distinguishes devices that use (1) heart rate measurements, (2) head nodding, (3) EEG recordings, (4) measurements to test fitness-for-duty, (5) computer vision monitoring the road, (6) computer vision monitoring the driver, (7) computer vision monitoring both the road and the driver, (8) the closure of the eyelids (PERCLOS), (9) eye movements, (10) steering movements, (11) computational models to predict fatigue, (12) measurement of body temperature, (13) skin conductance, (14) video recordings and human analysis, (15) activity tracking in combination with a fatigue model. Based on the criteria, a subset of these systems was selected. Title Fatigue and distraction detection Report R-2020-6 Page 5 Judged strictly on the eight criteria outlined above, fitness-for-duty tests achieved the best scores, specifically for validity, cost, and acceptability. These tests, however, do not provide real- time feedback and only focus on fatigue. The only systems that detect distraction directly are the computer vision systems and eye trackers, which both use one or more cameras to monitor the driver’s face (which may lead to privacy concerns and, hence, acceptability issues). Computer vision systems have the advantage (over eye trackers) that they can also monitor for phone use, smoking, emotion (e.g., road rage), eating and drinking, which may also affect driving. Indirect measurements of distraction may be obtained from computer vision systems that monitor the road, and systems that monitor steering movements, but whether these systems can provide feedback with sufficient time left to prevent a crash, is unclear. Computer vision systems, however, suffer from a lack of scientific evidence of their validity and robustness, and suppliers are often hesitant to share their own test results, because of fierce competition in the market. If the dominant goal is to detect fatigue, dry EEG systems that can be embedded inside a cap may provide a suitable alternative, as they have been tested in the scientific literature and show good validity, and there are suggestions that these systems may be sufficiently comfortable. Systems that monitor for eyelid closure have also been tested extensively in the literature and show good validity, but may be outperformed by computer vision and eye tracking systems that can also measure distraction. If computer vision systems, EEG systems and eyelid closure systems are found to cause too many acceptability