ISSN: 1402-1544 ISBN 978-91-7439-XXX-X Se i listan och fyll i siffror där kryssen är DOCTORAL T H E SI S Department of Business Administration, Technology and Social Sciences and Effects on Attention, Performance in a Camilla DrivingGrane Context Haptic Perception, Division of Human Work Sciences Haptic Perception, Attention, and Effects on Performance in a Driving Context ISSN: 1402-1544 ISBN 978-91-7439-490-0 Use of a Haptic Rotary Device in a Menu Selection Task Luleå University of Technology 2012 Camilla Grane HAPTIC PERCEPTION, ATTENTION, AND EFFECTS ON PERFORMANCE IN A DRIVING CONTEXT USE OF HAPTIC ROTARY DEVICE IN A VISUAL MENU SELECTION TASK CAMILLA GRANE LULEÅ UNIVERSITY OF TECHNOLOGY Department of Business Administration, Technology and Social Sciences Division of Human Work Sciences Engineering Psychology Printed by Universitetstryckeriet, Luleå 2012 ISSN: 1402-1544 ISBN 978-91-7439-490-0 Luleå 2012 www.ltu.se Till Love och Tuva med kärlek PREFACE This thesis work was part of research collaboration between Luleå University of Technology and Volvo Car Corporation partly financed by Vinnova PFF and the EFESOS project. I have always had the car industry, and specifically Volvo Car Corporation, in mind during this thesis work. One of my main goals has been to provide the industry with information that could be used to develop more usable and safer in-vehicle interfaces for the driver. Therefore, it is important to me to write this thesis with a broad group of readers in mind, researchers within my field as well as HMI designers and developers in the car industry. To this end, I have chosen to write this thesis in a somewhat more descriptive way so as to make my research more accessible and useful to a broad audience. ACKNOWLEDGEMENTS There are many people who made this work and my journey possible. It has been a challenging trip, most enjoyable sometimes but also very hard at other times. All of you who have witnessed this journey know how many laughs, sweat, and tears these pages include. I am most grateful for all your support throughout this journey. Specifically, I would like to thank my supervisor Peter Bengtsson. I will always remember your support and understanding through the hardest moments, both at work as well as when my son came early. I am also glad I had the best project members by my side. Our meetings and trips always gave me inspiration and energy. Robert Broström, thank you for believing in me and giving me this opportunity. Annie Rydström, my dearest friend and companion since 1997, I am so glad I could do this trip with you by my side. I also want to give a special thanks to Håkan Alm, Kjell Rask, Lena Abrahamsson and Jan Johansson for believing in me and giving me a chance to develop further. I want to thank Elisabeth Berg specifically for your advice and support. Finally, thanks to all my colleges at TP, my office mates, PhD-mates, lunch date companions, and friends. Thank you all for being there. My deepest thanks go to my extended family and to Johan, Love and Tuva specifically. You are truly the ones that made this possible. You have seen all sides of this trip and always stayed there sharing moments of joy and sorrow and bringing me down to earth, when needed. Your endless time, love and support are a large part of this thesis. ABSTRACT In-vehicle driving interfaces have become increasingly complex with added secondary task functions designed to make driving enjoyable and comfortable. A single display solution in combination with a haptic rotary device has the potential to reduce the clutter of buttons. With well-designed haptic information (i.e., cues that could be explored by touch), drivers should be able to find and select functions without taking their eyes off the road. However, when this thesis work started, few studies focused on the effects of adding haptic information to secondary tasks in cars. Clearly, research is needed that examines how adding more information to secondary tasks supports or distracts drivers. This thesis investigates haptic perception, attention, and effects on secondary tasks and driving performance for an interaction menu selection interface controlled by an in-vehicle haptic rotary device. The research questions addressed how and why performance would be affected by added haptic information. The causes of selective attention in a visual and haptic menu selection task were also investigated. Three experimental studies complemented with interviews and questionnaires were performed. Two of the studies included a simulated driving task. It could be concluded that an addition of haptic information to a visual menu selection interface could increase secondary task performance and were preferred with respect to usability issues. However, more complex haptic additions could also confuse the driver. This result depends on the context and differed between persons. From a driving performance perspective, both visual and cognitive demand affected the driving, but differently. These effects were less pronounced when both visual and haptic information was provided. Selective attention to haptic information seemed to be an effect of lacking expectations. By simply mentioning the haptic information before the test, a driver would pay closer attention to the haptic information. This result implies that drivers might learn to use more flexible and informative multimodal interfaces in the future if the interfaces emphasize and communicate the haptic cues. This implication would be interesting to study further. In addition, future studies may apply these results to more ecologically valid driving situations. APPENDED PAPERS Paper I Grane, C., & Bengtsson, P. (2005). Menu selection based on haptic and/or graphic information. In G. Salvendy (Ed.), Human Computer International 2005, [CD-ROM]. USA: Erlbaum. Paper II Grane, C., & Bengtsson, P. (2008). Serial or parallel search with a multi-modal rotary device for in-vehicle use. In W. Karwowski & G. Salvendy (Eds.), Applied Human Factors and Ergonomics Conference 2008, [CD-ROM]. USA: USA Publishing. Paper III Grane, C., & Bengtsson, P. (2012). Haptic addition to a visual menu selection interface controlled by an in-vehicle rotary device. Advances in Human-Computer Interaction, 2012, 12 pages. doi:10.1155/2012/787469 Paper IV Grane, C., & Bengtsson, P. (resubmitted 2012). Driving performance during visual and haptic menu selection with in-vehicle rotary device. Manuscript resubmitted for publication. Paper V Grane, C., & Bengtsson, P. (submitted 2012). Selective haptic attention and selective use of haptic information when interacting with visual-haptic interface during simulated driving. Manuscript submitted for publication. TABLE OF CONTENTS 1 Introduction ............................................................................................. 1 1.1 Aim and Purpose................................................................................... 3 1.2 Research questions and research structure .......................................... 3 1.2.1 The first research question 3 1.2.2 The second research question 3 1.2.3 The third research question 3 1.3 Limitations ............................................................................................ 4 2 Frame of reference ................................................................................... 5 2.1 Engineering psychology ........................................................................ 5 2.2 Haptic interfaces ................................................................................... 6 2.3 The driving context ............................................................................... 6 2.4 Perception ............................................................................................ 7 2.4.1 Haptic perception 7 2.5 Attention .............................................................................................. 9 2.5.1 Theories of selective attention 9 2.5.2 A neuroscience perspective of selective attention 10 2.5.3 Effects of selective attention 11 2.5.4 Selective attention to haptic information 12 2.5.5 Effects of selective attention during driving 12 2.5.6 Divided attention 13 2.5.7 Driver distraction 13 2.5.8 Theories of divided attention 14 2.5.9 The multimodal approach 15 2.5.10 The multimodal approach in a driving context 16 2.5.11 Integration of multimodal information 17 2.5.12 Modality dominance 18 2.6 Performance ....................................................................................... 19 2.6.1 Visual-haptic interface performance 19 2.6.2 Driving performance 20 3 Method .................................................................................................. 23 3.1 Participants ......................................................................................... 23 3.2 Experimental design ........................................................................... 23 3.3 Haptic rotary device ............................................................................ 24 3.4 Menu selection task ............................................................................ 24 3.5 Visual and haptic interface ................................................................. 25 3.6 Driving task ......................................................................................... 26 3.7 Interviews ........................................................................................... 27 3.8 Questionnaires ..................................................................................