EYES-FREE INTERACTION WITH AURAL USER INTERFACES Romisa Rohani Ghahari Submitted to the faculty of the University Graduate School in partial fulfillment of the requirements for the degree Doctor of Philosophy in the School of Informatics and Computing Indiana University December 2015 Accepted by the Graduate Faculty, Indiana University, in partial fulfillment of the requirements for the degree of Doctor of Philosophy. ____________________________________ Davide Bolchini, Ph.D., Chair ____________________________________ Michael Justiss, Ph.D. Doctoral Committee ___________________________________ Sarah Koskie, Ph.D. August 11, 2015 ____________________________________ Xi Niu, Ph.D. ____________________________________ Stephen Voida, Ph.D. ii © 2015 Romisa Rohani Ghahari iii Dedication To my parents. iv Acknowledgements It is a pleasure to thank those who have made this dissertation possible. First and foremost, I owe sincere and earnest thankfulness to my advisor, Dr. Davide Bolchini. Dr. Bolchini was the person who had the most influence on my academic profession. It has been an incredible experience working with him, and I always feel excited when thinking about how much I have grown during these years under his continuous support and inspiring instructions. I also want to thank Dr. Bolchini for his patient in helping me correct my mistakes. Without his help, I might not have been able to overcome the challenge so smoothly. Many thanks to my research committee members: Dr. Michael Justiss, Dr. Sarah Koskie, Dr. Xi Niu, and Dr. Stephen Voida. Thanks go out to Dr. Justiss and Dr. Koskie for letting me use the driving simulation lab facility and helping me to be familiar with the logistics of conducting a study in a simulation lab. Their guidance helped me enhance the experimental design found in Chapter 6 of this dissertation. I want to thank Dr. Xi Niu for her feedback on this dissertation although she served as my committee member for the last one year. Dr. Xi Niu‘s feedback allowed me to enhance the data analysis section. Thanks also need to go out to Dr. Stephen Voida for his continuous feedback on the whole dissertation. He had provided me with many constructive comments to strengthen the reasoning on the core part of my dissertation. I would also like to thank Dr. Mark Pfaff, who served as my research committee member for a year. Dr. Mark Pfaff helped me a lot shaping the experimental design and statistical tests I needed to perform throughout this dissertation. My life in the US would not have been as wonderful as it has been without the company of my colleagues and friends in the USER Lab. Thanks go out to Dr. Mexhid Ferati, Dr. Tao Yang, and Dr. Afarin Pirzadeh for teaching me both the experimental v design and statistical analysis skills. I always admire how knowledgeable they were in those fields. Thanks also need to go out to Dr. Jennifer George-Palilonis for helping me with the initial stages of this dissertation. Her background in journalism enabled me with discovering the news domain much better than discovering it on my own. I would also like to thank Dr. Arthur Liu, Siavash Mortazavi, Dr. Hossain Gahangir, Lindsay Kaser, Kevin Pritchard, Callie Archibald, Dr. Sung Pil Moon, Shiva Ladan, Prathik Gadde, Debaleena Chattopadhyay, and Yuan Jia for their help and ideas, all of which helped strengthen my dissertation. I wish you all good luck in your future endeavors. I would also like to acknowledge the support of the National Science Foundation and the Funding Opportunities for Research Commercialization and Economic Success at IUPUI Office of the Vice-Chancellor for Research. Thanks to all participants for their contributions to this research. Last, but not least, I would like to thank my parents and my sister for encouraging me to continue with my higher education, and for supporting me both emotionally and financially whenever I needed. I understand allowing me not to be by their sides for so many years was a tough decision, but thanks for letting me experience and accomplish this degree. vi Romisa Rohani Ghahari EYES-FREE INTERACTION WITH AURAL USER INTERFACES Existing web applications force users to focus their visual attentions on mobile devices, while browsing content and services on the go (e.g., while walking or driving). To support mobile, eyes-free web browsing and minimize interaction with devices, designers can leverage the auditory channel. Whereas acoustic interfaces have proven to be effective in regard to reducing visual attention, a perplexing challenge exists in designing aural information architectures for the web because of its non-linear structure. To address this problem, we introduce and evaluate techniques to remodel existing information architectures as ―playlists‖ of web content − aural flows. The use of aural flows in mobile web browsing can be seen in ANFORA News, a semi-aural mobile site designed to facilitate browsing large collections of news stories. An exploratory study involving frequent news readers (n=20) investigated the usability and navigation experiences with ANFORA News in a mobile setting. The initial evidence suggests that aural flows are a promising paradigm for supporting eyes-free mobile navigation while on the go. Interacting with aural flows, however, requires users to select interface buttons, tethering visual attention to the mobile device even when it is unsafe. To reduce visual interaction with the screen, we also explore the use of simulated voice commands to control aural flows. In a study, 20 participants browsed aural flows either through a visual interface or with a visual interface augmented by voice commands. The results suggest that using voice commands decreases by half the time spent looking at the device, but yields similar walking speeds, system usability and cognitive effort ratings as using buttons. vii To test the potential of using aural flows in a higher distracting context, a study (n=60) was conducted in a driving simulation lab. Each participant drove through three driving scenario complexities: low, moderate and high. Within each driving complexity, the participants went through an alternative aural application exposure: no device, voice- controlled aural flows (ANFORADrive) or alternative solution on the market (Umano). The results suggest that voice-controlled aural flows do not affect distraction, overall safety, cognitive effort, driving performance or driving behavior when compared to the no device condition. Davide Bolchini, Ph.D., Chair viii Table of Contents List of Tables ................................................................................................................ xiv List of Figures ................................................................................................................ xv Chapter 1. Introduction ................................................................................................... 1 Chapter 2. Review of Theoretical Background ................................................................ 5 2.1. Theoretical Background Behind Visual and Auditory Channels ............................ 5 2.2. The Value of Aurally Navigated User Interfaces ................................................... 9 2.3. Automated Browsing ...........................................................................................10 2.4. Voice User Interfaces and Their Application in Regard to Driving ........................11 2.4.1. Voice Input User Interfaces ...........................................................................11 2.4.2. Disadvantages of Voice Interaction ...............................................................13 2.4.3. A Design Method for Voice Commands ........................................................14 2.4.4. Guidelines for Effective Voice Commands ....................................................15 2.4.5. Voice Interaction in Driving Context ..............................................................16 2.5. Measuring Distraction Due to Interactions with Mobile Devices ...........................20 Chapter 3. Introducing the ANFORA Framework ...........................................................22 3.1. ANFORA Framework ..........................................................................................22 3.1.1. Full Flow for Prolonged Aural Experiences ...................................................23 3.1.2. Short Aural Explorations with Group Flow .....................................................23 3.2. Reifying ANFORA in the News Domain: ANFORA News ....................................26 3.2.1. The Four Different News Consumption Experiences of ANFORA News .......26 3.2.2. ANFORA News‘ User Profiles .......................................................................27 3.3. The ANFORA News Prototype ............................................................................28 3.3.1. Content, Styles and Formats ........................................................................31 3.3.2. Design Challenges for the Aural Experience .................................................32 Chapter 4. Preliminary Evaluation of the ANFORA Framework .....................................37 ix 4.1. Study Design .......................................................................................................37 4.1.1. Physical Setup ..............................................................................................37 4.1.2. Participants ...................................................................................................38