OPTIMIZING HUMAN PERFORMANCE IN MOBILE TEXT ENTRY STEVEN JOHN CASTELLUCCI A DISSERTATION SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY GRADUATE PROGRAMME IN COMPUTER SCIENCE YORK UNIVERSITY TORONTO, ONTARIO DECEMBER 2014 © STEVEN JOHN CASTELLUCCI, 2014 ABSTRACT Although text entry on mobile phones is abundant, research strives to achieve desktop typing performance “on the go”. But how can researchers evaluate new and existing mobile text entry techniques? How can they ensure that evaluations are conducted in a consistent manner that facilitates comparison? What forms of input are possible on a mobile device? Do the audio and haptic feedback options with most touchscreen keyboards affect performance? What influences users’ preference for one feedback or another? Can rearranging the characters and keys of a keyboard improve performance? This dissertation answers these questions and more. The developed TEMA software allows researchers to evaluate mobile text entry methods in an easy, detailed, and consistent manner. Many in academia and industry have adopted it. TEMA was used to evaluate a typical QWERTY keyboard with multiple options for audio and haptic feedback. Though feedback did not have a significant effect on performance, a survey revealed that users’ choice of feedback is influenced by social and technical factors. Another study using TEMA showed that novice users entered text faster using a tapping technique than with a gesture or handwriting technique. This motivated rearranging the keys and characters to create a new keyboard, MIME, that would provide ii better performance for expert users. Data on character frequency and key selection times were gathered and used to design MIME. A longitudinal user study using TEMA revealed an entry speed of 17 wpm and a total error rate of 1.7% for MIME, compared to 23 wpm and 5.2% for QWERTY. Although MIME’s entry speed did not surpass QWERTY’s during the study, it is projected to do so after twelve hours of practice. MIME’s error rate was consistently low and significantly lower than QWERTY’s. In addition, participants found MIME more comfortable to use, with some reporting hand soreness after using QWERTY for extended periods. iii ACKNOWLEDGEMENTS I would like to convey my sincerest gratitude to my supervisor, Scott MacKenzie, for his knowledge, guidance, and support. His generous funding of me via his NSERC grant made this research possible. I especially appreciate co-presenting his course Empirical Research Methods for Human-Computer Interaction at CHI for the past three years. Thank you to Wolfgang Stuerzlinger for your research advice, your advice on the design of MIME, and for suggesting the communication channel to log IME data in TEMA. Thank you to Burton Ma for serving on my supervisory committee on short notice. Thank you to Rob Allison, Xiangji (Jimmy) Huang, and Mark Green for taking time out of their busy schedules to read this dissertation, provide feedback, and serve on my examination committee. I would also like to thank Hamzeh Roumani for his support of my teaching skills, Ouma Jaipaul Gill for her guidance with administrative procedures, and Susan Cameron for ensuring I received reimbursement for conference and user study expenses as soon as possible. Thank you, Andrew Speers, for encouraging me to apply for the faculty teaching award. Thank you, Ilnaz (Elle) Ameli, for your generous help and guidance with statistics. Thank you, Michelle Brown, for your invaluable contributions to my user studies. Thank you, Kathy Weaver, for your expertise in box plots. Thank you, Robert iv Teather, for being a conference guru. Thank you, Victoria McArthur, for selecting me as a CHI Student Volunteer. I would also like to thank Andriy Pavlovych, Ahmed Arif, Shumon Zaman, and Hoda Dehmeshki for their constructive feedback on my research over the years. Most importantly, I would like to thank my parents, Betty and Sandro, for their love, encouragement, and support. v DISSIMINATION OF THIS DISSERTATION The following chapters (in whole or in part) of this dissertation have been previously published: Chapter 3 Castellucci, S. J. and MacKenzie, I. S., Gathering text entry metrics on Android devices. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems – CHI 2011, ACM (2011), 1507-1512. Castellucci, S. J. and MacKenzie, I. S., Encouraging consistency in mobile text entry evaluations. In CHI 2012 Workshop on Designing and Evaluating Text Entry Methods, ACM (2012), 43-46. Castellucci, S. J. and MacKenzie, I. S., Gathering Text Entry Metrics on Android Devices. In Proceedings of the International Conference on Multimedia and Human Computer Interaction – MHCI 2013, International ASET Inc. (2013), 119.1-119.8. Chapter 4 Castellucci, S. J. and MacKenzie, I. S., Touch Me, Hear Me, Feel Me: Feedback Preferences for Mobile Text Entry. In Proceedings of the International Conference on Multimedia and Human Computer Interaction – MHCI 2013, International ASET Inc. (2013), 123.1-123.6. vi TABLE OF CONTENTS Abstract ............................................................................................................................... ii Acknowledgements ............................................................................................................ iv Dissimination of this Dissertation ...................................................................................... vi Table of Contents .............................................................................................................. vii List of Tables ..................................................................................................................... xi List of Figures .................................................................................................................. xiii List of Acronyms ........................................................................................................... xviii CHAPTER 1 Introduction and Motivation ............................................................................ 1 1.1 Established Performance Metrics ....................................................................... 2 1.2 A Priori and Post Hoc Power Calculations ......................................................... 6 1.3 Dissertation Contributions .................................................................................. 6 1.4 Dissertation Organization ................................................................................... 9 CHAPTER 2 Reviewing Touch-Based Mobile Text Entry ................................................. 10 2.1 Character-Based Recognition ........................................................................... 12 2.2 Menu Navigation .............................................................................................. 18 2.3 Mid-Air Techniques .......................................................................................... 33 2.4 Tapping and Hybrid Techniques ....................................................................... 39 vii 2.5 Non-English Text Entry .................................................................................... 46 2.6 Design Rationale for MIME ............................................................................. 49 CHAPTER 3 Gathering Text Entry Metrics on Android Devices ....................................... 52 3.1 Motivation ......................................................................................................... 52 3.2 TEMA Features ................................................................................................. 53 3.3 TEMA Design ................................................................................................... 61 3.4 Encouraging Consistency in Mobile Text Entry Evaluations ........................... 63 3.5 Method .............................................................................................................. 65 3.6 Results and Discussion ..................................................................................... 68 3.7 Conclusion ........................................................................................................ 71 CHAPTER 4 Determining Feedback Preferences for Mobile Text Entry ........................... 72 4.1 Motivation ......................................................................................................... 72 4.2 Related Work .................................................................................................... 73 4.3 Method 1 (Survey) ............................................................................................ 74 4.4 Method 2 (User Study)...................................................................................... 75 4.5 Results and Discussion ..................................................................................... 77 4.6 Conclusion ........................................................................................................ 82 CHAPTER 5 Compiling Phrase Sets for Mobile Text Entry ............................................... 83 5.1 Motivation ......................................................................................................... 83 5.2 Corpora Sources ................................................................................................ 84 viii 5.3 Digram Frequencies .........................................................................................