EdgeWrite: A Versatile Design for Text Entry and Control Jacob O. Wobbrock July 2006 CMU-HCII-06-104 Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Brad A. Myers, Chair Scott E. Hudson Jennifer Mankoff Richard C. Simpson, University of Pittsburgh Shumin Zhai, IBM Almaden Research Center Submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Copyright © 2006 Jacob O. Wobbrock. All rights reserved. This work was supported in part by the National Science Foundation under grant UA-0308065. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author and do not necessarily reflect those of the National Science Foundation. Additional support was provided by General Motors, Microsoft, NEC Foundation of America, NISH, Synaptics, and A.T. Sciences. Keywords: Text entry, text input, unistrokes, gestures, assistive technology, computer access, universal access, universal design, motor impairments, situational impairments, mobile device, mobile phone, handheld, PDA, stylus, game controller, displacement joystick, isometric joystick, power wheelchair joystick, touchpad, trackball, on-screen keyboard, capacitive sensors, word prediction, word completion, goal crossing, physical edges, EdgeWrite, Pebbles, Fitts’ law, Hick-Hyman law, Steering law, Zipf’s law. ii Abstract This dissertation presents a versatile design for text entry and control called EdgeWrite. EdgeWrite was designed to provide accessible text entry on a variety of platforms to people with motor impairments and to able-bodied users of small devices. The EdgeWrite design includes a square input area, four corner sensors, corner- sequence recognition, physical edges, goal crossing, and unistroke segmentation. Advanced EdgeWrite features include continuous recognition feedback, non-recognition retry, slip detection, and word-level stroking—concepts not before realized in a text entry method. The EdgeWrite alphabet, which is the same for all EdgeWrite versions, was designed for maximum guessability and learnability through participatory design. A key result is that EdgeWrite is about as guessable as Palm OS Graffiti, a method lauded for its immediate usability. Multiple versions of EdgeWrite were built and user-tested, including Stylus EdgeWrite for the Palm OS, Joystick EdgeWrite for game controllers and power wheelchairs, Touchpad EdgeWrite, Trackball EdgeWrite, Isometric Joystick EdgeWrite for mobile phones, and EdgeWrite on four keys or sensors. For each version, empirical results from formal user studies were obtained. Of particular importance for users with motor impairments are the results for Stylus and Trackball EdgeWrite, which show marked improvements over existing techniques. In addition, five versions of EdgeWrite built by other researchers further highlight EdgeWrite’s versatility. As part of EdgeWrite’s ongoing evaluations, a new character-level error analysis for text entry input streams was developed. This error analysis and the algorithms that automate it constitute a methodological and theoretical contribution to the field of text entry evaluation and measurement. The thesis is: A versatile design for text entry and control called “EdgeWrite,” which uses physical edges, goal crossing, and a minimized need for sensing, is effective on handhelds and desktops for people with motor and situational impairments. iii Acknowledgements I would like to thank my advisor, Brad A. Myers, for his faithful support and sharp insight. Not one of my projects would have fared as well apart from his input. It has been a real pleasure working with him for five years. Along these lines, I also thank Scott E. Hudson for his helpful comments. My other committee members deserve special thanks for their time and energy in improving this work. Jennifer Mankoff was a source of directional help. Richard C. Simpson provided valuable access to subjects and resources. Shuman Zhai inspired me by the rigor of his methods and brilliance of his innovations. Along with Shumin Zhai, I owe an intellectual debt to I. Scott MacKenzie and R. William Soukoreff for escorting text entry into the 21st century. Without these scientists, my work would have been greatly impoverished. This work has corporate and governmental sponsors and collaborators who deserve thanks: the National Science Foundation, General Motors, Microsoft, NEC Foundation of America, NISH, Synaptics, and A.T. Sciences. A number of students, faculty, and others have influenced this work. In alphabetical order, they are: Gregory Abowd, Sonya Allin, Lisa Anthony, Chris Atkeson, Htet Htet Aung, Daniel Avrahami, Thi Truong Avrahami, Ryan S. Baker, Aaron Bauer, Gábor Blaskó, Matthew Blythe, Dan Boyarski, Connie Campbell, Duen Horng Chau, Ed Chi, Benjamin Cramer, Marian D’Amico, Kim D’Ewart, Victoria Danforth, Andrew Faulring, James Fogarty, Jodi Forlizzi, Susan Fussell, Darren Gergle, Iván E. González, Steve Hayashi, Mitch Hill, Bonnie John, Geoff Kembel, John Kembel, Silvia Kembel, Pedram Keyani, Sara Kiesler, Roberta Klatzky, Andrew Ko, Panu Korhonen, Robert Kraut, Queenie Kravitz, Johnny C. Lee, Edmund F. LoPresti, Tom Lorusso, Bob Milan, Jeffrey Nichols, Sue O’Connor, Dan Olsen, Roni Rosenfeld, Brandon Rothrock, Joe Rutledge, John SanGiovanni, A. Fleming Seay, Ted Selker, Howard Seltman, Michael Sinclair, John Springsteen, Jeffrey Stylos, Alan Tannenbaum, Janis Thoma-Negley, James A. Vroom, Elaine Wherry, Matthew Wick, Andrew Wilson, Alison N. P. Wobbrock, Nicholas J. Wobbrock, and John Zimmerman. Thank you, all. iv My gratitude goes to my church friends in both California and in Pittsburgh. Whether we journeyed only briefly or over the long haul, it was indeed the journey that mattered; and not because of the road we walked, but because of who we walked it with. I admire your commitment to faith. My family—Góralska and American—has earned my deep thanks for their encouragement, generosity, and devotion. My father and mother deserve great thanks for their lifelong love and support. Without them I would not have lasted long in this Ph.D. program. I count on my brother to keep me humble. I know he always will. Finally, to my wife Ala, to whom this dissertation is dedicated: You have been my strength, my will, and my rock. There is no one for whom I would more gladly punch keys. If I can be half as good a man as you are woman to me, I will be twice as good a man as any man could ever be. v Dedication To Ala Kocham cię vi Table of Contents Abstract iii Acknowledgements iv Dedication vi Table of Contents vii List of Figures xi List of Tables xxi 1 Introduction 1 1.1 Motivation..........................................................................................................1 1.1.1 The Need for Better Computer Access.................................................1 1.1.2 The Need for Mobile Device Accessibility ..........................................2 1.1.3 A Versatile Design for Text Entry and Control....................................4 1.2 The EdgeWrite Concept.....................................................................................6 1.3 Research Approach ............................................................................................7 1.4 Dissertation Organization ..................................................................................8 2 Related Work 11 2.1 Universal Design..............................................................................................11 2.1.1 Situational Impairments .....................................................................12 2.1.2 Mobile Device Accessibility ..............................................................13 2.2 Beating Fitts’ Law............................................................................................14 2.2.1 Edges in the Interface .........................................................................15 2.2.2 Goal Crossing.....................................................................................17 2.3 Input Devices and Techniques .........................................................................18 2.3.1 Multi-device Input..............................................................................19 2.3.2 Stylus Input.........................................................................................21 2.3.3 Word-level Stroking...........................................................................23 2.3.4 Displacement Joystick Input...............................................................24 2.3.5 Touchpad Input...................................................................................25 2.3.6 Trackball Input ...................................................................................27 2.3.7 Isometric Joystick Input .....................................................................29 2.3.8 Mobile Keypad Input..........................................................................30 2.4 Text Entry Evaluation......................................................................................33 2.4.1 Speed ..................................................................................................34 2.4.2 MSD and KSPC..................................................................................35 2.4.3 Uncorrected, Corrected, and Total Errors ..........................................36 3 The EdgeWrite Design 39 3.1 Background......................................................................................................39