Keyboards Redux: Fast Mobile Text Entry

Wearable Computing Editor: Thad E. Starner ■ Georgia Institute of Technology ■ [email protected]

Thad Starner

ext entry research is in vogue again, ple, when messages were sent across con- minute,1 more than six times slower than T and inventors are flocking to con- tinents by telegram, senders would use the Guinness record of 192 wpm for the ferences and meetings worldwide to abbreviations to keep the message cost desktop qwerty keyboard.2 show their methods. What’s causing low, and the recipient could still under- this resurgence of interest in a topic stand the message. Similarly, abbrevia- MULTITAP that’s been dormant for so long? tions are common in today’s short mes- The most common style of text entry sage service messaging. For example, using a mobile phone’s number pad is WIRELESS MESSAGING “CUL8R” means “see you later” in com- multitap, where multiple letters are In 2003, wireless messaging accounted mon parlance. These abbreviations, some mapped to the same key (see Figure 1). for US$16 billion in revenue for mobile of which can be quite complex and cryp- Holding the keypad toward you, you phone service providers. Some forecast- tic, have probably evolved both because can enter text with one or two hands ers predict that between 1 and 2 trillion of the slow typing speed the telephone using one or two fingers or thumbs. messages will be sent this year, and ana- keypad allows and because of the mes- You press the key to cycle through the lysts say that mobile email will be the saging culture itself. However, if mobile letters until the desired one appears next big market for the European indus- phone service providers want to enable onscreen. You can then proceed to the try. Other markets, such as mobile gam- mobile email, faster typing methods must next letter if it’s on a different key. If the ing and location-based services, will also be devised. desired letter is on the same key, you use text entry. However, two barriers The typing rates of today’s text-entry must wait a short time for the phone to exist for these markets: appropriate dis- methods using the mobile phone’s num- accept the letter. Alternatively, many plays and text-entry speed. ber pad are two to five times slower than phones use the “#” button to force the Mobile device displays have improved that expected with a desktop keyboard. phone to the next letter. For example, steadily. Several recent PDAs, such as the For example, in 2003 the world’s fastest you could type the word “deaf” enter- Sharp Zaurus 6000, feature color 640 × mobile “texter” typed 29 words per ing “3#332333.” Usability studies have 480 displays that are crisp and readable found average rates of 8 to 20 wpm3–5 in daylight. Mobile phone displays are for this method’s experienced users. also improving, driven by the integration (The rates vary owing to incentives of cameras and PDA-like functions. given during testing, training time, the Additionally, mobile head-up display phrase sets used, and a particular manufacturers such as MicroOptical, device’s haptics, including key size, key Microvision, and Interactive Imaging are spacing, and key resistance.) selling the most useable and affordable displays to date. Although still too ex- T9 pensive for consumers, these head-up Another common typing method on units are drawing interest from mobile mobile phones is T9, which uses a dic- phone manufacturers. tionary to disambiguate the user’s input. However, mobile text entry remains Just like multitap, multiple letters corre- a difficult problem for handset manu- spond to the same key. However, you facturers. In certain circumstances, terse Figure 1. A typical mobile phone press keys with the corresponding letters messages can be appropriate. For exam- number pad design. only once. As you enter the word, the

as inefficient for text entry (see www. jamestrusler.com).

LETTERWISE In contrast to the dictionary-based disambiguation T9 uses, Letterwise uses just the preceding letters to disambiguate the letter you’re typing. For example, if you type “th” and press “3,” Letterwise fills in an “e” as the next letter. If this interpretation is wrong, you can cycle through that number’s other possible letters—in this case “d” and “f,”—using a “next” key (such as “#”). Researchers have reported rates of 21 wpm with Letterwise.3 This method has the advantage of letting you type any letter combination because it doesn’t rely on a dictionary. Figure 2. (a) The Twiddler one-handed chording keyboard; (b) chording the letter “k.” with the Twiddler. TWIDDLER CHORDING Handykey’s Twiddler keyboard uses the same three column by four row button design that a mobile phone’s number pad does. You operate each row of keys with one of your four fingers (see Figures 2 and 3). You press multiple keys simultaneously to enter letters, numbers, and symbols—a process called chording. You can type any letter on the Twiddler by pressing one or two keys concur- rently.5 The Twiddler also allows multi- character chords (MCCs), where you type frequent words and letter sets such as “and,” “the,” and “ing” using a sin- gle chord. When I lecture on wearable computing, I’m always asked how fast I can type on the Twiddler and how long it Figure 3. You can produce letters, symbols, numbers, or words on the Twiddler by takes to learn chording. Two students, pressing one or more buttons. The alphabet is sequential and simple to learn. Starting Kent Lyons and Daniel Plaisted, took from the top left and going to the bottom right, the letters resulting from the chords an interest in these questions, and we above are the 26 letters of the alphabet, period, space, backspace, and “the.” designed experiments to determine the answer using methods similar to those Scott MacKenzie reported at the User phone continually guesses what word Researchers have observed average text Interface Software and Technology you’re typing. You can cycle through entry rates of 20 wpm when all words Conference (UIST) and the Conference potential words by pressing the “#” key. entered are in the phone’s dictionary.4 on Human Factors in Computing Sys- For example, you’d type “3323” for the However, T9 can be frustrating and slow tems (CHI) in recent years.3 Surpris- “deaf” example mentioned earlier. How- if you’re entering words the phone does- ingly, we found that the learning rate ever, the first word T9 would guess is n’t recognize. Interestingly, the world was quite fast. In less than seven hours, “dead.” You’d press the “#” key until texting rate record was set using multi- our 10 subjects reached an average rate the phone displayed the proper word. tap, and the record’s holder dismisses T9 of 26 wpm using the Twiddler. In par-

98 PERVASIVEcomputing www.computer.org/pervasive WEARABLE COMPUTING

TABLE 1 allel, we had the same subjects type Noteworthy text entry speeds. with multitap on the Twiddler keyboard using the same procedure, and Words per minute Method they reached an average of 20 wpm. We 192 Fastest qwerty speed2 might explain this high rate by the 120 Twiddler burst speed (author) Twiddler’s better spacing and button 70–90 Highly skilled secretary resistance versus other 3 × 4 button *6 keypads. Another contributing factor 67 Fastest Twiddler user average (25 hours) might be that we paid the subjects on 50 Highest US government pay grade for typists 6 the basis of their speed and accuracy. 47 Twiddler average (25 hours) However, the multitap rates stopped 40 Goal of semester typing class (60 hours) improving significantly, while the Twid- 35 Fastest Morse code keyer1 dler chording rates continued to 30 Average desktop rate (including hunt-and-peck typists) improve dramatically.5 29 Fastest mobile phone “texter” (multitap)1 To better predict what an expert 26 Twiddler average (7 hours) Twiddler user might achieve, we offered 21 Letterwise average (10 hours) our subjects the opportunity to continue 20 Multitap on Twiddler (7 hours)6 the study; five of the 10 subjects ac- 20 T9 “expert” average4 cepted. Fortunately, the average typing 16 ChordTap average (2.5 hours)7 rate of those five was similar to that of 3 the entire group. After 25 hours, the 16 Multitap average (10 hours) 7 typing rates stopped increasing dra- 11 Multitap average (2.5 hours) 4 matically, and the average rate was 47 8 Multitap “expert” average wpm. Interestingly, this amount of prac- <10 Multitap/T9/Letterwise/ChordTap novice tice is significantly less than the 60 hours *Times in parentheses indicate the subjects’ amount of training. done in a semester-long typing class, where the goal is 40 wpm. Another sur- prise is that the top Twiddler typist actu- comfortable with the device. However, cate the first letter in the “def” triplet ally exceeded my letter-by-letter (no service providers could also offer incen- that appears on the 3 key. You’d then MCCs) typing speed at 67 wpm (I’ve tives to increase their clients’ typing type “e” with the number 3 plus the been typing on the Twiddler for over 10 speed. For example, they might design a second chording key, “a” with the years).6 Although directly contrasting mobile phone that worked in both mul- number 2 and the first chording key, learning curves between studies is un- titap and chording modes. Users would and “f” with the number 3 and the wise, these resultant averages compare receive a $50 rebate whenever they ex- third chording key. Although Chord- favorably to even desktop speeds (Table ceeded 30 wpm while typing a wireless Tap probably wouldn’t be as fast as the 1 provides a survey of typing rates). This message, thus encouraging chording. Pre- Twiddler for an expert “chordist,” the result suggests that, with sufficient moti- sumably, with faster typing skills, users mapping is immediately accessible to a vation, Twiddler novices might reach would send more wireless messages. current multitap user. adequate desktop-level typing speeds Additionally, they might be more easily In their CHI paper, Wigdor and Bal- relatively quickly. coaxed into advanced services, such as akrishnan showed that ChordTap and The question remains, however, mobile email or wireless gaming. multitap’s initial rates were equivalent, whether mobile phone users will invest presumably giving users an immediate the time to learn chording. My group is CHORDTAP sense of reward. With practice, Chord- currently investigating ways to encour- Daniel Wigdor and Ravin Balakris- Tap rates became significantly higher age and increase learning speed using a nan presented an intriguing compro- than multitap rates, reaching 16 wpm typing tutor similar to that used for mise between walk-up usability and after 2.5 hours of testing. Meanwhile, desktop keyboards. You can find the faster text entry through chording at multitap users averaged between 11 and current version of “Twidor,” the Twid- the CHI 2004.7 Their method, Chord- 12 wpm by the study’s end. Although dler tutor, at www.cc.gatech.edu/ccg/ Tap, uses the current multitap layout the current data doesn’t allow much projects/twidor. plus three chording keys to disam- extrapolation over 20 wpm, the Chord- My students are also investigating how biguate each letter. To type the word Tap method suggests an interesting way to improve the Twiddler’s “walk-up” “deaf” as before, you’d press the num- to transition current mobile phone users usability so that novices can quickly feel ber 3 plus the first chording key to indi- into a chording framework.

JULY–SEPTEMBER 2004 PERVASIVEcomputing 99 WEARABLE COMPUTING WEARABLE COMPUTING

method is similar to “hunt-and- exist for keyboards. Speech interfaces next issue peck” typing on desktop key- are interruptive in many social situa- boards, which averages around tions, such as when taking private notes What security and privacy attacks 23 wpm.8 However, although I during a meeting. Additionally, many don’t know of formal longitu- people object to microphone use in can we expect now that mobile dinal studies on mini qwerty everyday interactions for privacy rea- keyboards, a model William sons. Even when allowed, transcripts of phones have become the world’s Soukoreff presents predicts a conversations are often incomplete, peak expert typing rate of 61 misleading, and difficult to search. dominant computer platform? wpm.9 What rate an average Finally, speech interfaces will always expert could attain with prac- suffer from error and ambiguity. Mean- tice is still an open question. while, mobile keyboards, especially one-handed ones, let you enter terse, MINI QWERTY KEYBOARDS BLIND TYPING private, and meaningful notes quietly Some phones and PDAs are now inte- One attribute text entry researchers are and with little error while you’re in grating miniature qwerty keyboards as investigating is “blind” typing, where public. part of their design. The difficulty with users receive little or no visual feedback Already, mobile keyboard use is surg- these keyboards, of course, is their but- as they type. Such touch typing enables ing in highly populated areas such as tons’ size and spacing. Most people can text entry in more varied situations, such Japan. According to Newsweek, more use only two thumbs while typing. This as teenagers sending mobile phone mes- email is sent via mobile phones than sages while avoiding detection home PCs in Japan.10 One reason given in class. Multitap, Twiddler is that a Japanese family can’t afford the UPCOMING EVENTS chording, and ChordTap allow space to dedicate a room to computer such blind entry. Owing to their use. This restriction limits the number Int’l Conf. Ubiquitous Computing (Ubicomp) dependence on context, T9 and of households with a PC. In contrast, 7–10 Sep. 2004; Nottingham, England Letterwise don’t allow touch you can use a mobile phone virtually http://ubiomp.org/ubicomp2004 typing for most users. Mini anywhere. Another reason for mobile qwerty keyboards should theo- phone email use in Japan is privacy. Int’l Conf. Multimodal Interfaces (ICMI/PUI) retically allow blind typing; Teenagers don’t want their parents 13–15 Oct. 2004; State College, Pa. however, anecdotal reports sug- monitoring their conversations, and www.icmiplace.org gest that the user makes enough mobile texting provides a convenient errors to require visual feedback mechanism for private communication. User Interface Software and Technology (UIST) for corrections. Interestingly, in 24–27 Oct. 2004, Santa Fe, N.M. our experiments with the Twid- www.acm.org/uist dler, blind typing had virtually no effect on either typing speed Int’l Symp. Wearable Computers (ISWC) or error rates. 31 Oct.–3 Nov. 2004; Washington D.C. or these reasons and others, I www.iswc.net MOBILE KEYBOARDS’ F believe the demand for mobile key- FUTURE MARKET boards will grow substantially. As com- Int’l Symp. Mixed and Augmented Reality (ISMAR) I’ve only covered a few puters spread throughout the third 3–5 Nov. 2004; Washington D.C. methods of text entry on mo- world in the guise of mobile phones, the www.ismar04.org bile devices, mostly focusing demand for more rapid means of text on the number pad. Other entry for email and instant messaging Int’l Conf. Pervasive Computing and methods include virtual key- will drive innovation. The first mobile Communications (PerCom) boards commonly included on phone service provider to enable fast 8–12 Mar. 2005; Kauai Island, Hawaii PDAs, projected keyboards text entry for large communities will www.percom.org such as Canesta’s design, and reap big rewards. Novel keyboard gesture recognition. Some pun- design might also provide product Int’l Conf. Pervasive Computing (Pervasive) dits believe that speech recog- “lock-in” or supplemental licensing rev- 8–12 May 2005; Munich, Germany nition will prevail on mobile enue. Imagine the profits if every com- www.pervasive.ifi.lmu.de platforms. However, I believe puter sold had to pay a licensing fee for a large market will always its qwerty keyboard.

100 PERVASIVEcomputing www.computer.org/pervasive WEARABLE COMPUTING

For inventors who think they have the solution to mobile input, I suggest reading recent papers by MacKenzie and his colleagues on text-entry evalu- ation methods.3,8,9 Or download the a monthly magazine of the IEEE Computer Society Twidor tutor, develop rapid-text-entry skills, and challenge me to a race at the gadget show during the next Interna- http://dsonline.computer.org tional Symposium on Wearable Com- IEEE Distributed Systems Online puters in Washington DC! brings you peer-reviewed articles, detailed REFERENCES tutorials, expert-managed topic areas, and 1. C. Folkard, ed., Guinness World Records, Bantam Dell, 2003. diverse departments covering the latest news

2. M. Young, ed., Guinness World Records, and developments in this fast-growing field. Bantam Dell, 1998.

3. I. MacKenzie et al., “LetterWise: Prefix- Based Disambiguation for Mobile Text Log on http://dsonline.computer.org Input,” Proc. 14th Ann. ACM Symp. User Interface Software and Technology, ACM for free access to topic areas on Press, 2001, pp. 111–120; http://doi.acm. org/10.1145/502348.502365. ❍ 4. C.L. James and K.M. Reischel, “Text Input Grid Computing for Mobile Devices: Comparing Model Pre- diction to Actual Performance,” Proc. ❍ Mobile & Pervasive Conf. Human Factors in Computing Sys- tems (SIGCHI 01), ACM Press, 2001, pp. ❍ 365–371. Distributed Agents

5. K. Lyons et al., “Twiddler Typing: One- ❍ Security Handed Chording Text Entry for Mobile Phones,” Proc. Conf. Human Factors in ❍ Computing Systems (SIGCHI 01), ACM Middleware Press, 2004, pp. 671–678. ❍ Parallel Processing 6. K. Lyons, D. Plaisted, and T. Starner, “Expert Chording Text Entry on the Twid- ❍ Web Systems dler One-Handed Keyboard,” to be pub- lished in Proc. IEEE Int’l Symp. Wearable Computing (ISWC 04), IEEE CS Press, ❍ Real Time & Embedded 2004. ❍ Dependable Systems 7. D. Wigdor and R. Balakrishnan, “A Com- parison of Consecutive and Concurrent ❍ Input Text Entry Techniques for Mobile Cluster Computing Phones,” Proc. Conf. Human Factors in Computing Systems (SIGCHI 01), ACM ❍ Distributed Multimedia Press, 2004, pp. 81–89. ❍ 8. E. Matias, I. MacKenzie, and W. Buxton, Distributed Databases “One-Handed Touch-Typing on a QWERTY Keyboard,” Human-Computer ❍ Collaborative Computing Interaction, vol. 11, no. 1, 1996, pp. 1–27. ❍ 9. R. William Soukoreff, Text Entry for Operating Systems Mobile Systems: Models, Measures, and Analyses for Text Entry Research, masters thesis, York University, 2002. To receive regular updates, email 10. K. Itoi, “Living the Wireless Lifestyle,” Newsweek Int’l, Atlantic edition, 7 June [email protected] 2004.

JULY–SEPTEMBER 2004 PERVASIVEcomputing 101