Chinese Input with Keyboard and Eye-Tracking
Total Page:16
File Type:pdf, Size:1020Kb
CHI 2001 • 31 MARCH – 5 APRIL Papers Chinese Input with Keyboard and Eye-Tracking - An Anatomical Study Jingtao Wang + Shumin Zhai ++ Hui Su+ + IBM China Research Lab ++ IBM Almaden Research Center No. 7, 5th Street, Shangdi 650 Harry Road Beijing, 100085, P. R. China San Jose, CA 95120, USA +86 10 6298 6677 {ext.309, 330} +1 408 927 1112 {jingtaow, suhui}@cn.ibm.com [email protected] ABSTRACT (13.6 vs. 32.5 cwpm for transcription and 7.8 vs. 19.0 Chinese input presents unique challenges to the field of cwpm for composition). Furthermore, the study also human computer interaction. This study provides an revealed many human-factors issues that were not anatomical analysis of today’s standard Chinese input previously realized. For example, many users found it process, which is based on pinyin, a phonetic spelling “harder to talk and think than type and think” and system in Roman characters. Through a combination of considered the keyboard to be more “natural” than speech human performance modeling and experimentation, our for text entry. study decomposed the Chinese input process into sub-tasks Once learned, touch-typing offers two critical and found that choice reaction time and numeric keying, characteristics: rapid speed and full attention focus on the two components resulted from the large number of screen. A skilled typist can type 60 words per minute, far homophones in Chinese, were the major usability beyond what handwriting could achieve [13]. The other bottlenecks. Choice reaction alone took 36% of the total perhaps even more important property is that touch-typing input time in our experiment. Numeric keying for multiple frees the user’s visual attention so she can focus on her task candidates selection tends to take the user’s attention away on the computer display. from the computer visual screen. We designed and implemented the EASE (Eye Assisted Selection and Entry) Numerous difficulties rise when the user’s language is system to help maintaining complete touch-typing Chinese (or many other non-alphabetic languages). experience without diverting visual attention to the numeric Currently the most popular method used for Chinese text keys. The EASE approach used a common selection key input in China mainland is pinyin input. Pinyin is the (spacebar) and implicit eye-tracking to replace the numeric official Chinese phonetic alphabet based on Roman keystrokes. Our experiment showed that such a system characters. For example, in pinyin the Chinese character could indeed work, even with today’s imperfect eye- (center, middle) is “zhong” and the pinyin for the word tracking technology. “ ”, consisting of two Chinese characters, is “Beijing”. About 97% of computer users in China use pinyin or some Keywords variations of it for daily input [3]. There have been other Chinese text input, pinyin input, gaze, eye-tracking, gaze- non-pinyin-based Chinese text input methods that encode tracking, multi-modal interface, performance modeling. the logographic Chinese characters, but the amount of learning and memorization has prevented them from INTRODUCTION becoming popular. Text entry is one of the most frequent human computer interaction tasks. Although great strides have been made The complication to pinyin input, however, is that most towards speech and handwriting recognition, typewriting Chinese characters are homophonic with many others. In remains and will likely be the main text entry method in the Mandarin Chinese, there are only about 410 distinct future. A recent study [5] showed that text entry by today’s syllables 1 [6]. In contrast, there are 6,763 Chinese continuous speech recognition is still far slower than typing characters in the national standard database GB2312. This means that on average, each syllable corresponds to 16.8 Permission to make digital or hard copies of all or part of this work for characters, notwithstanding the relatively small number of personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, 1 Unlike in English, a syllable in Chinese only takes two forms: a requires prior specific permission and/or a fee. vowel or a consonant-vowel combination. Note also that each SIGCHI’01, March 31-April 4, 2001, Seattle, WA, USA. syllable has four tones Copyright 2001 ACM 1-58113-327-8/01/0003…$5.00. anyone. anywhere. 349 Papers CHI 2001 • 31 MARCH – 5 APRIL characters with multiple pronunciations. As a result, when a If the intended character is not on the first page, Tr could be user types the pinyin of a character, e.g. “zhong”, the even longer. On average, however, the H value in equation computer software displays many candidate characters with (2) could be lower than 3 bits. First, as the user gains the same pronunciation, together with identifying numbers, experience, she may be able to anticipate where the correct typically in a “page” (usually a one-line graphical window). choice is likely to occur, hence changing the pi distribution For example, the first eight candidate characters for and reducing the entropy value H. Second, the choices in “zhong” could be pinyin software are usually listed by their frequency rank, giving a skewed distribution of pi values, which reduces the , H value in equation (2). which correspond to the following meanings: “1.center, There are limitations to the foregoing analysis of the 2.type, 3.heavy, 4.mass, 5.kind, 6.finale, 7.loyal, portion of choice reaction time in pinyin-based input. First, 8.swollen”. The user must then select a choice from the the input experience with each Chinese character varies multiple candidates by typing the identifier number, e.g. significantly both between users and within a user over “1”. When the candidates take more than one line, the user time. Second, the number, the sequence and the frequency has to scroll to another line of candidates by hitting a page distribution of the homophonic characters for each pinyin key. When the intended character is the first in a line, the syllable also change significantly. Third, the validity to user can select it by typing either number 1 or the spacebar. Hick’s law in the context of Chinese character recognition needs to be questioned. In short, while the Hick’s law HUMAN PERFORMANCE DIFFICULTIES IN PINYIN approach gives us some baseline points, it is still necessary INPUT to empirically measure the actual average choice selection The multiple choice selection process, in our view, is what time in Chinese input. makes today’s pinyin input much less efficient than typing Many methods have been invented to reduce the frequency in English. Our view is based on two human performance and number of choices in pinyin-based input. These factors. methods include the following five categories. Choice reaction time 1. Using additional keystrokes to represent shape or structure information of a Chinese character. The first factor is the information processing involved in multiple-choice reaction. It is possible to apply the Hick’s 2. Enable the user to input both characters and words. A law to estimate, in a first order approximation, the time Tr Chinese word is usually composed of one, two or three in such a process: Chinese characters. If the user types the pinyin of “ ” (Beijing) separately, i.e. ‘bei’ and ‘jing’, the T = Ic H (1) r two syllables have 29 and 40 candidates respectively. If where Ic is a constant between 150 to 200 ms [2] [12] and she types them as one unit, then only two candidates H is the information entropy (or uncertainty) in the n “ ” (1. Beijing, 2. Background) are the number of stimuli measured in bits: likely choices in daily language. n 3. Using a Chinese language model to reduce the = + (2) H ∑ pi log 2 (1 pi 1) uncertainty at the phrase or sentence level [14, 15]. The = i 1 language model can be either rule-based or bigram or pi is the probability for item i to be the target. In the current trigram-based, which are commonly used in speech context, it is the probability for choice i to be the intended recognition. character. In the worst case, when all choices are equally 4. Adding the Chinese intonation information after the probable, pinyin of a character. The four tones in Chinese H = log2 ( n + 1 ) (3) pronunciation are often encoded as 1-4 for additional entry. If we assume the target is in the first 7 choices appearing in the first line of display, then H = 3 bits; Tr = 450 ~ 600 ms. 5. Continuous completion. The system continuously composes possible choices based on the pinyin On average, each Chinese character’s pinyin has 4.2 Roman characters typed. As the pinyin stream grows longer, characters. For a skilled typist, an average keystroke takes the number of possible choices is reduced, until the 200 ms (corresponding to 60 wpm in English) [2, 13]. Her user decides to select a choice. pinyin typing time for each Chinese character is therefore around 200 X 4.2 = 840 ms. This means that in this case the Some of these measures help to reduce the choice reaction ratio between the time spent on actual pinyin typing (840 time, but still cannot completely eliminate it. ms) to the time to identify the correct choice (450 ~ 600 ms) is a startling 1: 0.53 ~ 1: 0.71. 350 Volume No. 3, Issue No. 1 CHI 2001 CHI 2001 • 31 MARCH – 5 APRIL Papers Numeric key typing are two shortcomings to use the eye gaze as a direct control The other problem in multiple homophonic character channel, regardless of the maturity of eye-tracking selection lies in the difficulty of touch-typing the numeric technology.