Chinese Phonetic Handwritten Text Input for Mobile Phones

Ethan Cheng Renaud Amar Nada Matic Synaptics Inc. 3120 Scott Blvd. Santa Clara, CA 95054, USA {echeng,ramar,nada}@synaptics.com

ABSTRACT keypad forces mapping of more than one character per key. We present a novel user-friendly Chinese text input method CHINESE TEXT INPUT SOLUTION for mobile phones and other small form-factor-devices. We designed and developed an efficient and user-friendly Our technique is based on handwritten phonetic input text input system for the Chinese alphabet on mobile rather than handwritten ideogram input. Using our system, phones. It is based on handwritten input, where the user the user is able to write any Chinese characters (Traditional enters phonetic symbols on a touch sensitive surface [4] or Simplified) using finger movements on the touch sensi- mounted under the standard phone keymat using a finger tive surface mounted under a standard cell phone keymat. (see Fig. 1). To write a Chinese word via its phonetic representation, users need to enter a sequence of phonetic symbols fol- Using our system, the user is able to input any combination lowed by the selection of the desired ideogram. The hand- of Chinese characters (Traditional or Simplified), Latin writing recognizer generates ideogram candidate list. alphabet, numbers and punctuation marks. In contrast to standard Chinese handwriting based text input systems [2], ACM Classification: H5.2 [User Interfaces]: Input Devices where the user needs to enter many strokes of the ideo- Keywords: Chinese text input, pinyin input, bopomofo grams, we designed a phonetic based handwritten input input, mobile phones, text messaging, on-line handwriting method: pinyin for Simplified Chinese and bopomofo for recognition, neural networks, TouchPad. Traditional Chinese. INTRODUCTION The pinyin phonetic based method uses the English alpha- As computer technology improves and becomes more bet to enter standard Mandarin pronunciations of Chinese widespread, writers of ideographic characters need more characters (see Fig 2.), while the bopomofo phonetic based friendly text input systems to support one of the most fre- alphabet uses another set of 37 phonetic symbols to docu- quent human computer interaction tasks. ment the same standard Mandarin pronunciation. There is a Chinese text input represents a particularly challenging straight one-to one correspondence between Bopomofo and task. On PCs text input is usually accomplished using ei- Pinyin symbols. ther a QWERTY typewriter-style keyboard with phonetic To enter Chinese ideogram via its phonetic representation a input method like pinyin and bopomofo or using handwrit- user enters a sequence of English or bopomofo symbols ing input on TouchPads or other types of writing pads followed by an “end of word” gesture or button press. where the user can trace Chinese ideograms using a finger Within the English/bopomofo symbol sequence, a system or stylus. Handwriting systems have to be coupled with has to rely on timeout to differentiate individual symbols. ideogram handwriting recognition software [2]. Small form-factor devices like mobile phones are likely to The small writing area available on mobile phones vs. the have space available for writing only a single alphabet size of the writing instrument (finger) prohibits accurate symbol, forcing all the alphabet symbols to be written on input and recognition of the many strokes of a typical ideo- top of one another. Recognition software displays multiple gram. The limited size of mobile phones cannot accommo- candidate characters [Fig. 2] with the same pronunciation. date a QWERTY style of keyboard. Mapping bopomofo The user then selects the desired ideogram using the and pinyin symbols to the standard 12-key mobile phone phone’s joystick. In addition, a whole phrase that starts with a particular ideogram can be selected from the list of Permission to make digital or hard copies of all or part of this work for phrases in the phrase dictionary. If the desired ideogram (or personal or classroom use is granted without fee provided that copies are phrase) is not among those displayed, the user needs to not made or distributed for profit or commercial advantage and that copies scroll through the complete list. bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior IMPLEMENTATION specific permission and/or a fee. We developed and ported our recognizers [2,4] for the pin- UIST’05, October 23–27, 2005, Seattle, Washington, USA. Copyright ACM 1-59593-023-X/05/0010...$5.00. yin and bopomofo input method to the Nokia 7650 smart phone that has been retrofitted with an integrated capaci- tive touch-sensitive input device (MobileTouch). The soft-

ware is running under the English version of the Symbian V6.1 Operating System. For more information about the Synaptics Mobile Touch hardware capacitive technology see [3].

Synaptics Chinese handwriting input technology should work on conventional mobile phones with touch sensitive hardware or on resistive transparent displays capable of capturing x and y coordinates of the pen or finger move- ment. Our accurate and robust recognizer technology is neural

network based [2,4]. Neural networks are often used to design systems for human interfaces. For example, recogni- tion systems for speech, handwriting and visual gestures all process human interface inputs. Example of on-line neural based recognition system can be found in [1]. The timeout was set to 0.3 seconds. This amount of time was selected as a reasonable compromise between speed of entry and accuracy. Decreasing the time interval leads to segmentation errors. Individual users can set the timeout interval to match their speed of writing. FUTURE WORK We need to conduct controlled experiments that will com- pare our handwritten phonetic text input technique to the commonly used key-based input techniques. Due to the Figure 1: Nokia 7650 Smart phone with touch- homophonic nature of Chinese the ideogram selection sensitive sensor under the standard keymat can process can be slow. We plan to optimize the selection capture x and y coordinates of finger movements. process by introducing intonation symbols, Chinese lan- Strokes of the individual phonetic symbols are sent guage model and context. to the recognizer. Recognizer generates a list of ideograms. User can select desired ideogram or ACKNOWLEDGMENTS phrase. We would like to thank our co-workers at Synaptics-UK office for help and support with hardware and software integration of Nokia 7650 smart phone with Mobile Touch sensor.

REFERENCES 1. Guyon I., Bromley J., Matic N., Schenkel M., and Weissman H. Penacee: A neural network system for recognizing on-line handwriting. In E. Domany, J. L. Van Hemmen, and K. Schulten, editors, Models of Neu- ral Networks, volume 3, pages 255--279, Springer.1995. 2. Nada P. Matic, John C. Platt, Tony Wang. QuickStroke: An Incremental On-line Chinese Handwriting Recogni- tion System. In ICPR International Conference on Pat- tern Recognition, pages 435-439, 2002 Figure 2: GUI for Chinese pinyin based text input. 3. MobileTouch: Top left is the recognized phonetic sequence. A se- http://www.synaptics.com/products/mobto.cfm lection of nine ideograms produced by recognizer is displayed below the phonetic sequence. Once a de- 4. U.S. Patent 5,812,698 Handwriting Recognition System sired ideogram is selected (red) the corresponding and Method. phrases for the ideogram are displayed on the right. Finally, the user can select a phrase and send it to the application.