CHIME: An Efficient Error-Tolerant Chinese Pinyin Input Method Yabin Zheng1, Chen Li2, and Maosong Sun1 1State Key Laboratory of Intelligent Technology and Systems Tsinghua National Laboratory for Information Science and Technology Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China 2Department of Computer Science, University of California, Irvine, CA 92697-3435, USA fyabin.zheng,[email protected], [email protected] Abstract Chinese Pinyin input methods are very importan- t for Chinese language processing. In many cases, Figure 1: Typical Chinese Pinyin input method for a correct users may make typing errors. For example, a user Pinyin (left) and a mistyped Pinyin (right). wants to type in “shenme” (什么, meaning “what” in English) but may type in “shenem” instead. Ex- isting Pinyin input methods fail in converting such to the corresponding Chinese words. In the running example, a Pinyin sequence with errors to the right Chinese for the mistyped Pinyin “shanghaai”, we can find a similar words. To solve this problem, we developed an Pinyin “shanghai”, and suggest Chinese words including “上 efficient error-tolerant Pinyin input method called 海”, which is indeed what the user is looking for. “CHIME” that can handle typing errors. By incor- When developing an error-tolerant Chinese input method, porating state-of-the-art techniques and language- we are facing two main challenges. First, Pinyins and Chi- specific features, the method achieves a better per- nese characters have two-way ambiguity. Many Chinese formance than state-of-the-art input methods. It can characters have multiple pronunciations. The character “和” efficiently find relevant words in milliseconds for can be pronounced as “hé”, “hè”, “huó”, “huò”, and “hú”. an input Pinyin sequence. On the other hand, different Chinese characters can have the same pronunciation. Both characters “人” and “仁” have the same pronunciation “rén”. The ambiguity makes it techni- 1 Introduction cally challenging to do Pinyin-to-Chinese conversion. Notice Chinese uses a logographic writing system, while English and that it is important for us to suggest relevant words to a user, many other romance languages use an alphabetic writing sys- since otherwise the user may be annoyed by seeing suggested tem. Users cannot type in Chinese characters directly using words that are irrelevant to the input sequence. Thus we need a Latin keyboard. Pinyin input methods have been widely to detect and correct mistyped Pinyins accurately. used to allow users to type in Chinese characters using such Another challenge is how to do the correction efficiently. a keyboard. Pinyin input methods convert a Pinyin sequence Existing input methods suggest Chinese words as a user types to the corresponding Chinese words, where Pinyins are used in a Pinyin sequence letter by letter. Studies have shown that to represent pronunciations of Chinese words. For instance, we need to answer a keystroke query within 100 ms in order suppose a user wants to type in the Chinese word “上海” (the to achieve an interactive speed, i.e., the user does not feel any city Shanghai). He types in the corresponding Pinyin “shang- delay. Achieving such a high speed is especially importan- hai”, and the input method displays a list of Chinese words t when the method is used by a Web server that can receive with this pronunciation, as shown in Figure 1 (left). The user queries from many users, and requires a method to find rele- selects the word “上海” as the result. vant Chinese words for a mistyped sequence efficiently. As users of other languages, Chinese users often make er- In this paper we develop a novel error-tolerant Chinese rors when typing in a Pinyin sequence. Existing Pinyin-based Pinyin input method that can meet both requirements of high methods have error-tolerant features such as fuzzy tone for accuracy and high efficiency. It is called “CHIME”, which Chinese southern dialects. However, there are many input er- stands for “CHinese Input Method with Errors.” CHIME rors that cannot be handled by these methods. In our example works on a Pinyin sequence as follows. For a mistyped Pinyin of “shanghai” (上海), the user may make typos, thus type in in the sequence that does not exist in a Pinyin dictionary, “shanghaai” instead. As shown in Figure 1 (right), typical we find similar Pinyins as candidates. We rank them using methods cannot return the right word. This limitation affects a noisy channel model and Chinese-specific features to find user experiences greatly, since he has to identify and correct those that are most likely what the user intends to type in for the typos, or cannot find the right word. this Pinyin. Finally, we use a statistical language model on In this paper, we study how to automatically detect and cor- the input sequence to find the most likely sequence of Chi- rect typing errors in an input Pinyin sequence, and convert it nese words. The rest of this paper is organized as follows. Section 2 3 Correcting a Single Pinyin reviews related work. Section 3 analyzes typing errors at the We first study the problem of how to correct a single Pinyin. Pinyin level. In Section 4, we study how to convert a Pinyin Given a dictionary D of Pinyins and an input Pinyin p that is sequence to a sequence of Chinese words. In Section 5, we not in the dictionary, our method CHIME finds a set of can- report experimental results. Section 6 concludes the paper didate Pinyins w 2 D that are most likely to be erroneously and discusses future work. typed in as p. First, we do a similarity search to find candidate Pinyins that are similar to the input Pinyin. Second, we rank 2 Related Work these candidates and find those with the highest probabilities to be what the user intends to type in. Chen and Lee [2000] used a statistical segmentation and a trigram-based language model to convert Chinese Pinyin se- 3.1 Finding Similar Pinyins quences to Chinese word sequences. They also proposed an Measuring Pinyin Similarity: As shown in [Cooper, 1983], error model for Pinyin spelling correction. However, they on- most typing errors can be classified into four categories: in- ly correct single-character errors, while our method can cor- sertions, deletions, substitutions, and transpositions. There- rect multi-character errors. fore, in this paper we use edit distance to measure the number Kwok and Deng [2002] proposed a method of extracting of errors between a candidate Pinyin and a mistyped Pinyin. Chinese Pinyin names from English text and suggesting cor- Formally, the edit distance between two strings is the mini- responding Chinese characters to these Pinyins. However, mum number of single-character operations required to trans- their approach can only convert Pinyin names to Chinese form one sting to the other. We generalize the definition of characters, while our method can handle general words. edit distance by allowing transpositions. There are studies on spelling correction mainly for English. We assume an upper threshold on the number of typing Brill and Moore [2000] introduce a noisy channel model for errors a user can make in a single Pinyin. In other words, for spelling correction based on generic string-to-string edit op- a mistyped Pinyin, we assume that the edit distance between erations. Without a language model, their error model gives the correct Pinyin and the input one is within the threshold. a 52% reduction in spelling correction error rate. With a Damerau [1964] showed that 80% of typing errors are caused language model, their model gives a 74% reduction in er- by a single edit operation. Using a threshold of 2, we can ror rate. Toutanova and Moore [2002] extend the work using find the correct Pinyin with a very high probability. For pronunciation information. They use pronunciation similar- example, for the Pinyin “sanghaai”, by finding those whose ities between words to achieve a better accuracy. Inspired edit distance to the input is within 2, we can find candidates by [Toutanova and Moore, 2002], we also take pronuncia- “shanghai”, “canghai”, and “wanghuai”. tion information in Chinese to gain better performance. More details are given in Section 3. Efficient Similarity Search: We adopt the state-of-the- [ et Cucerzan and Brill [2004] use information in search query art index structure and search algorithm proposed in Ji al. ] logs for spelling correction. Their approach utilizes an itera- , 2009 to find those similar candidate Pinyins efficiently. tive transformation of the input query strings into other cor- The main idea is to build a trie structure to index the Pinyins rect query strings according to statistics learned from search in the dictionary. As a user types in a Pinyin letter by letter, query logs. Sun et al. [2010] extract query-correction pairs the algorithm can on-the-fly do a similarity search to find by analyzing user behaviors encoded in the click-through da- those trie nodes, such that the edit distances between the ta, and then a phrase-based error model is trained and inte- corresponding prefixes and the input Pinyin are within a grated into a query speller system. Gao et al. [2010] make given threshold. From the trie nodes of these similar prefixes significant extensions to a noisy channel model and treat spel- we can find their leaf nodes as candidate similar Pinyins. woemng l checking as a statistical translation problem. Experiments Consider the sequence in our running example, “ gounai sanghaai shengchang niulai show that these extensions lead to remarkable improvements. le de ”. For each Pinyin, we find similar candidate Pinyins in the dictionary, whose edit [ ] Whitelaw et al. 2009 implement a system for spelling distance to the given Pinyin is no greater than 2.