OpenVanilla – A Non-Intrusive Plug-In Framework of Text Services Tian-Jian Jiang1,2, Deng-Liu, Kang-min Liu, Weizhong Yang3, Pek-tiong Tan4, Mengjuei Hsieh5, Tsung-hsiang Chang, Wen-Lien Hsu1,2 1. Department of Computer Science, National Tsing Hua University 2. Institute of Information Science, Academia Sinica 3. Department of Theatre, Taipei National University of Arts 4. Department of Physics, National Tsing Hua University 5. School of Information and Computer Sciences, University of California Irvine 128 Academia Road, Section 2, Nankang, Taipei 115, Taiwan {tmjiang,hsu}@iis.sinica.edu.tw ABSTRACT not to know which applications they are serving. Because Input method (IM) is a sine qua non for text entry of many of this quality, IMs can be seen as non-intrusive plug-ins Asian languages, but its potential applications on other lan- that adds functionalities to a running application. We will guages remain under-explored. This paper proposes a phi- explore the potential of this quality in the present research losophy of input method design by seeing it as a non- and show that IMs are not only useful for Asian languages, intrusive plug-in text service framework. Such design al- but also for European, African and even artificial lan- lows new functionalities of text processing to be attached guages. onto a running application without any tweaking of code. It should be noted that although alternatives to keyboard We also introduce OpenVanilla, a cross-platform frame- exist, none of them - speech recognition, handwriting rec- work that is designed with the above-mentioned model in ognition or optical characters recognition (OCR) - equals mind. Frameworks like OpenVanilla have shown that an keyboard in text entry efficiency and accuracy. As we input method can be more than just a text entry tool: it of- won't see the end of the keyboard's day anytime sooner, fers a convenient way for developing various text service input method will continue to play an important role in our and language tools. desktop environment. ACM Classification: H5.2 [Information interfaces and HUMAN FACTOR, USABILITY AND UI DESIGN ISSUES presentation]: User Interfaces. - Input devices and strate- The Keyboard Layout Problem gies, Interaction styles, Natural language. The QWERTY keyboard was designed for one single lan- General terms: Design, Human Factors guage. Modifications have been applied to the QWERTY Keywords: Input method, non-intrusive plug-in frame- keyboard to suit the needs of other European languages work, text service such as French QWERTY and German QWERTZ key- boards. This is the origin of the “keyboard layout” of mod- INTRODUCTION ern GUI environments. Most ideograph-based Asian languages consist of thou- sands of complex characters. It would be impractical to Although the idea of keyboard layout seems to have solved create a huge keyboard to map every possible character. the problem of language-specific text entry well, it has Modern GUI environments, be it Microsoft Windows, Mac limitations. First, it is impossible to enter more than one OS X, or X11, all come with build-in tools for transforming language with one single layout. For example, a French multiple composition keystrokes into one single ideograph. keyboard may not be capable of entering Polish text. Sec- These tools are known as input methods or IM for short. ond, it is impractical to attach many different keyboards to a single device. To have French and Polish keyboard at- IMs are often categorized into “radical-based” or “phonet- tached to one laptop computer is unimaginable. ics-based” methods. With radical-based input methods, users get a character by typing the composing radicals, Finally, many devices simply cannot accommodate a large whereas with phonetics-based ones users get a character by keyboard. For example, mobile phones usually have less typing the syllables. If there are homophones, a choice will than 20 keys. This is where keyboard layouts become un- have to be made, and the proper character is selected and feasible: there is simply not enough space. Using input entered. methods is the solution that is scalable up to multiple lan- guage text entry, and down to limited hardware device. As most modern GUI environments are designed with standardized IM APIs, applications usually do not have to Input methods like T9 [1] have achieved more balance worry if a user is entering Latin characters or Asian- situation between the trade-off of keystroke numbers and language texts (and, in the latter case, applications need not keystroke-character conversion collision rates after some to know which kind of input method the user is using). research of human factor and language model. It's valuable That is to say, text entry is transparent to the applications. to mention Hsu's keyboard layout [2], which maps Chinese Likewise, IMs work through a standardized API and need bopomofo phonemes to 25 keys by phonetic rules and simi- lar alphabets, as the following figure shows, it is more effi- ment for years. The last is no easier than its Microsoft or cient than any traditional layout, which occupies 40 keys. Apple equivalents, though. Such complexities and difficulties are the reason why many input method frameworks have been flourishing: both IIIMF and SCIM aim to relief the pain of IM development on X Window. The Japanese UIM aims to provide a unified interface to more than two dozens of different IM modules, so that the whole set can be easily ported. OpenVanilla Input method also solves dead-key problems on many key- serves Mac OS X and X Window well. In general, a boards of European languages. When the size of keyboard framework should provide a set of abstract API, and takes is limited, using dead keys to input several symbol is also either a dynamic-loading or client-server approach to work impractical. as a mediator between input method modules, operating system, and applications. A framework should also imple- Text Service ment a set of default widgets and event handlers. With such So far there are two flavors of definitions of text services. “facilities,” an IM developer can concentrate on algorithm Microsoft describes its Text Service Framework (TSF) by design with no further concerns on platform-specific de- “TSF provides a simple and scalable framework for the tails. delivery of advanced text input and natural language tech- Furthermore, when IMs convert source key codes to re- nologies. ... A TSF text service provides multilanguage quired target type, they are like output filters more in some support and delivers text services such as keyboard proces- sense. Actually, even using the same input method, output sors, handwriting recognition, and speech recognition.” On results still have chances to be tweaked by applying output the other hand, Apple defines its Text Service Manager filters for different purposes, e.g. conversion between Tra- (TSM) as “A text service is a specific text-handling task ditional Chinese and Simplified Chinese. such as spell-checking, hyphenation, and handling input of In short, given that problems of keyboard is still the main complex text.” Since this paper is focused on “keyboard” input device, human factors of input remain places for im- input method, we found it would be interesting to compare provement, many text processing requirements are still with spell-checking task. Although spelling checker is a unsatisfied, and input method development are not easy. handy function of MS Word, it is only available there. INTRODUCING OPENVANILLA However, it is possible to develop an input method for this OpenVanilla is the successor of two successful open source requirement, then a “check as you type” spelling checker projects on the Mac OS X platform: VanillaInput and will be born. SpaceChewing. Both projects were designed to provide Candidate List input methods that have been (and still are) inadquately During the text entry process, chances are that a sequence supported by Apple's built-in modules. OpenVanilla is de- of keystrokes maps to multiple characters or phrases. It is signed as an abstract text input/output service framework, especially so for phonetics-based input methods, such as and after two major releases (0.6 and 0.7) it currently en- Pinyin for Chinese or Kana for Japanese. A user must then joys a wide user base. pick up the exact character/phrase he or she wants from a From IM development's point of view, OpenVanilla is de- list of possible choices. Such interaction requires display- signed with the following two principles: ing those candidate characters/phrases on screen first and The framework and a set of various IM modules waiting for a choice. We call this special UI widget “candi- • should be easy to deploy. date list”. The framework should offer a unified, platform- Candidate list is an indispensable widget for Asian lan- • independent interface to save IM developers' times on guage text entry. However, it can have applications other investigating complex platform-specific issues. It than picking up a proper character; it could also serve as an should allow anyone with some basic knowledge of on-the-fly spelling checker for European languages. More C/C++ to be able to write his or her own IM module. generally, it is a context-sensitive UI widget for any type of text services. OpenVanilla is divided into two parts: platform-dependent loaders and (mostly) platform-independent IM modules. HOW INPUT METHOD WORKS OpenVanilla is actually a very thin layer of interface be- Most modern GUI environments offer a set of low-level tween the two parts, because it consists of only two C++ API for writing an input method module, but nothing more. header files. This makes OpenVanilla, especially its IM A developer that wishes to build up a fully functional IM modules, extremely easy to port and deploy.