Voice Key Board: Multimodal Indic Text Input
Total Page：16
File Type：pdf, Size：1020Kb
Voice Key Board: Multimodal Indic Text Input Prasenjit Dey Ramchandrula Rahul Ajmera Kalika Bali Hewlett-Packard Labs Sitaram Human Factors International Microsoft Research India nd 24, Salarpuria Arena Hewlett-Packard Labs 310/6, H.R Complex “Scientia”, 196/36 2 Main Adugodi, Hosur Road 24, Salarpuria Arena Koramangala, 5th Block Sadashivnagar, Bangalore-560030, India Adugodi, Hosur Road Bangalore 560095, India Bangalore-560080, India [email protected] Bangalore-560030, India [email protected] [email protected] [email protected] ABSTRACT intimidating because they use modalities which are natural to our Multimodal systems, incorporating more natural input modalities everyday human communication. We are therefore motivated to like speech, hand gesture, facial expression etc., can make human- explore the use of multiple modalities to address the problem of computer-interaction more intuitive by drawing inspiration from Indic text input. spontaneous human-human-interaction. We present here a multimodal input device for Indic scripts called the Voice Key Indic scripts such as Devanagari, used for Hindi and other Indic Board (VKB) which offers a simpler and more intuitive method languages, are defined as “syllabic alphabets” in that the unit of for input of Indic scripts. VKB exploits the syllabic nature of Indic encoding is a syllable, however the corresponding graphic units language scripts and exploits the user’s mental model of Indic show distinctive internal structure and a constituent set of scripts wherein a base consonant character is modified by graphemes. The formative principles behind them may be different vowel ligatures to represent the actual syllabic character. summarized as follows [2]: We also present a user evaluation result for VKB comparing it - graphemes for independent (initial) Vs (vowel) with the most common input method for the Devanagari script, - C (consonant) graphemes with inherent neutral vowel a the InScript keyboard. The results indicate a strong user - V indication in non-initial position by means of matras preference for VKB in terms of input speed and learnability. (V diacritics) Though VKB starts with a higher user error rate compared to - ligatures for C clusters InScript, the error rate drops by 55% by the end of the - muting of inherent V by means of a special diacritic experiment, and the input speed of VKB is found to be 81% higher called virama than InScript. Our user study results point to interesting research Most Indic scripts have of the order of 600 CV units and as many directions for the use of multiple natural modalities for Indic text as 20,000 CCV ones in theory, although only a much smaller input. subset (especially of CCV units) is used in practice. The V diacritics and ligatures for C clusters are not standardized in some scripts. Therefore, standard QWERTY keyboards as means of text Categories and Subject Descriptors input in Indic scripts such as Devanagari, Tamil etc., as well as H.5.2 [User Interfaces]: User interface management systems; other syllabic scripts like Sinhalese and Thai, face unique Voice I/O; Natural language; D.2.2 [Software Engineering]: challenges due to script complexity and the sheer size of the Design Tools and Techniques - user interfaces; syllable set. Complex keyboard layouts for Indic script input add to the already existing complexity and unfamiliarity of traditional General Terms: Algorithms, Human Factors desktop user interfaces, and further raise the barrier to adoption by lay users. Keywords: multimodal systems, text input, voice keyboard, human-computer-interaction, syllabic scripts, Indic text. In this paper we describe the Voice Key Board (VKB). In order to enter a CV character, the user presses the key corresponding to a 1. INTRODUCTION consonant C on the keyboard while simultaneously uttering the The lack of a good system for Indic text input has been one of the syllable CV in speech. This results in information from the two primary barriers to penetration of computing systems in India. modalities (keyboard and speech) which are both supplementary Keyboards for Indic text input have proven unnatural, and and complementary in nature. The accurate recognition of C from difficult to learn and use. Multimodal systems [1] which use the keyboard can be used to disambiguate recognition of the multiple modalities like speech, touch and gestures for spoken CV syllable. interaction, potentially require very little learning and are less The paper is organized as follows. We review related Indic text input work in Section 2. We describe the Voice Key Board Permission to make digital or hard copies of all or part of this work for system in Section 3. Some comparisons with an existing system personal or classroom use is granted without fee provided that copies are and user evaluation results are presented in Section 4. We not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or conclude with discussion, summary and next steps, in Section 5 republish, to post on servers or to redistribute to lists, requires prior specific and Section 6. permission and/or a fee. ICMI-MLMI’09, November 2–4, 2009, Cambridge, MA, USA. Copyright 2009 ACM 978-1-60558-772-1/09/11...$10.00. 313 2. RELATED WORK character. As with IndicDasher, the base character recognition is As described earlier, Indic scripts face an unique challenge due to avoided by using the pen location, and only the vowel modifiers script complexity, and the sheer size of the syllable set resulting drawn on top of the base characters are recognized. from various CV and CCV combinations. The T9 [7] input method has also been adapted for input of Indic To accommodate such a large character set, current techniques text on mobile phones. This requires one or several key-presses to employ either a keyboard where several key strokes may be specify the base character, and another single or several key- required to enter a desired character, or a handwritten character- presses on a different key to specify the vowel modifier until the recognition technique that recognizes entire characters. For desired syllable is displayed. The prediction mechanism displays example, Figure. 1 shows the process of input of the character a list of related words from which the desired word can be /kii/ in the Devanagari script used for Hindi by pressing two keys selected, or one can continue to type more characters to get more /ka/ (base character) and /ii/ (vowel modifier) in sequence. This refined choices. Spelling out-of-vocabulary word requires QWERTY keyboard layout, called InScript, may require as many toggling to complete typing mode from the predictive T9 mode. as 3-4 keystrokes to enter a single character representing conjuncts like /kta/, /pna/ etc. which are fairly frequent in syllabic Speech only solutions relying on speech recognition are also used languages such as Hindi, Tamil and Bengali [3]. for text entry as in a dictation systems. Though intuitive, this approach suffers from issues of recognition accuracy in other application scenarios. There are usability issues even with existing systems. Hesitations, interruptions, and background noise, for example, can lead to errors that are difficult to recover from, unlike keyboards. Further, development of commercial speech engines for Indic languages is still in its infancy. 3. VOICE KEY BOARD Voice Key Board (VKB) is proposed as a simpler way to support input of Indic characters, by using voice to augment keystroke input. Here, the necessary modification of base letters for Indic k + ii = kii scripts is achieved by simultaneously speaking out the syllable represented by the CV combination, as the base character C is Figure 1: Inscript Keyboard Layout being input either by pressing a key or by writing using a pen- based device (Figure. 2). Thus, entering /kii/ would now require a Not only does the keyboard approach provide incomplete single key-stroke, that is, the user now has to press the key “k” visibility of the entire character map at any given point of time and speak out “/kii/”. For a word like “bhaarat” (India), the but these keyboards are non-intuitive and can require an extensive number of keystrokes would reduce from six using a conventional practice period for achieving proficiency of use. Also, the QWERTY layout to just three. That is, instead of pressing two increasing demand for smaller and smaller devices is driving keys for “bh”, two for “aa”, and one each for “ra” and “t”, the keyboard design towards smaller, one-handed keypads that cannot user needs to press only three keys, one each for “bh”, “r” and “t” accommodate large character sets required for Indic scripts. This while speaking out the associated syllables. This method naturally poses a severe problem when handheld devices, such as PDAs uses the phonetic nature of Indic scripts where a one-to-one and mobile phones, have to be used for input of data in these relationship exists between the character and the sound it scripts. represents. Solutions using the handwritten character-recognition approach The solution proposed here requires the integration of an allow the user to write an entire character on a graphics tablet in a Automatic Speech Recognizer (ASR) with the text-input method. natural manner using a stylus. In this approach, a character- The recognition vocabulary of the ASR is dynamically recognition engine attempts to recognize the character from the determined and is limited in size by the base character keyed in, strokes entered on the tablet. However, current handwritten resulting in faster and better recognition. For example, when /kii/ character-recognition techniques give significantly low is entered by keying in the base letter “k” and speaking out recognition accuracies for Indic scripts due to their complex “/kii/”, the active vocabulary of the ASR would be limited to /ka/, shapes, and the considerable variety in writing styles.