Aiding Text Entry of Foreign Alphabets with Visual Keyboard Plus Lara Rennie Andy Cockburn Department of Computer Science and Software Engineering University of Canterbury Christchurch, New Zealand lre15, andy @cosc.canterbury.ac.nz { } Abstract guages and alphabets of the world. Millions of multi- lingual users, and those who use keyboards with dif- Computer keyboards are used to input hundreds of different ferent native mappings than their natural language languages using many different alphabets. Despite this diver- (as is often the case in Internet Cafes around the sity, the physical layout of keyboards is fairly uniform, with world), must alter the keybindings to allow them to keyboards generally containing approximately 80 keys spread work efficiently. across six rows (excluding cursor keys and numberpad). In En- glish speaking countries, the QWERTY layout is the de-facto Most operating systems allow users to alter the standard binding between the physical location of keys and the keybindings so that the alphabets of different lan- corresponding letters of the alphabet. guages can be entered efficiently. In addition, soft- To aid international and multi-lingual computer use, oper- ware systems such as Microsoft’s Visual Keyboard al- ating systems allow users to alter bindings between physical low users to view the relationship between they physi- keys and resultant characters, but this raises a problem for cal keys and underlying alphabet—Figure 1(a) shows users as the labels on the physical keys will not match those Microsoft’s Visual Keyboard with the Russian lan- of the bindings. Software user interfaces such as Microsoft's guage keybindings. Visual Keyboard (MVK) help users by providing a visual de- Despite the common use of modified multi-lingual piction of the keyboard's new bindings, but users still suffer an overhead in establishing the mapping between the physical and keybindings and associated Visual Keyboard soft- displayed keys. ware, there has been little prior research into their This paper describes a comparative analysis and empiri- effectiveness. In this paper we analyse and evalu- cal evaluation of three alternative techniques for helping users ate three aids for working with modified keybind- input non-standard alphabets using a standard keyboard. In ings. The three aids are stickers placed over the QW- particular we investigate whether our VKPLUS (Visual Key- ERTY keys showing the new symbols, Microsoft’s vi- board Plus) user interface, which displays both the physical key sual keyboard (MVK), and our modified visual key- labels and the new keybindings, improves text entry rates over board (VKPLUS), which displays both the new key- Microsoft's Visual Keyboard. The third technique, included for binding and the original ‘native’ labels on the physical baseline comparison, uses sticky-labels placed over the physical keyboard. keyboard. Results show that VKPLUS improves performance In our next section, we examine past methods for over Microsoft's system. facilitating input of non-English alphabets. We then describe our experiment, with a description and jus- Keywords: Text input, keyboard keybindings, visual tification of each of the three methods of support for keyboards, international alphabets modified keybindings that we are testing. A discus- sion is then given of their performance. Finally, we draw conclusions on the usefulness of each typing aid, 1 Introduction and make recommendations for those sectors where facilitating multi-lingual computing is of importance. Since the invention of typewriting by Christophrel Sholes, Carlos Glidden and Samuel Soule in 1867 (Baddeley 1998), the QWERTY keyboard has be- 2 Background come the standard method for English text entry, en- trenched after the 1915 adoption by training schools 2.1 Text Entry of non-English Characters of touch-typing (Yamada 1980) which meant users no longer used visual cues in typing, and hence required To meet the challenge of entering non-English char- stability and standardisation of layout to retain pro- acters on any system, many methods have been de- ficiency. veloped. The simplest of these is the use of the ‘insert The physical layout of keyboards is largely stan- symbol’ command available on most word-processing dard throughout the world, but the mapping be- systems. However, this has usually a limited charac- tween physical keys and underlying characters nat- ter set, is limited to a specific application, and is very urally varies dramatically across the different lan- time-consuming to use. It is suited to cases where one or two letters absent from the English alphabet are re- Copyright c 2005, Australian Computer Society, Inc. This pa- quired. Similarly, short-cut keys exist for symbols not per appeared at the 6th Australasian User Interface Conference directly provided by the keyboard, typically the ac- (AUIC2005), Newcastle. Conferences in Research and Practice in cents on vowels needed for other latin languages, such Information Technology, Vol. 40. M. Billinghurst and A. Cock- as the ´e used in typesetting French. This requires a burn, Eds. Reproduction for academic, not-for profit purposes permitted provided this text is included. significant memory load on the user when only six ex- tra symbols are required, as is the case for typing in French; using such a method for a language which has board layouts, although it could be utilised if it was an entire alphabet of non-Latin characters would be necessary to quickly train a particular group of users. extremely demanding for the user. It would also be The technology required is expensive, and not readily time-consuming, as for every symbol more than one available. It also slightly increases the physical effort key would need to be pressed. involved in pressing a key, although the users would To circumvent these problems, several methods of soon become familiar with this. Some inexpensive text entry for foreign languages have been developed support for typing using modified keybindings is also that avoid the use of the keyboard altogether. One currently available, such as keyboard stickers, which such method is that developed by Henry (1992). This can be placed over the current keyboard to enable bypasses the need to spend time learning a new key- the user to locate the desired symbol. Microsoft Cor- board by presenting a pool of text chunks that can poration has also produced a visual keyboard(MVK) be selected to create the sentences the user is try- to aid typing using different keyboard layouts. This ing to enter. This would include syllables, common shows the keyboard layout on the screen, to use as a complete words, as well as individual letters. A more map for finding any symbol on the physical keyboard common yet similar method is that of virtual key- (MSC 2000). When a key is pressed, the correspond- boards, where the user is presented with a keyboard ing key on the visual keyboard is highlighted. How- on the screen, and uses the mouse or a stylus to select ever, there has been no previous evaluation of these the appropriate key on the screen to make the symbol more accessible methods. appear. However, this is, as is any method eschewing 3 Evaluation the physical keyboard, inherently slow as it requires pointing between different letters, rather than util- The three support methods to be evaluated are pre- ising the natural dexterity of humans as does tra- sented below, and the theorised ways in which they ditional typing. Skilled users of virtual keyboards may assist typing are described. have hence been proven to be incapable of match- ing the speeds achieved by experienced typists on normal keyboards, with a maximum input speed of 3.1 Keyboard stickers 31.6 wpm attained on virtual keyboards in recent experiments by Shumin Zhai & Accot (2002), who Keyboard stickers aid the user in locating the correct also predicted that with the optimum keyboard lay- key to press for the desired symbol as they place the out currently proposed a maximum of 46.6 wpm could new symbols over each key in the keyboard. Typing be achieved. This means that although virtual key- ability would now be similar to those using a QW- boards may show some speed advantage initially, it is ERTY keyboard for the first time, as the user must preferable that the user learn to use a physical key- look at the keyboard itself to determine the location board to input the symbols. For expert users who of a key. An advantage of this method is that no have memorised the location of every symbol, per- mapping is required to translate between finding a formance would be equitable to that achieved by ex- symbol and knowing which key to press. However, pert users of the QWERTY interface, with any differ- there are concerns both that the fingers themselves ences attributable to the movement efficiency of the could obscure the symbols, making them difficult to keyboard layout for the new language. To this end, locate, and that learning to type in such a way would modifiable keybindings have been developed, taking not encourage the development of touch-typing with advantage of the speeds that can be attained from the new layout, thus limiting future efficiency. touch typing. The main keypad of the stickered keyboard used in However, for novice users the speed of text input the evaluation measured 28.5cm across by 9.6cm, with is determined mostly by the time taken to search keys measuring 1.2cm across by 1.3cm. Keyboard and find the required key, rather than the time stickers featured symbols 4-9mm high. taken to move their finger to press this key (Smith & Zhai 2001). This means that any support offered 3.2 Microsoft's Visual Keyboard - MVK in finding the appropriate key would be invaluable in assisting novice users come to terms with a new A visual keyboard may be a useful way of aiding keyboard layout.