Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

Survey of Eye-Free Text Entry Techniques of Touch Screen Mobile Devices Designed for Visually Impaired Users Sonali Banubakode1, Chitra Dhawale 2 [email protected] 2MCA Department, P.R. Pote College, Amravati, India [email protected]

Abstract: Now a days touch screen mobiles are becoming more popular amongst sighted as well visually impaired people due to its simple interface and efficient interaction techniques. Most of the touch screen devices designed for visually impaired users based on screen readers, haptic and different (UI). In this paper we present a critical review of different keypad layouts designed for visually impaired users and their effect on text entry speed. And try to list out key issues to extend accessibility and text entry rate of touch screen devices.

Keywords: Text entry rate, touch screen mobile devices, visually impaired users.

I. Introduction increasingly common in both sighted Visual impairment describes any and visually impaired people. These kind of vision loss, whether it‟s devices used not only for about total blindness or partial vision entertainment and communication loss. Some people are totally blind purpose but for learning, browsing, but many others have legal e-billing and many more. But they blindness. They don‟t have enough are highly visual demanding. Touch vision to see the object stands 20 feet screen is highly sensitive; it contains away from them (Arditi & lots of tiny icons and requires more Rosenthal, 1998). According to concentration as well as fast action. recent statistics of WHO (Chatterjee, It is not easy for any person with 1198) 285 million people are vision problem to handle such visually impaired. From which 39 devices satisfying above demands. million are blind and 246 have low Though there are some touch screen vision. devices available for vision impaired

Touch screen mobile devices known people, it remains inaccessible in as , becoming many ways. Most of the available devices use screen readers like Jaws,

1 Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

Apple Voice Over, etc; screen degree as to qualify as an additional magnifiers and support tactile support need through a significant feedback. These techniques help to limitation of visual capability support navigate the screens and resulting from either disease, trauma, search the desire words or icons. or congenital or degenerative Some devices come with different conditions that cannot be corrected types of keypad layouts other than by conventional means, such as traditional one. They also support refractive correction or medication TTS techniques and tactile feedbacks (WHO; Bill Text,2003-2004; Belote which help to enter text without & Lary,2006). watching the screen. Though these Problems faced by vision impaired ;n techniques try to make touch screen size which causes inconvenience in devices more users friendly for handling the device and some have visually impaired people, there are too small to select only one letter at a some of drawbacks like: lack of time. clearness in sound, more time Button size: Some devices have consumption in navigation, etc; fails too small buttons that either do not to increase accessibility of these click when pressed or adjacent other devices. Also the text entry rate is button is clicked. Thus provide not so good which reduces fast wrong feedback. responses on touch screen. Keypad layouts: Every device In next section, we discuss about provides different layouts of accessibility problems of keypads. Novice users have to spend devices faced by more time on learning and being visually impaired people. In section familiar to it. III, related work in this area is Menus: Large number of menus studied. In section IV, we discuss causes difficulties in understanding about various text entry evaluation and selecting. metrics. In section V, comparative Text size: Small text size is unable analysis of available devices based to read. on keypad layout is performed. Feedback: Some sound and tactile Section VI concluded our survey feedbacks are not clear to easily with providing some key issues to understand. improve text entry technique and Text entry rate: Low text entry accessibility of touch screen mobile rate causes obstacles in fast typing devices for vision loss users. and response. Time Delay: Some approaches II. Visual Impairment and have more time laps between key Accessibility Problems touch and recognition which causes Visual impairment ( or vision irritation and unwanted time loss of impairment ) is vision loss to such a expert users. 2

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

Cost: Disabled users either have to is displayed on screen, rather than purchase mobile devices developed using mouse or ; secondly, for them or have to download that does not require any various application required making intermediate device that would need the device accessible by paying large to be held in hand (other than stylus). amount. Every user can not afford it. 1). Single-touch based strategies: Basic touch screen functionality is I. Touchscreen Mobile Devices for single-touch, where you touch the the Visual Impaired Users screen like mouse moving around the From last two decades frequency of screen and „tap‟ the screen like a evaluating new text entry method mouse click. In this strategy a finger increases due to recent and or any (gesture) is heightened interest in touchscreen ( used to enter the text i.e. Smartphone) and use of SMS. According to statistics 9.8T SMS 2). Multi touch-based strategies: messages were sent in 2012 Multi-touch refers to ability of touch (Factbrowser.com, 2012). In 1997 sensing surface to recognize the first touchscreen was developed by presence of two or more points of Dr. Sam Hurst of Elographics, now contacts on the surface. This dual- known as Elo TechSystems (Brown point awareness is used for pinch to et.al, 1971). The design called, zoom or activating predefined “elograph”, was a computer input programs. devices that uses resistive A. Text Entry factors touchscreen technology. Generally Following are some factors on touchscreen has two main attributes: which affects text entry. first, one directly interacts with what

TABLE I: TEXT ENTRY FACTORS Text Entry Explanation Factors Speed/Efficiency Speed of text to be keyed by either using muiltitap or single touch Learnability Comfortness of user to learn the text entry mechanism. Simplicity simplicity to use text entry mechanism Navigation Comfortness for key selection while texting (eg. Punctuation, blank space, capitalization, etc) 3

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

It is seen that all these factors are searching for a specific character or related with keypad design and group of characters along the screen. hence text entry is strongly affected Also this approach suffers from by layout of keypad of mobile problem of segmentation, when the device. character is on the same key as previous one. B. Keypad Layout One of the important factors In 2011 Aakar Gupta and Navkar responsible for text entry speed is Samdaria implemented SVIFT keypad layout. Researchers proposed (Gupta & Samdaria,2011) which different keypad layouts for their uses vibrotactile technique for eye- applications. free text entry. The technique involves expanding the text-entry 1) keypad mode to a full screen format and This multi-touch approach is used by innovating over the old-style T9 and Sanchez and Aguayo in 2007. They telephone keypad design. It makes proposed 9-button virtual keyboard use of multiple input and feedback layout called as Mobile Messenger interactions – swipe, pause, circle, for the blind (Yfantidis & hand-waving, audio and vibrotactile. Evreinoy,2006). It is a messaging Its text entry speed is 4.75 wpm. But system for mobile devices having its input mechanism for special multiple characters on each key character is not efficient. Also supported by audio feedback. It has number appears in separate mode less number of targets which are which is not convenient to select. placed at easy- to-reference location 2) Stroke based Keypad e.g. near the edge of the screen that make blind user easily find them. In 1993 Goldberg and Richerdson But problem increases when first proposed Unistokes [13] based numbers of targets are increases. In keypad. They use Single-touch such target-based system text strategy for text entry. These strokes entering become more difficult. are very simple and user writes it without watching the stylus. Its text In 2008 Guerreiro et al. (Guerreiro entry rate was 16 wpm. But these et al ,2008) presents 12 button based strokes are different from regular approach of touch screen mobile handwritten or printed letters. Hence known as MultiTap. Each button they must be learned and an expert is features a set of characters. It proved needed for fast text entry. Also to be faster as it offers a more direct Unistrokes could not used for mapping between input and desired number, punctuation, or symbolic output. Its text entry rate was 0.88 characters. wpm with 15.28% error rate. But with MultiTap problem arises during 4 Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

In 1995 Palm Inc. also designed of list use L-select gesture. Its single-stroke alphabets recognition interface is entirely speech-based and system that uses Graffiti strokes it does not support any visual (Blickenstorfer,1995). It was used to feedback. Its text entry rate is 27 draw upper-case characters blindly with 14.1% error rate. The major with stylus on touch panel. It is problem with Slide Rule is that it has similar with hand written or printed no visual representation. It does not letters. Hence no need of experts. display item labels and its targets are small and densely pack. Tinawal and Mackenzie uses Graffiti strokes for eye-free text entry In 2011 Apple announced a system method for touch screen mobile called VoiceOver (Buzzi et.al, 2011) phones in 2009 (Tinawal & based on one graphical QWERTY Mackenzie, 2009) The entire screen . It offers a function is used as drawing surface. Graffiti to correct an error. User can adjust has strokes for number, punctuations, its speed depending upon preference. symbolic characters and mode Also when it is activated other phone switches. The entry is guided by related sounds are automatically speech, sounds and vibrations. The lowered. Its text entry rate was 0.66 unrecognized entry is guided by wpm and error rate is 9.7%. But it phone‟s vibration actuator pulses. Its displays a large number of targets in text entry rate is 7.30 wpm and error small size, which can be difficult to rate is 0.4%. But variations in some find, particularly for those who are strokes are problematic for not proficient with the QWERTY recognizer for ex. character „O‟, „T‟, layout. Similar to VoiceOver, „E‟ and „N‟ faces reorganization Oliveria et al. and Azenkot et al. both problem. evaluated accessible soft QWERTY 3) QWERTY Keypad keyboard (Oliveria et.al,2011; Azenkot et.al, 2012). It uses split-tap In 2008 Kane et al. proposed soft interaction. First tap produces a QWERTY keyboard approach Slide voice output of the touched character Rule (Kane et.al, 2008). It is gesture- and a second tap select the character. based approach for multi-touch text Its text entry rate was 2.11 to 3.99 entry. It was designed specifically wpm with 5.2% to 6.4% error rate. for list-based application like music player and phonebook. Slide Rule 4) Braille based Keypad uses four basic gesture interactions: In the year 2011, Joao Oliveira (1) use one-finger scan to browse introduced a single touch Braille list, (2) use second-finger tap to keypad based mobile texting scheme select item, (3) to perform additional called BrailleType (Oliveria et.al, action use multi-directional flick 2011) for visually impaired. It has gesture and (4) to browse hierarchy six large dot representing Braille

5 Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

cells. These dots are mapped to the Recently, in 2012 Mascetti et al. corners so that from edge of the developed TypeInBraille (Mascetti screen user easily find it. Each dot et al. ,2012). Here user types the gives auditory feedback on touching Braille cell one row (2 dots) at a it. As compared to traditional Braille time. Space is entered by a flick typewriter it has less number of keys. gesture. Its text entry rate is 6.3 wpm Its text entry rate is 1.45 wpm with with error rate 3%. Azenkot et al. 9.7% error rate. But it requires presented an IFD technique for text multiple gestures and inputs to input on touchscreen called access a specific character, which Perkinput (Azenkot et al. , 2012). It resulted in slower performances. Its uses 6 bit Braille encoding bits with timeouts causes error. The time audio feedback. It allows one-handed elapse is more which affect text and two-handed entry. Its text entry entry. This resulted in trying to rate is 6.1 wpm with 3.5% error rate. accept incorrect Braille cells. It lose 5) Different UI track of text. It doesn‟t explain reading of whole message. It is In 2008 Tiago Guerreiro et al. designed especially for visually proposed single touch gesture-based impaired people who know Braille. approach called NavTouch. (Guerreiro et al. , 2008). It is based In year 2011 Caleb Southern et al. on a navigational approach. To present six-key chorded Braille soft navigate alphabets use left to right keyboard for smartphone called gesture i.e. in horizontal direction BrailleTouch (Southern et al. , and to navigate between vowels 2011). They provide two hints to move the finger up and down i.e. spell or remember words and vertically. Speech and vibrotactile phrases. The first hint (chord 4-5-6) feedback is given. Its text entry rate spelled out the entire phrase, letter was 1.37 wpm with 9.87% error rate. by letter before the timer started. The But it gives slower performance second hint (chord 1-4-5-6) repeated because of its multiple gesture and the entire phrase. Its text entry inputs to access a specific character. speeds is 23.2 wpm with 14.5% error Also an accidental touch loses the rate. The difference between track of text and gives wrong result. BrailleType and BrailleTouch is that One other multi-touch text entry the latter requires the user to input all method is No-Look Notes dots for a character simultaneously. introduced in 2009 designed by But its software failed to accurately Matthew Bonner (Bonner et al. , recognize the flick gesture for the 2009). The characters are present in space character. This approach is pie menu supporting audio feedback. useful only for the user who knows Its text entry speed is 1.33 wpm with Braille. 11% error rate. But in No-look note,

the text entry takes much time 6

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

because all characters are not present so user has to search the desire one at a time; they are grouped into 8 by navigating through the interface character set. User faces difficulties on each time. in understanding pronunciations of I. Text Entry Evaluation characters. It does not have haptic The analysis of keypad design can be (vibration) feedback. The system performed by measuring the metrics was not design for symbols and of text entry and effectiveness of numbers entry. keypad layout. Another development is EasyWrite in 2011 by P.A.Candado et al. A. Text Entry Metrics (Candado et al. , 2011). It is a small For evaluation of text entry two virtual keyboard with less and bigger metrics are used. These are Speed keys. In EasyWrite alphabets are and Accuracy (Soukoreff & grouped into central key and user has Mackenzie , 2003). to navigate through directional keys. 1) Speed This approach is developed to Text entry speed is number of improve typing accuracy and characters entered per second. To accessibility of mobile device. Its calculate speed in Word Per Minute text entry rate was 2.7 wpm. Its (WPM) following formula is used: crucial aspect is that all characters are not appear for immediate choice,

(Transcribed text -1 ) ( 60sec onds time in sec onds) (1) (5 characters per word) 2) Accuracy Accuracy of text entry depends on number of error occurred during entering text. There are two method used to analyse text entry error: MSD error rate and KSPC. a) MSD error rate The Minimum String Distance (MSD) between the strings is number of primitives (insertion, deletion or substitution) to transfer one string to other . It is calculated by MSD(P,T, ) (2) MSDerrorrate= 100 max(P|,|T |)

Where P and T are the presented and transcribed text strings, and the vertical bars | | represent the length of the strings. b) Key Stroke Per Character (KSPC)

7

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

There are two classes of errors: 1) those are not corrected, and 2) those are corrected. MSD error rate measures not corrected error. To measure corrected error KSPC is used. KSPC is calculated as: ||Input Stream KSPC = (3) ||Transcribed Text Input Stream|

Where | | represent the length of comparative analysis of all this string (including insertions, available research is performed deletions, or substitutions). If text is according to keypad layouts in given entered without error, KSPC will be table (Table II). It is seen that for 1.00 (Soukoreff & Mackenzie , high text entry rate Unistroke is best 2003) one. But for minimum error rate, Graffiti is good one. Feedback is also B. Effectiveness of text entry important feature of mobile device; methods according to table only VoiceOver Theoretically the effectiveness of a and Graffiti have audio as well as keypad design used with predictive tactile feedback. From above disambiguation text entry method is discussion it should be clear that not measured by Disambiguation a single device has all features. So Accuracy and Key Stroke Per how can a visually impaired user Character. These two metrics used to choose a mobile device for effective tell how much effort user have to text entry? Hence, authors list out take for particular text entry method. some key issues that may help to Disambiguation Accuracy develop more effective and DA evaluates the probability of accessible as well as affordable text displaying desired word after any entry technique for mobile device. keystroke sequence is entered. When  The keypad should be of any ambiguous keystroke is entered, standard layout so that user with this process the matching word should familiar to it. with highest frequency of occurrence  Eliminate non-essential buttons will be displayed. Larger DA implies and menus. better keypad design.  Screen size should be of I. Discussion appropriate size. In above section authors discussed  Component like text boxes, problems faced by visually impaired users while using touch screen menus and buttons must be of devices, text entry factors and suitable size, so that they can be available research in this field. The easily seen and pressed. Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

 Screen reader and magnifier sound or vibrations. should be there so those users  For text deletion some gesture 8 with vision loss get advantage or tapping technique should be from it. provided.  If there are more buttons or The important thing is to consider menus appear on the screen that the design should be simple, clear and specific. then while interacting there should be tactile feedback to I. Conclusion sense the boundaries of each As available related work in the field one. Also screen boundaries of touch screen mobile devices for visually impaired users, we have should be represented by other found that different touch screen feedbacks. strategies, keypad layouts and  The screen should appear on explorer methods are used to make any direction so no need to hold the devices more accessible. Out of the device in specific direction. all available techniques either they  For text entry there should be are not easy to use or time word completion, letter consuming. Also their text entry rate is not efficient for fast texting. So prediction and message authors are suggesting some translation techniques. important factors which will help in  Accidental key pressing and developing more accessible, errors should be detected and affordable and fast text entry responses by feedback like technique for new user interface.

References Society, Toronto. Arditi , A., & Rosenthal, B. (1998). Belote, & Larry (2006, March 31). Developing an objective Low Vision Education and definition of visual impairment Training: Defining the in Vision. Proceedings of the Boundaries of Low Vision International Low Vision Patient. Retrieved from Conference,, 331-334. http://www.larrybelote.com/Fil Azenkot, S., Wobbrock, J., es/Low Vision Education and Prasain, S., & Ladner, R. Training/Extending the (2012). Input Finger Detection Boundaries of Services.Doc for Nonvisual Touch Screen Bill Text - 108th Congress (2003- Text Entry in Perkinput. Proc. 2004) - THOMAS (Library of Graphics Interface, Canadian Congress). (n.d.). Retrieved Information Processing from http://thomas.loc.gov/cgi-

9 Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

bin/query/z?c108:H.R.1902.IH: Zacarias, M., & Lobo, F. G. Blickenstorfer , C. H. (1995). (2011). EasyWrite: A touch- Graffiti; Wow!!!. Pen based entry method for mobile Computing Magazine, 30-31. devices. Portugues Foundation Bonner, M., Brudvik , J., for Science and Technology, Abowad, G., & Edwards, W. Portugal. (2009). No-Look Notes: Guerreiro, T., Nicolau, H., Jorge, J., Accessible Eyes-Free Multi- & Goncalves, D. (2008). Touch Text Entry. GVU centre NavTap: a Long term Study & School of Interactive with Excluded Blind Users. Computing, Georgia Institute INESC-ID, Porto Salvo, of Technology, Atlanta, USA, Portugal, 2780-2790. Sci. Rep. Guerrerio , T., Lagoa, P., Brewster, S., Hoggn , E., & Nicolau, H., Santana, P., & Jonston, J. (n.d.). Investigating Joaquim, J. (2008). Mobile the effectiveness of tactile Text-Entry Models for People feedback for mobile with Disabilities. European . Proc. of the Conference on Cognitive ACM, (2008), 1573-1582. Ergonomics 2008, Maderia, Brown, D., Steinbacher , C., Portugal. Turpin, T., Butler, R., & Gupta, A., & Samdaria, N. (2011). Balse, C. (n.d.). Company SVIFT: Swift Vision-Free Text- history from Elographics to Elo Entry for Touch Screens. TouchSystems, 1971 - present - Retrieved from Elo TouchSystems - TE http://www.cs.toronto.edu/aaka Connectivity. Retrieved from r/Publications/Svift-Report.pdf http://www.elotouch.com/Abou Kane, S., Bigham, J., & tElo/History/default.asp Wobbrock, J. (2008). Slide Buzzi, M. G., Buzzi, M., & Rule: making mobile touch Leporini, B. (2011). Interacting screen accessible to blind with the iPod via VioceOver: people using multi-touch accessibility and usability interaction techniques. issues. Proc INTERACT ASSETS?08, 73-80. 2011Workshop on Mobile Mascetti,, S., Bernareggi, C., & Accessibility,Lisbon. Belotti, M. (2011). Chatterjee, B. M. (1998). Handbook TypeInBraille: Quick Typing of Opthamology. on Smart Phones by Blind Condado, P. A., Godinho, R., Users. Universita Degli Studi Di Milano, Milan, Italy. Oliveira, J., Guerreiro, T.,

10

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

Nicolau, H., Joaquim, J., & BrailleTouch: Mobile Goncalves, D. (2011). Touchscreen Text Entry for the BrailleType: Unleashing Visually Impaired Users. Proc. Braille over touch screen of MobileHCI?12, Sep. 2012, mobile phones. INTERACT San Francisco, CA, USA, © ?11, Heidelberge: Springer, ACM. Retrieved from 978-1- 100-107. 4503-1105-2/12/09 Ricks, T., & Smith , P. (2007). Tinawal, H., & MacKenzie, I. Method for making a display (2009). Eye-Free Text Entry on with integrated touchscreen. U. a Touchscreen Phone. S. Patent Application International Conference Publication. Science and Technology, Scott Mackenzie, R., & Toronto, IEEE TIC-STH, 83- Soukoreff, W. (2002). Human- 89. Computer Interaction, 17, 147- WHO | Visual impairment and 198. blindness. (n.d.). Retrieved Soukoreff, R. W., & from MacKenzie, I. S. (2003). http://www.who.int/mediacentr Metrics for text entry research: e/factsheets/fs282/en/ an evaluation of MSD and Yfantidis, G., & Evreinoy, G. KSPC, and a new unified error (2006). Adaptive Blind metric. Proc. CHI ?03, 113- . 120. Universal Access in the Southern, C., Clawon, J., Frey, B., Information Society, 4, 328- Abowd, G., & Romero, M. 337. Retrieved from (2012). An Evaluation of www.factbrowser.com/facts

11

Covenant Journal of Informatics and Communication Technology (CJICT) Vol. 1, No. 2, December, 2013.

TABLE II: COMPARATIVE ANALYSIS OF TEXT ENTRY TECHNIQUES Text MSD Audio Keypad Device Explore Screen Tactile Entry Error Author Features Layout Selection Feedba Layout Name r Size Feedback Rate Rate ck (WPm) (%) 1) Twelve medium size buttons representing Guerreiro et 12 set of character. al. (2008) medium Split/ MultiTap 2) Split or double tap used to enter character. Scan Medium Yes No 0.88 15.28 size double 3) Offers a more direct mapping between input Telephon buttons and desired output. e Keypad Aakar Gupta 1) Vibrotactile technique is used. Similar and 2) Swipe, pause, circle, hand -waving, audio SVIFT phone Scan Split-tap Large Yes Yes 4.75 - NavkarSamdar and vibrotactile feedback is used. keypad ia (2011) 3) Increases speed and accuracy 1) Unistrokes alphabets are entered with Goldberg and stylus. Unistroke Single- Unisroke Richardson Stylus Small Yes No 32 - 2) Strokes are simple. Keypad stroke (1993) Stroke 3) Provide audio feedback. Based 1) Graffiti strokes are entered by stylus. Keypad Tinwala and 2) Strokes are for number, punctuations, Single- Graffiti Mackenzi symbolic characters and mode switches. Adaptive Stylus Large Yes Yes 7.60 0.4 stroke (2009) 3) Feedbacks are given by speech, sounds and vibrations. 1) Vision less keypad 2) Designed specifically for list-based Slide Kane et al. No visual Split/ application like music player and phonebook. Gesture Small Yes No 27 14.1 Rule (2008) display double 3) Use four basic gesture interactions for navigating and selecting characters. 1) Screen displays QWERTY keyboard layout 2) Offer a function to correct an error. QWERT 3) Give audio output. Spilt/ Y Keypad Apple co. QWERT VoiceOver 4) Speed can be adjusted depending upon Scan double- Medium Yes Yes 0.66 9.7 ltd.(2009) Y keypad preference. tap 5) Offers a more direct mapping between input and desired output Olieria et al Fixed 1) Provide soft QWERTY keyboard. Split/ QWERTY and Azenkot et QWERT Scan Small Yes No 2.11 5.2 2) Use split-tap interaction. double al. (2012) Y keypad 1) Screen displays 6 dots as Braille cells. Large Braille Braille Braille Joao Oilveria 2) Double tap is used to accept Braille press and Based cells ( 6 Scan Large Yes No 1.45 9.7 Type (2011) character. double Keypad dots) 3) Audio feedback is given. tap

12