JustSpeak: Enabling Universal Voice Control on Android Yu Zhong1, T.V. Raman2, Casey Burkhardt2, Fadi Biadsy2 and Jeffrey P. Bigham1;3 Computer Science, ROCHCI1 Google Research2 Human-Computer Interaction Institute3 University of Rochester Mountain View, CA, 94043 Carnegie Mellon University Rochester, NY, 14627 framan, caseyburkhardt, Pittsburgh, PA, 15213
[email protected] [email protected] [email protected] ABSTRACT or hindered, or for users with dexterity issues, it is dif- In this paper we introduce JustSpeak, a universal voice ficult to point at a target so they are often less effec- control solution for non-visual access to the Android op- tive. For blind and motion-impaired people this issue erating system. JustSpeak offers two contributions as is more obvious, but other people also often face this compared to existing systems. First, it enables system problem, e.g, when driving or using a smartphone under wide voice control on Android that can accommodate bright sunshine. Voice control is an effective and efficient any application. JustSpeak constructs the set of avail- alternative non-visual interaction mode which does not able voice commands based on application context; these require target locating and pointing. Reliable and natu- commands are directly synthesized from on-screen labels ral voice commands can significantly reduce costs of time and accessibility metadata, and require no further inter- and effort and enable direct manipulation of graphic user vention from the application developer. Second, it pro- interface. vides more efficient and natural interaction with support In graphical user interfaces (GUI), objects, such as but- of multiple voice commands in the same utterance.