Mobile Interaction Does Not Exist
Total Page:16
File Type:pdf, Size:1020Kb
Mobile Interaction Does Not Exist Joe Marshall Abstract Mixed Reality Lab Most mobile systems are ‘stop-to-interact’; designed for Department of Computer Science active interaction only when a user is standing still, University of Nottingham paying visual and mental attention to the device. Nottingham, NG8 1BB, UK However, people are increasingly carrying and using [email protected] devices while undertaking a wide range of movement activities, such as walking, cycling, running. Some Paul Tennent existing systems such as Apple’s Siri aim for hands and Mixed Reality Lab eyes free use, but they do not consider the wider Department of Computer Science challenges of interaction during movement. University of Nottingham Nottingham, NG8 1BB, UK We describe the challenges of system design for active [email protected] mobile interaction. These ‘interaction in motion’ challenges are discussed with reference to an extreme movement interaction situation – cold water swimming. Author Keywords Motion; interaction; interaction in motion; mobile; swimming Permission to make digital or hard copies of all or part of this work for ACM Classification Keywords personal or classroom use is granted without fee provided that copies H.5.m. Information interfaces and presentation (e.g., are not made or distributed for profit or commercial advantage and that HCI): Miscellaneous. copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, General Terms requires prior specific permission and/or a fee. Design CHI 2013 Extended Abstracts, April 27–May 2, 2013, Paris, France. Copyright © 2013 ACM 978-1-4503-1952-2/13/04...$15.00 Introduction touchscreens and audio feedback [20]. These work well Current mobile device user interfaces are primarily when there is a limited range of actions to be done, but based around users stopping and visually attending to a when a whole system has to be controlled eyes free, it touchscreen at any point they wish to interact with the becomes too complicated to use in the pocket, as device. Many applications present data to the user demonstrated both by the comparison with Apple’s without forcing them to use a screen, such as music, more generic VoiceOver accessibility software in the navigation instructions and the audio from phone calls. study above, and the long step by step lists of how to However, this is typically a one way transfer, with the do something as simple as set a clock alarm using screen being used to stop calls, change how music is eyes-free input on a mobile device eg [1]. being played, or set the navigation instructions. As such, these systems are designed around a ‘stop-to- Voice commands are also possible, typically in sync interact’ model; this is even made very explicit in many with a specific physical button action such as used in systems – for example the manual for the TomTom Go Apple’s Siri voice control [3]. These are becoming car navigation system states: usable in some situations, such as in cars, but in less controlled outdoor environments, they are extremely ‘Important: You should always plan your susceptible to environmental and wind noise. Siri, and journey before you start driving. It is its equivalent Google Voice require a connection to the dangerous to plan a route while driving.’ [23]. internet to function. While in theory our devices may be "always connected" in practice this may not be the case We would argue that these are not truly mobile - especially when participating in more extreme mobile systems; devices are increasingly portable, but activities (eg. fell running or cold swimming). performing any meaningful two way interaction while actively mobile is hard, and falls outside the design Finally, in applications which track movement such as parameters of the device. navigation systems, whilst the core of the application is based around stop-to-interact, the application may There are a small number of devices and systems which track the response of the user to the output which it is allow a limited amount of two way mobile interaction. presenting, which can allow the user to actively interact with the system by altering their movements. For Firstly, on many older mobile devices, there are specific example users of satellite navigation in cars may buttons for functions such as answering and ending choose to use their local knowledge and ignoring an calls on a mobile phone, changing tracks on an mp3 instruction from the device, and it will then respond by player, or announcing heart rate on a sporting device. recalculating the route taking account of the direction Due to the tactile nature of physical buttons, these can that the user chose [8]. Our previous work creating at least be used while walking and without looking at ‘intense experiences’ by encouraging extreme exertion the device. Recently, some developers have tried to [12] took this approach to the logical extreme, by create similar ‘in pocket interaction’ interfaces using creating a system entirely controlled by how the user chose to move. Other examples include audio based people in their (typically optional) movement mobile games such as ‘1831 Riot’ which used locational activities, enable the applications to be useful to audio to present a story based on a historical riot [9] them during their movement, and minimize and the commercial audio running game ‘Zombies Run’ interruption by applications during movement. in which zombies chase you as you go for a run. [2]. Whilst most current design assumes stop-to- Why is Interaction in Motion Necessary? interact, in reality people do engage in movement The reason for the general uptake of the ‘stop-to- activities whilst interacting. As designers, we should interact’ paradigm is clear; by assuming that people consider the reality of peoples' use of our designs. stop and are looking at a touchscreen when they The most obvious is the use of mobile phones in interact with mobile devices, we remove a vast number cars, which can potentially lead to accidents when of variables from the situation; this in turn allows the people are not paying full attention to the road highly configurable touch-screen interface to serve a [14]. As mentioned above, satellite navigation wide range of purposes, as can be seen by the vast devices have also cautioned against use while libraries of applications available for most modern driving, with some devices even automatically mobile device operating systems. disabling menus when the car is moving. The primary approach thus far in driving has been There are several reasons why designers might wish to legislation to ban use of mobile devices. However, go beyond stop-to-interact, and to design for active even walking can lead to social issues when people interaction whilst a user is moving, these include: don’t pay adequate attention to their surroundings and walk into others, or walk across the road Movement and exercise activities, such as walking, without looking (as observed in reports of several running, cycling, swimming can be intense, exciting pervasive games eg.[4,6]). As people move further, experiences [12]. As such, they are a positive thing faster and do more activities while they are in themselves; integrating physical movement into carrying complex mobile devices such as activities such as games can lead to enhanced smartphones, this disconnect between the device feelings of social connection amongst players [15], design and the use in practice is only likely to and has the potential to have physical and mental increase. health benefits for users. As an extremely simple example of design for Mobile devices are increasingly worn at all times, interaction in motion, we can consider one of the most with many people carrying them while they take common tasks on mobile devices, sending an SMS or part in everyday movement activities, such as email message whilst walking. This causes problems commuting by bike or exercising for fun. Designing solely because you are looking at the screen; people systems and applications that can be used during have their hands available to interact, so the touch these moments could potentially both support interface is not a problem. Potentially, we could apply a design used in driving, the rear-view mirror, except by purpose interaction during movement (as evidenced by using the camera on the device to show an onscreen the comparison in [20] between their PocketMenu for ‘front view mirror’, so that people can see what they music playback, and the use of Apple VoiceOver). While are about to walk into. Such apps do already exist on Siri and Google Voice may go some way to mitigate several platforms (eg. type'n'walk [10]) however, they this, we have seen that they are restricted by are as yet imperfect because on current devices the environment and require a constant network camera angle is not correct. To work effectively this connection. As well as application, we have to consider would require a small mirror to align the view forwards; the range of movement activities which we are potentially on a newer device, the camera angle could expecting from users, for example designing an be altered (or a specific camera could be fitted). This interface for use while running may be extremely would allow people to safely and politely walk whilst different to designing for use while swimming or texting. This works well for walking, although it is cycling. Again, it may be harder to design for a wide clearly not a one size fits all text input in motion range of activities rather than focus on a smaller range. solution, once someone runs, or cycles, they no longer have their hands free to interact.