sign in sign up
<<
Home , User interface

Attentive Interfaces If there was a Moore’s Law for user interfaces, it would state that the number of per user will double every two years. In the past four decades, we have moved from many users sharing a single mainframe through line interfaces, to a single user with a using a graphical user (GUI). Today, increasing numbers of users are surrounded ! By Roel Vertegaal, Guest Editor by multiple devices, such as BlackBerries, PDAs, and cell phones. As our devices connect to a global wireless network, we become members of a 24-hour global society—one where we are always connected, and always on. Although the trend to use more computing becoming part of our physical environment. devices may provide an opportunity for Although ubiquitous wall-sized displays may increased productivity, such benefit comes at a offer relief from the tunnel vision caused by our cost. That cost is the requirement to be avail- desktop screens, this technology brings with it able—at any time or place—in order to swiftly the capacity to further increase our information adapt to changes in our information environ- load. Consequently, growing demands on users’ ment. Rather than mitigate this cost, our com- have become a critical issue puting devices currently exacerbate it. This is in computing. because their user interfaces have not funda- Researchers are becoming aware of the fact mentally changed in 20 years: each device acts in that user attention is a limited resource that isolation, as if it were the user’s only device. As a must be conserved. designers and consequence, devices bombard users with engineers are beginning to design computing requests for attention, regardless of the cost of devices that negotiate rather than impose the their interruptions. At the same time, comput- volume and timing of their communications ers are increasingly enveloping our senses by with the user. Devices are augmented such that Illustration by Jean François Podevin

COMMUNICATIONS OF THE ACM March 2003/Vol. 46, No. 3 31 evices bombard users with requestsD for attention, regardless of the cost of their interruptions.

they become aware of competing requests for their attention have been referred to as Attentional User user’s attention. Information systems are being Interfaces. We note that models of the attentive developed that weigh the importance of the - behavior of users need not be fully predictive. This mation they supply with the estimated priorities in is because AUIs are able to gracefully negotiate the user activity. Displays are created that track the user’s amount of attention they receive from the user using focus of attention and adjust their renderings to pro- a similar to turn-taking behavior in conver- vide context in support of focused activ- sations. Instead of immediately taking the fore- ity. This area of work has created new interaction ground—interrupting the ongoing activity of the and styles that aim to focus—rather user—AUIs can progressively signal their requests for than distract—the user. Computing interfaces that attention. Initially this may happen through a chan- are sensitive to the user’s attention are called Atten- nel peripheral to the user’s activity. AUIs may then tive User Interfaces (AUIs); the user interface para- wait for user acknowledgment—provided through digm that is the topic of this special section. an attentive —before they take the fore- AUIs follow the general metaphor of a well-tuned ground. To achieve such behavior during mediated modern traffic light system. Modern systems use communications, AUIs may relay information about sensors in the road to determine where users will go the attention of their users to remote persons or next—their future focus. They employ statistical devices. Such interfaces are known as “Attention- models of traffic volume to determine the priority of based” or “Awareness” systems. user requests for intersection space. They use Finally, AUIs can also augment the user’s capacity peripheral displays—traffic lights—to negotiate for attention. Analogous to the Cocktail Party turn-taking activities of users sharing the limited real Effect, which allows people to focus on one among estate of an intersection. Likewise, AUIs may mea- several speakers during noisy meetings, AUIs can sure and model the focus and priorities of their user’s enhance information presented in areas where users attention. They structure their communication such focus their attention, while they attenuate peripheral that the limited resource of user attention is allo- detail. The accompanying figure summarizes the cated optimally across the user’s tasks. features noted here, and compares them to interac- AUIs use a range of novel input, interpretation, tion techniques currently used in GUIs. and output techniques to implement the examples The focus on design for user attention is what dis- mentioned here. To determine what task, device, or tinguishes the AUI paradigm from other, related person a user is currently attending to, AUIs employ areas of research in human-computer interaction. extra channels of input that measure overt charac- For example, Perceptual User Interfaces aim to opti- teristics of user attention (such as user presence, mize human-computer interactions by employing proximity, orientation, speech activity, or gaze). users’ verbal and nonverbal communication chan- Based on such input, AUIs infer knowledge about nels. AUIs employ such perceptual information with the priorities in the tasks that govern their user’s a specific intent to identify or optimize user atten- attention. By statistically modeling attention and tion. Similarly, Context-Aware Interfaces use the other interactive behaviors of users, AUIs may estab- context in which interactions take place to provide lish the urgency and relevance of information or more relevant services to the user. However, Con- actions they offer in the context of current activity. text-Aware Interfaces are attentive only when that Such interfaces with a deep understanding of user context is governed by user attention. The AUI par-

32 March 2003/Vol. 46, No. 3 COMMUNICATIONS OF THE ACM adigm also has roots in a history of research on information systems that track user interest and ren- human attention, , nonverbal communi- der suggestions using peripheral ticker-tape displays. cation, conversational interfaces, user modeling, and Horvitz et al. review principles for sensing and rea- awareness in collaborative systems. The phrase soning about a user’s attention with Bayesian - “attentive user interface” is used in this section as an els, and for using these inferences to identify ideal umbrella term that includes attentional and atten- decisions in messaging, interaction, and computa- tion-based systems, as well as what have been called tion. Baudisch et al. take a look at advances in atten- “attentive systems,” which observe user activity to tive display technologies, visualization systems that anticipate user needs. increase or decrease information density depending As with every emerging paradigm, AUIs bring on the focus of user attention. McCrickard and about new challenges for researchers. The articles in Chewar close this section by proposing a qualitative this section present some of the first milestones framework for evaluating AUI designs. c toward Attentive User Interfaces. Authors contribut- ing to the section include some of the world experts Roel Vertegaal ([email protected]) is a professor of human- on this topic who share their views and present their computer interaction and director of the Human Media Lab at prototypes. Zhai lays the foundation by taking an in- Queen’s University, Kingston, Ontario, Canada. depth look at IBM’s work on eye tracking—the most Permission to make digital or hard copies of all or part of this work for personal or relevant attentive input technique. He discusses how classroom use is granted without fee provided that copies are not made or distributed implicit measures of the user’s visual attention may for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute combine with manual action to optimize GUI inter- to lists, requires prior specific permission and/or a fee. actions. et al. discuss work at Queen’s Univer- sity and MIT toward AUIs that employ subtle turn-taking techniques to optimize user interactions with groups of computing devices. Maglio and Campbell describe their research on attentive agents, © 2003 ACM 0002-0782/03/0300 $5.00

1960s–1980s 1980s–1990s 1990s– 2000s–2010s This diagram shows how many-one one-one one-many many-many AUI interaction styles relate to those found in traditional GUIs. AUIs sense whether they are in the foreground of user attention by observing GUI the presence, activity, and eye-gaze behavior of the user. Undo In AUIs, menus and alerts Cut Copy are supplanted by a graceful Paste negotiation of turns with the Select all alert user through peripheral Clear channels. Foreground pointer alert focus vs. activities are distinguished background from background activities AUI using seamless transitions in the density of the information rendered, rather than by a discrete windowing system.

eye contact turn interruption high-detail focus vs. low-detail background

COMMUNICATIONS OF THE ACM March 2003/Vol. 46, No. 3 33