Adaptive click-and-cross The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters Citation Li, Louis, and Krzysztof Z. Gajos. 2014. "Adaptive Click-and-cross: Adapting to Both Abilities and Task Improves Performance of Users with Impaired Dexterity." In Proceedings of the 19th International Conference on Intelligent User Interfaces (IUI '14), Haifa, Israel, February 24-27, 2014: 299-304. Published Version doi:10.1145/2557500.2557511 Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:30846195 Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Open Access Policy Articles, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#OAP Adaptive Click-and-Cross: Adapting to Both Abilities and Task Improves Performance of Users With Impaired Dexterity Louis Li Krzysztof Z. Gajos Harvard SEAS Harvard SEAS 33 Oxford St., Cambridge, MA, USA 33 Oxford St., Cambridge, MA, USA [email protected] [email protected] ABSTRACT Approaches that modify the interaction in order to adapt the Computer users with impaired dexterity often have difficulty user’s abilities to the existing user interface make access pos- accessing small, densely packed user interface elements. Past sible without requiring substantial modifications to existing research in software-based solutions has mainly employed interfaces. However, these techniques may lack generality two approaches: modifying the interface and modifying the (e.g., area cursors and the bubble cursor enhance interaction interaction with the cursor. Each approach, however, has lim- only when clickable elements are sparsely laid out), or they itations. Modifying the user interface by enlarging interactive may reduce the efficiency of the interaction (e.g., the Click- elements makes access efficient for simple interfaces but in- and-Cross technique from Findlater et al. [3] replaces a single creases the cost of navigation for complex ones by displacing click with two operations: a click in the vicinity of the desired items to screens that require tabs or scrolling to reach. Mod- target followed by a crossing action to make a specific selec- ifying the interaction with the cursor makes access possible tion, Figure 1a). to unmodified interfaces but may perform poorly on densely In contrast, approaches that adapt the user interface to the packed targets or require the user to perform multiple steps. user’s abilities by modifying the user interface enable effi- We developed a new approach that combines the strengths cient access to each item, optimizing the interaction to each of the existing approaches while minimizing their shortcom- user’s strengths [8, 21]. However, adapting user interfaces to ings, introducing only minimal distortion to the original in- the abilities of users with impaired dexterity involves an im- terface while making access to frequently used parts of the portant trade off: such adaptations typically involve making user interface efficient and access to all other parts possible. clickable elements larger at the cost of increased navigational We instantiated this concept as Adaptive Click-and-Cross, complexity. This requires more scrolling and tab switching a novel interaction technique. Our user study demonstrates when navigating between user interface elements. Existing that, for sufficiently complex interfaces, Adaptive Click-and- approaches often enlarge all clickable elements — even those Cross slightly improves the performance of users with im- that users rarely access — because not enlarging them might paired dexterity compared to only modifying the interface or render them inaccessible. The increased navigational com- only modifying the cursor. plexity from such a broad approach is a source of inefficiency. Keywords: Accessibility; area cursors; adaptive user inter- face We set out to combine the strengths of the two approaches: ACM Classification Keywords making access possible and efficient while minimizing modi- H.5.m. Information Interfaces and Presentation (e.g. HCI): fications of the original design. To do this, we build on a third Miscellaneous adaptive approach: user interfaces that adapt themselves to the user’s task (e.g., [2, 6, 9, 18]). Such interfaces have been INTRODUCTION demonstrated to improve users’ performance by leveraging Computer users with impaired dexterity often have difficulty predictive models for each user’s actions to ease access to the with mainstream user interfaces, especially when these user features that the user is most likely to access next (e.g., by interfaces contain small, densely-packed interactive elements. copying them to a more easily accessible location, by making In the past few decades, a variety of software-based tech- them larger or more visually salient). niques have emerged to assist such users. These approaches Building on these three ideas of adaptation, we have devel- fall broadly into two categories: those that modify the user oped Adaptive Click-and-Cross. As illustrated in Figure 1, interface itself (e.g., ability-based user interfaces generated with Adaptive Click-and-Cross, user interface elements that with SUPPLE [8]) and those that modify the user’s interaction are predicted to be most frequently accessed by the user are with the mouse pointer (e.g., area cursor [15], bubble cur- enlarged and can be accessed efficiently with a single click sor [11], enhanced area cursors such as Click-and-Cross [3]). (adapting the interface to the user, adapting the interface to Permission to make digital or hard copies of all or part of this work for personal or the task). The remaining elements are left unmodified and can classroom use is granted without fee provided that copies are not made or distributed be accessed through the Click-and-Cross technique: the user for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the can click anywhere in the vicinity of the desired target and author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or subsequently refine the selection with a crossing interaction republish, to post on servers or to redistribute to lists, requires prior specific permission (adapting the user’s abilities to the interface). This approach and/or a fee. Request permissions from [email protected]. IUI’14, February 24–27, 2014, Haifa, Israel. achieves three things: it enables efficient access to frequently Copyright is held by the owner/author(s). Publication rights licensed to ACM. accessed user interface elements, makes access to all other ACM 978-1-4503-2184-6/14/02..$15.00. http://dx.doi.org/10.1145/2557500.2557511 (a) (b) Figure 1: Adaptive Click-and-Cross. (a) When users click near or directly on small targets, Click-and-Cross is triggered: a circular overlay of the nearby targets appears. Users can then cross through the corresponding arc to select the item. (b) Users can directly click on a large target to select it. elements possible, and minimizes the distortion of modifying the user interface. The results of our study with 12 participants of impaired dex- terity demonstrate that for a complex user interface (where enlarging all interface elements substantially increases the cost of navigation), Adaptive Click-and-Cross results in sig- nificantly shorter task completion times compared to either Figure 2: (a) In Adaptive Click-and-Cross, when target is bordered by an en- adapting the interface by enlarging all elements or Click- larged item, the target has a decreased amount of space for activating Click- and-Cross alone. We observed no significant differences in and-Cross. (b) Near the edge of the screen, the Click-and-Cross cursor only error rates or subjective preference across the three tech- displays a subset of the circle. niques. However, participants subjectively perceived the in- terface with all elements enlarged as more efficient than either functionality, though each user accesses a different subset [10, Click-and-Cross or Adaptive Click-and-Cross. 17]. This finding has been used to design user interfaces that enable efficient access to a subset of items that are predicted to be of most use to the user. For example, in split inter- RELATED WORK faces [5, 6, 18], the elements predicted to be most useful are Many existing software solutions improve accessibility by duplicated to a convenient location to support more immedi- modifying users’ methods of interaction. Such solutions may ate access. In contrast, morphing menus [1, 20], which have adapt the behavior of a pointing cursor to the user (e.g., been tested with able-bodied users, do not duplicate elements Steady Clicks [19], Angle Mouse [22]), or they may introduce but instead enlarge predicted items to enable efficient access. entirely new interaction techniques (e.g., area cursor [15], bubble cursor [11], enhanced area cursors [3]). ADAPTIVE CLICK-AND-CROSS Approaches that directly modify the user interface have also The original Click-and-Cross technique [3] is illustrated in been investigated. These approaches advocate adapting the Figure 1a: when the user clicks near or directly on a user in- user interface to users’ needs (e.g., EyeDraw [13] and Voice- terface element, a circular overlay is displayed with several Draw [12]). Although creating accessible designs that are (up to six in our implementation) of the closest interface el- well suited to a particular set of abilities can be time con- ements laid out along the circumference. Moving the mouse suming, previous work has begun to demonstrate how such such that it crosses the circumference triggers a “click” on the modifications could be automated [8]. corresponding element. If the first click was made by mis- take, performing another click inside the circle cancels the However, for complex user interfaces, adapting the user inter- interaction.