Effects of Hand Drift while Typing on Touchscreens Frank Chun Yat Li1 Leah Findlater2 Khai N. Truong1 University of Toronto University of Maryland University of Toronto ABSTRACT On a touchscreen keyboard, it can be difficult to continuously type without frequently looking at the keys. One factor contributing to this difficulty is called hand drift, where a user’s hands gradually misalign with the touchscreen keyboard due to limited tactile feedback. Although intuitive, there remains a lack of empirical data to describe the effect of hand drift. A formal understanding of it can provide insights for improving soft keyboards. To formally quantify the degree (magnitude and direction) of hand drift, we conducted a 3-session study with 13 participants. We measured hand drift with two typing interfaces: a visible conventional keyboard and an invisible adaptive keyboard. To expose drift patterns, both keyboards used relaxed letter Figure 1. Traces show hand drift on the adaptive keyboard. Each disambiguation to allow for unconstrained movement. Findings coloured dot is an average of contact points every 10 phrases. The show that hand drift occurred in both interfaces, at an average rate light letters in the background show the starting key positions. of 0.25mm/min on the conventional keyboard and 1.32mm/min on the adaptive keyboard. Participants were also more likely to drift review of keyboard layouts that have been optimized using Fitts’ up and/or left instead of down or right. law to improve typing performance with a single point of input like a stylus or thumb. Finally, for ten-finger typing, Findlater et Keywords: Hand drift, personalization, soft keyboard, adaptation. al. [4, 5] showed that a personalized and adaptive layout could improve typing performance. Index Terms:! H.5.2. [Information interfaces and presentation To formally study hand drift, we conducted a 3-session study (e.g., HCI)]: User interfaces—Input devices and strategies (one calibration and two test sessions) on a Microsoft Surface with two typing interfaces: a visible conventional soft keyboard 1 INTRODUCTION and an invisible adaptive keyboard. This setup allowed us to compare hand drift in visually constrained and unconstrained To achieve high typing speeds on physical keyboards, expert conditions. The adaptive keyboard adapted the location of keys to typists have long been touch typing, that is, typing without a moving average of the previously entered keys, following the frequently looking at the keys. Ten-finger touchscreen keyboards, user’s hands. To further expose the effect of hand drift, both in comparison, have not yet been able to match the performance keyboards used relaxed letter disambiguation; that is, participants of traditional physical keyboards [7]. The limited tactile feedback did not need to type with exact accuracy. of a touchscreen means that users often have to visually scan the Results showed that there was hand drift with both the visible keyboard to ensure they hit the intended key. Otherwise, their conventional and the invisible adaptive keyboards, though the hands may gradually misalign with the underlying soft keyboard. effect was stronger in the invisible one. With the conventional This phenomenon, called hand drift [3], has not been studied in keyboard, participants drifted an average rate of 0.21mm/min in detail. Although intuitive, there remains a lack of empirical data to session 1 and 0.25mm/min in session 2. In comparison, in the describe the effect of hand drift. A formal understanding of hand invisible interface, participants drifted an average of 0.65mm/min drift can provide insights for improving the design of soft in session 1 and 1.32mm/min in session 2 (Figure 1). Participants keyboards for touch typing. experienced hand drift even on the visible keyboard, which means A rare example of research involving drift is Azenkot et al.’s that users are likely to experience hand drift on any ten-finger soft Perkinput [3], a Braille text input method that accounts for hand keyboard, thereby restricting touch typing speed. Future soft drift if it were to occur. However, Azenkot et al. did not examine keyboards should have methods of supporting hand drift (e.g., to what effect drift does occur. Instead, researchers typically have adaptive keys) or deter hand drift (e.g., realign prompts). focused on other aspects of soft keyboard design. For example, 2 EXPERIMENT Sears et al. [11] examined the impact of key size on typing performance for keyboards large enough to support ten-finger We conducted a within-subjects study to examine the effects of typing. As well, language models have been employed to improve hand drift when typing on a visible conventional keyboard and an input by effectively increasing the contact area available for invisible adaptive keyboard. The visible interface allowed us to probable letters [1, 2, 6]. Zhai et al. [14] provides an excellent observe how hands drift on a visible layout and the invisible interface allowed us to study how hands drift naturally without 1 any visual constraints. {frankli, khai}@dgp.toronto.edu 2 [email protected] 2.1 Participants Thirteen participants (7 female) from the local university community were recruited using an online forum. The average age was 29 (SD=5.6). All participants were right-handed and reported regular use of standard rectangular desktop keyboards. No effects from handedness were mentioned in a previous study of adaptive were displayed, indicating the location of the space bar on the keyboards on the Microsoft Surface [4]; as such, we did not expect non-visual keyboard. These anchor keys disappeared after typing handedness to have a significant effect on drift rates. TextTest [13] started. To prevent severe misalignment, if no action was was used at the beginning of the study to measure participants’ performed on the touchscreen for more than 10 seconds, the entire typing speeds on a physical keyboard. Their average typing speed in keyboard reset to the participant’s initial personal layout and the adjusted words per minute (AWPM) was 81.3 WPM (SD=15.8). thumb anchor keys reappeared. AWPM is a measure of typing speed after adjusting for uncorrected We exposed the effect of hand drift with both typing interfaces typing errors [12]. by relaxing the letter disambiguation. Because the software was aware of what phrases the user was supposed to enter, users could 2.2 Apparatus type one key off on the keyboard or one letter off from the The experiment was conducted on a Microsoft Surface (now presented text but still produce the correct output. For example, if known as PixelSense) using custom software written in C# .NET the next expected letter is ‘d’, users could press any of the 4.0. The software logged all typing data. To minimize fatigue, the neighbouring keys (e.g., ‘f’, ‘x’, etc.) and ‘d’ would still be typed. setup was designed to be as ergonomically correct as possible. If the finger touch point did not match the next expected letter key Because the housing of the Surface would have prevented or any of its immediate neighbours, the system checked for a participants from sitting at it comfortably, we mimicked a match with the preceding or the second-next expected letter. If standing work desk. The height of an ergonomic standing desk still no matches were found, an asterisk was output for a typo (see should be such that the user can rest her elbow on the table the transcribed vs. presented text in Figure 2). This is to ensure without needing to bend her back [10]. We raised the Surface to that the effect of natural hand drift is exposed in the study. 130cm, fit for anyone up to 200cm tall; shorter participants stood The backspace and enter operators were implemented for both on a platform that was raised appropriately. The Surface typing interfaces as four or eight simultaneous finger presses. This recognizes touches from the narrow side of thumbs poorly; thus, replacement was made because in pilot studies we found that a we asked participants to wear rubber bands approximately 3mm in touchscreen button for backspace in its typical location (near the width around the pad of their thumbs to increase the sensitivity of ‘p’ key) did not work well for a non-visual keyboard. Pilot testers thumb inputs. The rubber bands were small, just tight enough not repeatedly triggered the backspace instead of ‘p’, and vice versa. to fall off easily and minimally obtrusive overall. Participants did not report that the bands affected their typing. 2.4 Procedures The experiment consisted of three 45-minute typing sessions, a 2.3 Typing Interfaces calibration session followed by two test sessions. Consecutive We implemented two typing interfaces: a visible conventional sessions by a given participant were scheduled between 2 and 72 keyboard and an invisible adaptive keyboard (Figure 2). For the hours apart. At the end of last session, we conducted a short conventional interface, we showed a picture of the native Surface debriefing interview to inquire about participants’ experiences. keyboard. This keyboard had a fixed dimension of 200×90mm. Participants were compensated $60 for the entire study. We used an invisible keyboard for the adaptive interface because 2.4.1 Calibrating a Personalized Layout previous work has shown visually adaptive keyboards to be distracting [4]. The initial position of each key was personalized The calibration session allowed us to collect data for based on typing data collected during a calibration session personalizing the initial layout of the adaptive keyboard. It also (described in a later section). That is, the center location of each allowed participants to become familiar with the study apparatus. key was placed at the median of all contact points that had been Participants completed two 20-minute typing blocks with a short recorded for that letter during the participant’s calibration session.