Turbulent Touch: Touchscreen Input for Cockpit Flight Displays Andy Cockburn, Carl Gutwin, Philippe Palanque, Yannick Deleris, Catherine Trask, Ashley Coveney, Marcus Yung, Karon Maclean

To cite this version:

Andy Cockburn, Carl Gutwin, Philippe Palanque, Yannick Deleris, Catherine Trask, et al.. Turbulent Touch: Touchscreen Input for Cockpit Flight Displays. International Conference for Human-Computer Interaction (CHI 2017), May 2017, Denver, Colorado, United States. pp.6742-6753. ￿hal-02649024￿

HAL Id: hal-02649024 https://hal.archives-ouvertes.fr/hal-02649024 Submitted on 29 May 2020

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés.

Open Archive Toulouse Archive Ouverte

OATAO is an open access repository that collects the work of Toulouse researchers and makes it freely available over the web where possible

This is an author’s version published in: https://oatao.univ-toulouse.fr/24688

To cite this version:

Cockburn, Andy and Gutwin, Carl and Palanque, Philippe and Deleris, Yannick and Trask, Catherine and Coveney, Ashley and Yung, Marcus and Maclean, Karon Turbulent Touch: Touchscreen Input for Cockpit Flight Displays. (2017) In: International Conference for Human-Computer Interaction (CHI 2017), 6 May 2017 - 9 May 2017 (Denver, Colorado, United States).

Any correspondence concerning this service should be sent to the repository administrator: [email protected] Turbulent Touch: Touchscreen Input for Cockpit Displays Andy Cockburn1, Carl Gutwin2, Philippe Palanque3, Yannick Deleris4, Catherine Trask2, Ashley Coveney2, Marcus Yung2, Karon MacLean5 1 University of Canterbury, Christchurch, New Zealand, [email protected] 2 University of Saskatchewan, Saskatoon, Canada, [email protected] 3 Université Paul Sabatier, Toulouse 3, France, [email protected] 4 Airbus Operations, Toulouse, France, [email protected] 5 University of British Columbia, Vancouver, Canada, [email protected]

ABSTRACT dials, keypads and wheels. While computer-based flight Touchscreen input in oc mmercial aircraft cockpits offers instrument displays are becoming increasingly prevalent potential advantages, including ease of use, modifiability, (i.e., the ‘glass cockpit’), these displays are almost and reduced weight. However, tolerance to turbulence is a exclusively used for data output, and user input to displayed challenge for their deployment. To better understand the objects is dependent on a separate indirect device. When a impact of turbulencen o cockpit input methods we conducted cursor is incorporated in the display, a trackball is typically a comparative study of user performance with three input used for item selection. methods – touch, trackball (as currently used in commercial aircraft), and a touchscreen stencil overlay designed to assist The reliance on physical controls is influenced by several finger stabilization. These input methods were compared factors, including the historical development of cockpit across a variety of interactive tasks and at three levels of environments, pilot expectations, and requirements for simulated turbulence (none, low, and high). Results showed safety, redundancy and adherence to standards imposed by that performance degrades and subjective workload regulatory authorities. For example, Boeing and Airbus increases as vibration increases. Touch-based interaction conform to the ARINC 661 standard [1] that stipulates was faster than the trackballn whe precision requirements behavioral requirements for GUI components in a Cockpit were low (at all vibrations), but it was slower and less Display System (CDS). The standard permits only a limited accurate for more precise pointing, particularly at high set of widgets, and it is slow to evolve – the explicit account vibrations. The stencil did not improve touch selection times, for cockpit touch input first appeared in the 2016 update [2]. although it di d reduce errors on small targets at high Touch interaction in commercial cockpits offers potential vibrations, but only when finger lift-off errors had been advantages to pilots, airlines, and aircraft manufacturers. eliminated by a timeout. Our work provides new information Pilots may gain from familiar, expressive, and direct means on the types of tasks affected by turbulence and the input for interaction in certain tasks. Airlines and manufacturers mechanisms that perform best under different levels of could gain from reduced hardware installation complexities. vibration. Replacing hard-wired physical controls with touchscreens Author Keywords could therefore ease development, facilitate upgrades, reduce Touch interaction; turbulence; aviation. weight, and improve pilot interaction. ACM Classification Keywords However, turbulence is a challenge for cockpit touchscreen H.5.m. Information interfaces and presentation (e.g., HCI): deployment. There are risks that the pilot’s ability to interact Miscellaneous. with the touchscreen may be substantially impaired or eliminated during periods of heavy cockpit vibration. INTRODUCTION Previous studies have shown that vibration can be a factor in Commercial aircraft cockpits are replete with physical touch input, but there are few studies that look at different controls, including many forms of switches, knobs, levers, types of interactive touch tasks or that consider how the problem of turbulence might be reduced. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are We therefore examined users’ ability to interact with various not made or distributed for profit or commercial advantage and that copies types of interactive objects at different vibration levels. We bear this notice and the full citation on the first page. Copyrights for components of this work owned yb others than ACM must be honored. used a motion platform to expose participants and a large Abstracting with credit is permitted. To copy otherwise, or republish, to touchscreen to three levels of simulated turbulence – none, post on servers or to redistribute to lists, requires prior specific permission low, and high. All tasks were completed using three different and/or a fee. Request permissions from [email protected]. means for input – a trackball (used in many commercial DOI: http://dx.doi.org/10.1145/3025453.3025584 aircraft), a 21.5 inch touchscreen, and the same touchscreen augmented with a guiding stencil overlay. The stencil overlay consisted of a 3mm-thick transparent Lexan sheet predominantly rely on mechanical switches, dials, and levers that entirely covered the touchscreen (see Figure 1). Holes with wiring redundancy (in case of failure on one channel). were cut through the sheet to permit interaction with The reliance on hardware stems from the aviation regulatory underlying widgets. The sheet’s 3mm thickness eliminated environment, including conformance with standards. capacitive sensing of the finger outside the cut-out regions. While physical controls retain many advantages for certain The stencil overlay provided two potential benefits for flight-critical tasks, there is substantial industry interest in turbulent touchscreen interaction. First, it might offer moving functions to cockpit display systems. Examples stabilization benefits because users can place their fingers or include the Lockheed Martin F-35 Lightning II stealth hand-edge on the stencil, without contact registration, and fighter, which has a large touchscreen display [9], Garmin subsequently move their finger to the target (by sliding over International’s patent for dual touch/cursor control of a CDS the stencil and ‘popping’ into a hole). Second, once within a [22], Rockwell Collin’s cockpit touchscreens [13] for light hole, users can further stabilize movement by pushing jets and turboprops, and the Thales Group “Avionics 2020” against the stencil’s edge. vision for the cockpit of the future [42]. See Kaminani [20] and Hamon [12] for reviews. In empirically examining touchscreen interaction in the cockpit, Stanton et al. [33] analyzed the effectiveness of four different input devices (trackball, rotary controller, touch pad and touchscreen) for menu navigation in non-turbulent environments. They concluded that different devices have different strengths, and that the touchscreen had the highest number of ‘best’ scores. Barbé et al. [4] and Huseyin et al. [17] examined ergonomic aspects of touchscreen positioning in airplanes and helicopters. Wang et al. [37] recently examined the influence of the size and shape (square versus rectangular) of touchscreen targets. Targeting time results Figure 1: Experimental set up. Participants sat on a motion were consistent with Fitts’ law, errors were higher with platform and made selections using a touchscreen or trackball. rectangular targets, and participants preferred larger targets A transparent stencil (shown) was overlaid on the touchscreen. to small ones. In another cockpit-directed targeting study, Results include the following findings: touch interactions Lewis et al. [24] compared left-handed targeting were faster and preferred at low vibrations; the stencil performance when using a touchscreen and trackpad (left- assisted touch selection of small targets at high vibrations; handed performance was studied based on the assumption the stencil increased accidental lift-off errors, but this can be that the right hand would be on a control stick, which is not accommodated by using a short timeout between touch the case for the captain in a passenger aircraft). The main selections; drag-based selections were highly inaccurate finding was that participants were faster when using the during vibration with touch and trackball; and multi-touch touchscreen than they were with the trackpad. Neither of pan-and-zoom interactions were much faster than the these studies examined the influence of turbulence. equivalent trackball method, even at high vibrations. In addressing issues arising from turbulence, at least two We make three main contributions: we demonstrate that patent applications disclose methods to facilitate the use of different tasks have very different tolerance to turbulence; touchscreens in the cockpit. Kolbe [21] describes a method we provide empirical data on the effectiveness of touch and for determining whether touchscreen events are invalid trackball input at high levels of vibration; and we test the following detection of a ‘movement indicative of turbulence’ value of a novel stabilizing stencil overlay for touchscreens. by analyzing the characteristics of the contact data. The Overall, we provide new information for designers who must method of Komer et al. [22] instead assures that an accommodate high vibration in touch-input. alternative cursor-based indirect method of input (such as a trackball) is available to supplement touchscreen control. BACKGROUND In an early empirical study of cockpit touch interaction, Aviation Cockpits and Controls The lifespan of a passenger aircraft is typically 20-30 years, Bauersfeld [5] compared two different touchscreen operating between 35,000 and 110,000 cycles dependent on activation modes for confirming selection actions – confirm longhaul or shorthaul use. During its lifetime, much of an on contact versus confirm on lift-off (i.e., when the finger is aircraft’s componentry will be replaced and updated, released from the surface). These modalities were compared including power-units, jet engines, and parts of the airframe. at two different levels of simulated turbulence, using a NASA-Ames flight simulator. Results confirmed the higher Updates to cockpit controls are also desirable, particularly accuracy of lift-off observed in prior findings in non- when new technologies become available. However, cockpit turbulent environments [30]. More recently, Hourlier et al. updates can be prohibitively expensive because controls [16] examined the perceived realism of cockpit turbulence simulations. Participants used an 11 point rating scale (from environments, as well as showing that touchscreen errors 0 for “not realistic at all” to 10 for “extremely realistic”) to increase with vibration. Similarly, Lin et al. [25] based their assess the perceived realism of turbulence that was simulated studies on ship motion, comparing the influence of low- using a hexapod robot similar to that used in our experiment. frequency vibrations on touchscreen, mouse, and trackball Mean ratings ranged from 6 for the simulation of light input using a Fitts’ Law target selection methodology. Their turbulence to ~9 for the simulation of severe turbulence. maximum vibration condition used root mean square (RMS) accelerations of 0.34m/s2, corresponding to the lower end of Huseyin et al. [18] recently reported results of a Fitts’ Law ‘a little uncomfortable’ on the ISO vibration discomfort study examining the influence of simulated constant G- index for low frequency vibration (ISO 2631-1). Their forces on touchscreen target acquisition. Participants wore a findings indicated that the touchscreen allowed fast but weighted bag on their wrist while acquiring 55px (15mm) inaccurate input and that the trackball was slow but accurate. and 75px (20mm) targets. Results showed that acquisition times increased with wrist weight. Finally, Dodd et al. [8] Neither of the ship studies examined the higher frequency examined the impact of moderate levels of simulated flight vibrations and much higher accelerations incurred during turbulence on touchscreen virtual-keypad input, with buttons flight. While the study of Dodd et al. [8] did examine touch of different sizes and separations. Data entry times, error input during simulated flight turbulence, they did not rates, fatigue, and perceived workload were all higher at examine other types of input activity beyond keypad entry moderate turbulence than in a no-turbulence condition. (such as dragging or multitouch pan/zoom operations), nor did they compare touch with other input methods. None of Touchscreen interaction and input stabilization Abundant research has examined the efficiency and accuracy the studies examines methods for remediating touch input of different methods for interacting with various forms of problems during turbulence. touchscreens. Early studies compared user performance TOUCHSCREEN STENCIL OVERLAYS (time and error rate) attained when different methods are The success of edge-based touch stabilization for users with used to terminate touch selections, such as first-contact, motor impairments suggests that related methods might slide-over, and lift-off [26, 30, 31]. Dragging actions with improve touch input in turbulent environments. In particular, different input devices have also been closely examined there are opportunities to assist stabilization on interface (e.g., [7, 10]). In general, studies suggest that direct finger widgets by placing a transparent stencil overlay on top of the pointing is fast but relatively imprecise unless augmented touchscreen. The idea of using a stencil or template to allow with a cursor-like indicator (e.g., [36]), that stylus input is users to feel touchable parts of a touch panel was first fast but can be inaccurate, and that indirect pointing methods described by Buxton et al. [6]. can be highly accurate. Zhai et al. [43] provide a Like Buxton’s template, our stencil contains holes that match comprehensive review of work on gestural touch interaction. the geometry of underlying touchscreen widgets, offering Several studies have examined methods for enabling two potential interaction benefits. First, users may be able to interaction by users with motor impairments, such as stabilize their hand and fingers during target approach by uncontrolled tremor. Barrier Pointing [11] proposed stylus- placing fingers or palm on top of the overlay, without based methods for target acquisition using bezel edges and triggering touch registrations on the touchscreen. Users corners of a mobile device to facilitate input stabilization. could then slide their finger into a hole to acquire a widget. Similarly, EdgeWrite [40] used the ridge edges of a small Second, once one or more fingers rest within a stencil hole, trackpad region to improve text-entry movements for users users could press against the stencil edge to stabilize their with limited motor control. Related studies have examined control over dragging actions (e.g., when operating a slider). trackball and joystick input for similar objectives [38, 39]. However, it is possible that the stencil may slow certain The work most closely related to our current study concerns interactions. For example, Avrahami [3] showed that touch computer input in motion environments. In an early study of target acquisition times were slower by up to ~90 ms when ship motion, McLeod et al. [27] showed that joystick input targets were located close to the edge of a displayed border. was negatively affected by motion across a variety of tasks. Stencil Thickness, Material, and Friction Hill and Tauson [15] showed that soldiers’ computer input in Multitouch touchscreens predominantly use projected vehicles (such as an armored personnel carrier) is negatively capacitive sensing to detect charged objects on, or in close influenced by vehicle vibration. Similar findings have been proximity to, the surface. Objects are not sensed beyond a shown for mouse input on trains [29]. few millimeters from the display – a 3mm-thick sheet of Yau et al. [41] examined user performance in abstract target Lexan polycarbonate prevented touches except where holes selection activities using three types of trackball and a were cut. Lexan has excellent optical clarity, is easily touchscreen in five different settings based on ship worked, has good abrasion resistance, and is widely used in movement (static, ‘heave’, ‘roll’, ‘pitch’ and ‘random’, all at the aerospace industry for aircraft window dust covers. 0.3Hz, with a maximum vertical displacement of ±100mm). It is likely that there is a friction sweet-spot that best affords Results indicated the superiority of the mouse in vibration contact stability without compromising control due to ‘stick- slip’ effects. Robinson et al. [32] examined surface friction EVALUATING INPUT METHODS IN TURBULENCE in cockpit touchscreen environments, and Levesque et al. We conducted an experiment to better understand the [23] examined methods for programmatically varying influence that simulated turbulence has on interaction. touchscreen contact friction. Participants completed a series of five interactive task types using each of three different input methods (trackball, Hole Size and Edge Profile touchscreen, stencil-touchscreen) at each of three different The shape of the edge of the stencil holes, and the size of the vibration levels (none, low, and high). Trackball holes with respect to the underlying widget, will influence performance serves as a baseline comparator, representing several aspects of interaction, including comfort, accuracy, current industry practice for cockpit interaction with CDSs. friction, and stability provided by the edge. We considered various edge profiles, including those shown in Figure 2. Turbulence was simulated using a motion platform. Participants wore a safety harness connected to the ceiling The bevel edge shown in Figure 2a should be relatively and sat on a seat mounted on the platform. The harness did comfortable, but it is likely to provide low levels of edge not encumber arm movement, and was present only to stability. The square edge shown in Figure 2b should be more prevent falling. The display, also mounted on the motion stable, less comfortable, and may create a small gap between platform, was placed approximately 45cm in front of the the stencil edge, finger, and contact surface. The ∑ profile participant’s chest, with the top edge of the display ~25cm edge shown in Figure 2c, is likely to be yet more stable, but below the user’s eye-line. This arrangement facilitates easy uncomfortable, and with a small possible gap between stencil touch interaction without interfering with the pilot’s view base and finger at the contact surface. The blade edge of from the cockpit windows [4]. Figure 2d is also likely to be stable and uncomfortable, with a potentially large gap from stencil base to finger contact. The objectives of the study were to compare and examine users’ performance (task accuracy and error rates) with the three devices for various activities during vibration. Task Types Five different types of experimental tasks were used with Figure 2: Candidate stencil edge profiles, from most (left) to each input device at each vibration level. These five task- least (right) comfortable (and least to most stable). types were as follows. Our experiments used the square edge shown in Figure 2b, Target selection – participants tapped/clicked on targets as representing a compromise between comfort, stability, and quickly and accurately as possible. There were three potential stencil-finger surface gap. To account for the movement amplitudes (96, 256 and 512 px) and two target anticipated gap, we made the stencil holes approximately 0.5 sizes (32 and 64 px). Figure 3a shows the arrangement of mm wider on each side than the target widgets (i.e., 1 mm targets, with the annotated braces showing the target diameter larger for circular targets). sequence – successive movement directions were N, E, S, N, Supporting Mode Transitions W, S for each of small, medium and long distances. Each Cockpit display systems often support modes that present target was highlighted green, reverting to blue when different interfaces, such as control panel (ECAM) in one successfully tapped. Each successful tap advanced to the mode and navigation information in another. Various next target, highlighting it green. Any error caused the techniques could be used to reconfigure the overlay when background to turn red for 500 ms. mode changes occur. If stencils were fabricated from a Keypad tasks involved entering and confirming each of four flexible material, the overlays could be stored on a roller different three-digit numerical values. Each target value was housed beneath the display. When a mode change occurred, cued at the top of the window (Figure 3b). Once the three actuators could roll the required stencil into place. digits were entered, the participant pressed “OK” to confirm Alternatively, actuators might pop-up a new stencil from a the value. Each button in the keypad was 104 px wide. ‘toast-rack’ of candidates when needed, leaving the flight Keypad interaction is a common activity for pilots, such as crew to snap the stencil into a housing. entering radio frequencies or waypoint coordinates. Input Redundancy Feedback for each keypress was shown immediately below While stencil overlays may assist touch input during the target value. Incorrect entries (on pressing ‘OK’) resulted turbulence, the need for safety critical operation requires in the cue and entered numbers highlighting red for 500 ms; high redundancy in the cockpit. Therefore, redundant users then had to correct their error (using the ‘Del’ key to methods of input would need to be supported, such as the delete erroneous digits) and correct their entry. Correct availability of a trackball and cursor to augment or replace a values were highlighted green for 500 ms, then the next touchscreen in the event of hardware failure. This need is target value was shown. Each three-digit target value directly addressed by a patent to Komer et al. [22]. comprised two digits from the edge of the keypad (1, 2, 3, 4, 6, 7, 9) and the digit ‘5’ (centre of the keypad). {14, 18} {15, 17}

{8, 12} {9, 11}

{2, 6} {3, 5}

(d) Dial

{1, 7, 13, 19} {4} {10} {16}

(a) Target Selection. Numbers indicate (b) Keypad target sequence. (c) Slider Figure 3: Target Selection (left), Keypad (right), Slider (bottom) and Dial (top) tasks shown in their approximately location within the experimental display. The Target Selection figure is augmented with braces showing the target sequence. Slider tasks involved setting a slider to a target value the left target). The first target (left) was displayed at a scale (Figure 3c). This task emulates a CDS version of setting a factor of 0.5. Touchscreen and stencil users zoomed into the physical lever to a target value. The target value was shown target through multitouch manipulation of the surface above the slider, and a slider permitted selecting values (‘pinch to zoom’ or its bimanual equivalent). Single finger between 0 and a maximum value shown at the right of the contacts panned the display at its current scale factor. slider (see Figure 3c). Lifting from the display, or releasing Trackball users panned by dragging the ball with the button the trackball button, completed the selection. Correct depressed, and zoomed by rotating the scrollwheel, with the selections were confirmed by green highlighting for 500 ms, cursor location serving as the scale orientation point. then the next trial began automatically. Incorrect selections The target turned green and became selectable at scale factor caused red highlighting for 500 ms, and the trial continued. 1.3 or greater. Having selected the left target by The slider length was 512 px. The number of candidate items tapping/clicking on it, it turned grey and the right-hand target within the slider varied across trials to control the precision was highlighted. Users then zoomed out to view the next required. The full set of trials comprised three repetitions at target, and combined pan and zoom actions to bring it to a each of the following resolutions: 2, 8, 32, 128 candidate items selectable scale factor. Software implemented maximum and in the slider. For each trial, the target value was randomly minimum scale factors of 3.0 and 0.04. Software prevented selected between 1 and the maximum value; and the slider panning to illogical regions, such as moving the left target was initially set to 0 for each trial. beyond the right display edge. The full set of trials consisted of four target selections, beginning with an initial selection Dial tasks involved setting a dial needle to a highlighted of ‘Left’ (data discarded, as unlike other trials it began with target in the dial (see Figure 3d). The needle could be set to the target in view). a location either by tapping/clicking in the target area (suitable when the target segment was large) or by dragging Procedure the needle with the finger/cursor. Software checked the On arrival, participants were briefly shown the motion correspondence between the needle and intended target when platform, harness, and apparatus. They were informed that the finger was lifted from the display (or trackball button the objective of the study was to compare the effectiveness released). If correct, the dial flashed green for 500 ms and the of different means for input in turbulent environments. next target segment was highlighted, with the needle initially Having put on the harness, they sat on the platform seat, and at due north. If incorrect, the dial flashed red for 500 ms and the ceiling-mounted securing cables were adjusted. the trial continued. The resolution of task precision was For familiarization, participants completed a training set of controlled across trials by varying the number of candidate trials consisting of all input methods and task type. The items in the dial. The full set of trials consisted of two motion platform was turned off during this familiarization. repetitions at each of the following resolutions: 4, 8, 32, 128. Participants then advanced to the main experimental tasks, Pan-zoom tasks required moving a yellow highlighted target which progressed according to the following algorithm: circle into the view and manipulating scale until the circle was sufficiently large to become selectable (it turned green). for each vibration ∈ {none, low, high}: for each input device ∈ {trackball, touch, stencil} The task emulates the activity of selecting destinations or for each task ∈ {targets, keypad, slider, dial, panzoom} waypoints on a map. The display contained two circles complete task trials labelled ‘left’ (on the left) and ‘right’ (horizontally right of complete subjective experience questionnaire Participants could orally request that any trial be abandoned. Design The experimenter then pressed a control key to advance to Data from each task type was separately analyzed for the next task, and the abandonment was logged. dependent measures trial time and error rate. Order of vibration level and input device was counter- Target selection data was analyzed using a 3×3×3×2 balanced. Tasks types were always completed in the order repeated-measures analysis of variance (RM-ANOVA) for targets, keypad, slider, dial, panzoom. factors vibration (none, low, high), device (touch, stencil, trackball), distance (96, 256, 512 px) and size (32, 64 px). Having completed all tasks with all three input devices at one Keypad and Pan-zoom data was analyzed using a 3×3 RM- vibration level, participants completed a subjective ANOVA for factors vibration and device. Dial and Slider experience questionnaire, recording NASA TLX measures data was analyzed using a 3×3×4 RM-ANOVA for factors for physical workload and frustration, subjective vibration, device, and resolution (4, 8, 32, 128 items for Dial; preferences, and other comments. While the participant 2, 8, 32, and 128 items for Slider). completed these worksheets, the experimenter loaded the next vibration profile into the motion platform. RESULTS The 18 participants produced data for a total of 7244 Apparatus successful trials across the five task types. Only 5 trials were Software for the experiment was written in Python 3.4, using abandoned (0.07%), all in high vibration. the kivy package for multitouch interaction in the pan-zoom tasks. Software ran on a Dell S2240T 21.5 monitor running Target Selection Tasks at 1920  1080 pixel resolution (4.03 px/mm). The monitor Selection time analysis and participant’s seat were mounted on a Mikrolar R-3000 RM-ANOVA showed a significant effect of vibration level hexapod motion platform. The experimenter sat alongside on error free selection time (Figure 4): F1.33,22.5 = 45.1, p < 2 the motion platform, configuring changes to motion profile .001, η G = 0.16; Greenhouse-Geiser correction applied for and input device when instructed by the software. sphericity violation. Mean selection times more than doubled To avoid continually switching between different stencils for as vibration increased from none (mean 1163 ms, sd 677), the five task-types, a single stencil was used for all task types. through low (1648 ms, sd 973), to high vibration (2360 ms, Each task type therefore inhabited a different region of the sd 1798). Distance had a significant effect on selection time display – target tapping was top-left; keypad top-right; dial (F2,34 = 4.3, p < .05), but the magnitude of its effect was 2 = 0.009), with mean times increasing through top-centre; slider bottom-centre (the transparent stencil small (η G overlay is visible in Figure 1). During pan-zoom tasks with 1610, 1684 and 1860 ms for 96, 256 and 512 px distances. the stencil, participants were instructed to use the dial hole Distance was not a factor in any significant interactions. for controlling interaction. Target size had a stronger and significant effect on selection 2 The motion pattern sent to the hexapod platform was time: F1,17 = 139.7, p < .001, η G = 0.12. Mean selection designed to induce non-periodic vertical displacements with times for small (32 px) and large targets (64 px) were 2137 and 1299 ms, respectively. a mean motion frequency of 3.1 Hz (max 5 Hz). Movement amplitudes were configured to produce weighted RMS There was no significant main effect of device: F2,34 = 1.15, accelerations in the ‘very uncomfortable’ range (ISO2631-1, p = .33. Mean selection times were similar with stencil and 2 1.25 < aT < 2.5 m/s ) for the high vibration condition and touch, at 1648 and 1686 ms respectively, and slightly higher 2 ‘uncomfortable’ for low vibration (0.8 < aT < 1.6 m/s). for trackball (1820 ms). However, there was a significant Accelerometer sampling on the seatpan confirmed x, y, z and device × size interaction (F2,34 = 3.46, p < .05). This effect total (aT) acceleration values of 0.15, 0.05, 1.08, and 1.10 was largely caused by the relatively similar mean selection m/s2 respectively for the low vibration condition, and 0.72, 2 0.07, 2.11 and 2.15 m/s for the high vibration condition. 4000 Stencil Touch Trackball These frequency and acceleration levels are much higher 3500 than previous studies of ship motion (e.g., [25, 41]). 3000 Trackball input was provided through a Logitech M570 2500 (milliseconds)

device, which includes a scrollwheel that was used for 2000 time controlling zoom-level in the pan-zoom tasks. 1500 Participants 1000 selection Eighteen volunteer participants were recruited for the study 500 – 11 female, aged 19-40 (mean 27). All were familiar with 0 Mean touchscreen input methods, reporting a mean daily use of ~2 None Low High None Low High hours. Participation in the experiment lasted for 1 hour, and 32 64 was compensated with a $10 payment. Figure 4. Error-free selection times for small (left) and large (right) targets by device and vibration. Error bars ±1 s.e.m. times for the three devices with small targets (2089, 2193, 0.16 Stencil Touch and 2128 ms for stencil, touch, and trackball respectively), 0.14

whereas with large targets, the trackball was slower (mean 0.12 rate 1510ms) than stencil and touch (1207 and 1180 ms). 0.1 0.08 error There was also a significant three way interaction of device 0.06

× size × vibration: F4,68 = 3.27, p < .05. For small targets Mean 0.04 (Figure 4, left), when there was no vibration, mean selection 0.02 times with stencil were slower than touch, but as vibration 0 increased stencil selections became faster than touch. 32 64 32 64 32 64 However, for larger targets (as shown in Figure 4, right), the None Low High benefits of the stencil over touch at high vibrations did not Figure 5. Error rates for stencil and touch for differently sized emerge; presumably because the targets were large enough targets at each vibration level. Error bars show ±1 s.e.m. to be easily acquired without a stabilizing edge. Also, the 0.9 benefits of self-stabilization with the trackball become Stencil 0.8 apparent at high levels of vibration, particularly with small Touch targets. In summary, there is cross-over effect for small 0.7 0.6 errors

targets (that is largely absent with large targets) –

of 0.5 unstabilized touch is fastest when vibration is absent, but it becomes slowest when vibration is present. 0.4 0.3 Proportion Error analysis 0.2 Two different interaction events could be interpreted as 0.1 errors: 1. the user misses the target, instead hitting ‘dead 0 30 40 50 60 70 80 90

space’ between active elements in the interface; 2. the user 100 110 120 130 140 150 160 170 180 190 200 300 400 500 hits an incorrect target. Our error analysis focuses on Milliseconds since correct selection incorrect target selections for two reasons. First, the stencil Figure 6. Cumulative proportion of errors with stencil and essentially eliminates the possibility for hitting interface touch across time since correct selection. dead-space because the holes are only present over active elements. Second, interactions on dead-space are likely to stencil to 1.6% (and 2.8% for touch); and for small targets at have lower consequences than incorrect selections. high vibration, the stencil error rate was 2.5%, compared to Incorrectly or accidentally hitting a target could have serious 10.4% for touch. flight consequences, such as unknowingly changing a mode, To confirm that stencil errors can be eliminated by whereas hitting dead-space has no effect. addressing ‘bounce’ errors on lift off, we conducted a small Across all trials, an incorrect target was selected in 2.9% of subsequent study of stencil and touch Target Selection with trials. There were zero incorrect target selections with the four participants at high vibration. We used the same stencil trackball, even at high vibration. as the original study, but increased the target sizes to 54 (small) and 96 px (large) to make them easier to acquire with There were a total of 45 errors with the stencil (rate 4.67%) touch (subjective comments, reported later, indicated touch and 40 with touch. Figure 5 summarizes error rate across was too hard at high vibration). Each participant completed conditions. One surprising observation from this analysis four repetitions of the same target pattern used in the original was that stencil errors were higher than touch. experiment, producing data for 576 trials (4 participants, 2 Timing analysis showed the cause of this anomaly. It devices (stencil and touch), 4 repetitions, 18 targets). With revealed that a large proportion of stencil errors were caused the 200 ms delay between contacts, there were 11 incorrect by a second selection of the target shortly after its correct target selections with stencil (3.8%), compared to 8 with selection. Figure 6 shows the cumulative proportion of touch (2.8%). If all touch contacts off the intended target stencil and touch errors occurring within 500 ms of a correct were interpreted as errors (including all wrong target and selection – 64% of stencil errors and 28% of touch errors dead-space selections), the total number of errors with stencil occurred within 200 ms of correctly hitting a target. These was 31 (mean 0.11 errors/selection) compared to more than errors occur during the ‘lift off’ action, and can be attributed six times as many with touch (198, 0.69 errors/selection). to vibration causing accidental activation during ‘lift off’. Keypad Tasks Dragging the finger up the stencil edge during lift-off RM-ANOVA of keypad tasks showed significant effects of appears to increase the incidence of these errors. device (F2,34 = 38.8, p < .001) and vibration (F2,34 = 20.0, p These stencil accuracy problems are easily remediated by < .001) on the mean time to enter and confirm a three-digit discarding contacts that occur within 200 ms of a previous target number. There was no significant device × vibration contact. Discarding these ‘bounce’ contacts within 200 ms of interaction (p = .53). As shown in Figure 7, mean a previous touch reduces the overall mean error rate for the performance was very similar between stencil and touch 7000 2.5 Stencil Touch Track Stencil Touch Track 6000 (ms)

2 selection 5000 time

per 1.5 4000 entry

count 1 3000 error

number 2000

0.5 1000 Mean Mean 0 0 2 8 32 128 2 8 32 128 2 8 32 128 None Low High None Low High Figure 7. Mean time (± 1 s.e.) to enter a 3-digit number. Figure 8. Mean errors (± 1 s.e.) per slider selection trial. (3398 and 3371 ms, respectively), and trackball was Setting values via sliders therefore seems to be an substantially slower (5166 ms). The similarity of task time inadvisable interaction for a cockpit environment, where for stencil and touch suggests that users were able to accuracy and error-free interaction is required. confidently hit the large key targets without relying on the Dial Tasks stencil edge, even when vibration was high. Like slider tasks, dial tasks required maintaining a dragging Error rates were low. An incorrect number was submitted (by state while manipulating the location of the arrow. Any pressing the ‘OK’ button) 12 times from 542 tasks (2%): 6 unintended lift-off caused an error. In light of the slider times with touch, 4 times with trackball, and twice with results, it is therefore unsurprising that dial task performance stencil. The ‘DEL’ key was used to correct an incorrectly was also dominated by errors, with an overall mean of 0.24 entered digit 48 times (from 1626 required digit entries) – 24 errors per trial. There were no significant effects on task time with touch, 4 with trackball, and 20 with stencil. (means of 3.8, 3.5 and 3.4 s for stencil, touch and trackball). Slider Tasks Mean errors per trial were lower with the trackball (0.06) The performance of slider tasks was dominated by errors. than stencil (0.34) or touch (0.32). At high vibrations with Errors were high in all conditions, but particularly so for 128 candidate items, all devices had high error rates: 0.33 precise selections in high vibration settings. Figure 8 shows with trackball, 1.89 with stencil, and 1.69 with touch. the mean number of errors per trial, which ranged from 0.019 Pan-Zoom Tasks for trackball selections with 2 candidate items and low Pan-zoom tasks are interesting because completing them on vibration, through to a maximum of 1.97 for touch selections a touchscreen requires manipulating dragging actions (i.e., with 128 candidate items and high vibration. There were no pinch to zoom and pan). However, unlike slider and dial significant effects on task time. Mean selection times were tasks, lift-off actions are anticipated as part of successful task 1.9, 1.8 and 1.8 s with stencil, touch and trackball. completion – users are expected to repeatedly clutch their Even when there was no vibration, all of the devices pinching/depinching gestures to attain the required zoom produced high error rates (means of 0.37, 0.54 and 0.7 for level. Also, completing zoom actions with the trackball did touch, trackball, and stencil). The particularly high error rate not require maintaining a dragging state because zooming for stencil (~1.5 per trial) with 128 items and no vibration could be directly controlled by rotating the scrollwheel. may be due the finger contact forming unanticipated shapes Figure 9 summarizes selection time results for the pan-zoom (triggering unintended movement) during lift-off due to task. As indicated by the figure, there was a significant main pressure against the stencil edge. effect of device (F2,34 = 42.8, p < .001), with stencil and At high vibrations, the error rates were extremely high touch having relatively similar performance (9.76 and 8.49 s (overall mean of 0.9 errors per trial, ranging from 0.6 for respectively, s.d. 3.4 and 2.8 s), and track much slower at trackball to 1.12 with touch). One factor contributing to 14.5 s (s.d., 4.3). Vibration also had a significant effect on these high error rates is the mechanical difficulty of selection time (F2,34 = 5.98, p < .01), with mean times maintaining contact pressure on a touch-surface (or trackball increasing through conditions of none (10.2 s, s.d. 3.8), low button) to continue the dragging state while undergoing (10.7 s, s.d., 4.2) and high (11.9 s, 5.0). There was no device vibration. The target selection and keypad tasks allowed × vibration interaction (p = .3). users to make discrete ‘stabs’ at the surface or button, Subjective Responses reducing the temporal window for vibrations to influence Analysis of subjective responses reinforce the objective data interaction. Dragging actions, in contrast, necessarily have a reported above. We used Friedman tests to analyze prolonged duration, increasing the opportunity for vibration participants’ ratings of physical workload and frustration to adversely affect execution. (reported on 7-point Likert scales) for target selection tasks across vibration and device. As shown in Figure 10 (left), participants’ ratings of physical workload increased contact with the sheet to avoid accidental activations. significantly as vibration increased, from a mean of 2.35 at Explicit training that the Lexan sheet would prohibit contact no vibration, through 3.83 at low vibration, to 4.56 at high registration may have influenced participants’ strategies and vibration – 2 = 39.8, p < .001. There was no significant improved performance with the stencil. effect of device on physical workload (p = .09). Frustration ratings (Figure 10, right) showed significant effects of 6 Stencil Touch Track high) 2 2 5 vibration ( = 34.7, p < .001) and device ( = 16.1, p < 7

.001) with touch having the highest overall frustration rating low,

4 (1 (4.0), followed by stencil (3.2) and track (2.9). 3 rating 20000 2 Stencil Touch Track 1 workload

(ms)

15000 0 Mean time None Low High None Low High 10000 Physical Frustration selection

5000 Figure 10. Mean responses (±1 s.e.) for physical workload and

Mean frustration by vibration (7 point Likert scales, 1 low, 7 high).

0 Most participants quickly learned that when vibration was None Low High present they needed to stabilize their hand and finger in order Figure 9. Mean selection times in pan-zoom trials for each to complete tasks with touch and stencil. Typically they did input device at each vibration level. Error bars show ±1 s.e.m. so by placing their fingers or thumb on the screen edge (off the contact surface) and spanning their hand to make contact Participants’ comments further emphasized that unstabilized with the target on the display. Target selection tasks were touch interaction was difficult, whereas the trackball and located on the left side of the display (see the holes in stencil were easier to control in high vibration: Figure 1). Most users therefore placed the thumb of their S1: “Stencil help to control where I want to point out on the screen … right hand on the left or top display edge, and spanned to the touchscreen is hardest to control especially for the small object.” targets with their index or middle finger. The same strategy S6: “Stencil was best because its borders offered a control or support was used for both touch and stencil. for the tasks that were difficult under vibration (e.g., the tapping). The touchscreen was worst because even a little vibration threw me off.” Slider tasks were located near the bottom of the display (Figure 3). This placement allowed most participants to S12: “The touchscreen was impossible in the ‘tapping’ run. My arm was out of my control … The stencil made it hard to mess up.” grasp the screen edge with their thumb and span their index S13 “trackball [best for high vibration] because it was easiest, least finger to the slider widget. Keypad tasks were located near jerky (i.e., most stable), most precise”. the right edge of the display, and some participants grasped S18 “I found the stencil helps a lot when I want to press exact places. the right edge of the display with the fingers of their right It’s a combination of benefits of using touchscreen (handy) & hand and spanned their thumb to the targets. However, improves the exactness.” keypad targets were sufficiently large for most participants Several participants commented that when vibration was to complete selections without stabilization. absent, touch interaction was the easiest input method and DISCUSSION that the trackball was slow and awkward. Seven key findings emerge from the results: S1: “trackball takes time to move the pointer to where I want” 1. There was a wide variance in the different tasks’ overall S13 “trackball was slowest and least user-friendly.” tolerance to vibration. Tasks with low pointing precision One participant commented that the stencil was distracting: requirements (e.g., Keypad) could be accomplished successfully with any device under all vibration S17 “the limitations in space [with the stencil] made it more difficult to conditions; others (particularly Slider) were difficult in concentrate on the task. It’s like doing more things at once.” all cases (even without vibration). Participant Strategies 2. When pointing precision requirements were low, touch- Participants were not instructed on how best to use each of based interactions (stencil or touch) were generally faster the input devices, nor on how to stabilize their input. than trackball input. This includes the selection of large Although we had anticipated that participants would rest targets (e.g., the 104 px wide keys in the keypad task) and their fingers, palm, or hand on the stencil surface to stabilize coarse pan-zoom actions, even at high vibration levels. their input, only one participant did so (S11). In hindsight, 3. However, when selecting small targets in a vibrating this is understandable – the Lexan sheet minimally environment, trackball input was faster than unstabilised influenced the appearance of widgets on the display, and touch-based input. consequently, participants were likely resistant to making 4. Trackball input was accurate, even at high vibrations. 5. Drag-based selections, in which lift-off/release there. Additionally, at least one participant referred to visual confirmed selection, were inaccurate with touch and ‘distraction’ caused by the overlay. For these reasons, it trackball. Errors were high when there was no vibration, seems that the extra bezel ridges described above, may be and became extremely high when vibration was present. preferable to the transparent overlay examined in our study. 6. The stencil was generally unsuccessful. The device × size Study limitations and further work × vibration interaction in target selection tasks (Figure 4) To our knowledge, this is the first study of touch interaction suggests that the stencil may improve touch interaction in cockpits that covers a range of important task types and with small targets at high vibrations, but further work is different vibration levels, as well as examining a potential needed to confirm this effect. Also, the stencil relies on remediation technique. As touchscreens are increasingly a timeout after each selection (~200 ms), without which used in aircraft, there is need for further research on the it is susceptible to finger lift-off errors during vibration. problems of turbulent input and their remediation. Froehlich et al. [11] used a similar timeout method to eliminate ‘bounce’ errors by users with tremor. Our study used a limited range of vibration profiles, partially 7. Multi-touch pan and zoom selections were much faster due to the maximum displacement (150 mm) attainable by than equivalent interactions with a trackball (panning and our motion platform. Further studies could examine different scrollwheel zooming) at all vibration levels. vibrations, such as higher amplitudes and varied frequencies. Another obvious area for further study concerns the impact Touch vibration tolerance of target distribution on performance with the various Prior to conducting the study, we suspected that participants devices (e.g., sparsely versus densely packed targets). would have extreme difficulty in completing tasks in the touch condition at high vibration. However, results showed Participants in our study were predominantly university that participants were resilient, maintaining their ability to students. While this demographic is familiar with interact by stabilizing their hand on the screen edge. touchscreen interaction, none were pilots, and they were not explicitly trained on how to best exploit the evaluated Our experimental conditions facilitated this hand technologies. There are therefore opportunities to study stabilization because the smallest targets (and therefore whether comparative user performance with stencils and hardest to acquire) were located near the screen edges. Based trackballs changes when users are well trained. Similarly, on our observations of strong reliance on hand stabilization, testing with real pilots and real cockpit display systems will we suspect that performance with touch would have been help validate findings for commercial use. Also, our trackball much slower and more error prone if the target widgets had input used default control-display gain functions provided by been placed nearer the center of the display. Central the device manufacturer, which are likely to differ from the placement would reduce the availability (or effectiveness) of proprietary functions used by aircraft manufacturers. the hand-spanning strategy we observed, with the fingers or thumb placed on the screen edge. Finally, we plan to test these results in other settings where touchscreens are being introduced into environments that can However, flight displays could easily be constructed to experience turbulence. For example, touchscreens are facilitate hand stabilization. Each display could be small, or becoming common in cars and motorbikes – and here, the on a large display, touch-inactive horizontal and vertical added concern of visual attention becomes an important bezel ridges for finger/thumb placement could be overlaid on issue. In further studies we will test whether the tactile the display, with the distance between the ridges configured feedback provided by the stencil reduces the amount of time to permit easy hand-spanning to targets. There are also users must attend to the display during vibration episodes. interesting opportunities for augmenting touchscreens with graspable or tactile widgets that could be positioned on or CONCLUSION near the display (e.g., [14, 19, 28, 34, 35]). Touchscreen interaction in the cockpits of commercial aircraft offers potential advantages to aircraft manufacturers, Lessons from the stencil airlines, and pilots. However, turbulence is a challenge for Results from the stencil are interesting, suggesting both their deployment. Our studies of touch interaction in strengths that can be deployed, and weaknesses that should simulated turbulence indicate that users can successfully be avoided. In terms of strengths, several participants stabilize discrete touch input (i.e., target selections) by referred to the intended benefits of stabilization. Also, while touching the side of the screen with their fingers or thumbs. raw wrong-target errors were increased by the stencil, timing For small targets, a stencil overlay assisted users in hitting analysis indicated that once lift-off errors were eliminated targets. Drag-based selections that confirmed selection on (by disabling touch for 200 ms following a selection), the lift-off/release were highly error prone during simulated stencil had similar wrong-target error rates as unstabilised vibration, both with touch and trackball. However, multi- touch, and dramatically fewer off-target errors. touch pan-and-zoom selections were much faster with touch However, participants seemed reluctant to place their fingers than trackball. or hand onto the stencil surface, possibly because they mistakenly believed that their contacts would be registered REFERENCES 12. Hamon, A., Palanque, P., Andrea, R., Barboni, E., 1. ARINC661 Cockpit Display System Interfaces to User Cronel, M. and Navarre, D. Multi-Touch interactions for Systems. ARINC Specification 661, AEEC, 2002. control and display in interactive cockpits: issues and a 2. ARINC661 Cockpit Display System Interfaces to User proposal. Proceedings of the International Conference Systems ARINC Specification 661, supplement 6, on Human-Computer Interaction in Aerospace, ACM, AEEC, 2016. Santa Clara, California, 2014, 1-10. 3. Avrahami, D. The Effect of Edge Targets on Touch 13. Hardinger, C.E. Built to touch. Rockwell Collins Performance. Proceedings of the 33rd Annual ACM Horizons. Vol. 17(1). 2011, 18-19. Conference on Human Factors in Computing Systems, 14. Harrison, C. and Hudson, S.E. Providing dynamically ACM, Seoul, Republic of Korea, 2015, 1837-1846. changeable physical buttons on a visual display. 4. Barbé, J., Chatrenet, N., Mollard, R., Wolff, M. and Proceedings of the SIGCHI Conference on Human Berard, P. Physical ergonomics approach for touch Factors in Computing Systems, ACM, Boston, MA, screen interaction in an aircraft cockpit. Proceedings of USA, 2009, 299-308. the 2012 Conference on Ergonomie et Interaction 15. Hill, S. and Tauson, R. Soldier performance issues in C2 homme-machine, ACM, Biarritz, France, 2012, 9-16. "On the move". 10th International Command and 5. Bauersfeld, K.G. Effects of turbulence and activation Control Technology Symposium, McLean, VA, USA, method on touchscreen performance in aviation 2005. environments. Psychology, San Jose State University, 16. Hourlier, S., Guérard, S., Barou, J.L. and Servantie, X. 1992. Testing Touch Screens in Realistic Aeronautic 6. Buxton, W., Hill, R. and Rowley, P. Issues and Turbulent Conditions (Light to Severe). SAE techniques in touch-sensitive tablet input. SIGGRAPH International Journal of Aerospace 8, 2 (2015), 243-247. Comput. Graph. 19, 3 (1985), 215-224. 17. Huseyin, A., Fischer, J.E. and Rodden, T. Designing 7. Cockburn, A., Ahlström, D. and Gutwin, C. touch-enabled electronic flight bags in SAR helicopter Understanding performance in touch selections: Tap, operations. HCI-Aero: International Conference on drag and radial pointing drag with finger, stylus and Human-Computer Interaction in Aerospace, ACM, mouse. International Journal of Human-Computer Paris, France, 2016. Studies 70, 3 (2012), 218-233. 18. Huseyin, A., Fischer, J.E. and Rodden, T. Future Flight 8. Dodd, S., Lancaster, J., Miranda, A., Grothe, S., Decks: Impact of +Gz on Touchscreen Usability. HCI- DeMers, B. and Rogers, B. Touch Screens on the Flight Aero: International Conference on Human-Computer Deck: The Impact of Touch Target Size, Spacing, Touch Interaction in Aerospace, ACM, Paris, France, 2016. Technology and Turbulence on Pilot Performance. 19. Jansen, Y., Dragicevic, P. and Fekete, J.-D. Tangible Proceedings of the Human Factors and Ergonomics remote controllers for wall-size displays. Proceedings of Society Annual Meeting 58, 1 (2014), 6-10. the SIGCHI Conference on Human Factors in 9. Fletcher, M. and Huffman, D. Panoramic cockpit Computing Systems, ACM, Austin, Texas, USA, 2012, displays for tactical military cockpits. Javidi, B., Son, J.- 2865-2874. Y., Thomas, J.T. and Desjardins, D.D. eds. Proc. SPIE 20. Kaminani, S. Human computer interaction issues with 7690, Three-Dimensional Imaging, Visualization, and touch screen interfaces in the flight deck. in Digital Display 2010 and Display Technologies and Avionics Systems Conference (DASC), 2011 Applications for Defense, Security, and Avionics IV, IEEE/AIAA 30th, (2011), 6B4-1-6B4-7. 769016, Orlando, Florida, 2010, 769016--769016- 21. Kolbe, D.M. Method for determining valid touch screen 769017. inputs. Patent: Application CA2792590A1. 2012. 10. Forlines, C., Wigdor, D., Shen, C. and Balakrishnan, R. General Electric Company Direct-Touch vs. Mouse Input for Tabletop Displays. in 22. Komer, J.L., Gepner, J.E., Hogan, R.K. and Mabie, T.D. Proceedings of the 25th international conference on Avionics control and display unit having cursor control Human factors in computing systems, ACM, (2007), mode of operation. Patent: US8626360B2. 2014. 647-656. Garmin International Inc. 11. Froehlich, J., Wobbrock, J.O. and Kane, S.K. Barrier 23. Levesque, V., Oram, L., MacLean, K., Cockburn, A., pointing: using physical edges to assist target acquisition Marchuk, N.D., Johnson, D., Colgate, J.E. and Peshkin, on mobile device touch screens. Proceedings of the 9th M.A. Enhancing Physicality in Touch Interaction with international ACM SIGACCESS conference on Programmable Friction. in Proceedings of CHI'11: ACM Computers and accessibility, ACM, Tempe, Arizona, Conference on Human Factors in Computing Systems, USA, 2007, 19-26. ACM, (2011), 2481-2490. 24. Lewis, R. The Effect of Handedness on use of Touch Factors in Computing Systems, ACM, Atlanta, Georgia, Screen versus Touch Pad. Proceedings of the 5th USA, 2010, 2223-2232. International Conference on Application and Theory of 35. Ullmer, B., Dell, C., Gil, C., Cornelius Toole, J., Wiley, Automation in Command and Control Systems, ACM, C., Dever, Z., Rogge, L., Bradford, R., Riviere, G., Toulouse, France, 2015, 115-120. Sankaran, R., Liu, K., Freeman, C., Alvin Wallace, J., 25. Lin, C.J., Liu, C.N., Chao, C.J. and Chen, H.J. The DeLatin, M., Washington, C., Reeser, A., Branton, C.W. performance of computer input devices in a vibration and Parker, R. Casier: structures for composing environment. Ergonomics 53, 4 (2010), 478-490. tangibles and complementary interactors for use across 26. Mack, R. and Lang, K. A Benchmark Comparison of diverse systems. Proceedings of the fifth international Mouse and Touch Interface Techniques for an conference on Tangible, embedded, and embodied Intelligent Workstation Windowing Environment. in interaction, ACM, Funchal, Portugal, 2011, 229-236. Proceedings of Human Factors Society 33rd Annual 36. Vogel, D. and Baudisch, P. Shift: A Technique for Meeting, (1989), 325-329. Operating Pen-Based Interfaces Using Touch. in 27. McLeod, P., Poulton, C., Du Ross, H. and Lewis, W. Proceedings of CHI'07: ACM Conference on Human The influence of ship motion on manual control skills. Factors in Computing Systems, ACM Press, (2007), Ergonomics 23, 7 (1980), 623--634. 657-666. 28. Miruchna, V., Walter, R., Lindlbauer, D., Lehmann, M., 37. Wang, L., Cao, Q., Chang, J. and Zhao, C. The Effect of Klitzing, R.v., Johansson, E. and Müller, J. GelTouch: Touch-key Size and Shape on the Usability of Flight Localized Tactile Feedback Through Thin, Deck MCDU. The Eighth International Conference on Programmable Gel. Proceedings of the 28th Annual Advances in Computer-Human Interactions, IARIA, ACM Symposium on User Interface Software & Lisbon, Portugal, 2015, 234-238. Technology, ACM, Daegu, Kyungpook, Republic of 38. Wobbrock, J., Chau, D. and Myers, B. An Alternative to Korea, 2015, 3-10. Push, Press, and Tap-tap-tap: Gesturing on an Isometric 29. Narayanamoorthy, R. and Huzur Saran, V. Error Joystick for Mobile Phone Text Entry. in Proceedings analysis of task performance with laptop in vibration CHI 07: ACM conference on Human factors in environment. Computers in Human Behavior 27, 6 computing systems, ACM, (2007), 667-676. (2011), 2263--2270. 39. Wobbrock, J. and Myers, B. Trackball Text Entry for 30. Potter, R.L., Weldon, L.J. and Shneiderman, B. People with Motor Impairments. in Proceedings of Improving the accuracy of touch screens: an CHI'06 ACM Conference on Human Factors in experimental evaluation of three strategies. in Computing Systems, ACM Press, (2006), 299-308. Proceedings of the SIGCHI conference on Human 40. Wobbrock, J., Myers, B. and Kembel, J. EdgeWrite: A factors in computing systems, ACM, (1988), 27-32. stylus-based text entry method designed for high 31. Ren, X. and Moriya, S. Improving selection accuracy and stability of motion. in Proceedings of the performance on pen-based systems: a study of pen- 16th annual ACM symposium on User Interface based interaction for selection tasks. ACM Trans. Software and Technology, ACM Press, (2003), 61-70. Comput.-Hum. Interact. 7, 3 (2000), 384-416. 41. Yau, Y.-J., Chao, C.-J. and Hwang, S.-L. Effects of 32. Robinson, T., Grabski, G., Green, J., Jacobson, M., Input Device and Motion Type on a Cursor-Positioning Byrne, C. and Harper, D. Physical Touch Aspects of the Task. Perceptual and Motor Skills 106, 1 (2008), 76-90. Touch Interface for Flight Deck Applications. SID 42. Zammit-Mangion, D., Becouarn, L., Aymeric, B., Symposium Digest of Technical Papers, 2014, 1618– Fabbri, M. and Bader, J. A Single Interactive Display 1621. Concept for Commercial and Business Jet Cockpits. in 33. Stanton, N., Harvey, C., Plant, K. and Bolton, L. To 11th AIAA Aviation Technology, Integration, and twist, roll, stroke or poke? A study of input devices for Operations (ATIO) Conference, American Institute of menu navigation in the cockpit. Ergonomics 56, 4 Aeronautics and Astronautics, 2011. (2013), 590-611. 43. Zhai, S., Kristensson, P.O., Appert, C., Anderson, T.H. 34. Tuddenham, P., Kirk, D. and Izadi, S. Graspables and Cao, X. Foundational Issues in Touch-Surface revisited: multi-touch vs. tangible input for tabletop Stroke Gesture Design — An Integrative Review. displays in acquisition and manipulation tasks. Foundations and Trends in Human–Computer Proceedings of the SIGCHI Conference on Human Interaction 5, 2 (2012), 97-205.