Pilot Performance, Workload, and Situation Awareness During Lunar Landing Mode Transitions
Total Page：16
File Type：pdf, Size：1020Kb
JOURNAL OF SPACECRAFT AND ROCKETS Pilot Performance, Workload, and Situation Awareness During Lunar Landing Mode Transitions Christopher J. Hainley, Jr.∗ Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 Kevin R. Duda† Charles Stark Draper Laboratory, Inc., Cambridge, Massachusetts 02139 and Charles M. Oman‡ and Alan Natapoff§ Massachusetts Institute of Technology, Cambridge, Massachusetts 02139 DOI: 10.2514/1.A32267 Aircraft and spacecraft pilots frequently change their level of supervisory control between full autopilot and other modes, providing varying levels of manual control. Therefore, multimodal systems must transition “gracefully,” meaning without unsafe decreases in flight performance or unacceptable changes in workload or situation awareness. Thirteen subjects flew a fixed base simulation of the NASA Constellation Program Altair lunar lander that transitioned from full autopilot to one of three flight-director-guided rate-command attitude hold manual control modes. After training, each subject flew 24 approaches, half of which included a landing point redesignation at the time of the mode transition requiring the pilot to null additional guidance errors. Bedford subjective workload and two-choice embedded secondary task response times were used to quantify temporal changes in mental workload. Situation awareness transients were detected by analysis of a tertiary task, verbal callouts of altitude, fuel, and terrain hazards. Graceful transitions were particularly difficult because Altair’s large inertia made the plant dynamics relatively sluggish. Transitions to manual control increased subjective and objective workloads and decreased callout accuracy in proportion to the number of flight control axes being manually commanded. I. Introduction Automation design heuristics [4], theories of supervisory control – ANY large transport aircraft and spacecraft, such as the [5], and level-of-automation (LOA) taxonomies [6 8] were initially M Apollo Lunar Landing Module and the U.S. Space Shuttle, developed for teleoperation, powerplant, and air traffic control were designed to be capable of almost fully automatic landing. Future applications but only bluntly characterize the manual control options spacecraft may be capable of fully automatic landings when used for available in modern autoflight systems. Employing a high LOA uncrewed missions (e.g., [1]). However, in crewed systems, pilots lowers the mental workload and improves the overall situation generally insist on the availability of manual control modes, allowing awareness (SA), but remaining engaged at a low LOA keeps the pilot them to change from full supervisory mode to some level of manual aware of the aircraft state and more able to detect and respond to control, e.g., while reprogramming an approach route or to diagnose a certain failures (e.g., [9]). It has long been recognized that the LOA system failure [2]. For example, airline pilots can choose to engage must vary during operational use depending on situational demands. However, some aircraft autoflight system modes are clumsy and heading, altitude, and/or thrust autoflight modes or control each of – them manually. NASA spacecraft human rating requirements [3] unpredictable [10], complicating mode awareness [11 14] and dictate that astronauts must be able to manually override higher- resulting in transitions that surprise even experienced operators level software control/automation and control spacecraft attitude and [10,15]. Mode transitions reduce operator performance in error flight path. Following the dictum “fly as you train, train as you fly,” detection and error mitigation in automated systems [2,9,16,17]. The need for aircraft and spacecraft automations that support graceful both astronauts and aircraft pilots practice transitioning between reversion from automatic to manual flight has long been recognized automation modes and often choose to fly part or all of an approach (e.g., [18–20]). Previous studies of the effects of the LOA (e.g., [21]) using manual modes, often aided by flight director guidance. assess performance, mental workload, and situation awareness but have focused on steady-state behavior rather than the transient. In aviation, pilots and their instructor evaluators typically speak of Received 27 October 2011; revision received 21 December 2012; accepted the “gracefulness” of a specific mode transition, referring to the Downloaded by MASSACHUSETTS INST OF TECHNOLOGY (DRAPER LAB) on June 19, 2013 | http://arc.aiaa.org DOI: 10.2514/1.A32267 for publication 31 December 2012; published online 19 June 2013. Copyright undesirable decrease in operator performance, increase in workload, © 2013 by The Charles Stark Draper Laboratory, Inc. Published by the and change in situation awareness associated with a mode transition, American Institute of Aeronautics and Astronautics, Inc., with permission. even when the transition is expected, and so automation surprise is Copies of this paper may be made for personal or internal use, on condition not an additional factor. Because each dimension of this gracefulness that the copier pay the $10.00 per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923; include the code construct can be quantified, it provides a useful three-dimensional 1533-6794/13 and $10.00 in correspondence with the CCC. metric for the evaluation of the transient effect of mode transitions. *MIT Research Assistant, Man-Vehicle Laboratory, Department of To date, no studies have attempted to quantify such transient effects Aeronautics and Astronautics, 77 Massachusetts Avenue 37-219; currently of the reversion to manual flight in terms of concurrently measured Draper Laboratory Fellow, Charles Stark Draper Laboratory, Inc. manual control performance, subjective and objective mental † Senior Member of the Technical Staff, Human-Centered Engineering workloads, and situation awareness and how these depend on Group, 555 Technology Square; AIAA Life Sciences and Systems Technical transition characteristics such as the change in the number of manual Committee. Senior Member AIAA. ‡ control loops the operator must control. This study examined these Senior Research Engineer and Senior Lecturer, Man-Vehicle Laboratory, Department of Aeronautics and Astronautics, 77 Massachusetts Avenue 37- questions in the context of a lunar landing scenario, but there are 219. potentially broader applications. §Research Scientist, Man-Vehicle Laboratory, Department of Aeronautics Automation of control functions previously assigned to a human and Astronautics, 77 Massachusetts Avenue 37-219. operator often improves the overall system performance and AIAA Early Edition / 1 2 AIAA Early Edition / HAINLEY ET AL. reliability, can significantly decrease operator workload [22], and can use of a timed two-choice response task to probe short-term improve failure detection [23,24]. These benefits typically come at variations in mental workload, detailed next, is new. the cost of potential operator complacency [9], skill degradation [2], SA can be generally defined as “the perception of the elements in and the inability of operators to diagnose and respond to subsystem the environment with respect to time and/or space, the com- failures [25]. To enhance operational flexibility and operator prehension of their meaning and the projection of their status in the confidence, a variety of automation modes are typically provided. near future” [40,41]. The subject’s perception of their own awareness Within a particular system mode, the role of the human operator may has been assessed using of subjective scales (e.g., Situation range from the supervisory level (e.g., human specifies a series of Awareness Rating Technique [42]), but such ratings are limited by the high-level goals and simply monitors the execution) to a level in subject’s own perceptions of reality. Similarly, ratings by expert which the human both sets goals and executes low-level manual observers (e.g., SALSA [43]) are constrained because observers may commands. This range of interactions has been referred to as the LOA have only limited knowledge of the operator’s concept of the situation [8]. Systems frequently have numerous modes; for example, the [21]. Objective measures are traditionally obtained via analysis of autoflight systems on large commercial aircraft have several dozen answers to questions administered during simulation freezes (e.g., heading-, altitude-, and trust-related modes. Systems are typically Situation Awareness Global Assessment Technique [41]) or by mea- designed so that some modes operate concurrently, and certain mode suring the reaction time to probe questions related to the displayed transitions occur automatically or can be only selected if certain information (e.g., SPAM [44]). However, we wanted to be able to conditions are met. Variouserror conditions can cause the automation assess short-term temporal changes in critical SA variables without to initiate a reversion to manual flight, but the operator typically the intrusion of simulation freezes or distracting probe questions or retains authority to choose the subsequent LOA (“adaptable reliance on posttrial memories. We therefore analyzed the timeliness automation” [26]). The alternative of designing the automation so and accuracy of the required verbal callouts of altitude, fuel, and terrain, which our pilot subjects made during their approach to that it monitors or anticipates pilot performance, workload, and “ ”