Contextual Constraints and Information Sources

NASA/TM—2017–219539

Autonomous, Context-Sensitive, Task Management Systems and Decision Support Tools II: Contextual Constraints and Information Sources

Barbara K. Burian NASA Ames Research Center

Janeen A. Kochan Aviation Research, Training, and Services

Kathleen L. Mosier San Francisco State University

Ute Fischer Georgia Institute of Technology

September 2017 NASA STI Program…in Profile

Since its founding, NASA has been dedicated • CONFERENCE PUBLICATION. to the advancement of aeronautics and space Collected papers from scientific and science. The NASA scientific and technical technical conferences, symposia, information (STI) program plays a key part in seminars, or other meetings sponsored helping NASA maintain this important role. or co-sponsored by NASA.

The NASA STI program operates under the • SPECIAL PUBLICATION. Scientific, auspices of the Agency Chief Information technical, or historical information Officer. It collects, organizes, provides for from NASA programs, projects, and archiving, and disseminates NASA’s STI. The missions, often concerned with NASA STI program provides access to the subjects having substantial public NTRS Registered and its public interface, the interest. NASA Technical Reports Server, thus providing one of the largest collections of aeronautical and • TECHNICAL TRANSLATION. space science STI in the world. Results are English-language translations of published in both non-NASA channels and by foreign scientific and technical NASA in the NASA STI Report Series, which material pertinent to NASA’s mission. includes the following report types: Specialized services also include creating • TECHNICAL PUBLICATION. Reports of custom thesauri, building customized completed research or a major significant databases, and organizing and publishing phase of research that present the results of research results. NASA programs and include extensive data or theoretical analysis. Includes For more information about the NASA STI compilations of significant scientific and program, see the following: technical data and information deemed to be of continuing reference value. NASA • Access the NASA STI program home page counterpart of peer-reviewed formal at http://www.sti.nasa.gov professional papers but has less stringent limitations on manuscript length and extent • E-mail your question via to of graphic presentations. [email protected]

• TECHNICAL MEMORANDUM. • Phone the NASA STI Help Desk at Scientific and technical findings that are (757) 864-9658 preliminary or of specialized interest, e.g., quick release reports, working papers, and • Write to: bibliographies that contain minimal NASA STI Information Desk annotation. Does not contain extensive Mail Stop 148 analysis. NASA Langley Research Center Hampton, VA 23681-2199 • CONTRACTOR REPORT. Scientific and technical findings by NASA-sponsored contractors and grantees.

NASA/TM—2017–219539

Autonomous, Context-Sensitive, Task Management Systems and Decision Support Tools II: Contextual Constraints and Information Sources

Barbara K. Burian NASA Ames Research Center

Janeen A. Kochan Aviation Research, Training, and Services

Kathleen L. Mosier San Francisco State University

Ute Fischer Georgia Institute of Technology

National Aeronautics and Space Administration

Ames Research Center Moffett Field, California

September 2017

iii

Trade name and trademareks are used in this report for identification only. Their usage does not constitute an official endoresement, either expressed or implied, by the National Aeronautics and Space Administration.

Available from:

NASA STI Program STI Support Services Mail Stop 148 NASA Langley Research Center Hampton, VA 23681-2199

This report is also available in electronic form at http://www.sti.nasa.gov or http://ntrs.nasa.gov/

Table of Contents

Acronyms and Definitions ...... vi List of Figures and Tables ...... vii 1. Introduction ...... 1 2. Contextual Constraints and Conditions ...... 2 2.1 Flight Operations Constraints and Conditions ...... 8 2.1.1 Time ...... 8 2.1.2 Risk ...... 8 2.1.3 Pilot/Operator Workload and Psychophysiological State ...... 9 2.1.4 Aircraft System and Component Status ...... 9 2.1.5 Phase of Flight ...... 9 2.1.6 Regulations, Procedures, Company Procedures and Policies ...... 9 2.1.7 Flight Parameters ...... 10 2.1.8 Equipage and Maintenance Status ...... 10 2.1.9 Environmental and External Conditions ...... 10 2.1.10 Critical Events ...... 10 2.1.11 Aircraft Habitability ...... 11 2.2 Constraint Dimensions ...... 11 2.2.1 Hard vs. Soft Constraints ...... 11 2.2.2 Fixed vs. Variable Constraints ...... 11 2.2.3 Known vs. Unknown Constraints ...... 12 2.3 Constraint Inter-relatedness ...... 12 2.3.1 Constraint Inter-relation Use Case: Flight Path Management ...... 12 3. Data, Information, and sources ...... 15 3.1 iAtom Identification ...... 15 3.2 iAtom Categorization and Description ...... 16 3.3 iAtom Relationship to Contextual Constraints ...... 20 4. Conclusion ...... 21 5. References ...... 22 Appendex A. Information Sources and iAtoms ...... 25 Apendix B. iAtoms Reference Documents ...... 26

Acronyms and Definitions

2D ...... 2-dimensional 3D ...... 3-dimensional A4A ...... Airlines for America AGL ...... above ground level AOA ...... angle of attack AP ...... autopilot AQP ...... Advanced Qualification Program ATA ...... Air Transport Association ATC ...... air traffic control CAS ...... calibrated air speed CFR ...... Code of Federal Regulations CIAIAC ...... Comision de Investigacion de Accidentes e Incidentes de Aviacion Civil DME ...... Distance Measuring Equipment E&E ...... electrical and environmental bay ECAM ...... Electronic Centralized Aircraft Monitoring System EICAS ...... Engine Indicating and Crew-Alerting System FAA ...... Federal Aviation Administration FL ...... flight level FMS ...... flight management system FPM ...... flight path management FPMSC ...... Flight Path Management Steering Committee ft ...... feet GPS ...... global positioning satellite system IA ...... inspection authorization ILS ...... instrument landing system MEL ...... minimum equipment list MMEL ...... Master Minimum Equipment List MSL ...... mean sea level NASA ...... National Aviation and Space Administration NAV ...... naviation Navaids ...... navigation aids NTSB ...... National Transportation Safety Board

O2 ...... oxygen PBN ...... performaced-based navigation QRH ...... quick reference handbook RAIM ...... Receiver Autonomous Integrity Monitoring RNAV ...... Area Navigation RPM ...... revolutions per minute RTCA ...... Radio Technical Commission for Aeronautics VOR ...... VHF (very high frequency) Omni-directional Radio- range

Figures and Tables

Figure 1. FPM primary parameters and first layer constraints ...... 13 Figure 2. FPM primary parameters and first and second layer constraints ...... 14 Figure 3. Some inter-relations between second layer constraints and amoung second and first layer constraints ...... 14

Table 1. Flight Operations Contraints and Conditons ...... 4 Table 2. iAtom Functional Groups ...... 17 Table 3. iAtom Organization, Definition, and Characteristics ...... 19 Table 4. iAtoms Associated with Geographic Location ...... 21

vii

Autonomous, Context-Sensitive, Task Management Systems and Decision Support Tools II: Contextual Constraints and Information Sources

Barbara K. Burian1, Janeen A. Kochan2, Kathleen L. Mosier3, and Ute Fischer4

1. Introduction Advanced automation and autonomous systems have been increasingly used to reduce pilot workload, help them better manage their tasks, and support their decision making during both normal and non-normal operations (Banks & Lizza, 1991; Champigneux, 1995; Matheus et al., 2005). Unfortunately, these systems and tools have often in the past led to pilot confusion (Mosier & Skitka, 1996; Sarter, Woods, & Billings, 1997) or have been narrowly developed and have failed to fully support and address the pilots’ needs (Banks & Lizza, 1991; Miller & Hannen, 1999). However, recent advances in artificial intelligence, machine learning, data mining and extraction, and especially in sensor technology have resulted in the availability of a vast amount of digital data and information and the development of advanced automated reasoners. This creates the opportunity for “the development of a robust dynamic task manager and decision support tool that is context sensitive and integrates information from a wide array of on-board and off aircraft sources—a tool that monitors systems and the overall flight situation, anticipates information needs, prioritizes tasks appropriately, keeps pilots well informed, and is nimble and able to adapt to changing circumstances” (Mosier, Fischer, Burian, & Kochan, 2017, p. 6).

This is the second of two companion reports exploring issues associated with autonomous, context- sensitive, task management and decision support tools. In the first report (Mosier et al., 2017), we explored fundamental issues associated with the development of an integrated, dynamic, flight information and automation management system. We discussed human factors issues pertaining to information automation and reviewed the current state of the art of pilot information management and decision support tools. We also explored how effective human-human team behavior and expectations could be extended to teams involving humans and automation or autonomous systems.

In this report, we extend this work to focus on two critical aspects of integrated, dynamic, flight information and automation management systems: 1) the constraints and conditions that drive the dynamic prioritization and presentation of data and information to the pilots, and 2) specific data and information to be accessed, monitored, integrated, and displayed in such a system. Although most of the topics discussed in the companion report (Mosier et al., 2017) have wide applicability to many work domains where information automation and autonomous systems are used, this document

1 NASA Ames Research Center. 2 Aviation Research, Training, and Services. 3 San Fracisco State University. 4 Georgia Institute of Technology.

focuses on issues specific to the aviation domain and particularly to the flight deck. Even so, the topics reviewed in this report (e.g., relevant constraints and conditions) can be modified for extension and application to other socio-technical work domains.

As in the companion report (Mosier et al., 2017), in this document we will use the terms “automation,” “autonomous systems,” and “agents” interchangeably, even though they are technically distinct from one another. Additionally, we refer to human actors only as “humans” and use the term “agent” only to refer to highly sophisticated automation and autonomous systems. Additionally, for simplicity, “dynamic, integrated information, task management and decision support tool” will often be referred to as “the system” or “the tool” throughout this document.

2. Contextual Constraints and Conditions As has been described elsewhere (Burian, Mosier, Fischer, & Kochan, in press; Burian & Martin, 2011) and in the companion report to this document (Mosier et al., 2017), pilots currently must access, interpret, and integrate a large amount of data and information from a wide variety of sources to perform even some of the most basic flight tasks. Some of this data and information change dynamically, moment by moment, such as flight instrument and navigation displays, flight deck cues and alerts (Berman et al. 2017), weather radar and environmental conditions, surrounding aircraft traffic and terrain, and clearances and information received from air traffic control (ATC). Other information is static but can be changed as it is edited or updated (though not in real time) such as navigation charts and procedures and checklist items. Yet other information, such as that included in aircraft performance tables and systems manuals, never changes (absent an emergency or non- normal event) once an aircraft has been developed and certified. Some of this data and information, whether dynamic or static, exist on-board or come from the aircraft itself while other data and information come from off-aircraft sources (Bailey, Prinzel, Kramer, & Young, 2011).

A significant task then, for developers of decision support systems, is to determine what data and information are relevant at a particular time (Abbott, McKenney, & Railsback, 2013) and to “…integrate, summarize, distribute, format, abstract, prioritize, categorize, calculate, process, and display [that] information in a variety of ways to support flight crew tasks” (Letsu-Dake et al., 2015, p. 3D1). Moreover, the system should neither overwhelm the pilots (Abbott et al., 2013; Letsu-Dake et al., 2015) nor compromise their situation awareness (Durso & Sethumadhavean, 2008; Endsley, 2000). In other words, information supporting pilot task management should be automatically made available when it is needed, be trustworthy and clearly interpretable, and non-relevant information should be withheld or suppressed (Bailey et al., 2011; Mosier et al., 2017).

In our companion report (Mosier et al., 2017), we identified a number of fundamental decisions that must be made by the developers of these systems prior to the initiation of development. These decisions, in part, pertain to the answers to the following questions: • What pilot cognitive processes (e.g., attention, information integration, analysis, option generation) should be supported by the system? • What is the appropriate level of support that the system should provide? In other words, how directive should the guidance be and should the system be able to autonomously complete certain actions, especially when pilots do not complete a critical task within a specified period of time? • How would different types of data and information best be presented?

• Which types of information should be constantly displayed (if any), which should be automatically displayed when relevant/needed, and which (if any) should never or rarely be displayed automatically but available for access whenever it is desired by the pilot? • Who or what should control the flow of information? • What factors should be taken into account at any given moment to determine information relevance? • How should relevant information be prioritized and should prioritization schemes be able to change moment-by-moment? If so, what factors should drive this re-prioritization and how should information be displayed during re-prioritization so that pilots do not become confused or “get lost”?

The answers to some of these questions, particularly those in the last two bullets, are partially determined by the constraints and conditions that exist moment-by-moment throughout the flight, the relative values placed on these constraints by the developers, and their association with tactical and strategic pilot task management and decision making requirements (Burian & Martin, 2011; Burian et al., in press). A constraint, by definition, imposes a limitation on or defines what tasks can or should be accomplished at a particular time—and, by extension, what information is needed. Constraints are a function of the many dynamic contextual conditions and situations that exist with the passage of time. Constraint values (i.e., statuses) are used in conjunction with if-then algorithms, fuzzy and Bayesian logic, machine learning and artificial intelligence to develop automated reasoners, which drive the system behavior regarding what information is needed, when it is needed, and how it should be prioritized (Abbott, Jones, Consiglio, Williams, & Adams, 2004; Burian et al., in press; Estes et al., 2016). Therefore, one of the early steps in the process of developing such a tool to support flight operations is to identify the various constraints and conditions that might exist over the course of a flight; we have identified 11 such inter-related categories of constraints and conditions (see Table 1; Burian et al., in press; Mosier et al., 2017). Most of these constraints exist concurrently and it is the myriad of ways in which their dynamic values may be combined and interact that makes these systems such powerful tools but also which makes their development so challenging. We explore each of these 11 constraints in more depth below.

Table 1. Flight Operations Constraints and Conditions Constraint or As a Function of Types System Functioning Conditions Time • Amount of time until X (countdown timer) • Monitors the passage of time relative to flight schedule, phases of • Amount of time since X (count up timer) flight, and progress along route of flight • Time window during which X can occur or • Monitors windows during which X condition can occur or X task be accomplished can be, or is most appropriately, accomplished • Monitors length of time pilot/operator takes to accomplish actions/tasks presented on system displays • Displays information that is tailored to the situation & tasks to be accomplished • Prompts pilot/operator to take action Risk • Safety risk • Monitors flight parameters and compares against pre-determined • Economic risk limits for operation of aircraft and company policies and • Productivity procedures • Displays information that is tailored to the situation providing relative “closeness” to exceeding established criteria (once a particular threshold has been met) • Prompts pilot/operator with which action(s) to achieve flight behavior/ parameters that are within desired “risk” envelope Pilot/operator • Lengths (L) and • Procedure and task length and complexity • Monitors pilot/operator workload and sensed psychophysiological characteristics, complexity (C) • Alertness, duty time, circadian rhythm, states workload and of N tasks to fatigue, work schedule (which leg in the trip, • Displays information and integrates to support decision making psycho- accomplish in how many trips back to back, hours of rest) • Prioritizes information and suggested actions based upon task physiological time (T) • Pilot physical health importance state • Measured • Pilot experience on aircraft type and with • Suggests which tasks to shed, if necessary, during extremely time physiological equipment/automation critical situations parameters • Pilot training status • Takes over pre-determined tasks Aircraft • Gear, flaps, spoilers, elevators, ailerons, brakes • Monitors and indicates system and component status, system and • Electrical system and E&E bays, hydraulics, malfunctions, and failures component pressurization, air systems, engines, oil, fuel • Calculates impact on: planned route of flight, expected aircraft status • Doors, windows, access panels, lavatories, performance, flight parameters, aircraft integrity and galleys, cargo/cargo bay airworthiness, and informs pilot/operator; suggests tasks/actions • Computers, displays, inflight entertainment as needed system, sensors, alerting system, communication systems, radios, navigation equipment (continued on next page)

Table 1. Flight Operations Constraints and Conditions continued Constraint or As a Function of Types System Functioning Conditions Aircraft • Gear, flaps, spoilers, elevators, ailerons, • Monitors and indicates system and component status, system and brakes malfunctions, and failures component • Electrical system and E&E bays, hydraulics, • Calculates impact on: planned route of flight, expected aircraft status pressurization, air systems, engines, oil, fuel performance, flight parameters, aircraft integrity and • Doors, windows, access panels, lavatories, airworthiness, and informs pilot/operator; suggests tasks/actions galleys, cargo/cargo bay as needed • Computers, displays, inflight entertainment system, sensors, alerting system, communication systems, radios, navigation equipment Phase of flight • Time • Amount of time until X • Monitors passage from one phase of flight to another and • Location • Amount of time since X compares against flight schedule and clearance; suggests • Time window during which X can occur or changes/actions, if needed be accomplished • Uses phase of flight and time to guide presentation of information • Procedures (relative to location and phase of and actions from checklists and procedures flight) • Tailors information to the needs of the flight phase tasks Regulations, • Adherence to industry and company policies • Monitors aircraft behavior and flight progress and compares procdures, and procedures such as: against industry, aircraft operating, and company policies and company ‒ stable approach criteria procedures procedures ‒ PBN criteria • Informs pilot/operator if the system is unable to operate the and policies ‒ RNAV departure and approach criteria aircraft according to the criteria and why ‒ precision approach criteria, etc. • When deviating from policies/ procedures during manual flight, informs pilot/operator of deviation and what action(s) to take to get back into compliance with policies and procedures • Informs pilot of possible consequences for specific deviation from policies/procedures when such a deviation exists (during both automated and manual flight)

(continued on next page)

Table 1. Flight Operations Constraints and Conditions continued Constraint or As a Function of Types System Functioning Conditions Flight Time and phase of • Altitude • Monitors flight parameters and compares against aircraft parameters flight and • Heading capabilities and limitations, flight schedule, and clearance; adherence to • Indicated airspeed informs pilot/operator and, if needed, suggests changes/actions policies/ • Ground speed • Automatically changes datacomm/radio and navigation procedures • Vertical speed frequencies, altimeter settings, and, if necessary, transponder • Distance from A (geographic location, codes and informs pilot of changes airports, Navaids, terrain features, obstacles) • Warns pilot when changes to flight parameters are imminent • Progress on route based upon route of flight or phase of flight changes • Datacomm/radio and navigation frequencies • Altimeter settings • Transponder codes Equipage and • Aircraft equipment list (type of engines, • Evaluates aircraft equipage (including MEL status) and equipment maintenance autopilot, avionics suite, etc.) capabilities against go/no go criteria/MMEL, procedural status • FMS database version requirements, and airworthiness directives and informs pilot of • Navigation database version operational issues • Maintenance history and logs • In reference to version/update status of databases, MEL’d items, • MEL’ed items & maintenance history/logs, the system selects relevant non- normal checklist items for presentation (when needed) and informs pilot of any related operational issues (continued on next page)

Table 1. Flight Operations Constraints and Conditions continued Constraint or As a Function of Types System Functioning Conditions Environmental • Winds and turbulence • Monitors environmental and external conditions and informs and external • Weather and precipitation pilot/operator as appropriate; suggests actions as necessary conditions • Icing conditions and aircraft icing • Compensates for effects of environmental and external conditions in • Air pressure determinations of/suggestions for revised flight path, approach to • Temperature and dew points aircraft operation and procedures, alternate landing sites, etc. • Airport location and status and runway conditions • Navaids (location and functioning status) • GPS/RAIM integrity • Traffic • Terrain • Other geographic features (bodies of water, cities) • Obstacles (towers, buildings, birds, animals on the runway, etc.) • Availability of ground services (ATC, dispatch, ground operators, maintenance facilities, emergency personnel, hospitals, etc.) Critical events Time and pilot • Emergency situations • Informs pilot/operator that a situation outside of the realm of normal workload/psycho- • Abnormal situations operations exists and what it is, with an initial assessment of level of physiological state • Delayed/compressed actions or maneuvers criticality (emergency, abnormal, off-nominal) and one or more • Collision/near collision • If appropriate, displays system schematics with indication of non- of the following: normal state or conditions and effects of response actions (in real time) Aircraft system/ • Integrates appropriate information and actions from relevant checklists component status, with actions required for overall situation management and presents for flight parameters, accomplishment/ consideration as appropriate or when relevant phase of flight, • Takes over some pre-determined tasks environmental/ • Informs pilot/operator of aircraft limitations and capabilities relevant to external remaining expected flight tasks conditions Aircraft Critical events and • Temperature • Monitors status of aircraft habitability and pushes information about habitability time • Atmospheric pressure maintenance of crew and passenger health as necessary

• Smoke • Suggests actions (e.g., donning O2 masks and goggles, dropping • Toxic fumes passenger O2 masks) as needed

2.1 Flight Operations Constraints and Conditions 2.1.1 Time Time is the most inescapable and limiting constraint that exists in flight operations as well as operations in many other high consequence work domains (e.g., medicine, nuclear reactors, etc.). Most actions must occur at a particular time or during a “window of time” relative to other conditions or actions (Burian, Christopher, Fry, Pruchnicki, & Silverman, 2013). For example, in aviation, at least one engine must be started before an aircraft can begin to taxi. Similarly, a flight must receive an ATC clearance before it can (legally) takeoff and flight crews must lower the landing gear prior to the aircraft touching down on the runway (if they wish to avoid damaging the aircraft). Thus, pilot actions are dependent upon how much time has passed since some things have occurred, how much time remains before something else occurs, and/or the window of time during which a task can or must be accomplished.

Therefore, a dynamic, integrated information, task management and decision support tool must monitor the passage of time relative to a number of factors including the flight schedule, phases of flight, and progress along the route of flight and track the accomplishment of tasks in comparison to them. By this statement and through the rest of our discussion below it will become obvious that the constraint of time is highly related to almost all the other constraints in our list.

2.1.2 Risk In flight operations, five types of risk have been identified: physical (i.e., safety), professional, productivity, economic, and social (Orasanu, Fisher, & Davison, 2004). In terms of constraints, we are concerned with safety risk, economic risk, and productivity risk. Some safety risks may be associated with relatively hard parameters such as those circumscribed by flight control laws (e.g., normal law5) or stabilized approach criteria. Other parameters are more “aspirational” and depend upon appropriate pilot judgment and behavior relative to air carrier or company policies. Economic risk during flight operations often depends upon human judgment and involves flight-related decisions that might increase costs, such as requesting a reroute that adds to fuel costs, or cause harm to the company’s image and reduce repeat business, such as long delays or mishandled passenger disruptions. Some productivity risks overlap with economic risks, such as a reroute that not only adds to fuel costs but also increases the length of a flight. Other productivity risks pertain to actions or events that increase workload or are less efficient.

During the development of a dynamic, integrated information, task management and decision support tool, as much as possible, specific behavior and actions associated with safety, economic, and productivity risks—derived from procedures, policies, and pilot practices (Degani & Wiener, 1994)—will have to be identified and then quantitatively defined. The system will then use those definitions or “hard targets” when determining what information is needed and when relative to a given risk. For example, the system might automatically prompt the pilot for actions and decisions that fall within a pre-defined acceptable risk envelope or inform pilots when they are getting close to or have crossed over into unacceptable risk territory.

5 Normal law is a control law that exists on Airbus fly-by-wire aircraft whereby flight computers provide flight envelope protection against unsafe pitch attitudes, bank angles, and overspeeding, among other things.

2.1.3 Pilot/Operator Workload and Psychophysiological State Although currently somewhat limited, in the future, sensor or behavioral data associated with pilot workload and psychophysiological state may be available. These data could be used for determining information needs and the amount and kind of task support that might be necessary. Workload is the more easily defined variable, even today, as a function of the length, complexity and number of tasks that must be performed within a given period of time. Pilot psychophysiological state includes such things as alertness and physical and emotional health—aspects of pilot functioning that are currently difficult to automatically sense on the flight deck. Other factors associated with expected pilot functioning related to training status, level of experience, and work schedule are more easily captured for system use.

2.1.4 Aircraft System and Component Status Today, the status of aircraft systems and components is heavily sensed in modern aircraft and sensor data drive the presentation of alerts, advisory messages, integrated electronic checklists (if equipped), and electronic system schematic information, as well as general system and component status, such as engine pressure ratio, oil temperature, amount of remaining fuel, and degree of gear and flap extension. These data could be easily incorporated into a dynamic, integrated information, task management and decision support tool.

2.1.5 Phase of Flight Phase of flight is a function of time and aircraft location (both vertically and laterally) along a route, including taxiing and ground operations at airports. The different phases are relatively well defined in aviation (e.g., taxi out, cruise, approach) and can provide the framework or structure for a dynamic, information and task management system since specific tasks and procedures are associated with each phase.

2.1.6 Regulations, Procedures, Company Procedures and Policies A variety of rules and expectations govern or guide what actions pilots carry out and when and how they are enacted. Some have the force of law, such as federal aviation regulations, or carry with them the expectation of strict adherence, such as altitudes and headings to fly during instrument departures and arrivals. Company dictated procedures and policies, such as stabilized approach criteria, also have the expectation of adherence though non-adherence generally resulting in fewer or lesser adverse costs.

Regardless of their source, regulations, all types of procedures, and company policies can be incorporated into a dynamic, integrated information, task management and decision support tool to guide pilot behavior and actions. One challenge for system developers, however, is to ensure that pilots understand which guidance based on these sources is expected to be more rigidly followed, except under exceptional circumstances such as an inflight emergency, and which may be more flexibly applied across many situations. In either case, it is important for the system to keep the pilot abreast of possible consequences for deviating from specific regulations, procedures, and policies relative to contextual factors that exist at the time.

2.1.7 Flight Parameters “Flight parameters” refers to current aircraft performance (e.g., altitude, ground speed), as well as location (e.g., distance from a waypoint, progress on route), and various instrument settings (e.g., active radio frequencies, transponder code). Flight parameters are a function of the intersection of three other constraints: time, phase of flight, and adherence to regulations, policies, and procedures. As with aircraft system and component status, flight parameters are usually well sensed and/or indicated to pilots in current modern aircraft and could be easily incorporated into a dynamic, information and task management system.

2.1.8 Equipage and Maintenance Status In contrast to flight parameters and aircraft system and component status, modern aircraft generally lack well sensed or clearly indicated information related to specific equipment aboard an aircraft (such as which version of avionics software is installed) or its maintenance status (e.g., how long until a “C” maintenance check is due, compliance with airworthiness directives). Some of this information has little relevance for pilots during their day-to-day operations so may appropriately be withheld or not made easily accessible. Other information has great relevance (e.g., minimum equipment list [MEL] status of equipment) but may only be available through reference to a handwritten logbook.

Similar information also has significant implications for flight operations. For example, pilots may need to know the model and brand of engines on the aircraft to determine the correct actions to take if one of them fails. Likewise, knowledge of an aircraft’s equipage and its related capabilities is required to receive different levels of service under NextGen. Information such as this, as well as that currently in handwritten logs, could be integrated into and referenced by a dynamic, information and task management system so that only pertinent actions are presented and pilots are informed about NextGen service level availability.

2.1.9 Environmental and External Conditions A wide range of environmental and conditions external to an aircraft has importance for flight operations and pilot decisions. These include winds, precipitation, convective activity, icing conditions, air pressure, aircraft traffic, terrain, obstacles, global positioning system (GPS) signal integrity, and the location and functionality of navigation aids (Navaids), among others. Known fixed features, such as mountains and towers, and sensed transitory features, such as aircraft traffic and weather conditions, are important additions to dynamic, information and task management system and decision support tools; the presence, status, or absence of these conditions over the course of a flight could have significant operational ramifications.

2.1.10 Critical Events It is in helping pilots respond to and manage critical events that integrated, dynamic, information and task management system and decision support tools reveal some of their greatest value. Some critical events are transitory in nature (such as a near miss or a missed approach) or might even be relatively benign and easily dealt with (Burian, Barshi, & Dismukes, 2005). Other critical events can be quite perilous, however, and a well-designed information and task management system can help pilots prioritize their actions, focusing on that which is most essential moment by moment. At times these actions will be in direct response to the event and focus on fixing, “safeing,” or taking a malfunctioning system off-line. At other times these actions will focus on “flying the aircraft” tasks,

such as communicating with ATC, coordinating with company dispatch, or preparing for an emergency landing.

Different critical events pose different types of constraints: limiting or restricting some actions, adding others, and integrating them to be responsive to the demands of overall flight operations and the contextual factors that exist. Thus, critical events serve to constrain or shape information, task management, and decision support needs as a function of the amount of time that is available (often an unknown quantity), the pilots’ taskload related to the event itself as well as to normal flying tasks, the pilots’ psychophysiological state, and one or more of the following: the phase of flight during which the event occurs, the aircraft system or component involved (if any), flight parameter status, and environmental or external conditions encountered.

2.1.11 Aircraft Habitability Aircraft habitability involves the degree to which the internal aircraft environment is fit to support human life. It involves the temperature and pressure within the aircraft, amount of oxygen, and presence of smoke or toxic fumes. Changes to aircraft habitability are typically the result of a critical event, such as an in-flight fire or a pressurization leak, and the amount of time available before crew and passengers begin to lose consciousness or perish may be quite limited.

As can be see through this discussion, many times a constraint actually reflects the inter-relationship or combination of several factors. For example, a critical event involves the condition itself, which may or may not involve a degraded status of an aircraft component, but also the phase of flight during which it occurs and how that may affect the degree to which safety might be compromised (safety risk) and the amount of time available to respond to it.

2.2 Constraint Dimensions These 11 constraints differ relative to three different dichotomous dimensions: • hard versus soft • fixed versus variable • known versus unknown

2.2.1 Hard vs. Soft Constraints A hard constraint relates to something that must occur, such as extending the flaps in preparation for takeoff—not doing so could result in the aircraft failing to gain lift and crashing (NTSB, 1988; CIAIAC, 2008). In contrast, soft constraints pertain to actions or conditions that are desirable though are not necessarily critical and often are related to company policies, such as making an “in-range” call when cleared for an approach.

2.2.2 Fixed vs. Variable Constraints A constraint is fixed if it delineates only one point in time when something can occur, such as responding to a microburst with windshear on final approach. Variable constraints, however, define actions that can occur within a window of time rather than at only one point (Burian et al., 2013), though the window could be relatively short as well as relatively long. Setting the takeoff flaps, for example, can actually be accomplished anytime between when ground crew are clear of the aircraft prior to push-back at the gate all the way up until the aircraft rotates on the takeoff roll (though it should be noted that it is exceedingly bad practice and dangerous to extend the flaps while the

aircraft is actually on the takeoff roll). So, where during this window should a dynamic, integrated, context-sensitive information, task management and decision support system instruct or remind the crew to set the flaps? The existence of other related constraints may have some bearing on when this occurs; for example, sensors identifying slush on the taxiway might cause the system to delay the instruction to set takeoff flaps until the aircraft has arrived at the runway to eliminate the possibility of slush being thrown up and freezing on the extended flaps during taxi.

2.2.3 Known vs. Unknown Constraints In the third dimension, known constraints are those that are understood or can be predicted; all aspects associated with the other two dimensions for a particular constraint must be understood in order for it to qualify as being “known.” In other words, a known constraint is one that is understood to exist and is known to be hard or soft and fixed or variable. Unknown constraints are understood to exist but the limits related to that constraint are unknown. A good illustration of an unknown constraint is Time during an inflight fire event—the amount of time available to complete a safe landing and evacuation is unknown while the event is unfolding.

Therefore, developers of a dynamic, decision support tool and information and task management system will have to contend with the various constraints and their qualities relative to the three dimensions, how they are inter-related to other constraints and conditions, and determine which has priority in any given moment.

2.3 Constraint Inter-relatedness In Table 1 and our earlier review of each of the 11 flight operations constraints and conditions, we discussed the ways in which some constraints were inter-related with each other; indeed, few constraints ever exist in isolation. Although instructive in the abstract, it can be helpful to explore even a simplified use case to further our understanding of just how vast the web of constraint inter- relatedness can be and how challenging the task of prioritization among inter-related constraints and conditions can be. For this purpose, let us consider the constraints and conditions associated with the most basic flight task: “flight path management—the planning, execution, and assurance of the guidance and control of aircraft trajectory and energy, in flight or on the ground” (Delta Air Lines Flight Path Management Steering Committee [FPMSC], 2015, p.1). In simplified terms, flight path management in the air and on the ground is determining where you want the aircraft to go and how it should get there, putting it there, and then ensuring that it stays there (Delta Air Lines FPMSC, 2015).

2.3.1 Constraint Inter-relation Use Case: Flight Path Management To explore flight path management (FPM) constraint inter-relations, even in a simplified manner, we first must consider the constraints most central to FPM—in this case, they are four flight parameters: aircraft speed, heading or track, altitude, and geographic location. The next step is to add first layer constraints taken from the 11 previously identified and consider how they are connected to each of the four primary parameters, as shown in Figure 1. (For ease in viewing, not all the interconnections are depicted.)

Figure 1. FPM primary parameters and first layer constraints.

First layer constraints in Figure 1 are those we believe to be most closely associated or inter-related with the first four—the four flight parameters—and they are external environmental and external conditions (wind, terrain/obstacles, traffic), aircraft system and component status (thrust, weight, configuration, angle of attack [AOA]), and aircraft equipage and maintenance status (equipage), and includes such things as the aircraft’s flight management (FMS) and autoflight systems.

Second layer constraints—those that are highly associated but less so than first layer constraints— can then be considered and are included in Figure 2. For the purposes of this use case exploration, all of the second layer constraints we have selected pertain to limitations and performance expectations codified in federal aviation regulation and practical test and airman certification standards, procedures (both published and company), and company policies. In Figure 3, we have added a few additional lines (in black) to illustrate some of the inter-relationships that exist between and among first and second layer constraints, not just with the four primary flight parameters. These additional inter-relations must also be considered.

Figure 2. FPM primary parameters and first and second layer constraints.

Figure 3. Some inter-relations between second layer constraints and among second and first layer constraints.

Readers should note that a large number of constraints have not yet been added to the figure for consideration. For example, phase of flight certainly has a connection to many of the FPM constraints identified (e.g., aircraft configuration, traffic, terrain regulations, procedures), as do the levels and types of risks that exist, the amount of time available, pilot workload and physiological state (e.g., fatigue), and the presence of any type of non-normal condition or emergency (i.e., a critical event). Because FPM is one of the most basic and critical of all flight operations tasks, we believe that developers of dynamic, information and flight management and decision support tools would do well to start with this task when considering constraints and conditions and how they are inter-related and how information and tasks associated with them should be prioritized in different contexts.

The second major topic covered in this report pertains to the specific data and information to be accessed, monitored, integrated, and displayed in an integrated, dynamic, information and task management and decision support tool. We turn to this topic now.

3. Data, Information, and Sources One of the authors (Kochan) took on the herculean task of identifying much of the data and information—and where it comes from (i.e., sources)—that could be integrated and automatically presented or made available on request by a dynamic, information and task management and decision support tool (see Appendix A). The task was to not only identify but also to categorize, define, and otherwise characterize the full range of data and information for inclusion in the system. The only rule we developed prior to this undertaking was that the data and information identified should be at the smallest discrete but coherent and interpretable unit possible. For example, an altitude of “FL 210” is one discrete, but smallest interpretable unit: “FL” alone is not specific enough (it denotes any flight level [FL] at or above 18,000 ft. mean sea level [MSL]), and “210” alone could refer to a heading, a flight number, or any number of other things (e.g., the first officer’s weight, the number of passengers on board, etc.); similarly “F” or “2” or “0” by themselves are meaningless. Thus, relative to presenting altitude information in terms of feet (as opposed to hectopascals or meters), numbers are required representing feet and some other indicator (e.g., ft., FL) to distinguish feet measurement above ground level (AGL) or MSL. We call these small, discrete data and information units “information atoms” or iAtoms. As described later, time and other resource limitations precluded our ability to identify all the iAtoms in some data/information categories.

3.1 iAtom Identification A number of documents were consulted to generate the list of iAtoms. The first was the Air Transport Association (ATA) Specification 100 (ATA, 1999) and its revision, the iSpec 2200 Information Standards for Aviation Maintenance (Airlines 4 America, 2016) published by Airlines for America (A4A), the successor to the ATA. Online master MELs for several transport category aircraft were also reviewed (see http://fsims.faa.gov/PICResults.aspx?mode=Publication&doctype=MMEL). These sources were particularly helpful in identifying iAtoms associated with aircraft structures and systems (things of greatest concern to maintenance personnel) but did not include a wide array of other data and information needed by pilots such as alerts, checklist items, or environmental/meteorological conditions.

Current flight regulations and advisory guidance from the Radio Technical Commission for Aeronautics (RTCA), Federal Aviation Administration (FAA) Advisory Circulars, and selected Aircraft Flight Manuals (see Appendix B) were consulted to identify other types of data and information, particularly those that are incorporated into current flight deck displays. With an eye to the future, NextGen Portfolios and Operational Improvements (Bolton, Bristol, & Hickey, 2014) were also reviewed and iAtoms necessary for NextGen technologies and operations were identified. iAtoms from these NextGen documents relate in large part to meteorological/environmental conditions and to communication, navigation and surveillance.

Normal checklists, quick reference handbooks (QRHs), and systems and training manuals for a number of aircraft types from different manufacturers were also reviewed. A literature review was also conducted, which yielded a few previously identified sets of flight deck data and information (Bailey et al., 2011; FAA, 2013; FAA Flight Standards Service, 2008; Schvaneveldt, Beringer, Lamonica, Tucker, & Nance, 2000). Finally, Dr. Kochan used her personal experience as a transport category aircraft pilot, airframe and powerplant mechanic (A&P) with inspection authorization (IA), and FAA designated pilot examiner, to add to the list of identified iAtoms.

3.2 iAtom Categorization and Description Identified iAtoms were sorted and organized into the 10 functional groups listed in Table 2.

Table 2. iAtom Functional Groups Functional Group Description Examples of iAtoms Aircraft structure and Basic aircraft structure, furnishings, and Engine ignition control position systems systems data and information

Aircraft flight deck Data and information found on the flight True airspeed deck used in operation of aircraft and control of flight path

Communication, Data and information associated with ILS glidepath indication navigation, and communication, navigation, and surveillance surveillance

External environment External factors such as airport design, Winds aloft (at a particular altitude) terrain, obstacles, weather, and other meteorological phenomena

Alerts, warnings, and Compilation of most common alerts, Windshear alert (aural) errors warnings, and errors (such as missing information)

Documents, Documents and reference material such Engine start procedure documentation, and as aircraft manuals, performance tables, procedures handbooks, and checklists

Personnel: flight and Physiological, cognitive, psychological, Time elapsed since last rest period cabin crew, social interactions, and workload of on- maintenance and board and on-ground crew members and ground personnel personnel

Aircraft cabin and Data and information associated with Galley and lavatory water quantity cargo compartment cabin equipment, cargo, and luggage (checked and carry on)

Maintenance Data and information pertaining to the Status of compliance with maintenance of the aircraft or associated airworthiness directive XX equipment

Ground and servicing Items associated with ground operations Number of cans of orange juice on and servicing of the aircraft board

Because of time and resource limitations, we focused on identifying iAtoms for the following functional groups: Aircraft Structure and Systems; Aircraft Flight Deck; Communication, Navigation, and Surveillance; External Environment; Alerts, Warnings, and Errors; and Documents, Documentation, and Procedures. Additionally, and again because of resource limitations, only major types of documents and documentation that exist (e.g., non-normal checklists, minimum equipment list) were identified and included in the Documents, Documentation and Procedures functional group, rather than all the iAtoms that exist within those documents and procedures. A color-coded tab was created for each of the 10 functional groups in a Microsoft Excel™ workbook (see Appendix A).

In the five functional groups in which specific iAtoms were identified (all groups listed above except for the Documents, Documentation, and Procedures group), the iAtoms were organized, defined, and characterized relative to the fields in Table 3.

Table 3. iAtom Organization, Definition, and Characteristics Relevance for System Column Heading Description Example* Development Super category A major overarching category Electrical Facilitates understanding of based on root ATA codes or dynamic, integrated other categorization scheme information system such as checklists or Quick architecture Reference Handbooks Category (ATA code, A category of information Electrical power Facilitates understanding of if available) sources under a Super (24) dynamic, integrated Category; may have an information system associated ATA code or sub- architecture code Sub-category (ATA A system, component, or Generator drive Facilitates understanding of sub-code, if available) procedure specific data or (24-10) dynamic, integrated information source information system architecture iAtom A small, discrete but Generator drive Information/data for display meaningful unit of information indication or use by the system or data for integration, use, and/or presentation iAtom definition/ Description of the iAtom Indication of Facilitates understanding of features mechanical devices iAtom and its use in the that drive the system generators at a desired RPM** Units iAtom unit of measurement (if Not applicable Necessary for use by the applicable) system Static/dynamic Whether the iAtom is static Dynamic Necessary for system to be (not expected to be changing considered a decision in real-time) or dynamic support tool and to provide (changing in real-time) support at appropriate level Current source System, computer, facility, or Electrical system Where iAtoms will come person providing source of from for integration into data or information system Current location of Current location of the iAtom Varies: typically Facilitates understanding of information/data (e.g., specific cockpit display EICAS, ECAM or how iAtoms are currently or document source) similar display used Warnings/cautions/ Any existing warning or Generator drive System developer awareness alerts alertinf information associated with the iAtom Available guidance/ Associated regulatory and 14 CFR 23 Additional sources of regulatioins advisory guidance materials 14 CFR 25 information relative to (14 CFR, RTCA, Advisory electrical systems iAtoms, their use, and their Circulars, etc.) and equipment presentation or display Needed guidance New regulations or guidance Not applicable Gaps in needed guidance needed for the system material or regulations * Example taken from the Aircraft Structures and Systems functional group. ** RPM (revolutions per minute).

During system development, additional considerations of iAtoms will also be necessary, such as: • Presentation style: iAtom “pushed” only when relevant, iAtom information to be made available only when requested by pilot (Cybenko & Brewington, 1999), or iAtom is constantly presented • Modality and location: visual, aural, haptic, localization, display unit on which it appears, location on display unit, etc. • Correspondence: iAtom always or often presented with what other iAtoms and under what conditions • Features: text/alphanumeric, graphic, picture, color, 2D, 3D, volume, aural sound/spoken language, font size, bolded, flashing, etc. • Thresholds: conditions, situations and parameter values that trigger iAtom presentation • Source: new sources for iAtom availability, if different from current sources • Verification criteria (dynamic iAtoms only): criteria which must be met to ensure that dynamic iAtom data and information from sources is up-to-date and accurate • Priority weighting: weighting score given to iAtom to be used for making prioritization decisions regarding presentation of iAtoms relative to others

3.3 iAtom Relationship to Contextual Constraints Earlier we considered the FPM use case to illustrate inter-relationship among different types of constraints. We began by identifying the primary constraints most closely related to FPM: aircraft speed, heading or track, altitude, and geographic location. Recall that constraints are actually thresholds or limits that exist. For example, imagine that we are in an aircraft that is flying at flight level 360 (36,000 feet, MSL). The winds that we encounter at that altitude (i.e., winds aloft), both their speed and direction relative to our track, will affect the speed with which our aircraft travels over the ground. If the winds are coming directly against us (a direct head wind) and they are strong, they will constrain or limit our forward speed more than if they are light. If the winds aloft are instead a tailwind for us (i.e., push us from behind), they are not a constraint at all on our forward speed, and actually can help us arrive at our destination earlier.

The value of the winds aloft speed that we encounter (e.g., 45 knots), by itself, is just information: an iAtom. But when considered relative to our aircraft, and our goal to get from one place to another, that wind now possibly becomes a constraint. Just how constraining or limiting depends upon the strength of the wind (the iAtom value of the wind), but also its direction relative to our aircraft and also other factors such as how much thrust our aircraft is producing, our aircraft’s weight, and a variety of other related factors, such as how aerodynamic the design of our aircraft is for that altitude. Thus, one or more iAtoms is associated with each constraint or condition depending on current flight operation goals and associated tasks.

Table 4 shows several iAtoms (taken from Appendix A) that are related to the FPM geographic location parameter constraint; at any point in time one or more might be needed for display by a dynamic, integrated, flight information and automation management system.

Table 4. iAtom Associated with Geographic Location Sub-Category Category (ATA Code, Super Category (ATA Sub-Code, iAtom if available) if available) Surveillance Location Altitude Altitude

Surveillance Location Satellite referenced Latitude/longitude

Surveillance Location Navaid referenced VOR radial/distance

Surveillance Location Radar referenced Map icon (radar)

Surveillance Location Landmark referenced Direction/DME from landmark

Surveillance Location Point-in-space Map icon (4D trajectory) referenced

This is just a small sample of the iAtoms that might be presented relative to just one constraint associated with FPM tasks. Consider just how many other iAtoms that also need to be conveyed at a single point in time associated with all the other pilot tasks and constraints that exist then!

The development and design process for autonomous, integrated context-sensitive, task management and decision support tools is tremendously complex and is most easily accomplished if considered in sections and by constraint type, such as by phase of flight, suggested earlier. Airlines participating in the Advanced Qualification Program (AQP) have already completed detailed task analyses for their flight operations (e.g., Lanzano, Seamster, & Edens, 1997) and aviation researchers have done the same for various studies (e.g., Burian et al., 2013; Hooey, Foyle, & Andre, 1997). After system developers have built on and extended existing task analyses by phase of flight, they should identify iAtoms associated with those tasks for incorporation into the system (see for example, Schvaneveldt, Beringer, & Lamonica, 2001). In parallel, developers must also identify the constraints and conditions associated with and impinging upon the completion of the tasks identified. With tasks, iAtoms, and constraints and conditions identified, developers can then begin the process of designing the system and displays in keeping with the role the system is expected to play and how it is to function and behave (Mosier et al., 2017).

4. Conclusion In the companion report to this document (Mosier et al., 2017), we made the case that autonomous, context-sensitive, task management and decision support tools hold great promise for supporting increasingly complex operations in aviation and other high-risk, dynamic, socio-technical work domains. We described some of the many issues that will need to be addressed during the first stage of developing these systems. These issues center on the role that the system is to have relative to human operators and how it should function to most effectively fulfill that role.

In this report we explored in more depth the contextual constraints that will drive the dynamic behavior of such systems and the types of data and information that will be sensed, collected,

integrated and displayed on the flight deck. These are two critical initial steps in the development of these systems following conception and functional requirement definition. It is likely that by considering the contextual constraints within specific operational contexts and identifying and prioritizing the data and information needed, developers will find themselves engaging in an iterative process whereby constraints, data and information, and system functionality and role will be further defined with increasing specificity.

Thus, these reports serve as the foundation—but only the beginning—in understanding and approaching the development and design of these highly complex but extremely powerful systems.

5. References Abbott, K. A., McKenney, D., & Railsback, P. (2013). Operational use of flight path management systems–Final report of the Performance-based operations Aviation Rulemaking Committee. Commercial Aviation Safety Team Flight Deck Automation Working Group. Abbott, T. S., Jones, K. M., Consiglio, M. C., Williams, D. M., & Adams, C. A. (2004). Small aircraft transportation system, higher volume operations concept: Normal operations. (NASA/TM-2004-213022.) Hampton, VA: NASA Langley Research Center Airlines 4 America. (2016). iSpec 2200: Information standards for aviation maintenance 2016.1. Washington, DC: A4A Publications. Bailey, R. E., Prinzel, L. J., Kramer, L. J., & Young, S. D. (2011). Concept of operations for integrated intelligent flight deck displays and decision support technologies. (NASA/TM-2011- 217181.) Hampton, VA: NASA Langley Research Center. Banks, S. B., & Lizza, C. S. 1991, Pilot’s associate: a cooperative, knowledge-based system application. IEEE Expert, 6, 18 – 29. Berman, B. A., Kochan, J. A., Burian, B. K., Pruchnicki, S., Christopher, B., & Silverman, E. (2017). Alerts and cues on the flight deck: Analysis and application in training. NASA Technical Memorandum. Moffett Field, CA.: NASA Ames Research Center. Bolton, E. L., Bristol, T., & Hickey, J. J. (2014). NextGen priorities joint implementation plan: Executive report to Congress. Federal Aviation Administration. Washington, DC. Burian, B. K., Barshi, I., & Dismukes, K. (2005). The challenge of emergency and abnormal situations. NASA Technical Memorandum, (NASA/TM 2005-213462). Moffett Field, CA: NASA Ames Research Center. Burian, B. K., Christopher, B., Fry, D., Pruchnicki, S., & Silverman, E. (2013). Jet single-pilot simulation study scenario overviews, task analyses, and concurrent task timelines. NASA Technical Memorandum, (NASA/TM-2013-216608). Moffett Field, CA.: NASA Ames Research Center. Burian, B.K., & Martin, L. (2011). Operating documents that change in real-time: Dynamic documents and user performance support. In Guy Boy (Ed.), The handbook of human-machine interaction: A human-centered design approach, pp. 107-130. Surrey, England: Ashgate. Burian, B. K., Mosier, K. L., Fischer, U. M., Kochan, J. A. (in press). New teams on the flight deck: Humans and context-sensitive information automation. In E. Salas & F. Jentsch (Eds.) Human

factors in aviation (3rd ed.). New York: Academic Press. Champigneux, G. (1995). In AGARD Lecture Series 200: Knowledge-based functions in aerospace systems (pp. 5-1 – 5-10). Neuilly-Sur-Seine, France. Comision de Investigacion de Accidentes e Incidentes de Aviacion Civil [CIAIAC]. (2008). Accident involving a McDonnell Douglas DC-9-82 (MD-82) aircraft, registration EC-HFP operated by Spanair, at Madrid-Barajas Airport, on 20 August 2008 (Report A-032/2008). Retrieved from http://www.fomento.es/NR/rdonlyres/EC47A855-B098-409E-B4C8- 9A6DD0D0969F/107087/2008_032_A_ENG.pdf Cybenko, G., & Brewington, B. (1999). The foundations of information push and pull. The mathematics of information coding, extraction and distribution (pp. 9-30). New York: Springer. Degani, A., & Wiener, E. L. (1994). Philosophy, policies, procedures, and practices: The Four "P"s of flight deck operations. In N. Johnston, N. McDonald, and R. Fuller (Eds.), Aviation psychology in practice (pp. 44-67). Hants, England: Avebury Technical. Delta Air Lines Flight Path Management Steering Committee. (2015). Delta Air Lines Flight Path Management Steering Committee Report: Recommendations to Improve Flight Path Management. Internal Technical Report: Delta Air Lines.

Durso, F. T., & Sethumadhavan, A. (2008). Situation awareness: Understanding dynamic environments. Human Factors, 50(3), 442-448. Endsley, M. R. (2000). Theoretical underpinnings of situational awareness: A critical review. In M. R. Endsley & D. J. Garland (Eds.), Situation awareness, analysis, and measurement (pp. 317- 341). Mahwah, NJ: Erlbaum. Estes, S. L., Burns, K. J., Helleberg, J. R., Long, K. M., Pollack, M. E., & Stein, J. L. (2016, November). Digital copilot: Cognitive assistance for pilots. Proceedings of the AAAI Fall Symposium Series. Arlington, VA. FAA Flight Standards Service. (2008). Federal Aviation Administration Joint Aircraft System/Component Code Table and Definitions. October 27, 2008, Regulatory Support Division, Aviation Data Systems Branch, AFS-620, Oklahoma City, OK, retrieved from http://av- info.faa.gov/sdrx/documents/JASC_Code.pdf Fischer, U., Orasanu, J., & Davison, J. (2003). Why do pilots take risks? Some insights from a think- aloud study. In M. J. Cook (Ed.), Proceedings of the Human Factors of Decision Making in Complex Systems Meeting (pp. 44-46). Dundee, Scotland: University of Abertay. Hooey, B. L., Foyle, D. C. & Andre, A. D. (2000). Integration of Cockpit Displays for Surface Operations: The final stage of a human-centered design approach. SAE Transactions: Journal of Aerospace, 109, 1053-1065. Lanzano, J., Seamster, T., & Edens, E. (1997). The importance of CRM skills in an AQP. In R. Jensen (Ed.), Proceedings of the 9th Symposium of Aviation Psychology (pp. 574–579). Columbus, OH: Ohio State University Letsu-Dake, E., Rogers, W., Whitlow, S. D., Nelson, E., Dillard, M., Dorneich, M. C., & Dudley, R. (2015, September). Flight deck information automation: A human-in-the loop in-trail procedure simulation study. In Digital Avionics Systems Conference (DASC), 2015 IEEE/AIAA 34th (pp. 3D1-1). IEEE.

Matheus, C. J., Kokar, M. M., Baclawski, K., Letkowski, J. J., Call, C., Hinman, M., & Boulware, D. (2005). Lessons learned from developing SAWA: A situation awareness assistant. (Technical report). Rome, NY: Air Force Research Laboratory. Miller, C. A. & Hannen, M. D. (1999). The rotorcraft pilot’s associate: Design and evaluation of an intelligent user interface for cockpit information management. Knowledge-Based Systems, 12, 443 – 456. Mosier, K., Fischer, U., Burian, B. K., & Kochan, J. (2017). Autonomous, context-sensitive, task management systems and decision support tools I: Human-autonomy teaming fundamentals and state of the art. NASA Technical Memorandum, Moffett Field, CA: NASA Ames Research Center. Mosier, K., & Skitka, L. (1996). Human decision makers and automated decision aids: Made for each other? In R. Parasuraman & M. Mouloua (Eds.), Automation and human performance: Theory and applications (pp 201-220). Mahwah, NJ: Lawrence Erlbaum Associates. National Transportation Safety Board [NTSB]. (1988). Northwest Airlines, Inc., McDonnell Douglas DC-9-82, N312RC, Detroit Metropolitan Wayne County Airport, Romulus, Michigan, August 16, 1987. Aircraft Accident Report (NTSB/AAR-88/05). Washington, DC: National Transportation Safety Board. Orasanu, J., Fischer, U. & Davison, J. (2004). Risk perception and risk management in aviation. In R. Dietrich and K. Jochum (eds.), Teaming up: Components of safety under high risk (pp. 93- 116). Aldershot, UK: Ashgate. Sarter, N. B., Woods, D. D., & Billings, C. (1997). Automation surprises. In G. Savendy (Ed.), Handbook of human factors/ergonomics (2nd ed., pp. 1926-1943). New York: Wiley. Schvaneveldt, R. W., Beringer, D. B. & Lamonica, J. A. (2001). Priority and organization of information accessed by pilots in various phases of flight. International Journal of Aviation Psychology, 11(3), 253-280 Schvaneveldt, R., Beringer, D. B., Lamonica, J., Tucker, R., & Nance, C. (2000). Priorities, Organization, and Sources of Information Accessed by Pilots in Various Phases of Flight (DOT/FAA/AM-00/26), Office of Aviation Medicine, Washington, DC. Retrieved from https://www.faa.gov/data_research/research/med_humanfacs/oamtechreports/2000s/media/00_26. pdf

Appendix A. Information Sources and iAtoms

Here is the link to the iAtom spreadsheet: https://hsi.arc.nasa.gov/flightcognition/Publications/FAIMTM2_AppA_iAtoms.xlsx

Appendix B. iAtoms Reference Documents

The following documents were used in the creation of the iAtoms listing.

Document Document Description Date 14 CFR 25 Airworthiness Standards: Transport Category Airplanes 02/17/2017 14 CFR 25.771 Pilot compartment 02/17/2017 14 CFR 25.777 Cockpit controls 02/17/2017 14 CFR 25.1302 Installed systems and equipment for use by the flightcrew 02/17/2017 14 CFR 25.1303 Flight and navigation instruments 02/17/2017 14 CFR 25.1309 Equipment, systems, and installations 02/17/2017 14 CFR 25.1321 Arrangement and visibility 02/17/2017 14 CFR 25.1322 Flightcrew Alerting 02/17/2017 14 CFR 25.1329 Flight guidance system 02/17/2017 14 CFR 25.1333 Instrument systems 02/17/2017 AC 20- 131A1 Airworthiness Approval of Traffic Alert and Collision Avoidance 03/29/1993 Systems (TCAS II) and Mode S Transponders AC 20-138D1 Airworthiness Approval of Positioning and Navigation Systems 03/28/2014 AC 20-151B1 Airworthiness Approval of Traffic Alert and Collision Avoidance 03/18/2014 Systems (TCAS II), Versions 7.0 & 7.1 and Associated Mode S Transponders AC 20-1751 Controls for Flight Deck Systems 12/08/2011 AC 25-11B1 Electronic Flight Deck Displays 10/07/2014 AC 25.1302-11 Installed Systems and Equipment for Use by the Flightcrew 05/03/2013 AC 25.1309-1A1 System Design and Analysis 06/21/1988 AC 25.1322-11 Flightcrew Alerting 12/13/2010 AC 25.1329-1C1 Approval of Flight Guidance Systems 10/27/2014 AC 90-100A1 U.S. Terminal and En Route Area Navigation (RNAV) Operations 03/01/2007 AC 90-105A1 Approval Guidance for RNP Operations and Barometric Vertical 03/07/2016 Navigation in the U.S. National Airspace System AC 120-109A1 Stall and Stick Pusher Training 11/24/2015 Boeing 737 NG Flight Manual 06/15/2012 Boeing 737 NG Operating Manual Volume 1 04/07/2009 Boeing 777 Flight Manual 10/28/2011 Boeing 777 Operating Manual Volume 2 06/27/2012 DOT/FAA/TC/44 Human factors considerations in the design and evaluation of flight November DOT-VNTSC- deck displays and controls, Version 1.0. 2013 FAA-13-09 FAA Human The Interfaces Between Flightcrews and Modern Flight Deck Systems 06/18/1996 Factors Team Report FAA DTFAWA- FAA, (2013). Aircraft Access to System-Wide Information 2013 10-E-00030 Management (AAtS) Concept of Operations.. Booz, Allen Hamilton, McLean, VA. iSpec 2200 Airlines 4 America (2016). iSpec 2200: Information Standards for 2014.1 Aviation Maintenance 2016.1. Washington, DC: A4A Publications.

Document Document Description Date https://publications.airlines.org/CommerceHomepage.aspx PS-ANM25-162 Low-Speed Alerting and/or Protection Proposed

RTCA DO-318 Safety, Performance and Interoperability Requirements Document 09/09/2009 for Enhanced Air Traffic Services in Radar-Controlled Areas Using ADS-B Surveillance (ADS-B-RAD)

RTCA DO-348 Safety, Performance and Interoperability Requirements Document 03/18/2014 for Traffic Situation Awareness with Alerts (TSAA)

RTCA DO-360 Standards Development Activities for using Near Real-Time 09/22/2015 Aircraft-Derived Data in Future Applications

1 FAA Advisory Circulars (AC), 2 FAA Policy Statement Number (PS)