AIAA Guidance, Navigation, and Control Conference AIAA 2012-4986 13 - 16 August 2012, Minneapolis, Minnesota Validation of LOC-I Interventions Judith Bürki-Cohen* and Andrea L. Sparko† USDOT/RITA/Volpe National Transportation Systems Center, Cambridge, MA 02142 The basic tenet of this paper is that today’s national airspace systems, at least in advanced industrial countries, qualify as so-called Highly Reliable Systems (HRS). In an HRS, even the type of accident that causes the most fatalities is a rare event. This means that in an HRS, the avoidance of accidents is a frequent event. Therefore, the best way to improve an already highly reliable system would be to learn from the cases where accidents have been avoided. This is not possible, however, because you can’t learn from what is unknown. Instead, safety managers resort to retrospective analyses of the most deadly accidents overall. In an unreliable system, it makes sense to correct what is wrong. In an HRS, however, any mitigation efforts that arise from rare, unpredictable, and often unique events carry great danger to upset the balance of the HRS. Such interventions must be scrupulously vetted, in a series of steps that become increasingly costly as the series progresses. This paper makes some suggestions for these steps. If the anticipated benefit from the intervention is not worth the cost of such a thorough review for unintended consequences, then it may be better to accept the existing high reliability of the system as good enough and leave the system unchanged. Nomenclature AA = American Airlines AAMP = Advanced Aircraft Maneuvering Program AF = Air France AFM = Atmospheric Flight Mechanics AOA = Angle of Attack ARC = Aviation Rule making Committee ASAP = Aviation Safety Action Program ASRS = Aviation Safety Reporting System ATC = Air Traffic Control AURTA = Airplane Upset Recovery Training Aid CFIT = Controlled Flight Into Terrain CVR = Cockpit Voice Recorder EASA = European Aviation Safety Agency FAA = Federal Aviation Administration FDR = Flight Data Recorder FOQA = Flight Operation Quality Assurance FFS = Full-Flight Simulator GNC = Guidance, Navigation, and Control HBAT = Handbook Bulletin for Air Transportation HPM = Human Performance Model HITLS = Human-In-The-Loop Simulation HRS = Highly Reliable System IATA = International Air Transport Association ICATEE = International Committee for Aviation Training in Extended Envelopes * Principal Technical Advisor, Aviation Human Factors Division, RVT-82, 55 Broadway. Senior Member AIAA. † Engineering Psychologist, Aviation Human Factors Division, RVT-82, 55 Broadway. Member AIAA. IQTI = IATA Training and Qualification Initiative LOC-I = Loss of Control In Flight LOSA = Line Operations Safety Audit MST = Modeling and Simulation Technologies MTOW = Maximum Takeoff Weight NAA = National Aviation Authorities NASA = National Aeronautics and Space Administration NTSB = National Transportation Safety Board OEM = Original Equipment Manufacturer SAFO = Safety Alert for Operators SOP = Standard Operating Procedures SME = Subject Matter Expert TNO = Netherlands Organization for Applied Scientific Research VMS = Vertical Motion Simulator I. Introduction Loss of Control In Flight (LOC-I) accidents have been identified as the number one reason for loss of life in today’s world-wide air transportation system.1 Many national and international efforts are underway to mitigate this state of affairs. To name just a few: the International Committee for Aviation Training in Extended Envelopes (ICATEE) under the auspices of the United Kingdom’s Royal Aeronautical Society; the European Aviation Safety Agency’s (EASA) “Gain 60 Seconds” initiative; the United States (U.S.) Federal Aviation Administration’s (FAA) FAA/Industry Stall/Stick Shaker Training Working Group convened in the wake of the 2009 Colgan Air/Continental Connection Flight 3407 accident (which resulted in a Draft Advisory Circular AC 120-STALL); and an Aviation Rule making Committee (ARC) on Stick Pusher and Adverse Weather. The International Air Transport Association’s (IATA) Qualification and Training Initiative (IQTI) is also concerned with LOC-I. This paper is one of over 30 papers addressing LOC-I presented in six LOC-I sessions at the combined Atmospheric Flight Mechanics (AFM), Guidance, Navigation, and Control (GNC), and Modeling and Simulation Technologies (MST) American Institute of Aeronautics and Astronautics conferences. These sessions were organized by Dennis Crider of the National Transportation Safety Board (NTSB) and Christine Belcastro of the National Aeronautics and Space Administration (NASA) and the first author. The same team organized similar sessions in 2008 and 2010. With all these efforts underway, the industry will soon be bombarded with a plethora of proposed LOC-I awareness, prevention, and recovery strategies, ranging from procedural to training to technical solutions. The main purpose of this paper is to discuss how these methods could be validated. Before embarking on a discussion of the issues involved with validating the proposed LOC-I mitigations, however, we would like to sound a word of caution. With all this energy and expertise focused onto one problem, it is easy to lose sight of a very important fact: We are living the safest period in the history of civil aviation transportation.2 In fact, there are those who say that we have reached the point of diminishing return and that efforts to make the system any safer would require such resources as to bankrupt the industry.2 Figure 1 illustrates that in any system and for any tool, the cost-safety benefit function will eventually asymptote, and those last 10 percent or so to achieve perfect safety remain elusive without exorbitant investment. This investment, in fact, may drain the resources of an organization to a point where safety may be compromised. These are questions for those charged with balancing the value of human life with the need for a thriving airline industry indispensable for international trade and access to remote areas. The concern of this paper is that if you already have such a highly reliable system (HRS), any intended improvement to the system must be very carefully validated to ensure that it does not upset the balance of the system and introduce harmful unintended consequences. The first rule for any interventions must be, especially for an already very safe system, DO NO HARM. 2 American Institute of Aeronautics and Astronautics Figure 1. Hypothetical cost-safety curves for three types of mitigation. II. A Cautionary Tale A prime example of how a well thought-out and intended effort may have contributed to the worst accident in U.S. aviation history can be found in the events that led up to the crash of American Airlines (AA) Flight 587. In this accident, the pilot’s rudder reversals in response to what is presumed to have been a mild wake vortex led to the separation of the Airbus’ 300-600 tail fin. The airplane plunged into the ground, killing 260 people on-board and five on the ground.3 In response to rising concerns with LOC-I accidents and several NTSB recommendations triggered by accidents (e.g., A-94-173 by the United Express Flight 6291 stall on approach to Columbus, OH), AA took proactive measures.4 Even before the FAA’s Flight Standards Handbook Bulletin for Air Transportation (HBAT) 95-10, “Selected Event Training,” was issued on August 16, 1995, AA initiated the development of its Advanced Aircraft Maneuvering Program (AAMP). According to the NTSB AA 587 accident report,3 AA diligently involved the original equipment manufacturers (OEMs) of most of the airplanes represented in its fleet, inviting comments, traveling to the Boeing Company, and even organizing a two-day AAMP Industry Conference with participation from FAA, NTSB, Boeing, McDonnell Douglas, Airbus, and the U.S. military. AA was generally hailed for its initiative, although the issues of rudder and the fidelity of the simulator outside its validated flight envelope were raised even during this development and vetting phase. In fact, less than three months after the AAMP Industry Conference, in August 1997, FAA, Boeing, and Airbus sent a joint letter to AA stating that the AAMP was already “excellent,” but elaborating on the danger of rudder reversals to the structural integrity of the tail fin. The FAA and the OEMs specifically recommended that “the hazard of inappropriate rudder use” during wake turbulence “should also be included in the discussion.” AA replied that it did so, and that the booklet handed to pilots (including the AA 587 pilots) during AAMP ground school including the recommendation “High AOA [Angle of Attack] maneuvering=RUDDER” was “not a standalone document and nothing should be inferred without listening carefully to the presentation.” AA also stated that the danger associated with rudder excursions at high AOA was “clearly exemplified by” showing NTSB videos on two LOC-I accidents.4 3 American Institute of Aeronautics and Astronautics However, more than simply opinions might have alerted AA to the danger of unintended consequences of its AAMP rudder instructions. Just two weeks before the AAMP industry conference, on May 12, 1997, AA Flight 903 experienced an upset causing serious injury to a passenger and minor injury to a flight attendant. The pilot of AA 903 also had applied full rudder to control roll. The fin of this Airbus 300-600, however, held steady, and the flight crew recovered the airplane in time. In response to this accident, AA’s managing director of flight operations- technical stated in a memorandum to AA’s chief pilot and vice president of flight that AAMP’s roll control instructions in unusual attitude recoveries were “not only wrong, [but] exceptionally dangerous. American Airlines is at grave risk of a catastrophic upset.”4 The lessons of the AA 587 accident illustrate how difficult it is, even with the best of intentions, to get it right, especially when you are dealing with an already very safe system. We reiterate, first do no harm. To avoid that, listen to any cautions. Beware of single-shot narrow solutions.