Commercial Aviation Safety Team and Joint Safety Analysis Teams

Raymond E. King, Psy.D. Major, USAF, BSC Chief, Research Branch HQ Air Force Safety Center Policy, Research, and Technology Division

ABSTRACT Commission and the NCARC both The number of commercial airplanes in service recommended that, to find a way to reduce will nearly double by the year 2015, going from aviation accidents, the FAA work with the airline about 12,000 airplanes today to over 23,000 in industry to establish some form of strategic 2015. On a worldwide basis, the data suggest safety plan. nearly a hull loss accident per week by the year 2015 at the current accident rate, which has In 1997, at an NCARC hearing, FAA and airline plateaued over the last decade. Many ideas for industry representatives testified about how they enhancing safety focus on technology were investigating the root causes of aviation improvements to airplanes. While such accidents. The FAA Deputy Director of Aircraft improvements are important, it should be noted Certification Service, Beth Erickson, testified for that their impact would not be significant unless the agency: "We had learned from past efforts they can be implemented on the existing airplane that safety improvements were better fleet. A large portion of the airplanes that will be accomplished when we worked with competent operating in 2007 have already been built, and aviation authorities--pilot unions, airlines, most of the rest have already been designed. The aircraft and aerospace manufacturers, and so data show there are significant factors outside of forth--all pooling our expertise to come up with the airplane design itself that influence the the best way to deal with safety issues. So we worldwide accident rate. were very excited by the NCARC suggestion that the FAA work with the airline industry, INTRODUCTION especially using a data-based approach to discover what critical interventions would make On April 14, 1998, at Reagan National Airport in a major difference in aviation safety." Erickson Washington, D.C., then-Vice President Albert says that the White House Commission Gore, Transportation Secretary Rodney Slater, recommendations also helped the FAA decide FAA Administrator Jane Garvey, NASA Acting that working with the airline industry was a more Deputy Administrator Jack Daily, and an practical way to achieve safety goals. audience of top aviation officials gathered to commit publicly to a new government/industry Early in 1996, a broad segment of the aviation aviation safety initiative: Safer Skies. The Vice- community established the Integrated Safety President's statements that day established a Strategy Team (ISST). After the NCARC issued public promise that the aviation industry and the its report, the FAA, NASA, and the DoD U.S. government would conduct a decade-long (represented by USAF) joined with the ISST to effort to reduce the fatal accident rate in aviation form the Commercial Aviation Strategy Safety by 80 percent by 2008. Team (CASST) in October 1997. The group intended to combine the accident investigation The roots of Safer Skies were planted in 1995, in research of the aviation industry and then-Transportation Secretary Federico Peña's government. decision to call for "zero accidents." These roots were nurtured in the reports of both the 1996 COMMERCIAL AVIATION SAFETY TEAM White House Commission on Aviation Safety (CAST) and Security Chaired by Vice President Gore and the 1997 National Civil Aviation Review John O'Brien, director of ALPA's Engineering Commission (NCARC). The White House and Air Safety Department, who served as a member of the NCARC and worked with CASST, explains that “CASST brought in

1

officials from the FAA and NASA to form the implementing the interventions. The JSIT is Joint Safety Analysis Steering Committee responsible for assessing the feasibility of JSAT (JSASC), later to be renamed the Commercial recommendations and developing any Aviation Safety Team (CAST). Throughout appropriate implementation plans. this transition, CAST members had decided there was considerable merit in this aviation THE DEPARTMENT OF DEFENSE’S RISK industry/government process--especially one with recommendations and interventions based The Department of Defense (DoD) is a major on data analysis.” CAST set a goal to of stakeholder in the National Airspace System. reducing the overall fatal accident rate by 80 The DoD’s risk, however, is necessarily different percent by 2007. for the majority of operations and risk tolerance is higher for military-unique operations, to JOINT SAFETY ANALYSIS TEAMS (JSATs) include:

CAST soon developed a process that included • Engine failures (fighters) Joint Safety Analysis Teams (JSATs), whose • Collisions with the ground members were technical experts from the parent • Pilot-induced control loss CAST member organizations, and whose • Midair collisions. purpose is to analyze aviation safety data. The CAST process for defining and implementing a Also, the USAF, over the past decade operated data-driven safety enhancement plan began with 14 civil-derivative and off-the-shelf aircraft in selection of the highest leverage areas of interest: “commercial equivalent” roles. The human Controlled Flight into Terrain (CFIT), Approach factors issues are very similar: and Landing Accidents, Loss of Control, and Runway Incursion. CAST members include • Fatigue representatives from the FAA, JAA, FSF, ATA, • Safety Culture ALPA, APA, RAA, NASA, and DOD • Human Error (represented by the USAF), and airplane and engine manufacturers. Each JSAT’s analysis is METHODOLOGY used to develop intervention strategies that become part of coordinated airline- The various JSAT teams (Controlled Flight into industry/government safety action plans. CAST Terrain, Approach and Landing, Runway set up a Joint Safety Implementation Team Incursion, Pilot-Induced Loss of Control, and (JSIT) to work on implementation of the JSAT Weather) recognized that interventions are developed interventions. This was a dramatic extremely limited in their potential to bring departure from past government/industry actions, about a change unless they are based on a because it provided an effective problem-solving thorough understanding of the underlying causes. mechanism to aid in implementation. Accordingly, a the teams generated sets of

supporting research recommendations. The JOINT SAFETY IMPLEMENTATION TEAMS JSAT methodology combines detailed case (JSITs) studies, a high-level data analysis, and expert

judgement. The case studies employ an event- JSATs are the first aviation industry/government sequence analysis, while the high-level approach bodies to have teeth, because strategies they involves statistical data and data from other develop are based on a government/industry sources. Based on these different sources, the consensus. These intervention strategies then are JSAT developed interventions that addressed turned over to the Joint Safety Implementation specific case-study accidents. Each intervention , which develops plans for the Team (JSIT) was rated for three characteristics: power; aviation industry and government to implement confidence; and applicability. The JSAT then these strategies. The JSAT combined the FAA's computed an "Overall Effectiveness" (OE) score JSASC process with the industry group process or ranking, ranging from 0 to 6. OE scores of CASST to evolve a process that considers the primarily reflect the estimated effectiveness of facts of an accident, plus what went wrong and each intervention in preventing the respective why, and suggests solutions and how to case-study accidents against which they were implement them. The process stipulates that the rated. JSATs recognized that singular and JSAT will not address the feasibility or costs of

2

isolated interventions generally are less effective in reducing accidents than are approaches that 1.) Situational Awareness Technologies integrate related interventions. The industry should develop and implement technologies that enhance flight crew awareness The JSAT also included interventions that of aircraft flight path and position geographically addressed organizational culture, systematic use and relative to terrain. of digital flight data, no-blame internal reporting systems, etc. Such interventions may not 2.) Stabilized Approaches produce their full benefits by the 2007 target, or To address the problems of unstabilized the analysis of past accidents may not adequately approaches and loss of vertical situational assess the full potential of some interventions to awareness, the industry should develop and break complex causal chains in future accidents. implement precision or precision-like approach Consequently, some recommended interventions, capability (glidepath guidance) to all runways particularly those that address research and without established precision approach development, were not assigned OE ratings. procedures. Instead of OE ratings, later JSATs prioritized the research interventions. 3.) Go Around To reduce the risk of accidents associated with The interventions that received the highest OE unstabilized and rushed approaches, airline ratings provide the foundation for the operators should: recommendations, each of which calls for several • Establish policies, parameters, and training actions by airlines/operators, manufacturers, to recognize unstabilized approaches and regulators, or others. In addition, the other factors and implement a go-around recommended strategies include interventions gate system that received lower OE ratings. Three additional • Institute a true no-fault go-around policy data-related interventions are included in the • Incorporate, in initial and recurrent training, recommendations due to their strong synergistic ways to recognize multiple cues that will effects. require a go around • Ensure that flight crews are trained to think All recommendations require the active in terms of “I will go around unless...,” participation of regulatory authorities. Such rather than “I will land unless…” participation may include developing technical • Enhance ATC training to emphasize dangers standards, approving procedures, or overseeing of rushed approaches and the performance implementation. In addition to the regulators, characteristics of modern jet transports many recommendations identify other members • Base runway selection on the most current of the aviation community that must take action wind available if the recommendations are to be effectively implemented. The recommendation strategies 4.) Standard Operating Procedures are presented with their constituent interventions • An industry/government team should be in a non-prioritized order. established to develop a template for

standard operating procedures best practices, The team analyzed the accident data set to including guidance on what SOPs should develop interventions that would have cover, development methodology, and how worldwide application. Full implementation of to train for, and monitor, procedural recommendations will therefore involve compliance. manufacturers, operators, air traffic service • Research should be undertaken to better agencies and regulatory agencies throughout the understand the underlying reasons/causes for world. procedural non-compliance

5.) Safety Culture INTERVENTIONS • Airlines/operators should and regulatory

agencies must encourage a culture that The following is a synopsis of broad intervention enhances safety. strategies: • Incorporate a company self-audit process From the Approach and Landing JSAT: and a cost analysis tool regarding the high

3

economic and psychological costs of 2.) Training Intervention Strategies accidents In general, training history was very poorly • Emphasize safe arrivals over timely arrivals documented in most of the accident data and discontinue on-time arrival tracking for reviewed. There were several training or airlines. Do not punish missed approaches. evaluation flight accidents reviewed as a part of Base reward system on something other that the data set. In most instances involving training completion of a route segment. issues, team members developed interventions • Allow voluntary removal from flight status based upon their experiences with regard to best due to illness and/or emotional distress. Be practices, which have proven effective in sensitive about fatigue and circadian rhythm. reducing or preventing accidents. • Implement policies regarding crew pairing • Airlines/Operators should establish a CRM Since this was the Loss of Control JSAT, the training program and regulators should training interventions focused primarily on require and ensure the initial training is preventing loss of control and regaining control. provided prior to line flying. Require Several major carriers have recently introduced recurrent CRM training. upset recovery training. However, the team felt • Parent Airlines/Operators should adopt a that on a worldwide basis this continues to be a program to ensure the same level of safety in neglected area. The team also felt that current regional partners regulations and training regimes which focus on "approach to stall" inadequately prepare the 6.) Operational Feedback pilots to deal with recovery from a full-stall • To enable airlines/operators to identify condition. safety issues and trends, monitor procedural compliance and initiate corrective actions Many of the accidents involved issues of prior to accident occurrence, the following automation, in which pilots were confused by the interventions should be implemented: automation, did not understand or were not aware of what automation was doing or how to • Airlines/operators should implement Flight control the automation. Once control was lost, Operations Quality Assurance (FOQA) pilots did not clearly understand how to regain programs control of the aircraft through manual control, • Airlines/operators and regulators should disabling the automation, or reestablishing implement a no-blame safety reporting and automated control. data sharing system with appropriate

protection from litigation and prosecution. Pilot training was a topic in almost every

accident analysis. There were two fundamental From the Pilot-Induced Loss of Control JSAT: issues involved:

1.) Design Intervention Strategies • Design interventions encompass strategies Current simulators are limited in their effectiveness to train pilots to recognize and ranging from developing and expanding new recover from a full stall. technologies, such as flight envelope protection, • to adapting and retrofitting older, proven systems Pilots have not had adequate training to to existing aircraft. Additional interventions develop skills and confidence to recognize attempt to deal with aircraft and systems and recover from aircraft upset. certification. Team members expressed a great deal of concern that newer aircraft and systems, Simulator data packages do not include data certified under derivative criteria, are not representing portions of the flight envelope, required to comply with higher standards of including stall. This precludes realistic pilot reliability and redundancy currently required of training in the dynamics of stall and stall new designs. New technologies and existing recovery. The team recommends that the JSIT technologies were also evaluated as to whether evaluate expansion of the flight envelope data they should or could be adapted to existing package and simulator capability in order to aircraft. improve pilot training in the stall regime.

The team concluded that upset recovery training for pilots is essential to reduce the number of loss-of-control accidents. The team believes that

4

simulators can be used more effectively to cases where the information provided by the develop skills and confidence for pilots to manufacturer had not been incorporated into the recognize and recover from aircraft upset. The operating procedures of the airlines. The team team recommends that upset recovery training recommends that the JSIT identify and become a mandatory part of initial and recurrent implement best practices for coordination pilot training programs. between regulators, manufacturers, and operators that ensures critical safety information is • Stall Recognition and Recovery distributed and implemented in a timely fashion • Upset Recovery Training to those who need it. • Automation Training • Instructor and Check Pilot Qualification and There continues to be an industry-wide problem Training of pairing minimally qualified pilots. It was the • Improved Training, Policies, and Procedures opinion of the team that this issue needs • Simulators continual oversight by both operators and regulators. 3.) Practices, Policies, and Procedures • Intervention Strategies Standard Operating Procedures • A large number of interventions deal with Automation Policy practices, policies, and procedures. Few of the • Integration of Manufacturer Procedures Into accident reports contained specific information Flight Operations regarding this topic area. The team members had • Crew Qualifications to rely on their experience to provide • Currency and Accuracy of Information interventions that reflect the best practices in the • Service Bulletins industry. The team recommends that accident • Manuals investigation reports should explicitly address practices, policies, and procedures. 4.) Data Intervention Strategies There are a limited number of intervention Standard Operating Procedures (SOP) either did strategies related to data collection. The analysis not exist or were not followed in many accidents. of the team was complicated by the fact that Assuming pilots do not intentionally violate many of the accident aircraft had no or limited existing SOPs, it is crucial to understand the data recording available. In many of the accident underlying cognitive processes contributing to aircraft, the data recorders were not functioning procedural non-compliance. Intervention #204 or the data available was so limited that it was calls for important research aimed at almost useless. understanding this issue. The team also recommended statutory support In most accidents involving automation, there for ongoing analysis of data from nonvolatile was an absence of an automation policy. Pilots memory systems, such as FOQA or BASIS, and became confused about the automation and did self-reporting programs, such as ASAP or ASRS, not have any policy of monitoring or to identify and eliminate problems before they disengaging that would have helped them result in accidents. These programs need to be prevent the accident. In many cases, airlines protected by statute to ensure that they can fulfill have automation policies that include phrases their purpose of sharing safety data and identify like "Use the appropriate level of automation," precursors to accidents in a non-punitive but they provide no guidance or training as to atmosphere. how to decide what the appropriate level of automation is for a given set of circumstances. 5.) Regulatory Intervention Strategies This situation leads to pilot confusion and Almost every intervention assumed a regulatory inaction and contributes to automation-related role. Effective implementation of any particular accidents. intervention will require active regulatory participation if not direct implementation There were several cases where service bulletins responsibility. The regulatory interventions had not been implemented or where they had encompass both the "rules" and the oversight been implemented and there was no information responsibility of the regulatory agencies on a provided to the pilots. There were also several global basis:

5

• Regulations, Rules, and Advisory Circulars • Certification • System Reliability • Training • Oversight • SOP • Airline Operations • Currency and accuracy of Information

CONCLUSION

The CAST and JSATs illustrate the ability of industry and government to work together effectively with the goal of increasing global aviation safety.

6