FAA/EUROCONTROL ATM Safety Techniques and Toolbox
Total Page:16
File Type:pdf, Size:1020Kb
FAA/EUROCONTROL ATM Safety Techniques and Toolbox Safety Action Plan-15 EUROCONTROL Issue 2 October 3rd, 2007 Version 2.0 i 3/10/07 Table of Contents Summary 3 1.0 Objective 4 2.0 Organization of Report 4 3.0 Review of Safety Initiatives 4 3.1 EUROCONTROL Safety Assessment Methodology Initiative 4 3.2 FAA NAS Modernization System Safety Program Initiative 5 3.3 FAA/EUROCONTROL Safety Assessment Methodology Joint Initiative 5 3.4 Selection of techniques in this report 6 4.0 Overview of Safety Assessment 7 5.0 Overview of Toolbox of techniques 19 6.0 Analysis of Toolbox of techniques 28 7.0 Examples of Applications 124 8.0 Additional Information 128 9.0 Toolbox references 129 Appendix A – Analytic Techniques Supporting Analysis of flight-recorded data (FOQA, ASAP), radar-track data (PDARS), and textual data (e.g., ASRS and ASAP) 140 Appendix B - Acronyms/Abbreviations 162 Appendix C – Participants 164 Version 1.0 ii 27/10/04 Summary This document contains some of the best safety assessment techniques currently available for Air Traffic Management applications, based on the joint experience of the FAA and EUROCONTROL 1 and based on a review of more than 500 safety techniques as used in nine different industries. The result is a set of twenty-seven techniques that can be used by safety practitioners and managers to evaluate and improve safety in Air Traffic Management. The document begins by outlining a simplified eight-stage safety assessment approach and then provides the required safety assessment techniques in a consistent template format. This template format answers basic questions such as where the technique comes from, and its maturity and life cycle stage applicability, as well as more detailed insights into the technique's process and data requirements, and practical and theoretical advantages and disadvantages. The overall approach in this document is biased towards concept design and development phases, since the significant and fast-evolving changes ongoing today in ATM represent the major driver for system safety assessment. Nevertheless, most of the techniques can be (and often are) just as easily applied to existing systems. A good number of the techniques themselves deal with Human Factors and human error aspects of safety, as the human element is a critical determinant of safety in current and future ATM, and cannot be ignored in safety assurance activities. Some outline examples of actual safety assessment approaches using these techniques are provided to show how techniques may be selected from the toolbox. Lastly, some key web addresses and supporting information are given for those who require further information. 1 These two organisations would also like to acknowledge the invaluable support of other organisations including NASA, NLR, CENA, and NATS (UK). Version 2.0 3 3/10/07 1.0 Objective The globalization of ATM systems demands that common safety techniques or tools be identified to support a more efficient interoperability of safety analysis. The objective of this report is to summarize and discuss both common and unique FAA and EUROCONTROL safety techniques. These safety techniques are those judged to be the best currently available. The safety techniques identified in this report are enablers to develop safety material identified in the FAA’s System Safety Management Program (SSMP) or EUROCONTROL Safety Assessment Methodology (SAM). Additionally, this report attempts to increase awareness of these techniques to assist safety practitioners in the air traffic community in conducting their respective safety analysis activities. Moreover, this report will attempt to provide guidance to analysis teams in the selection of effective and applicable techniques. The application of common safety techniques will allow ATM service providers to leverage their skills, knowledge, and experience with respect to global operations and systems. Safety management across ATM systems will therefore improve as safety practitioners implement common techniques, terms, and results. This report is the first major attempt to evolve a common inter-operable safety approach. 2.0 Organization of Report The report begins with a brief safety assessment initiative history of both service providers (FAA & EUROCONTROL) ATM. Section 4.0 provides an overview of a generic system safety assessment methodology, introducing a eight stage safety process and techniques. Section 5.0 provides a matrix of techniques to assist in initial tool selection. Section 6.0 presents twenty-seven selected techniques, each in a consistent template format. Section 7.0 provides five case studies showing that techniques may be consolidated and used together in an integrated fashion to answer safety questions. Section 8.0 briefly considers future developments in the Toolbox, and Section 8.0 the References for the techniques in the Toolbox. Appendix A provides some further templates for tools used to support detailed analysis of flight data, radar-track data, and text data analysis. Appendix B contains a list of acronyms and abbreviations used in this report. 3.0 Review of Safety Initiatives (EUROCONTROL & FAA) 3.1 EUROCONTROL Safety Assessment Methodology Initiative EUROCONTROL is an organization concerned with the safety of European ATM, and aims to support and harmonize approaches across different European Member States. EUROCONTROL has a vision of future ATM that includes many new airspace and advanced controller-tool concepts, and aims to ensure that this future vision is at least as Version 1.0 4 27/10/04 safe, and preferably safer, than current levels in Europe, even given projected significant increases in air traffic volume. In 2002-4 EUROCONTROL therefore undertook a major review of more than 500 safety assessment techniques from nine different industries [Review of techniques for SAM, 2004]. These techniques ranged from ‘traditional’ techniques examining hardware reliability to techniques focusing on human behavior and software safety. The purpose of the Safety Methods Survey project was to make an as complete inventory as possible and to identify from these the techniques and methods (including those developed in other domains and industries such as nuclear, chemical, telecommunication, railways, software design, but excluding commercially available tools) for its formal Safety Assessment Methodology (SAM) applications. From the inventory of more than 500 techniques, a selection was made that appeared most relevant to support the SAM in the short term (with minimal adaptation). In this report the selection of techniques for integration in the FAA tools has been based on broader criteria and this resulted into the selection of fifteen techniques from the ones selected for SAM on the short term, and a similar number of additional techniques. EUROCONTROL aims to ensure a high degree of safety in the Agency’s activities and a formal and systematic approach to safety management with the implementation of a Safety Management System (SMS). Local SMSs in the different Service Business Units (SBUs) and Operational Service Units (OSUs) of the Agency that adequately relate to the safety criticality of the activities and functions are being implemented. Additionally, a process is ongoing to adapt the Agency SMS to the activities at the Experimental Centre for the development of new ATM concepts. 3.2 FAA NAS Modernization System Safety Program Initiative The FAA System Safety Management Program (SSMP) and System Safety Handbook (SSH) for the acquisition of new systems, establishes a plan to ensure system safety is effectively integrated into NAS (National Airspace Structure) Modernization. The SSMP and SSH identify various hazard identification techniques and provide specifics on how to apply these techniques to ATM systems. The FAA’s Air Traffic Organization (ATO) has been evolving towards a Safety Management System (SMS). The SMS provides guidance to the service provider to ensure hazards to the operation, system, and/or procedures are identified in a systematic, disciplined manner implementing defined hazard analysis tools. The SMS identifies various safety techniques (included in Table 1) to ensure that whoever performs the hazard analyses shall select the tool that is most appropriate for the type of system being evaluated. 3.3 FAA/EUROCONTROL Safety Assessment Methodology Joint Initiative Both the FAA and EUROCONTROL have been working to maintain and improve the effectiveness of safety assessment. In April 2003, these two organizations identified the Version 1.0 5 27/10/04 roles, responsibilities, tasks and deliverables with respect to Coordinating Safety R&D, Understanding System Safety, and Assessing and Improving Safety as outlined in the FAA/EUROCONTROL R&D Committee Safety Action Plan (AP-15). This current report represents one of the first major outputs from this Action Plan. Its primary target audience is safety practitioners and safety managers in ATM, but it should also be useful for informing project and program managers developing future ATM concepts, and managers and safety personnel at operational facilities who need to manage the safety of existing operations. 3.4 Selection of techniques in this report In order to obtain a techniques toolbox, which is the main aim of the current report, the EUROCONTROL inventory of over 500 techniques has been used again as a starting point. However, the current criteria for selection from these 500+ are slightly different than in [Review of techniques for SAM, 2004], namely: • The technique should be currently in use; • The technique is judged by the AP15 group as being of value; • The technique is missing in the 500+ review, but satisfies the first two criteria. This resulted in a list of 27 selected techniques. For these techniques this report provides explanatory material in the form of a template, and these 27 are listed below (in alphabetical order of their best known acronym): 1. Air-MIDAS 2. Air Safety Database 3. ASRS (Aviation Safety Reporting System) 4. Bias & Uncertainty Assessment 5. Bow-Tie Analysis 6. CCA (Common Cause Analysis) 7. Collision Risk Models 8. ETA (Event Tree Analysis) 9. External Events Analysis 10.