Risk Indicators and Their Link With Air Carrier Safety The relationship between risk indicators and safety is explored by comparing the accident rate of air carriers with the characteristics of those carriers at the time of an accident. Based on these analyses, a number of risk indicators are identified. by David C. Biggs, Gordon B. Hamilton and Robin Lee Monroe Sypher:Mueller International, Ottawa, Canada This article is based upon the results of a study jury rates requires data over long time periods undertaken by Sypher:Mueller International1 for (up to 10 years or more in Canada). In addi- Transport Canada to evaluate the use of risk indi- tion, the data give poor indications of changes cators for monitoring aviation safety in Canada in risk from year to year. and targeting resources to improve safety. It only considers the safety of commercial operations and Examining less serious accidents can be of great focuses on the larger operators who carry the vast value because the differences between the se- majority of the passengers. The feasibility of us- quence of events leading to a major disaster ing the resultant indicators to identify high-risk and to a minor accident are often small. The operations is demonstrated. minor accidents may be the result of problems that could eventually lead to a serious acci- dent2. Aircraft accident rates based upon number Aviation Safety Measuring of departures rather than hours flown are a Methods Examined better measure of safety, because most acci- dents occur during the taxiing, takeoff/land- The primary bench mark of passenger trans- ing and ascent/descent phases of flight3. Acci- portation safety is the probability of death, or dent rates based on hours flown can be mis- injury, as a result of traveling2. Fatality and leading, especially when comparing different injury rates are estimates of these probabili- types of operations. The infrequency of acci- ties based upon past experience. While fatal dents also makes analysis by numbers of them commercial airline accidents are rare events, a difficult, but there are enough jet aircraft acci- single accident can result in a large number of dents to allow trends in safety to be identified deaths significantly altering the fatality rates. over three or four years in the United States Consequently, trend analysis of fatality or in- and over five to seven years in Canada. F LI GHT SAFETY FOUNDATI O N • F LI GH T SA F E TY D I G E S T • DECEMBER 1991 1 Alternative Measures of The margin of safety is the extra cushion in Safety Explored the system to avoid accidents, which would not be needed if everything worked as planned. With the deregulation of the airlines and per- To the extent that the system becomes less ceptions of its possible effect on the safety of tolerant of failure, or one in which failures airline operations, there has been an increas- occur more frequently, the margin of safety ing need for improved understanding of the decreases and the risk of an accident increases. factors affecting safety, as well as the ability to predict areas where safety may The margin of safety concept pro- vides an alternative method of as- deteriorate. This, combined with … there has restrictions on resources available sessing changes in the level of safety. for aviation inspection and enforce- been an in- Unlike accident and fatality rates, characteristics used to measure the ment, and the need to make better creasing need use of those resources, has led to margin of safety relate directly to the development of alternative ways for improved system components. This makes of measuring safety. These involve them useful for monitoring purposes, the concept of the margin of safety understanding identifying the causes of changes and the use of risk indicators 2,3,4,5. of the factors in safety levels and evaluating ac- tions to improve particular system The aviation system is comprised affecting components. Also, since the char- of a number of components that safety … . acteristics relate to system compo- include airport operators, the air nents, and component failure rates navigation system, weather services, aircraft are greater than accident rates, safety prob- operator personnel and management, aircraft lems can be identified in a more timely man- and aircraft manufacturers. Aircraft accidents ner. are rare events that are typically the result of several concurrent mechanical, human or tech- Indicators measuring the margin of safety should nological failures in components of the avia- be related to safety. After some period of reli- tion system. Any slight deterioration in the able, consistent data collection, it should be separate components increases the probabil- possible to establish a link between the indi- ity of failure (or error) and eventually leads to cator and accident, fatality or injury rates. an increase in the accident rate. However, it may be years before changes in the accident rate due to the component deterioration can Identify Risk Indicators be separated from random fluctuations in ac- cident rates. Potential safety indicators must be measur- able factors associated with or causally related Accidents, despite being the result of a num- to accidents, fatalities or injuries. Non-acci- ber of failures in system components, do not dent data can be divided into two types: provide adequate early warning of deteriora- tion in the system components. Alternative • Incidents — events where the safety of measures are required to identify changes in the aircraft and passengers was affected, the safety level of each component as they but no accident occurred (e.g., engine occur. To develop these measures, the concept shutdown, forced landing due to me- of the margin of safety is employed. chanical difficulties, severe turbulence, etc.), and Aviation safety does not rely on the total elimi- nation of failures and errors in mechanical, • Business stress or “leading” indicators human or technological components of the sys- — properties of an operator or the sys- tem. A margin of safety is built into the system tem which may affect safety (e.g., fi- allowing component failures (or errors) to oc- nancial performance, mixture of aircraft cur without an accident necessarily resulting. types, etc.). 2 F LI GHT SAFETY FOUNDATI O N • F LI GH T SA F E TY D I G E S T • DECEMBER 1991 Incidents are events where a failure or error improving the effectiveness of the inspection occurs in a system component, but no acci- program. dent occurs. By their very nature, incidents are more frequent than accidents and provide Risk indicators are only useful if they are re- more timely evidence of changes in safety lev- lated to safety and, at least in the long term, a els. However, there are a number of problems link can be established between them and the with incident indicators. Despite their obvi- accident rate. Following are the results of analyses ous link with safety, little correlation was found investigating the relationship between a num- between incidents and accidents by the U.S. ber of risk indicators and the accident rate of Aviation Safety Commission2 and Office of Tech- air carriers. An analysis of flight crew charac- nical Assessment6. This could be due to the teristics related to safety could provide valu- desirable effect of identification of incidents able evidence for assessing risk indicators as leading to improvements in the safety of that the flight crew are by far the most common component and therefore fewer accidents, or casual factor in accidents6,7. However, insuffi- due to problems with current incident data. cient data were available on which to assess Incidents comprise a large set of component the effect of pilot characteristics on accident failures that vary greatly in the degree to which risk. they affect safety. Subjective judgment is often used in determining when to classify an event as an incident, and this affects the consistency Air Carrier Characteristics and of reporting the incident. In addition, the reli- Accident Risk Linked ability is dependent upon accurate reporting of the incident which varies greatly with the Air Carrier Data type of incident and the reporting requirements. An analysis of air carrier characteristics and Leading indicators could be used for identify- their accident rates was conducted using 21 ing areas where safety may be deteriorating Canadian air carriers during the years 1983 to before an accident occurs. These indicators can, 1988. Carriers were selected solely on the ba- for example, take the form of measures of busi- sis of number of passengers — not their acci- ness stress (e.g., financial indicators), proper- dent rates, financial status, types of aircraft ties of carriers (e.g., mixture of aircraft types operated, etc. These carriers included all ma- and frequency of recurrent pilot training) or jor jet (Canadian level 1) carriers during years characteristics of the air traffic con- 1983 to 1987, all commuter (level 2) trol (ATC) system. As with incident … incidents carriers in 1987 and four large con- indicators, consistency and reliabil- tract/charter (level 3) carriers (Sta- ity of reporting should be consid- are more fre- tistics Canada levels given in “Ca- ered when choosing properties to be quent than nadian Civil Aviation,” Statistics used as indicators. Canada Catalogue 51-206). The 21 accidents and carriers and their relevant statistics One approach to improving safety provide more are listed in Table 1. Accident data is to determine the characteristics of for these carriers were obtained from the various components that are as- timely evi- the Air Safety Investigation System sociated with high-risk operations. dence of (ASIS), an accident/incident data- Identification of these undesirable base maintained by the Transporta- properties can be used to improve changes in tion Safety Board of Canada (TSB).