Airline Safety Assessment Mechanism

REPORT DECEMBER 418 2016

Airline safety assessment mechanism

safe travel Acknowledgements

Safety Committee

Aviation Subcommittee

Photography used with permission courtesy of ©PPAMPicture/iStockphoto and ©Paiwei Wei/iStockphoto (Front cover)

Feedback

IOGP welcomes feedback on our reports: [email protected]

Disclaimer

Whilst every effort has been made to ensure the accuracy of the information contained in this publication, neither IOGP nor any of its Members past present or future warrants its accuracy or will, regardless of its or their negligence, assume liability for any foreseeable or unforeseeable use made thereof, which liability is hereby excluded. Consequently, such use is at the recipient’s own risk on the basis that any use by the recipient constitutes agreement to the terms of this disclaimer. The recipient is obliged to inform any subsequent recipient of such terms.

This publication is made available for information purposes and solely for the private use of the user. IOGP will not directly or indirectly endorse, approve or accredit the content of any course, event or otherwise where this publication will be reproduced.

The contents of these pages are © International Association of Oil & Gas Producers. Permission is given to reproduce this report in whole or in part provided (i) that the copyright of IOGP and (ii) the sources are acknowledged. All other rights are reserved. Any other use requires the prior written permission of IOGP.

These Terms and Conditions shall be governed by and construed in accordance with the laws of England and Wales. Disputes arising here from shall be exclusively subject to the jurisdiction of the courts of England and Wales. REPORT DECEMBER 418 2016

Airline safety assessment mechanism

Revision history

VERSION DATE AMENDMENTS

2.0 May 2009 Update of Report 303

3.0 December 2016 See p.6 for the main changes Airline safety assessment mechanism 4

Contents

1. Introduction 8

1.1 Why an airline safety scoring system is needed 8 1.2 The need for formal risk assessment 9 1.3 Origins of the IOGP mechanism 10 1.4 Application of the mechanism 10

2. Summary 11

2.1 Score components of the airline safety assessment mechanism 11 2.2 Illustrative results 12

3. Safety Factors/Multipliers (SF) 14

3.1 Definition of accident rates (AR) 15 3.2 Accidents to be included in the analysis 15 Types of operation included 16 Types of operation excluded 16 Types of events included 16 Types of events excluded 16 3.3 Accident Severity Weighting 18 Fatal accidents (OF1) with >20 fatalities, weighted by a factor of 3.0 18 Fatal accidents (OF2) with >10 & ≤20 fatalities, weighted by a factor of 2.5 18 Fatal accidents (OF3) with ≤10 fatalities, weighted by a factor of 2.0 18 Serious accidents (OF4), weighted by a factor of 1.0 18 Minor accidents (OF5), weighted by a factor of 0.25 18 3.4 The ‘additional’ accident 20 3.5 Effective Accident Rate 21 3.6 Measuring number of flights 22 3.7 The ‘Safety factor’ concept 23

4. Airline Factors (AF) 24

4.1 Aircraft Fleet Age (AF1) 24 4.2 Airline Fleet Composition (AF2) 28 4.3 Aircraft Equipment (AF3) 31 4.4 Conduct of Operations (AF4) 32 4.5 Partnerships and alliances (AF5) 34 4.6 Airline Financial Standing (AF6) 35 4.7 Airline Maturity (AF7) 36 4.8 Airline Security (AF8) 37 Airline safety assessment mechanism 5

5. Country Factors (CF) 38

5.1 Regulatory Oversight (CF1) 38 5.2 National Safety Influences (CF2) 41 5.3 Air Traffic Environment (CF3) 42 5.4 Airfield Environment (CF4) 43 a. Terrain (CF4a – Table 17) 43 b. Climate (CF4b – Table 18) 44 5.5 Country Security (CF5) 44

6. Implementation of the mechanism 45

6.1 Calculating airline safety scores 45 6.2 Safe travel policies 46 6.3 Single sector journeys 49 6.4 Multi-sector journeys 49

Appendix A. Glossary of terms and abbreviations 51

Glossary of terms 51 Abbreviations 55

Appendix B. ASAM Spreadsheet Examples 57

Appendix C. Potential data sources and bibliography 61

C.1 Accident data and analysis 61 C.2 Airline schedules 61 C.3 Airline Fleet Data 61 C.4 Airline statistical data (including airline flight statistics) 62 C.5 Regulatory information 62 C.6 Security data 62 C.7 General airline data 63 C.8 Travel advice and information 63 C.9 Best practice 63 Airline safety assessment mechanism 6

Background to Version 3

This, IOGP ASAM Version 3, supersedes Version 2 published in 2009.

The IOGP Aviation Subcommittee requested the re-convening of the previous Working Group to determine what changes might be considered necessary to update and improve the previous model to take account the advancement of the theory behind the methodology and in the underlying data, and the practical aspects of populating the mechanism.

The main changes made to the ASAM as the consequence of the review were as follows: • Based on a further extensive analysis of commercial passenger aircraft accidents over the past 10 years, it was concluded that there was a strong correlation between aircraft accident rates and aircraft age out to 40 years, after which the data points became limited and inconclusive. The aircraft age factor was therefore modified to reflect a straight-line reduction from 1.00 to zero over 40 years. • The Fleet Mix scores (AF2) have also been revised to take account of the changes in technology levels as new generation types that are continuously being introduced. Less modern types have in general been adjusted downwards in 0.25 score intervals, thereby increasing the numbers of older types scoring zero for AF2. Each 0.2 interval corresponds to a different decade of aircraft development. A new Table 6 reflects the revised Fleet Mix type scores. • The Aircraft Equipment (AF3) default value has also been modified to reflect the fact that aircraft invariably have some degree of the relevant equipment fit to a minimum value of 0.5 plus 0.5 of the Fleet Mix (AF2) score. • The Airline Finance score (AF6) had previously proved impractical to populate with meaningful data. Therefore a simplified Finance Score, based on the airlines past five years reported operating profit was adopted as a means of scoring those airlines for which such data was available. The scale adopted is from 1.00 to zero in line with the percentage operating profit over the range +10% to minus 10%. The default value for the majority of airlines with no meaningful finance data remains 0.50. • Previously the Country Regulation Factor (CF1) assessment was somewhat arbitrary. Now that ICAO publishes online summaries of its ongoing audits of all Member States oversight, it is possible to have a more reliable source of country’s regulatory performance and this has been adopted based on ICAO percentage scores. Provision is also made for a 25 percentage point reduction to apply to countries on a one-off basis, based on any further identified regulatory deficiency. This includes any EU Country Ban, a US FAA IASA Category 2 rating and ICAO’s own cautionary ‘Red Flag’ associated with certain country audit scores. Airline safety assessment mechanism 7

Other aspects of the ASAM were unchanged, including the calculation of the Safety Multiplier and the relative weighting of the various Airline and Country Factors included in the mechanism.

A number of editorial and style changes have also been made to the report. Airline safety assessment mechanism 8

1. Introduction

1.1 Why an airline safety scoring system is needed This report contains an updated mechanism (Issue 3) for assessing the safety of scheduled commercial airlines. The mechanism is based on earlier work carried out in this area by organizations represented on the working group that drafted this report.

Statistically the chance of being involved in an accident with a scheduled airline is small – less than one per 100,000 flights – and the chance of being fatally or seriously injured is an order less than this at around 1.3 per million flights. The best charter airlines, those operating in all but name on a scheduled basis, achieve similar levels of safety to the best-scheduled carriers.

While specifically aimed at scheduled airlines engaged in regular public transport, the mechanism is also applicable to those non-scheduled carriers that operate to a recognizable schedule such as European inclusive tour-type operations. It is not applicable however, to ad hoc charter operators, where the route system operated varies on a weekly or more frequent basis. Combination passenger/cargo flights may also be included, but not usually all-cargo flights.

On a distance-flown basis, air travel has become one of the safest forms of travel. However, because of the longer distances involved in air travel compared to most surface journeys, it is the accident rate per flight that is of most concern. This is especially so for frequent travellers whose journeys often involve multiple-sector flights around the world; in extreme cases, accident rates for such flights can be more than 25 times the industry norm, reflecting unsafe operations and putting passengers and crew alike at risk.

This geographic context is important since, although it continues to become safer overall, airline safety varies widely around the world by airline as well as by the aircraft types used. Generally, operations by well-established, developed-world, airlines are safer than those by less-established, developing-world operators. However, much of the evidence for this has remained un-analysed. Assessing the risks involved in using different airlines is based largely on reporting of a few well- publicised accidents viewed in isolation from their proper statistical context. Airline safety assessment mechanism 9

There is also the problem of the smaller operators. The fact that they might not have experienced an accident may not be statistically significant and over-reliance on this fact may conceal underlying problems with air safety with, to use a popular phrase – “an accident waiting to happen”. Larger operators do not have this problem: the accident might indeed be waiting to happen, but its occurrence does not generally significantly affect the overall accident rate, although it might hit the headlines.

Charter operators, where they are used on a regular basis, may be subject to safety audit before contracts are placed. This is standard practice in the oil and gas industry and IOGP has in place its Aircraft Management Guidelines for charter services. In contrast, scheduled airline users – even those with a large corporate travel budget – have no automatic right to audit the operations of the airlines which they use. They have to rely on regulatory oversight, limited company knowledge and often-imperfect public perceptions of airline safety.

1.2 The need for formal risk assessment The selection of safe travel modes forms part of the duty of care companies owe to their employees and contractors when arranging business travel. Failure to attend to such concerns can have serious consequences.

Any business that require significant amounts of business travel should have in place a properly-constructed travel plan as part of its wider health and safety policy. Before developing such a plan for air travel by scheduled airlines, to be incorporated in a company safety management system, it is necessary to undertake a formal risk assessment.

Many countries require formal risk assessment as part of health and safety legislation but, even if this is not a specific legislative requirement, the assessment process makes good sense in developing robust safety systems. It would also form part of a duty of care defence in any case of litigation.

An important part of the overall hazard management involves undertaking a formal safety audit of an airline. This is mostly impossible to achieve however, primarily because most airlines would not permit such audits by customers, but also because of the prohibitive cost of primary and repeat audits.

An alternative approach is to conduct a desk-top risk assessment using a safety grading mechanism; this approach seeks to assess the relative risk of flying on specific airlines by the adoption of a grading system using data from the airline and its operating environment. The ready availability of data on commercial airlines makes it both possible and practicable to carry out such an exercise. Airline safety assessment mechanism 10

1.3 Origins of the IOGP mechanism Under the auspices of the IOGP Aviation Subcommittee, a working group brought together interested parties and specialists working in the field to develop the airline safety assessment mechanism presented here.

The safety assessment mechanism is derived from a number of different mechanisms which were available to members of the working group.

The overall project was formulated through a series of workshops. Participants included IOGP Aviation Subcommittee Members plus invited members from other organizations representing airlines, independent consultants, the telecommunication industry, the broadcast media and an international funding agency.

1.4 Application of the mechanism This report and its associated spreadsheet provide a framework for carrying out a rational and structured safety assessment of a scheduled commercial airline.

In order to carry out the assessment, a range of information is required which is specific to the airline and flight to be assessed. This information might be available within the company carrying out the assessment or via recognized data sources (many of which are listed in this report), or could be obtained through a contracting organization that specializes in compiling the relevant information.

The Safety Mechanism outlined in this report is just one element to be taken into account in assessing airline risk to passengers. In particular it should be noted that achieving a low Safety Score does not necessarily imply that an airline is unsafe to use or should necessarily be categorized as “high risk”. Often a low score may be entirely due to the fact that an airline has only limited operating experience, such as in the case of recent start-ups or long-haul operators flying only a few such flights per week. In such cases it may be more appropriate.

On occasion, Members of the IOGP safety assessment mechanism working group may be able to offer advice on the use of the mechanism. Airline safety assessment mechanism 11

2. Summary

2.1 Score components of the airline safety assessment mechanism The mechanism adopted uses a score derived from various operating and other parameters that may either be calculated from detailed information on the airline and its circumstances or set to various conservative default values based on known industry standards and practices.

Details of the various score components are set out in the following sections.

In brief, the airline safety assessment scores (AS) are derived from the following three main elements • an airline safety factor (SF) or multiplier, calculated between zero and one, based on the reported accident rate over a maximum ten-year period of scheduled airlines operations, also taking into account the severity of accidents in terms of loss of life and injury and damage to aircraft • an airline factor (AF) ranging from zero to ten made up of various specific- to-the-airline management factors, operating parameters and operational environment components • a country factor (CF) ranging from zero to five that takes into account national factors of regulation, security and safety culture of the airline’s home country.

The factors in the airline and country scores have made provision for a number of default values to be used where information is not currently available. The purpose of the default values is to achieve a reasonably conservative approximation of the appropriate score, without unduly penalizing the airline concerned.

Airline overall score (AS) is given by the formula:

AS = SF × (AF + CF)/1.5 where

SF = safety factor

AF = sum of weighted scores AF1 to AF8

CF = sum of weighted scores CF1 to CF5

Figure 1: Mechanism formula Airline safety assessment mechanism 12

The safety factor is multiplied by the sum of the airline and country scores divided by 1.5 to provide an overall airline score between zero and ten. • Ten represents an airline with a perfect safety record and other favourable attributes. • Zero represents a high-risk airline where the overall risk of an accident approximates to less than one per 4,000 flights.

While it is quite possible to achieve a zero score (and some airlines do), it is not possible to achieve a perfect ten because of the principle of an additional accident built into the system.

Whatever the actual airline accident score, the number is increased by 1.0, representing a further serious-injury accident. This means that airlines’ safety records and their resulting safety scores can be considered in the light of what becomes hindsight in the unfortunate (and it should be emphasized, statistically unlikely) event that they do have an accident in the immediate future. In effect, this introduces a sensitivity element into the mechanism.

A glossary of terms and abbreviations is provided in Appendix A.

2.2 Illustrative results At any time, there are over 1,000 airlines engaged in scheduled operations currently flying about 30 million flights each year.

Appendix B to this report presents the airline safety assessment mechanism in the form of a spreadsheet, and illustrates the calculation of example scores for 15 fictitious airlines. There is no significance in the location of these airlines, other than achieving a wide geographic spread and illustrating the range and diversity of the scores that it is possible to achieve using the scoring system.

The example safety assessment scores shown in Appendix B are derived from a Microsoft™ Excel spreadsheet model described later which may be used as an aid to compiling such scores. Further guidance on the use of the mechanism spreadsheet is contained in section 6.

Table 1 contains a summary of the airline safety scores derived from Appendix B. Airline safety assessment mechanism 13

Table 1: Example airline safety scores Airline Country World Region Safety Airline Country Overall Name factor factor factor score (SF) (AF) (CF) (AS) Airline 1 Congo-Brazzaville Africa 0.831 5.30 0.94 5.19 Airline 2 Brazil Latin America 0.935 5.65 1.58 6.77 Airline 3 Colombia Latin America 0.797 4.82 1.52 5.05 Airline 4 France Western 0.000 4.01 3.13 0.00 Europe Airline 5 Bhutan Asia/Pacific 0.780 3.96 2.27 4.86 Airline 6 Bulgaria Eastern 0.579 4.54 2.79 4.25 Europe Airline 7 Ethiopia Africa 0.561 0.83 1.36 1.23 Airline 8 Iran Middle East 0.452 1.33 2.35 1.67 Airline 9 Malaysia Asia/Pacific 0.817 1.50 2.55 3.31 Airline 10 Sudan Africa 0.452 1.20 1.36 1.16 Airline 11 USA North America 0.690 3.91 2.91 4.71 Airline 12 Russia Former 0.123 3.62 2.71 0.78 Soviet Union Airline 13 Kazakhstan Former 0.379 3.82 2.12 2.25 Soviet Union Airline 14 Solomon Islands Australasia 0.690 5.01 2.28 5.03 Airline 15 Myanmar Asia/Pacific 0.916 2.84 2.22 4.63 NOTE: AF and CF are presented as shown in the accompanying spreadsheet. Both have been divided by 1.5.

The above, and all similarly calculated scores, are arrived at only for the purposes of indicating relative risk based on the identified and quantified factors described in this report. They do not constitute advice to use, or not use, particular airlines.

Decisions on airline use are for individual travel organizers to take based on their corporate safe-travel policies. The guidelines have been developed simply to aid that process. Airline safety assessment mechanism 14

3. Safety Factors/Multipliers (SF)

Airline safety is typically measured either by the accident rate per the number of flights/hours flown, or by the rate per passenger-kilometres flown. Since a high proportion of accidents are associated with the take-off and landing phases of flight, and relatively few such events with the en-route phase, it is considered most appropriate to base the mechanism on the accident rate per 100,000 landings. This measure is used throughout this report.

Only operational accidents – those that involve an intended or actual flight – are counted in the analysis. Non-operational accidents that occur on the ground before flights are initiated or after flights are completed, such as when aircraft are parked, or during maintenance or towing are therefore excluded from consideration. Injuries or deaths to stowaways outside the pressure bulkheads, particularly those entering wheel-wells as the aircraft waits at the runway hold, are not included in the analysis.

Accidents are a relatively rare event in scheduled airline operations and, fortunately, fatal and serious injury accidents are even rarer. The scheduled airline industry average rate currently stands at around less than 0.50 per 100,000 flights. The majority, about two-thirds of reported accidents, involve no injury or only minor injuries to occupants and only around 28% result in fatalities. Most serious injury accidents also involve fatalities, with about 6% resulting in serious injuries to occupants without causing deaths.

The safety mechanism presented in this report considers five classes of accidents: • accidents involving more than 20 fatalities from the aircraft’s occupants are given a weighting of 3.0 • accidents involving between 11 & 20 fatalities from the aircraft’s occupants are given a weighting of 2.5 • accidents involving 10 or less fatalities from the aircraft’s occupants are given a weighting of 2.0 • accidents involving serious injury are weighted 1.00, and • minor or no-injury accidents are weighted with a factor of 0.25.

A very small minority of accidents, less than 1%, involve death or injury to non- occupants. These are not relevant for the purpose of determining air travel safety or grading accidents for severity.

Sources for accident information include the World Aircraft Accident Summary – (CAP479 or ‘WAAS’), maintained for the UK CAA by Ascend Worldwide, the Aircraft Loss Record, published by the same company, as well as Aviation Safety Net – ASN (see Appendix C). Airline safety assessment mechanism 15

These three sources cover the great majority of significant events involving fatalities and injury, and aircraft hull loss. Much of the information contained in them is common to all three, but WAAS and ASN also contains details of accidents to some smaller aircraft types excluded from the Loss Record, while the Loss Record also contains many more reports of non-operational accidents resulting only in aircraft damage. Other sources, including National accident investigation reporting, provide a further valuable source of accident data.

Further information on lesser accidents and incidents is also available from national aviation authorities. While data on incidents – events where safety is compromised but that do not result in accidents – ought to provide a better statistical basis for measuring actual air safety, incident data collection and dissemination is much less consistent than actual accident reporting. Most accidents are now well reported although there remain problems with under- reporting in some areas. (For example, this was in the past a problem in China and the Former Soviet Union.)

3.1 Definition of accident rates (AR) It was determined by the working group that the accident rate per 100,000 landings should be used for the mechanism based on reported operational accidents involving fatalities to passengers or crew, serious injuries to either group, and minor or nil injury accidents.

3.2 Accidents to be included in the analysis It is necessary to precisely define which operations and accidents are to be included in this analysis and it is important that the accident data and number of flights are consistent and relate to the same operating periods. For example, if the analysis is for scheduled passenger operation, then only scheduled passenger aircraft flights and scheduled passenger accidents should be included.

The mechanism uses the most recent 10-year period of airline operations as the basis for calculations. Where an airline has operated for less than 10 years, accident and flight data for the maximum available period of operation should be considered. Where an airline has operated for a longer period, accident and flight data for the period prior to the most recent 10 years operations should be excluded. Airline safety assessment mechanism 16

Following further review of the available data and definitions contained in the Ascend Worldwide Aircraft Loss Record, for scheduled passenger airlines the following types of operations and events were included and, by definition, all others excluded. The selection process is also shown in the form of a flow chart (Figure 2).

Types of operation included Commercial scheduled passenger operations, including combination passenger/ cargo services and military passenger operations conducted on a commercial basis.

Types of operation excluded All non-scheduled operations, except where cycle data is also included in the CASE database, any all-cargo operations, and all non-commercial flights by private business and government operators – the latter group including purely military operations.

Types of events included Occurrences during the most recent 10-year operational period of operational accidents, including those involving unlawful action against aircraft, pilot training accidents due to technical malfunction or poor flight management, and ferry flight accidents.

Types of events excluded Non-operational accidents, (unmanned accidents on the ramp or in maintenance where there is no intention to fly the aircraft) and test flights. Airline safety assessment mechanism 17

All accidents was the flight on which the accident occurred: Excluded accidents

Commercial or Non-commercial, military/commercial? No purely military

Yes

Passenger or All cargo passenger/combi? No

Yes

Scheduled or non-scheduled Non-scheduled with cycle data? No (no cycle data)

Yes

Non-operational (parked, Operational? No towing or in maintenance)

Yes

Non-revenue Ferry flying? No (except positioning flights)

Yes

Technical malfunction or Training (excluding poor flight management technical malfunction or on training flights? No poor flight management)

Yes

Included accidents

NOTE: In the event that the mechanism user wishes to include other types of operation, such as non- scheduled revenue passenger or cargo flights, then he or she may do so. However, he or she should always ensure that the aircraft flight data and accident data are fully consistent.

Figure 2: Accident selection flowchart Airline safety assessment mechanism 18

3.3 Accident Severity Weighting The early scoring systems weighted all included accidents equally (included accidents are those selected in accordance with Figure 2 and which occurred during the most recent 10 year period of airline operations (or the maximum available operations period if less than 10 years).

In view of the greater concern over accidents involving death and serious injury to occupants, it was decided to weight accidents on the basis of their severity. Alternative weighting schemes were considered, but the following scheme was adopted because of its simpler and more transparent method of calculation:

Fatal accidents (OF1) with >20 fatalities, weighted by a factor of 3.0 An accident involving fatalities of greater than 20 of the aircraft’s occupants, regardless of the extent of damage to the aircraft.

Fatal accidents (OF2) with >10 & ≤20 fatalities, weighted by a factor of 2.5 An accident involving fatalities of more than 10 but less than 21 of the aircraft’s occupants, regardless of the extent of damage to the aircraft.

Fatal accidents (OF3) with ≤10 fatalities, weighted by a factor of 2.0 An accident involving fatalities of 10 or less of the aircraft’s occupants, regardless of the extent of damage to the aircraft.

Serious accidents (OF4), weighted by a factor of 1.0 Accidents involving serious injuries to passengers or crew and any accident involving major loss to the aircraft.

Minor accidents (OF5), weighted by a factor of 0.25 An accidents resulting only in minor injuries or no injuries at all to the occupants and only minor damage to the aircraft. Airline safety assessment mechanism 19

Definitions of severity

Fatal accident

Any accident involving fatalities to passenger or crew, regardless of the damage to the aircraft.

Serious accident

Any non-fatal accident involving serious injuries to passengers or crew and any accident involving major loss to the aircraft.

Minor accident

Accidents resulting only in minor injuries or no injuries at all to the occupants but substantial damage to the aircraft.

The definitions of major and minor aircraft loss are those provided by Ascend Worldwide: • major aircraft loss: repair costs of $1.0 million or more, or 10% or more of the value of the aircraft • minor aircraft loss: repair costs less than $1.0 million or less than 10% of the value of the aircraft.

The Weighted Number of Accidents (WNA) is thus given by the formula:

Weighted Number of Accidents = the sum of the included accidents’ individual severity weightings.

WNA = (OF1 x 3.0) + (OF2 x 2.5) + (OF3 x 2.0) + (OF4 x 1.0) + (OF5 x 0.25) Airline safety assessment mechanism 20

Examples:

Airline ‘A’ has experienced 12 operational accidents: two of them ‘fatal’ (OF3), ten of them ‘minor’ and no ‘serious’ accidents. Its unweighted number of accidents is simply 12. Its weighted accident score however, is:

WNA = (0 × 3.0) + (0 × 2.5) + (2 × 2.0) + (0 × 1.0) + (10 × 0.25) = 6.5

Airline ‘B’ has also experienced 12 accidents but, in its case, three of them fatal (OF1), three of them fatal (OF2), three are serious and three of them are minor accidents. Its weighted number of accidents is therefore:

WNA = (3 × 3.0) + (3 × 2.5) + (0 × 2.0) + (3 × 1.0) + (3 × 0.25) = 20.25

The WNA of airline ‘B’ therefore reflects the greater severity of its accidents, and this will be reflected in its overall safety evaluation score.

3.4 The ‘additional’ accident Most large airlines have experienced a number of accidents which, depending on their severity, produces a valid and statistically significant accident score, averaging 2.0 per operator. Only 20% of the 125 or so largest airlines averaging over 1,000 flights per week have reported no accidents over the past 10 years.

Many medium-sized airlines, and most small ones, have never reported an accident. Especially in the case of the smaller airlines, this fact is not statistically significant as the expected number of accidents for their scale of operations based on average accident rates and their sectors flown is most likely to be zero or 1 as the average operator score is only 0.44. A notional additional accident for all airlines is therefore included to highlight the sensitivity of their score to such events.

The additional accident is assumed to be a serious one, weighted 1.0, striking a balance between the relatively small impact of mainly minor/no injury accidents scoring 0.25, and the more significant effect of much fewer fatal accidents scoring 2.0 to 3.0. Airline safety assessment mechanism 21

3.5 Effective Accident Rate The Effective Accident Rate (EAR) is taken over the most recent 10 years of airline operation and is defined by the formula:

EAR = {Weighted Number of Accidents + 1} per 100,000 landings

Thus, Effective Accident Rate = • 3.0 times the number of included fatal accidents (OF1), plus • 2.5 times the number of included fatal accidents (OF2), plus • 2.0 times the number of included fatal accidents (OF3), plus • 1.0 times the number of included serious accidents (OF4), plus • 0.25 times the number of included minor accidents (OF5), plus • 1.0 for one additional serious accident), all divided by:

the total number of airline landings over the most recent 10 year period of operations, divided by 100,000, where all accidents occurring prior to the 10-year period are excluded.

The effect of any new accidents may be mitigated by the extent to which any accidents that occurred in the first year of the 10-year period drop out as the 10- year rolling operations period advances. However, at very small scales of operation, the effect of just one single accident on the safety score of the airline concerned can be quite dramatic, which weights the conclusions somewhat against smaller airlines. Startup airlines will be similarly affected. Intuitively this is correct, as smaller and newer airlines will have significantly less operating experience than larger ones, and can be at higher risk during their earlier years of operation.

Effective Accident Rate

(per 100,000 landings)

is defined by the formula

EAR = (WNA+1) / total landings over past 10 years/100,000

where

WNA = Weighted Number of Accidents

All accidents prior to the most recent 10 year period of operations are excluded. Airline safety assessment mechanism 22

3.6 Measuring number of flights Calculation of Effective Accident Rate requires the measurement or estimation of the airlines operations over the preceding 10-year period.

Where an airline has operated for less than 10 years, then the maximum available period of operation should be considered. Care needs to be taken in aligning the operating period with the accident data so that the true accident rate is always calculated.

A number of sources exist for this data (see Appendix C), including: • the airline’s own annual report statistics, which may be available via the Internet directly or via an on-line information service such as Flightglobal and trade journals • figures reported to regulatory authorities and contained in the relevant country’s civil aviation statistics (e.g. UK CAA Airline Statistics) • industry statistics collated by IATA in its World Air Transport Statistics (WATS) or by regional airline trade associations such as the AEA or ATA • national airline operating statistics collated by ICAO, and available via Flightglobal, or regional inter-governmental bodies such as ECAC • Jet Operating Statistics issued by Ascend Worldwide, which gives take-offs and landings for the last 10 years.

As a last resort, the numbers of flights/landings may be estimated based on the number of flights reported in the published Official Airline Guide (OAG) and Innovata schedules, or alternatively based on the currently operating passenger fleet and typical annual utilization rates for the relevant aircraft types. Airline safety assessment mechanism 23

3.7 The ‘Safety factor’ concept A number of ways of taking account of accident rates in the scoring system were considered.

These included having a separate, additive, score-element forming part of the total score, and the use of a multiplier applied to the other score elements.

After discussion in the working group, it was determined that in view of the overwhelming importance of past safety performance as a guide to future safety levels, the use of a safety factor was preferable to a single score element.

This was so because the multiplier approach had the effect of returning very low or even zero scores if the airline safety record is very bad. Depending to some extent on the relative weightings of the score elements, the additive approach still leaves the accident-prone airline with a substantial score.

The factor chosen takes the form of a parabolic sliding-scale factor that reduces in value as accident rate increases, eventually reaching zero at unacceptably high accident rates.

Expressed mathematically the curve is given by the formula:

Safety factor (SF) = {1.0 – 0.2 × the SQRT of the Effective Accident Rate per 100,000 landings}

The safety factors to be applied for a range of Effective Accident Rates, using the scale formula, are illustrated in Table 2.

Table 2: Examples of relationship between EAR and Safety Factor Effective Accident Rate (EAR) Safety factor (SF) Zero 1.000 a 1 0.800 4 0.600 8 0.400 16 0.200 25 or more 0.000 b a A maximum score is not possible in practice because of the effect of the additional accident. b Negative scores are mathematically possible but these are cut off at zero score. Airline safety assessment mechanism 24

4. Airline Factors (AF)

The airline factor component of the overall safety score is intended to reflect those items which are specific to the particular airline under consideration.

Each airline’s score is based upon a total of nine different components each of which are weighted to reflect their relative importance. This gives an overall score for airline factors that can vary between zero and ten depending on the precise mix of values involved.

The resulting airline factor score is subsequently divided by a factor of 1.5 in order to provide two-thirds of the total score. Country factors provide the remaining one- third of the overall score.

The nine airline score factors are described in the following paragraphs.

4.1 Aircraft Fleet Age (AF1)

Potential score Weighting factor Default value 0 to 1.0 2.0 None

Research for the original ASAM mechanism established that older aircraft have poorer levels of safety than those more recently built. The early assessment models used a straight-line relationship between brand-new aircraft and those aged 30 years or more.

More recently and based on substantial number of accidents, it was considered appropriate to apply an AF1 score of 1.0 for aircraft up to 10 years old and zero for aircraft more than 20 years old. Between 10 and 20 years aircraft age the score was reduced in a straight line from 1.0 to zero. However, for this latest ASAM the actual relationships was examined more closely the opportunity has been taken to review the relationship based on an expanded and more up-to-date accident dataset that has been maintained for IOGP over the past 10 years.

The accompanying chart (Figure 3) was produced to show in more detail the results of this examination. It is based on a sample of some 1,340 accidents to operators included at the time in the IOGP database.

Airline safety assessment mechanism 25

Figure 3: Accident rate versus aircraft age

Figure 3 shows the distribution of numbers of accidents by aircraft age at the time of each accident (in red) and the equivalent distribution of flights flown, also by age of aircraft, in dark blue. Dividing the values in the aircraft distribution by the flight distribution produces the green curve. From this green curve, it can be seen that there is a significant degree of correlation between aircraft age and accident rates. The correlation is, of course, imperfect, but is considered sufficiently good for the present purpose.

Inverting the green curve produces the orange line to give an indication of the age score. However, to avoid the need to use a complex mathematical formula, the orange curve has been approximated to a series of straight lines shown in light blue/cyan. Airline safety assessment mechanism 26

The data used to derive Figure 3 is shown in Table 3. The age score line for individual aircraft has been converted into Table 4. Values may be interpolated between integer years for greater precision if desired.

Table 3: Accident analysis by aircraft age Age range Percentage of Percentage of Relative 1.00 / Proposed flights accidents accident rate accident rate score 0 – 5 10.7% 4.2% 0.43% 20.0 1.000 5 – 10 13.1% 7.8% 0.89% 9.70 0.875 10 – 15 12.4% 11.1% 1.48% 5.82 0.750 15 – 20 11.8% 14.0% 2.24% 3.83 0.625 20 – 25 11.2% 15.4% 3.03% 2.84 0.500 25 – 30 10.5% 13.8% 3.45% 2.49 0.375 30 – 35 9.9% 10.8% 3.65% 2.35 0.500 35 – 40 9.2% 8.3% 4.20% 2.04 0.250 40 – 45 5.8% 5.2% 4.82% 1.78 0.000 45 – 50 2.8% 3.4% 6.85% 2.36 0.000 50 – 55 1.4% 2.2% 9.91% 0.87 0.000 55 0.7% 1.5% 15.17% 0.57 0.000 60 plus 0.5% 2.5% 29.81 0.29 0.000 Total/average 100.0% 100% 10.0% 10.0 0.000

The relevant fleet is the then current in-service scheduled passenger fleet over the 10 years; taken as July 2005 to June 2015. Aircraft known to be out-of-service, or used for other purposes, such as cargo were excluded from the fleet calculation. Airline safety assessment mechanism 27

Table 4 may be used to calculate the Aircraft Fleet Age Factor, AF1, for a particular airline, as illustrated by the accompanying example.

Table 4: AF1 Lookup table – Aircraft fleet age factors Aircraft age (years) Age factor 0 1.00 4 0.90 8 0.80 12 0.70 16 0.60 20 0.50 24 0.40 28 0.30 32 0.20 36 0.10 40 plus 0.00

Example: calculation of Aircraft Fleet Age score (AF1)

The age factors for the individual aircraft given in Table 4 are combined to give an average fleet age score (AF1) for the airline in question shown in the following example:

Airline ‘A’ has an in-service passenger fleet comprised of 30 Type A aircraft of 0 to 10 years, with an average age of 6 years, 20 aircraft of Type B between 10 and 20 years old averaging 14 years, and 10 aircraft of Type C over 20 years of age with an average age of 26 years.

The fleet age factor score is derived as shown in the following Table 5.

Table 5: Example derivation of aircraft fleet age score (AF1) Type Number of Average age Age score Age band group aircraft score Type A 30 6.0 0.85 25.50 Type B 20 14.0 0.65 13.00 Type C 10 26.0 0.35 3.50 Total/average 60 12.0 0.70 42.00 Airline safety assessment mechanism 28

4.2 Airline Fleet Composition (AF2)

Potential score Weighting factor Default value 0 to 1.0 1.0 None

An important distinction needs to be made between the average age of a fleet and its relative design age in terms of the design certification standard. The previous age factor, AF1, reflects the average age of the individual aircraft. AF2, however, takes account of the fleet design age:

Aircraft types are categorized into one of several score groups based on the decade of certification as follows: • 2010 onwards scores 1.0 • 2000 to 2010 scores 0.8 • 1990 to 2000 scores 0.6 • 1980 to 1990 scores 0.4 • 1970 to 1980 scores 0.2 • pre-1970 scores 0.0 • aircraft certificated by non-Western authorities are downgraded by one category • aircraft certificated prior to 1990, but which are still in production are upgraded by one category.

Fleet composition values are calculated based on the numbers of aircraft in each type score category. As with Fleet age (section 4.1), the relevant fleet is the current in-service scheduled passenger fleet. Aircraft known to be out-of-service or used for other purposes should again be excluded from the fleet calculation.

Table 6 gives a list of current aircraft types categorized by score groups. The table will need to be further modified by mechanism users as new aircraft types entering scheduled service are identified and classified, and more existing types gradually become outdated and receive lower scores than at present. Airline safety assessment mechanism 29

Table 6: Aircraft Type Scores (AF2) – Individual Aircraft Type Factors 1.00 0.80 0.60 0.40 0.20 Airbus A320neo Airbus A320ceo Airbus A300-600 / Airbus A340-500 / 600 Airbus A300B2 / B4 (A319 / 320 / 321) (318/319/320/321) 600R Airbus A330-700 / Airbus A330-200 / Airbus A380 Airbus A310-200 / 300 Antonov An-32 800 neo 300ceo Airbus A350-800 / ATR42-600 Airbus A340-200 / 300 Antonov An-140 Boeing 747-200 900 / 1000 Boeing 737Max ATR72-600 Antonov AN148 / 158 ATR42-200 / 300 BRAD Dash 7 (-7 / -8 / -9) Boeing 777X Boeing 747-8i ATR42-500 ATR72-200 Fokker F28-4000 Boeing 787 Boeing 777-200LR / ATR72-500 BAE 146 Ilyushin IL-114 (all models) 300ER BRAD CRJ 700 / 900 BRAD CS100 Avro RJ BAE ATP Lockheed L1011 / 1000 BAE Jetstream 31 / BRAD CS300 Embraer E-170 / 175 Boeing 717-200 MDC DC-10 32 / 41 Boeing 737NG Comac C919 Embraer E-190 / 195 Beech 1900 MDC DC-9-50 (600/700 /800 /900) Boeing 737 Classic Embraer E175-E2 Sukhoi SSJ100 Boeing 747-400ER (300 / 400 / 500) Embraer E190/195-E2 Xian MA700 Boeing 777-200/300 Boeing 747-300 Fairchild Metro 23 Irkut MC-21 BRAD CRJ100 / 200 Boeing 747-400 Mitsubishi MRJ BRAD Dash 8-Q400 Boeing 757-200 / 300 Yun-7-100 Boeing 767 (all Dornier 328-100 models) BRAD Dash 8-100 / Dornier 328-300 (Jet) 200 / 300 Embraer ERJ-135 / Casa CN-235 140 / 145 MDC MD-90-30 Embraer EMB-120 Saab 2000 Fokker 50 Xian MA600 Fokker 70 / 100 Ilyushin IL-96 MDC MD-11 MDC MD-80 Series Saab 340A/B Tupolev TU-204 / 214 Tupolev TU-334 Xian MA60 All other older types are considered to score zero, and most such types will have long-since been retired from service. Any other modern types not included above will need to be assessed by mechanism users based on their knowledge and experience. All other transport aircraft are assumed to score zero; this includes obsolete airliners, as well as all-single- engined, piston-engined or un-pressurized aircraft and all aircraft under 15 seats in a normal passenger configuration.

Airline safety assessment mechanism 30

Example: calculation of airline fleet composition score

The factors for the individual aircraft types given above are combined to give an average fleet composition score for the airline in question per the following example:

Airline A’s fleet consists of twenty MD-90-30 aircraft, fifteen 737-300 series, ten 757-200s, ten 737-200 Advanced and five 737-200 non-advanced aircraft. The total fleet (TF) is thus 60 aircraft.

The fleet composition factor is derived in Table 7.

Table 7: Example – Derivation of Fleet Composition Score (AF2) Aircraft type Number Type factor Cumulative type score MD-90-30 20 0.60 12.00 737-300 15 0.40 6.00 757-200 10 0.40 4.00 737-200A 10 0.00 0.00 737-200 5 0.00 0.00 Total/average 60 AF2 = 0.37 22.00 Airline safety assessment mechanism 31

4.3 Aircraft Equipment (AF3)

Potential score Weighting factor Default value 0 to 1.0 1.5 0.50 + 50% of fleet composition score AF2

Scores are applied based on the equipment fit of the airline’s aircraft fleet. Specific items of equipment are given component scores in accordance with Table 8, which are added together to provide a single score factor for the airline. All transport aircraft must carry a basic standard of avionics for communication, navigation and certain additional equipment, as required by the safety regulator for the national airspace concerned.

The minimum level of equipment to be carried is a function of national regulatory requirements for the class of aircraft and its operational capability, as well as the standards for foreign airspace that the aircraft operate in.

Standards differ amongst the various aircraft types, for example between long and short-haul (or domestic versus international-route aircraft), so an average figure needs to be taken across the fleet.

In the absence of adequate information on an airline’s equipment standards suitable default values for the above are taken to be 0.50 plus half of the relevant fleet composition factor (see 4.2 above), as there is a close correlation between the age of a design and the level of equipment carried.

The equipment list will undoubtedly develop, in which case changes will need to be made to the individual equipment factors. For example, one emerging equipment is Head-up Guidance (HGS).

Table 8: AF3 Lookup table – Aircraft equipment factors (AF3) Equipment type Equipment factor CVR 0.05 FDR 0.05 FMS 0.10 FOQA 0.20 EGPWS 0.35 GPWS 0.20 TCAS 0.10 TCAS2 0.25 Maximum possible score 1.00 Note that GPWS and EGPWS are alternative systems, as are TCAS and TCAS2. See Appendix B for an explanation of the abbreviations and other terms use. Airline safety assessment mechanism 32

4.4 Conduct of Operations (AF4)

Potential score Weighting factor Default value 0 to 1.0 3.0 0.50

The score should be assessed based on a number of operations factors. These include the airline’s underlying safety culture and the safety management systems in use. They extend to the standard of the pilot flight and simulator training being conducted, the latter including the application of crew resource management (CRM) and line-oriented flight training (LOFT), and the quality assurance systems in use.

The maximum possible score elements are as recommended in Table 9.

Table 9: AF4 Lookup table – Conduct of operations factors Safety function Maximum operations factor Crew training procedures 0.25 Maintenance procedures 0.20 Quality assurance in operations and engineering 0.25 Safety culture and management 0.30 Maximum possible score 1.00

While, ideally, such an assessment should be based on a full operational, technical and safety audit, in practice this is often not possible, requiring the use of less formal or anecdotal information. If no such information is available, other than by conjecture, then the default value of 0.50 should be applied, giving the subject airline a weighted score of 1.50.

One element becoming more and more important is the IATA Operational Safety Audit (IOSA) Programme. This audit is seen as an additional assurance factor for all airlines and adds value to the IOGP Mechanism by virtue of the fact that a site visit has been made by a team of accredited auditors.

The IOSA Programme is an internationally recognized and accepted evaluation system designed to assess the operational safety management systems of an airline. IOSA uses internationally recognized quality audit principles, and is designed so that audits are conducted in a standardized and consistent manner and renewed every two years. Airline safety assessment mechanism 33

Inherent in the IOSA Programme is a degree of quality, integrity and security such that mutually interested airlines and regulators can all comfortably accept IOSA audit reports. As a result, the industry will be in a position to achieve the benefits of cost-efficiency through a significant reduction in audit redundancy.

With the implementation and international acceptance of IOSA, airlines and regulators have achieved the following benefits: • the establishment of the first internationally recognized operational audit standards • a reduction of costs and audit resource requirements for airlines and regulators • continuous updating of standards to reflect regulatory revisions and the evolution of best practices within the industry • a quality audit programme under the continuing stewardship of IATA • accredited audit organizations with formally trained and qualified auditors • accredited training organizations with structured auditor training courses • a structured audit methodology, including standardized checklists • elimination of audit redundancy through mutual acceptance of audit reports • development of auditor training courses for the airline industry.

Where the airline has passed a full IATA/IOSA audit then it should score 1.0 (un-weighted), except where IOSA audits report are qualified by excluding parts of the operation (e.g. domestic flights) and particular types of aircraft from their scope (mainly FSU and older Western types as well as some regional and business aircraft). Where this is the case, the AF4 score should be reduced to 0.75 (un-weighted).

In prospect for mainly smaller operators, IATA is developing its a simplified audit Standard Safety Assessment scheme (ISSA) mainly for the operators of small aircraft, i.e. those less than 5,700 kg maximum take-off weight that are presently excluded from IOSA audits. ISSA will also provide a one-time audit for operators of larger aircraft as a precursor to eventual full IOSA registration after two years. Airline safety assessment mechanism 34

4.5 Partnerships and alliances (AF5)

Potential score Weighting factor Default value 0 to 1.0 1.0 None

Airlines can benefit commercially and technically from their associations with other airlines, particularly where these are established major players in the industry.

Table 10: AF5 Lookup table – Partnership and alliance factors Safety function Maximum operations factor Significant code-shares 0.333 Technical co-operation 0.333 Strategic alliance 0.333 Maximum possible score 1.000 None of the above 0

Three levels of commercial and/or technical co-operation have been established. Each of them contributing one-third of the overall partnerships’ score, as follows: • the existence of one or more relevant and worthwhile code-share arrangement, such as with a well-established major international airline • the particular airline’s involvement and closer co-operation in technical alliances aimed at improving standards and performance • the airline’s membership of a wider international strategic alliance such as Oneworld, SkyTeam or Star Alliance. Airline safety assessment mechanism 35

4.6 Airline Financial Standing (AF6)

Potential score Weighting factor Default value 0 to 1.0 0.5 Total 0.5

The financial and management score is to be assessed on a range of financial standing and general management factors including the degree to which the airline is commercially aware, the past record of its key managers, the insurance underwriters view in terms of risk and its financial health. In practice this has proved difficult so an alternative method based on recent airline profitability has been devised, as detailed in Table 11, which measures performance over a number of years. Not least among the problems is the absence of reliable financial information except for a minority of larger airlines that publish reliable financial data, so that unless some well-substantiated views on an airline financial status are available, the user may resort to the default value of 0.50.

Table 11: AF6 Lookup table – Airline financial standing factors Financial performance in Typical profit margin Financial factor recent years Good 10% or more 1.00 Fair 5% 0.75 Close to breakeven 0% 0.50 Poor –5% 0.25 Bad –10% or less 0.00 Users may interpolate between the values shown if reliable financial data is available. Airline safety assessment mechanism 36

4.7 Airline Maturity (AF7)

Potential score Weighting factor Default value 0 to 1.0 0.5 None

Established airlines exhibit greater maturity as a result of their greater corporate knowledge and the on-going learning process of management and operational personnel. They should be safer as a result. The learning process is essentially logarithmic, with the greatest gains in the early years of operation, flattening-out after a while, although the possibility of the process being reversed is not excluded. This might occur in times of change such as during mergers and retrenchments, when some of the imparted knowledge may be lost to the airline.

Various maturity periods have been considered. Some early models used a relatively short, straight-line, learning period of five years and other linear and logarithmic timescales running out as long as 30 years were considered. Since use of the logarithmic curves is arithmetically complex, after further consideration, a linear scale was adopted assuming an airline reaches full maturity after ten years. This provides a reasonable representation of a long-term exponential learning curve as shown in the accompanying Figure 4.

Selected maturity curve Typical Learning curve

0.0 0102030 Airline age (years)

Figure 4: Airline maturity model (AF7) Airline safety assessment mechanism 37

The resulting airline maturity scores are as shown in Table 12. Values may be interpolated between integer years for greater precision.

Table 12: AF7 Lookup table – Airline maturity scores Airline age (years) Airline maturity score less than 1 0.00 1 0.10 2 0.20 3 0.30 4 0.40 5 0.50 6 0.60 7 0.70 8 0.80 9 0.90 10 plus 1.00

4.8 Airline Security (AF8)

Potential score Weighting factor Default value 0 to 1.0 0.5 100% of CF5

A number of organizations provide information and advice on airline security issues. Some of these are listed in Appendix C.

The assessment mechanism provides five categories of airline risk ranging from high to low. These are aligned with appropriate airline score factors ranging from zero to 1.00 as shown in the Table 13.

Alternatively default values may be used based on country risk factors (see 5.5).

Table 13: AF8 Lookup table – Airline security scores based on risk categories Airline risk category Airline security score Extreme risk 0.00 High risk 0.25 Medium risk 0.50 Low risk 0.75 Insignificant risk 1.00 Airline safety assessment mechanism 38

5. Country Factors (CF)

The country component of the overall safety score is intended to reflect those items that are specific to the country where the particular airline under consideration is based. Each country’s score is based upon a total of five different factors each weighted in accordance with their relative importance. This gives an overall score for country factors that can vary between zero and five depending on the precise mix of values involved.

The resulting country factor score is divided by the value 1.5, in order to provide one-third of the total score. The airline factors described earlier provide the remaining two-thirds.

The five airline score factors are described in the following paragraphs:

5.1 Regulatory Oversight (CF1)

Potential score Weighting factor Default value 0 to 1.0 0.5 0.50

The original systems from which this mechanism is derived relied on general judgement on the likely standard and effectiveness of safety regulation based largely on historic powers and the involvement of the relevant country’s regulator in setting its own aircraft certification standards. Since then, the number of industry safety assessment programmes has increased significantly, with programmes in place at the FAA, ICAO, IATA, and across Europe.

In 1998, the International Civil Aviation Organization (ICAO) established a universal safety oversight audit programme (USOAP), comprised of regular, mandatory, systematic and harmonized safety audits to be carried out by ICAO on its member states.

Since January 1999, the Safety Oversight Audit Section of the Air Navigation Bureau of ICAO has been conducting safety oversight audits of the civil aviation authorities of member countries in relation to personnel licensing, operation of aircraft, and airworthiness. The audits are designed to determine the status of States' implementation of the critical elements of a safety oversight system and the implementation of relevant ICAO Standards and Recommended Practices (SARPS), associated procedures, guidance material and safety-related practices.

Initially this information was not available publicly and subsequently only became available for a limited number of countries. However, more recently the summary findings have become available online, including measures of the extent of each member country's compliance with ICAO standards across the range of subject headings. Airline safety assessment mechanism 39

The aspects of regulation covered by the USOAP results cover eight areas: Legislation, Organization, Licensing, Operations, Airworthiness, Accident Investigation, Air Navigation Services and Aerodromes. Each national authority is assessed in percentage terms for its compliance across the eight headings and, for the purposes of the ASAM, these are converted to a score value as indicated in Table 14.

Table 14: CF1 Lookup table – Regulatory score based on ICAO USOAP findings ICAO score average Regulatory oversight factor 100% 1.00 50% 0.50 0% 0.00 Scores reduced by 0.25 in the event of any one of an ICAO ‘Red Flag’, an EU Country Ban or an FAA IASA Category 2. Intermediate scores pro rata for all other scores.

The relevant information for the above Table 14 is located at http://www.icao.int/safety/Pages/USOAP-Results.aspx

This information provides an up-to-date evaluation of the effectiveness of 186 member countries compliance with ICAO regulations.

At the time of publishing, nine member countries have no assessments recorded and a few non-members are excluded from the system. Most non-member territories are however covered indirectly by one or other member state, particularly by those of France (French overseas territories), The Netherlands (for the former Netherlands Antilles), United Kingdom for its dependant territories, and the USA for some of its former overseas territories. The authority for one group of nine former UK territories rests with a single Eastern Caribbean CAA based in Antigua.

Overall, the ICAO system provides a sound basis for grading country’s regulatory oversight, at least for those countries with airlines serving the US. However, other ratings may be used to supplement USOAP.

At the time of writing, there is an evolving situation as far as regulatory oversight is concerned. Not all countries have been assessed under the various schemes and, as an alternative to the shown default value, it is also acceptable for the purposes of the ASAM to rely on the individual user’s judgement, for example by analogy with neighbouring countries, or the scores of other directly comparable countries. Airline safety assessment mechanism 40

Other potential sources of guidance include the US FAA's IASA system and the European Commission (EC), which for some years has published ‘banned lists’ of airlines prohibited from entering their airspace. The EC banned list mainly comprises of airlines subject to a country ‘blanket ban’, which excludes all airlines from a particular country, with a few exceptions in some cases. However, such bans may not always relate only to safety issues, and audit by aviation specialists might well permit restricted use. These bans put IOGP member companies in the position that they need to assess the risk where there is the need to fly on a banned airline overseas, when that airline might be going about its normal business in its own country or region despite the EC blacklist.

Where airlines may have multiple regulators, they should attract the lowest scoring regulator’s score.

One disadvantage with the USOAP system is that the review cycle to update the audit findings takes up to ten years. To partially overcome this, ICAO takes account of more recent findings and where it finds a clear risk to air safety, issues a ‘Red Flag’ against the countries concerned.

For the purposes of the ASAM the regulatory scores are to be reduced by a score of 0.25 in the event of a Red Flag being indicated. A similar reduction should be applied in the event of an EC country ban and/or a US IASA Category 2 classification. For the avoidance of doubt, any such reduction should be applied only once to a country; multiple downgrading only confirming that some safety problem exists.

Each IOGP Member is recommended to devise a strategy to minimize the risk from banned airlines abroad by either electing not to use the airline completely (whatever the final score derived from this mechanism), or minimize the risk through selective travel policies limiting passengers to particular aircraft types or routes.

If significant limitation on operations is likely, for example with a country where all airlines are banned, a request to audit by the IOGP Member remains the final option, although a ‘right of audit’ does not exist, the airline might see the sense in permitting access to establish faith in its safety management.

The current EC banned list and FAA IASA country list can be found at: • http://ec.europa.eu/transport/air-ban/pdf/list_en.pdf, and • http://www.faa.gov/safety/programsD;finitiatives/ oversight/iasa/ Airline safety assessment mechanism 41

5.2 National Safety Influences (CF2)

Potential score Weighting factor Default value 0 to 1.0 0.5 Regional default values

This proved to be the most difficult item to assess. The working group discussed wide-ranging possible national approaches to safety at length without satisfactorily resolving the issue. Some of the topics raised were as diverse as crime rates, road traffic accidents, building site safety, taxi drivers, the wearing of seat belts, vehicle driver behaviour generally, and national queuing habits.

The Flight Safety Foundation (FSF) is an international not-for-profit organization dedicated to improving all aspects of flight safety. Based on an assessment of the FSF’s recent findings on regional accident rates, the working group has generated a series of safety influence scores based on regional differences, individual regions and certain sub-regions, as follows. In the absence of deeper knowledge, these may be used as default scores.

Table 15: CF2 Lookup table – National safety influence scores Regions affected Regional accident rate Safety influence score EASA regulated countries, North 0.0 1.0 America and Australasia Eastern Europe (non EASA), Russia, 0.1 0.8 Middle East and South Africa Asia and the Indian Ocean 0.2 0.6 Central Asia, Caucasus, Latin 0.3 0.4 America and the Caribbean Africa, excluding South Africa 0.5 0.0 Airline safety assessment mechanism 42

5.3 Air Traffic Environment (CF3)

Potential score Weighting factor Default value 0 to 1.0 1.25 N

The working group examined the effect of various types of air traffic service on accident risk. This was based on a comprehensive analysis by the Flight Safety Foundation (FSF) of landing and terminal area accidents.

From the FSF analysis, the working group was able to isolate and then combine the increased risk factors due to the absence of various types of services. These risk factors were then weighted according to the affected number of movements in each region to establish an overall risk factor for each main geographic region.

The effect of four types of approach aid or procedure were considered, as follows: • The absence of STARs (Standard terminal arrival routes) • the absence of a visual approach guidance system such as VASIS or PAPIS • the absence of precision approach radar (PAR), and • the use of non-precision approaches.

Western Europe and Asia/Pacific proved to be the least risky regions from an air traffic perspective and Latin America and Africa the riskiest. North America and Eastern Europe were about average. Table 16 shows the risk factors and associated ATC scores for the identified regions.

Table 16: CF3 Lookup table – Air traffic environment score World region Relative risk from lack of air ATC score traffic services Asia-Pacific 30% 1.00 Western Europe 31% 1.00 North America 94% 0.80 Eastern Europe 104% 0.80 Middle East 165% 0.60 Latin America 401% 0.00 Africa 424% 0.00 Average 100% 0.80 Airline safety assessment mechanism 43

5.4 Airfield Environment (CF4)

Potential score Weighting factor Default value 0 to 1.0 1.25 None

The airfield environment score was divided into two elements: the first related to terrain or topography, the second due to climate. The two elements should be added to give a maximum possible airfield environment score of 1.0. a. Terrain (CF4a – Table 17) The FSF study referenced in 4.3 also identified the effect of terrain in the region of airports on relative risk and the working group was able to apply the same process to arrive at the regional risk factors due to terrain.

Table 17: CFa Lookup table World region Relative risk from lack of air Airfield terrain scores traffic services Eastern Europe 14% 0.50 North America 74% 0.40 Western Europe 82% 0.40 Middle East 90% 0.40 Africa 156% 0.20 Asia-Pacific 225% 0.10 Latin America 282% 0.00 Average 100% 0.250 Airline safety assessment mechanism 44

b. Climate (CF4b – Table 18) The Flight Safety Foundation analysis did not cover the effects of climate. However, following discussion within the working group, it was decided to rationalise the various types of climate and their associated weather conditions into just three categories. Individual corporate users should base their weather categories on the freely available sources of climatological information, for example Microsoft™ Encarta World Atlas. Regions with rapidly changing weather associated with unstable air masses and/or the inter-tropical convergence zones, are scored lower than regions with moderate ranges of temperature and low precipitation rates.

Table 18: CF4b Lookup table – Airfield climate scores Type of climate Airfield climate scores Moderate ranges of temperature and precipitation 0.50 Year-round desert or arctic, but clear weather conditions 0.25 Seasonal extremes of temperature and precipitation 0.00

5.5 Country Security (CF5)

Potential score Weighting factor Default value 0 to 1.0 0.5 None

A number of political risk consultants provide information on country security risk factors. Some of these are listed in Appendix C. It is also considered acceptable for users to utilise their own, in-house, country risk assessments where these are available.

The working group examined various country risk scores, which ranged from a few countries graded extreme risk or similar, to a larger number categorized as having minimal or insignificant risk. Scores between zero and 1.0 were allocated to the various risk categories as shown in Table 19).

Table 19: CF5 Lookup table – Country security scores based on nominal categories (CF5) Country risk category Country security score Extreme risk 0.00 High risk 0.25 Medium risk 0.50 Low risk 0.75 Insignificant risk 1.00 Airline safety assessment mechanism 45

6. Implementation of the mechanism

6.1 Calculating airline safety scores The airline safety assessment mechanism described in this report is intended for use by IOGP Members. IOGP is not providing a populated database, so it will be necessary for users of the system either to purchase the data from a supplier or to have the system populated for a selection of airlines appropriate to their operational business requirements. This can be undertaken either in-house or by consultants acting for the member.

Members wishing to use the IOGP mechanism to create their own tool are free to do so. The type of information required to evaluate safety scores is available from a wide range of potential sources relating to airline operations and accidents, airline fleets and other relevant data. Several of these sources are listed in Appendix C. Some of this information is proprietary and involves costs for data access.

Outputs from a specimen spreadsheet model for evaluating safety scores, using the same methodology as the subscription website, is provided in Appendix B. This example model consists of a single Microsoft™ Excel file divided into individual worksheets plus a graphic chart for visualizing the results. Appendix B also shows the overall form of a typical spreadsheet model. The example assessments in Appendix B are intended as a help for members who wish to develop and implement a workable system of their own and, as such, are populated with dummy data.

Appendix B includes: • a results chart (Table B.1) showing indicative comparative overall scores for 15 notional airlines. (The number of airlines to be considered in an actual model will depend on the prospective user’s requirements.) • a corresponding summary worksheet (Table B.2) that identifies the airlines to be analysed, their countries and their world regions, and an overview of the safety scores of the notional airlines • An example Airline Scores’ sheet (Table B.3) that shows a format that could be used for calculating and totalling the weighted individual airline (AF) score elements • A typical Country Scores’ sheet (Table B.4) that could be used for calculating and totalling the weighted individual country score (CF) elements • A Safety Factors worksheet (Table B.5) that could be used to calculate the Safety Multiplier for each airline. Airline safety assessment mechanism 46

Additional knowledge by local company staff of relevant airlines can also be taken into consideration if considered appropriate and companies may want to introduce additional generalized rules in areas such as fleet composition.

All the example worksheets are colour-coded to indicate what happens in each box: • dark blue text on light grey background cells are used for headings • blue text on light blue background is used for data to be input by the user • blue text on a light green background is used for the various weighting factors.

In the example, default values for the individual score elements are also included for use where appropriate.

There are a number of key points to bear in mind when populating the mechanism. These have been highlighted at relevant points in the main text, but are repeated below in Table 20 for ease of reference.

The importance is stressed of maintaining consistency between the aircraft accident data and aircraft flight cycle data used. As far as practicable, these must be for the same class of operations and same time period.

6.2 Safe travel policies The IOGP Airline Safety Assessment Mechanism provides a means for companies to assess the relative risk factors associated with particular airlines. The results on their own do not constitute advice to travel on or to avoid travelling on particular airlines.

The mechanism is intended for use as part of a wider system of establishing safe air travel policies, and it remains the responsibility of individual companies to establish such policies, having regard to over-all travel risks and to the specific air travel risks associated with its business operations.

Normally this would require a graduated approach to the use, or otherwise, of particular airlines based on their safety scores and other relevant information available to the company. Airline safety assessment mechanism 47

A number of decisions are possible based on the safety mechanism results. These might include: • preferential and/or unrestricted use of potentially safer, higher scoring, airlines, possibly incorporating IATA/IOSA registration • more restricted use of moderately scoring airlines, for example where an unrestricted category airline is not available • limited use of low scoring airlines, such as only where required by operational necessity • outright bans on the use of the highest risk, lowest scoring, airlines.

Additional knowledge of relevant airlines by local company staff may also be taken into consideration, if appropriate, and companies may want to introduce additional generalized rules in areas such as fleet composition.

Table 20: Populating the mechanism – Summary Item Description Score range Weighting factor Default values Possible data sources Scoring basis AF1 Aircraft fleet age 0 to 1.0 2.00 None †CASE, ACAS, CAPA Fleets Linear 0 to 40 years old (Table 4) AF2 Fleet composition 0 to 1.0 1.00 None †CASE, ACAS, CAPA Fleets Based on type numbers and scores (Table 6) AF3 Aircraft equipment 0 to 1.0 1.5 0.5 +50% of AF2 Airlines, plus published operating rules Basic values need to be based on in-depth industry knowledge (Table 8) AF4 Conduct of operations 0 to 1.0 3.00 0.75 Ideally based on audit Per Table 9 + 10SA audit Airline safety assessment mechanism AF5 Partnerships and 0 to 1.0 1.00 None †Flightglobal, or trade journals. Three equal elements, per Table 10 alliances AF6 Financial standing and 0 to 1.0 0.50 0.50 Various sources, including airlines' Based on recent profitability per Table 11. Default quality of management Annual Reports and Accounts, IATA and to 0.50 if no good data available. ICAO Financial Statistics, FlightGlobal and, Euromoney Airline Analyst AF7 Airline maturity 0 to 1.0 0.50 None Year of commencing service: Flight Increases linearly from 0 to 10 years old per Global and ACAS CAPA Fleets Table 12 AF8 Airline security 0 to 1.0 0.50 Same as CF5 †Security specialists and consultants Default based on degree of country risk (Table 13) CF1 Regulatory oversight 0 to 1.0 1.50 0.5 ICAO USOAP results. EU banned list or Based primarily on ICAO USOAP compliance FAA IASA Categories gradings converted per Table 14. May be amended to take account of other data, for example EU Banned lists or FAA IOSA Categories. 48 CF2 National safety 0 to 1.0 0.50 By region (see N/A Use FSF safety study values per Table 15 influences section 5.2) CF3 ATC Environment 0 to 1.0 1.25 None N/A See Table 16 CF4 Airfield environment 0 to 1.0 1.25 None See Tables 17 and 18 CF5 Country security 0 to 1.0 0.50 None †Security specialists and consultants Per Table 19 OF1 Fatal accidents Number of 3.00 None †WAAS and/or Aviation Safety Net See definition section 3.3 >20 fatalities occurrences in previous 10 years OF2 Fatal accidents Number of 2.50 None †WAAS and/or Aviation Safety Net See definition section 3.3 >10 ≤20 fatalities occurrences in previous 10 years OF3 Fatal accidents Number of 2.00 None †WAAS and/or Aviation Safety Net See definition section 3.3 1-≤10 fatalities occurrences in previous 10 years OF4 Serious accidents Number of 1.00 None †WAAS and/or Aviation Safety Net See definition section 3.3 occurrences in previous 10 years OF5 Minor accidents Number of 0.25 None †WAAS and/or Aviation Safety Net See definition section 3.3 occurrences in previous 10 years Airline safety assessment mechanism 49

It is for the individual user to choose from these and any other policy guidelines that it might formulate and set the appropriate score levels.

Other factors the user may wish to consider include: • restrictions in the amount of travel • restriction on the use of some parts of an airline operation, for example certain types of aircraft, certain routes • the alternative of chartering an aircraft • the possibility of conducting a safety audit of the airline concerned • the surface transport alternatives, and their associated risks • alternatives to travel such as video-conferencing or changing meeting locations • the numbers of sectors to be flown per trip or in total over a given period.

6.3 Single sector journeys In assessing the scores for airlines operating internationally, the country factors for both departure and arrival countries should be calculated, and the average of the two computed for calculating the airline safety score.

6.4 Multi-sector journeys The mechanism outlined is specifically for comparing the relative risk on single sectors on each of the airlines being considered. However, it should be borne in mind that the risk from scheduled air travel is principally related to the take-off and landing phases of flight and therefore can be approximated based on the number of flights flown by an individual.

When considering a particular scheduled airline trip, it is necessary to consider both the choice of airlines and the number of flight segments involved.

The basic scores relate to the relative risk of a single flight, and cannot simply be combined to assess the overall risk of a multi-sector trip. However, it is possible to combine scores by the following process. • Appropriate accident rates and safety multipliers can be calculated by aggregating the accident occurrences and flight operations of the carriers involved. These should be weighted in proportion to the numbers of sectors to be flown on each airline. • The sum of the relevant airline and country factors are also taken as the weighted averages for the airlines and countries concerned. These are denoted respectively as Combined Airline factor (CAF) and Combined Country factor (CCF). Airline safety assessment mechanism 50

• A new combined safety factor (CSF) can then be derived by adding the Equivalent Accident Rates (EAR) for the sectors. These are based on the weighted number of accidents but modified for all but the final sector to exclude the additional accident (see 3.4). The standard formula for converting accident rates to safety factors is applied (see Figure 5). Although this is not strictly accurate, since it ignores second order effects and includes the additional accident for the last sector only, it is an acceptable approximation. • The combined safety factor is multiplied by the combined airline and country factor (CAF plus CCF) divided by 1.5 to obtain a combined airline score (CAS) for the relevant number of flights. This score may then be compared directly with other single sector or multiple sector journey scores.

The above process is shown as a series of formulae in Figure 5.

Each journey may also be judged according to company policy for single journey airline scores. For example if the minimum score for a single flight is to be taken as 5.0, then any combination of flights scoring less than 5.0 should also be rejected.

Figure 5: Combined score formulae Airline safety assessment mechanism 51

Appendix A. Glossary of terms and abbreviations

Glossary of terms

Aircraft equipment factor (AF3) the airline factor based on the equipment standard of the airline’s in-service passenger fleet

Airfield environment factor (CF4) the country score element relating to the relative regional risk of airports based on equally-weighted terrain and climate factors (CF4a and CF4b)

Airline factor (AF) the total of the weighted airline score elements AF1 to AF8 for an airline

Airline fleet age factor (AF1) Airline factor based on the average age of the airline’s in-service passenger fleet

Airline fleet composition factor (AF2) Airline factor based on the aircraft types in the airline’s in-service passenger fleet

Airline financial standing factor (AF6) the airline score element relating to the financial standing and quality of management of an airline

Airline maturity factor (AF7) the airline score element relating to the longevity of airline operations from zero to a maximum of 10 years

Airline safety score (AS) the overall safety score of an airline derived from its safety factor, airline score and country score

Airline security factor (AF6) the airline score element relating to the level of security afforded by the airline ranging from insignificant to extreme risk Airline safety assessment mechanism 52

Air traffic environment factor (CF3) the country score element relating to the quality of the air traffic services provided in the particular country where an airline is based

Combined airline factor (CAF) the average airline factor (AF) for a series of flights

Combined airline score (CAS) the overall combined score for a series of flights

Combined country factor (CCF) the average country factor (CF) for a series of flights

Combined safety factor (CSF) the combined safety factor applied to a series of flights

Conduct of operations factor (AF4) Airline factor based on an assessment of the airline’s underlying safety culture and the safety management system in use, with particular focus on training and quality assurance

Country security factor (CF5) the country score element relating to the level of security in a particular country ranging from insignificant to extreme risk

Country factor (CF) the total of country score elements CF1 to CF5 for airlines based in that country

Effective accident rate (EAR) the weighted qualifying accident rate per 100,000 landings or flight cycles measured over a maximum 10-year period, allowing for one additional serious accident

Fatal Accidents aircraft accident occurrences involving fatalities to the occupants (passengers or crew) Airline safety assessment mechanism 53

Fleet Age (FA) Average aircraft fleet age since new in years

Minor Accidents non-fatal aircraft accident occurrences involving no serious injuries to the occupants (passengers or crew) and only minor damage to the aircraft

Minor injury an injury sustained by a person in an accident or incident and which: • requires hospitalization for less than 48 hours; or • results in painful injury or strain; or • results in simple fracture of finger, toes, or nose; or results in minor burns.

National safety influences factor (CF2) the country score element relating to specific national safety influences

Occurrence factors (OFI, OF2, OF3, OF4 & OF5) the respective numbers of fatal, serious and minor qualifying operational accidents experienced by an airline

Occurrence score (OF) the sum of occurrence factors OF1 to OF5 – the total of qualifying accidents for an airline over the most recent one-year period

Partnership and alliances factors (AF5) the airline score element relating to the airlines commercial partnerships such as code-shares and its involvement in wider airline alliances

Qualifying accident an accident occurrence that meets the criteria discussed in 3.2 for inclusion as part of an airline’s accident score

Regulatory oversight factor (CFI) the score element relating to the quality of the national regulatory oversight of an airline Airline safety assessment mechanism 54

Safety Factor/Multiplier (SF) the safety score multiplier based on the 10-year average weighted accident-rate

Serious Accident a non-fatal aircraft accident occurrence involving serious injuries to the occupants (passengers or crew) or major damage to the aircraft

Serious injury an injury sustained by a person in an accident and which: • requires hospitalization for more than 48 hours, commencing within seven days from the date the injury was received; or • results in a fracture of any bone (except simple fracture of fingers, toes or nose); or • involves lacerations which cause severe haemorrhage, nerve, muscle or tendon damage; or • involves injury to any internal organ, or loss of sight in one eye, or loss of hearing; or • involves second or third degree burns, or any burns affecting more than 5% of the body surface; or • involves verified exposure to infectious substances or harmful radiation.

Total fleet (TF) Total airline passenger fleet (aircraft numbers)

Weighted Number of Accidents (WNA) the weighted number of qualifying accidents measured over a maximum 10-year operations period Airline safety assessment mechanism 55

Abbreviations AEA Association of European Airlines

ATA Air Transport Association of America

ATC Air Traffic Control

ATI Air Transport Intelligence

ATM Air transport movement (an aircraft arrival or departure)

ATC Air traffic control services

BCAR British Civil Aviation Requirements

CAA Civil Aviation Authority (UK)

CASE Client Aviation System Enquiry from Airclaims Ltd (now Ascend)

CAT Clear air turbulence

CRM Crew resource management

CVR Cockpit voice recorder

ECAC European Conference on Civil Aviation

EGPWS Second generation, enhanced ground proximity warning system

FAA Federal Aviation Administration (USA)

FAR Federal Aviation Regulations

FDR Flight data recorder

FMS Flight management system

FOQA Flight operations quality assurance

FSF Flight Safety Foundation

GPWS First generation Ground Proximity Warning System

IASA International Aviation Safety Assessment programme (by FAA)

IATA International Air Transport Association

ICAO International Civil Aviation Organization Airline safety assessment mechanism 56

IOGP International Association of Oil & Gas Producers

IOSA International Operational Safety Audit (IATA)

ISSA International Standard Safety Assessment (IATA)

ITCZ Inter-tropical Convergence Zone

JAA Joint Aviation Authority

JAR Joint Aviation Regulations

LOFT Line oriented flight raining

NAO National airworthiness organization (generic)

NTSB National Transportation Safety Board (USA)

OAG Official Airline Guide

PAPIS Precision approach path indicators

PAR Precision approach radar

SID Standard instrument departure

STAR Standard terminal arrival route

TCAS First generation traffic alert and collision avoidance system

TCAS 2 Second generation traffic alert and collision avoidance system

USOAP Universal Safety Oversight Audit Programme (ICAO)

VASIS Visual approach slope indicators

WAAS World Aircraft Accident Summary CAP 479 (published by Ascend on behalf of the UK CAA)

WATS World Air Transport Statistics (by IATA) Airline safety assessment mechanism 57

Appendix B. ASAM Spreadsheet Examples

Figure B.1: Results chart

Table B.2: Overall Safety Factor AS

Study Safety Airline Country Overall Airline Name Country World Region Date factor (SF) factor (AF) factor (CF) score (AS) Jul 15 Airline 1 Congo-Brazzaville Africa 0.831 5.30 0.94 5.19 Jul 15 Airline 2 Brazil Latin America 0.935 5.65 1.58 6.77 Jul 15 Airline 3 Colombia Latin America 0.797 4.82 1.52 5.05 Jul 15 Airline 4 France Western Europe 0.000 4.01 3.13 0.00 Jul 15 Airline 5 Bhutan Asia/Pacific 0.780 3.96 2.27 4.86 Jul 15 Airline 6 Bulgaria Eastern Europe 0.579 4.54 2.79 4.25 Jul 15 Airline 7 Ethiopia Africa 0.561 0.83 1.36 1.23 Jul 15 Airline 8 Iran Middle East 0.452 1.33 2.35 1.67 Jul 15 Airline 9 Malaysia Asia/Pacific 0.817 1.50 2.55 3.31 Jul 15 Airline 10 Sudan Africa 0.452 1.20 1.36 1.16 Jul 15 Airline 11 USA North America 0.690 3.91 2.91 4.71 Jul 15 Airline 12 Russia Former Soviet Union 0.123 3.62 2.71 0.78 Jul 15 Airline 13 Kazakhstan Former Soviet Union 0.379 3.82 2.12 2.25 Jul 15 Airline 14 Solomon Islands Australasia 0.690 5.01 2.28 5.03 Jul 15 Airline 15 Myanmar Asia/Pacific 0.916 2.84 2.22 4.63 Airline safety assessment mechanism 58

Table B.3: Airlines Scores AF

Overall Airline Fleet Fleet Equipment Operations Partnerships Finance Maturity Security Airline Name Age Mix Standard Score & Alliances Score Score Score Score Airline AF AF1 AF2 AF3 AF4 AF5 AF6 AF7 AF8 Factor Weightings 10.0 2.0 1.0 1.5 3.0 1.0 0.5 0.5 0.5 Airline 1 7.95 0.85 0.74 0.87 1.00 0.33 0.25 1.00 0.50 Airline 2 8.48 0.90 0.77 0.89 1.00 0.33 1.00 1.00 0.50 Airline 3 7.24 0.54 0.66 0.83 0.75 1.00 0.50 1.00 0.50 Airline 4 6.01 0.83 0.50 0.75 0.75 0.00 0.00 0.20 0.75 Airline 5 5.94 0.37 0.54 0.77 0.75 0.00 0.50 1.00 1.00 Airline 6 6.81 0.58 0.68 0.84 0.75 0.33 0.50 1.00 0.75 Airline 7 1.25 0.28 0.00 0.50 0.50 0.00 0.50 1.00 0.25 Airline 8 2.00 0.53 0.50 0.75 0.75 0.00 0.50 0.40 0.50 Airline 9 2.25 0.53 0.50 0.75 0.75 0.33 0.75 1.00 0.75 Airline 10 1.80 0.79 0.70 0.85 0.90 0.00 0.50 0.40 0.00 Airline 11 5.86 0.66 0.67 0.83 0.50 0.00 0.50 1.00 0.75 Airline 12 5.44 0.66 0.67 0.83 0.50 0.00 0.50 0.40 0.50 Airline 13 5.74 0.66 0.67 0.83 0.50 0.00 0.25 1.00 0.75 Airline 14 7.51 0.87 0.75 0.88 0.75 0.33 0.50 1.00 0.75 Airline 15 4.25 0.13 0.21 0.61 0.63 0.00 0.50 1.00 0.50 Airline safety assessment mechanism 59

Table B.4: Country Scores CF

Country Name Country Regulation Safety Operations Airfield Security World Region (alphabetical) Score Score Influences Score Score Score Country Factor CF CF1 CF2 CF3 CF4 CF5 Weightings 5.0 1.5 0.5 1.25 1.25 0.5 Bhutan Asia/Pacific 3.41 0.41 0.60 1.00 0.60 1.00 Brazil Latin America 2.37 0.86 0.40 0.00 0.50 0.50 Bulgaria Eastern Europe 4.19 0.82 0.80 0.75 1.00 0.75 Colombia Latin America 2.27 0.80 0.40 0.00 0.50 0.50 Congo-Brazzaville Africa 1.42 0.19 0.00 0.00 0.70 0.50 Ethiopia Africa 2.04 0.69 0.00 0.00 0.70 0.25 France Western Europe 4.70 0.96 1.00 1.00 0.90 0.75 Iran Middle East 3.53 0.88 0.80 0.60 0.65 0.50 Kazakhstan Former Soviet Union 3.18 0.90 0.40 0.50 1.00 0.75 Malaysia Asia/Pacific 3.82 0.76 0.60 1.00 0.60 0.75 Myanmar Asia/Pacific 3.32 0.52 0.60 1.00 0.60 0.50 Russia Former Soviet Union 4.07 0.82 0.80 0.75 1.00 0.50 Solomon Islands Australasia 3.42 0.37 1.00 1.00 0.60 0.75 Sudan Africa 2.04 0.78 0.00 0.00 0.70 0.00 USA North America 4.37 0.91 1.00 0.80 0.90 0.75 Airline safety assessment mechanism 60

Table B.5: Safety Multiplier SF

Date Next No. of 10 yr Average Airline Safety Years Commenced Maturity Maturity Accidents Accident Annual Name Multiplier flown Operations Update due Included Severity Flights Used Airline 1 0.831 10 n/a Mature n/a 5 6.75 109,158 Airline 2 0.935 10 n/a Mature n/a 1 0.25 119,770 Airline 3 0.797 10 n/a Mature n/a 1 0.25 12,150 Airline 4 0.000 2 Aug-13 No Aug-15 0 0.00 1,560 Airline 5 0.780 10 Jun-05 Mature n/a 0 0.00 8,251 Airline 6 0.579 10 n/a Mature n/a 1 3.00 9,038 Airline 7 0.561 10 n/a Mature n/a 0 0.00 2,080 Airline 8 0.452 4 May-11 No May-15 0 0.00 3,328 Airline 9 0.817 10 n/a Mature n/a 0 0.00 11,971 Airline 10 0.452 4 May-11 No May-15 0 0.00 3,328 Airline 11 0.690 10 n/a Mature n/a 5 2.00 12,495 Airline 12 0.123 4 Dec-11 Yes Dec-15 0 0.00 1,300 Airline 13 0.379 10 Jun-01 Mature n/a 1 2.00 3,112 Airline 14 0.690 10 n/a Mature n/a 5 2.00 12,495 Airline 15 0.916 10 n/a Mature n/a 1 0.25 71,158 Airline safety assessment mechanism 61

Appendix C. Potential data sources and bibliography

C.1 Accident data and analysis World Aviation Accident Statistics published as CAP479 by Ascend Worldwide, a Flightglobal company, for the UK CAA, World Business Centre 2, 1st Floor, Newall Road, Heathrow Airport, London, TW6 2SF, UK, http://www.ascendworldwide.com

Aircraft Loss Record published by Ascend Limited as part of its on-line aviation database (see contacts above).

Aviation Safety Net (ASN) published on-line by Harro Ranto, The Netherlands at http://aviation-safety.net

Flight Safety Digest published by the Flight Safety Foundation, ISSN 1057 5588, 601, Madison Street, Alexandria, VA 22314, USA, http://www.flightsafety.org

Aviation Disasters (third edition) by David Gero, Published by Haynes Publishing, ISBN 1 85260 602 9, Sparkford, Yeovil BA22 7JJ, UK, http://www.haynes.co.uk

World Directory of Airlines Crashes, by Terry Denham, Published by Patrick Stephens, ISBN 1 85260 554 5, Sparkford, Yeovil BA22 7JJ, UK, http://www.haynes.co.uk

C.2 Airline schedules Innovata, part of Flightglobal, Quadrant House, Sutton, Surrey, SM2 5AS, UK http://www.innovata-llc.com

The Official Airline Guide (OAG), 450 Capability Green, Luton, Bedfordshire, LU1 3LU, UK, http://www.oag.com

C.3 Airline Fleet Data Contained in the on-line database system (formerly CASE) published by Ascend Worldwide (see above).

CAPA Fleets published on-line by CAPA Centre for Aviation, Level 4, Aurora Place, 88 Phillip Street Sydney, NSW 2000, Australia http://centreforaviation.com/data/fleet/

Alternative sources of airline fleet data include

The ACAS database published by Flight-global http://www.flightglobal.com/products/ACAS

JP Airline-Fleets international, ISBN 978 385 758 1410, published annually until 2014 by Flightglobal. Back issues available from: http://www.flightglobalshop.com Airline safety assessment mechanism 62

C.4 Airline statistical data (including airline flight statistics) International Civil Aviation Organization (ICAO) Digest of Statistics, Series T – Traffic - Commercial Air Carriers, Montreal, Canada, http://www.icao.org

International Air Transport Association (IATA), World Air Transport Statistics (WATS) BP 33, Aeroport, Geneva, CH- 1215, Switzerland http://www.iata.org/services/statistics/Pages/index.aspx

C.5 Regulatory information ICAO Universal Safety Oversight Audit Programme http://www.icao.int/safety/Pages/USOAP-Results.aspx

UK Civil Aviation Authority, Safety Regulation Group, Aviation House, Gatwick Airport, West Sussex RH6, UK http://www.caa.co.uk/Safety-initiatives-and-resources/

US Federal Aviation Administration, 800 Independence Avenue, Washington DC 20591, USA. http://www.faa.gov

International Airline Safety Assessments (IASA). http://www.faa.gov/about/initiatives/iasa/

IATA International Operational Safety Assessment http://www.iata.org/whatwedo/safety/audit/iosa/Pages/registry.aspx

The European Commission Banned Airline List http://ec.europa.eu/transport/modes/air/safety/air-ban/search_en.htm

C.6 Security data Own company security department

Ackerman Group, Inc, Risks Forecast Service, 1666 Kennedy Causeway, Suite 506, Miami Beach FL33141, USA. Tel +1 (305) 865 0072, email: [email protected] http://www.ackermangroup.com

Air Data Research. 9865 Tower View, Helotes TX 78023, USA. Tel +1 (210) 695 2204, fax +1 (210) 695 2301, email: [email protected] http://www.airsafety.com Airline safety assessment mechanism 63

The Anvil Group, Vicarage House, 58-60 Kensington Church Street, London W8 4DB. Tel +44 (0)20 7938 4221. http://www.anvilgroup.com

ASI Group - Global Risk Management Services, 2925 Briar Park Drive, Suite 1100, Houston TX 77042, USA. Tel +1 (713) 430 7300, fax +1 (713) 430 7318. http://www.airsecurity.com/aviation.asp

Control Risks Group, Cottons Centre, Cottons Lane, London SE1 2QG, UK. Tel +44 (0)20 7970 2100, email: [email protected] http://www.crg.com

Kroll Associates, 900 Third Avenue, 8th Floor, New York, New York 10022, USA. Tel +1 (212) 593 1000, fax +1 (212) 593 2631 http://www.kroll.com/en-us/security-risk-management

C.7 General airline data Flight International & Airline Business magazines, both published by Reed Business Information. Much of the information they contain is also available as an on-line service provided by Flightglobal. http://www.flightglobal.com

Air Transport World, published by Penton Aviation. 1911 N. Fort Myer Drive, Arlington, VA 22209, USA, Tel +1 (301)755-0200, fax +1 (913)-514-3909. http://www.atwonline.com

C.8 Travel advice and information UK Foreign & Commonwealth Office, London, UK. http://www.fco.gov.uk

Travel Document Systems, 925 15th Street NW, Washington DC 20005, USA, Tel +1 (202) 638 3800, fax +1 (202) 638 4674. https://www.gov.uk/foreign-travel-advice

Central Intelligence Agency, Washington, USA, http://www.cia.gov

US Department of State Travel Alerts and Warnings, Washington, USA. http://travel.state.gov/content/passports/en/alertswarnings/worldwide-caution.html

C.9 Best practice Report 590, Aircraft Management Guidelines, 2016 revision of IOGP report 390, forthcoming http://www.iogp.org/Our-library Registered Office Brussels Office Houston Office City Tower Bd du Souverain,165 16225 Park Ten Place www.iogp.org 40 Basinghall Street 4th Floor Suite 500 14th Floor B-1160 Brussels Houston, Texas 77084 London EC2V 5DE Belgium United States United Kingdom T +44 (0)20 3763 9700 T +32 (0)2 566 9150 T +1 (713) 338 3494 F +44 (0)20 3763 9701 F +32 (0)2 566 9159 [email protected] [email protected] [email protected]

The often unspoken requisite for all travellers is safe arrival at his or her destination on each and every occasion a journey is undertaken. For personal travellers, this is an individual concern but for business travellers it is also a matter for the businesses involved, whether the traveller is a full time employee or a contractor travelling for that business. Under the auspices of the IOGP Aviation Subcommittee, a working group brought together interested parties and specialists working in the field to develop the airline safety assessment mechanism presented in this report.