Quantitative Risk Management

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben SeiteCpt 1. ManfredDeutsche Mueller Lufthansa AG Head of Flight Safety Research Lufthansa AG Lecturer on Risk Management, Univ. Bremen SAMSYS - Safety Performance Index (SPI) Quantitative risk assessment and their protective measures in aviation

More than 25 University Experts

Institute for Production an Logistics GmbH Institute of Aeronautics and Astronautics affiliated institute at the University of Bremen Chair of Flight Guidance & Air Transportation

Section Operation / Safety Project Leader Institute of Strategic Management

Software Development Institute of Flight System Dynamics Institute of Ergonomics Institute of Mathematics 2 Is flying an „ultra-safe“ system ? 40 years ago driving to the airport was the most dangerous part of an Airline-flight Hull Loss per Million Sectors (1958-2011)

12 …last 40 years in Germany: 10 Increase in car-safety ~40 times 8 Increase in flight safety ~4 times 6

4 2

0 1 1960 6 11 16 21 198026 31 36 41 2000 46 51 Risk of driving to the airport in Germany versus an average IATA-Flight (2014)

Pfatal car ride: 7,2 x 10e-8 (1 in ~15 Million)

Pfatal flight: 0,3 x 10e-6 (1 in ~3 Million)

driving to the airport is now 5 times safer than an Ø Airline-Flight

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 6 Deutsche Lufthansa AG >400 cars Why don‘t we see the problems ?

The number of all Airline-flights worldwide per year equals the number of car-trips in Germany within 4 hours 6 years of worldwide Airline flights => 1 day car traffic in Germany:

~5 fatalities in cars per day, (without country roads ~2 fatalities daily)

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 8 Deutsche Lufthansa AG Pilot Manager Ship owner

Passenger Economical pressure

Operational topics after deregulation… every 90 hours

one per month How dangerous are various jobs? Number of fatal accidents per 100.000 workers per year

fisherman 123 window cleaner 10 seaman 68 garbageman 9 railroad worker 31 farmer 8 scaffold builder 30 construction 8 roofer 25 worker lumberjack 20 crane driver 8 docker 19 chemical worker 8 miner 15 Fireman 7 truck driver 15 policeman 7 canal worker 14 metalworker 6 farm hand 12 painter 6 railroadman 11 soldier 6 roadworker 10 electrician 5

University of Oxford Continental CEO Jeff Smisek

(cost saving: worked one year without getting salary)

2010 „Manager of the year“

2004 bis 2009 „Most Admired Global Airline“

„Highest Ranked Network Airline“

increasing economical pressure Continental Airlines

Safety is our top priority: Fligths can stop for extra fuel en route if necessary!

Continental Airlines fuel emergencies

(less than 30 minutes of fuel!) at Newark Airport

2005: 19 2006: 42 2007: 96 Challenge for Flight Operations in the 21st Century

Defining an acceptable level of Safety ALoSP (Limit)

measuring the actual Identifying risk level Deviations (Δ)

Corrective action(s) ALoSP (Acceptable Level of Safety Performance)

How safe should it be?

What should be the safety target for an Airline flight?

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 17 Deutsche Lufthansa AG ALoSP (Acceptable Level of Safety Performance) versus Lufthansa accident rate

LH: ALoSP 10-8 in average one total loss every ~100 years EU: ALoSP 10-7 in average one total loss every ~10 years IATA-Rate: 0,5 x 10-6 in average one total loss every ~2 years

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 18 Deutsche Lufthansa AG Challenge for Flight Operations in the 21st Century

Defining an acceptable level of Safety ALoSP (Limit)

measuring the actual Identifying risk level Deviations (Δ)

Corrective action(s) How dangerous are various jobs? Number of fatal accidents per 100.000 workers per year

fisherman 123 window cleaner 10 seaman 68 garbageman 9 railroad worker 31 farmer 8 scaffold builder 30 construction 8 roofer 25 worker lumberjack 20 crane driver 8 docker 19 chemical worker 8 airline pilot 15 Fireman 7 miner 15 policeman 7 truck driver 15 metalworker 6 canal worker 14 painter 6 farm hand 12 soldier 6 railroadman 11 electrician 5 roadworker 10

University of Oxford How dangerous are various jobs? Number of fatal accidents per 100.000 workers per year

fisherman 123 window cleaner 10 cargo pilot 90 garbageman 9 seaman 68 farmer 8 railroad worker 31 construction 8 scaffold builder 30 worker roofer 25 crane driver 8 lumberjack 20 chemical worker 8 docker 19 Fireman 7 miner 15 policeman 7 truck driver 15 metalworker 6 canal worker 14 painter 6 farm hand 12 soldier 6 railroadman 11 electrician 5 roadworker 10

University of Oxford TOP 10 MOST DANGEROUS US-JOBS IN 2010 (with fatal work injury rate)

1. Fishermen 116,0 2. Logging workers 91,9 3. Airplane pilots 70,6 4. Farmers and ranchers 41,4 5. Mining machine operators 38,7 6. Roofers 32,4 7. Sanitation workers 29,8 8. Truck drivers and delivery workers 21,8 9. Industrial machine workers 20,3 10. Police officers 18,0 Why is this accepted by the authorities ? Safety Managemet System (SMS)

ICAO DOC 9859 / 01.01.2009: …an Airline has to define an acceptable level of safety (ALoSP)

Definition: An acceptable level of safety is a safety level which is acceptable for the respective Airline. (e.g. 10E-5)

Do not rely on oversight authorities! LH-Group:1 per month

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 24 Deutsche Lufthansa AG Aviation Accident Statistic

Kind of Operation

LH-Safety Target

Best Practice

IATA Ø

Risk

10 e-3 10 e-4 10 e-5 10 e-6 10 e-7 10e-8

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 25 Deutsche Lufthansa AG log-scale Aviation Accident Statistic

Kind of Operation

LH-Safety Target

Best Practice

IATA Ø

Business

Risk Glider flying competition 10 e-3 10 e-4 10 e-5 10 e-6 10 e-7 10e-8

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 26 Deutsche Lufthansa AG log-scale Theoretical backgroud

STRUCTURE defines RISK

Prof. Dr. Franz Porzsolt, Univ. Ulm

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 27 Deutsche Lufthansa AG Status Status T A P B

H+ H+

H+

H+ H+

angle deviation < 0,000001°

H+

H+ H+ H+ H+

H+

H+ H+

H+

H+ H+ H+ H+

H+

He++

H+

H+ H+ Transition Probability (PT)

-8 PT = 10 => He++ H+ + H+ hydrogen nucleus Transition Sunlight for Probability 5 x 107 years

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 33 Deutsche Lufthansa AG Quantitative Study

Downhill Skiing approx. 8 Mio. skiers in Germany approx. 80 Mio. skiing days per winter

Skiing Accidents 80.000 severe injuries per winter personal risk 10-3 per skiing day approx. 4 victims daily per skiing area

Transition Probability PT Did your car´s safety belt ever save your life?

TP ~10% of the audience

PT 10e-5: required in 1 out of 100.000 car rides

per person: ~500 rides per year: Ø every 200 years

Error interval: minimum ONE, maximum less than ALL Technische Universität Berlin

Calculation Model

Accident Probability

Accident

Number of flights Incidents correlating Top Event world wide to a Top Event

Incident Probability Transition Probability

Accident ASR DLH ?

Number of flights Lufthansa Incidents DLH SPI corr. to a Top Event Incident Probability Enis Aksu FRA CF Accident Probability DLH Technische Universität Berlin

SPI calculation for a specific aircraft Aviation Data Aircraft Manufacturer Provider Accident Database series and event (Airbus) (OAG, Innovata, (AccidentECCAIRS, ASN etc.) Ascend)

Probability Probability Incident worldwide Accident worldwide PIW = IW / FW PAW = AW / FW Calculation of worldwide Incidents in specific Incidents Flights Accidents Aircraft Series and Event worldwide worldwide worldwide IW FW AW

IW = IA x (FW / FA) x ASF x UCF Source External

Transition Airline Safety Probability Factor PT ASF

Unreported Cases Factor Incidents in Flights in Accidents in Airline UCF Airline Airline (theoretical) ASRIA FA AA = IA x PT

Probability ProbabilityAccident

Incident in Airline Accident in Airline Airline ( Lufthansa ) Airline PIA = IA / FA PAA =DLH AA / FA ? (Lufthansa)

Incidents correlating to a Top Event Mid Air Collision with TCAS Standard Events ASR Datenbank IATA, ANSP‘s, TCAS-OWN etc. PILOT AIRPROX Top Event TCAS-ATC, ACAS ALERT OTHER PILOT LOSS OF SEPARATION MAC AIRPROX (NO NEAR MIDAIR- TCAS) COLLISIONS TCAS FALSE WARNINGS

TCAS NUISANCE OWN

TCAS NUISANCE OTHER

Enis Aksu FRA CF Technische Universität Berlin

Number of TCAS / AIRPROX Events all Lufthansa Fleets

120 111 107 100 99 94 97 80 80 80

60 TCAS-OWN PILOT 61 62 TCAS-ATC,OTHER PILOT 40 AIRPROX (NO TCAS)

20 20 18 19 17 14 10 12 11 8 7 8 67 58 0 3 2 3 2002 2003 2004 2005 2006 2007 2008 2009 2010

Enis Aksu FRA CF Technische Universität Berlin

Mid Air Collision Risk

1 Accident

23.260.877 18.609

Honeywell Study -4 8,0 x 10 -5 TP 5,4 x 10 Airline Safety Factor x 2,4

ASR Accident 208 DLH ? ≈ 80 % 729.590 -8 240 SPI = 1,9 x 10 3,3 x 10-4 Enis Aksu FRA CF Accident Probability DLH Evidence Based Risk Management

Every Flight is confronted with a number of various threats. The summ of all different threats has to be <10-8. The probability of a single threat has to be significantly lower than10-8. How can we calculate the Total Risk? Technische Universität München

Overall Accident Risk of an Airline Flight

• Failure Scenarios: , ,…,

• Overall Failure Probability𝐹𝐹1 𝐹𝐹2 : 𝐹𝐹𝑛𝑛 Pr = Pr

= 𝐹𝐹1 Pr 𝐹𝐹1 ∪ 𝐹𝐹2 ∪ ⋯ ∪ 𝐹𝐹𝑛𝑛 Independent Risks: − 𝐹𝐹1 ∩ 𝐹𝐹2 ∩ ⋯ ∩ 𝐹𝐹𝑛𝑛 = 1 1 Pr 1 Pr … 1 Pr

− −Pr 𝐹𝐹1 + Pr− +𝐹𝐹2 Pr( −) 𝐹𝐹𝑛𝑛 Approximation for Aviation≈ 𝐹𝐹Risks1 : Pr𝐹𝐹2F =⋯Pr 𝐹𝐹𝑛𝑛+ Pr + Pr( ) 𝐹𝐹1 𝐹𝐹2 ⋯ 𝐹𝐹𝑛𝑛 LH-ALoSP Risk Addition

-8 RG ~ 10 Σ

-9 -9 -9 -9 RA ~ 10 RA ~ 10 RA ~ 10 RA ~ 10

Σ e.g. 10 Groups (1st Sub level)

-10 -10 -10 -10 -10 RB ~ 10 RB ~ 10 RB ~ 10 RB ~ 10 RB ~ 10 e.g. 10 Sub-Groups (2nd level)

Overall risk = Sum of all single risk factors (~100)

Prof. Dr. Daniel Straub Institute for Risk Analysis Technical University Munich

03.11.2011 SaMSys LH-ALoSP Risk Addition

-8 RG ~ 10 Σ

-9 -9 -9 -9 RA ~ 10 RA ~ 10 RA ~ 10 RA ~ 10

Σ e.g. 10 Groups (1st Sub level)

-10 -10 -10 -10 -10 RB ~ 10 RB ~ 10 RB ~ 10 RB ~ 10 RB ~ 10 e.g. 10 Sub-Groups (2nd level)

e.g.: P enginge failure on final ~10-7 accepted failure rate (Human): ~10-3

This failure rate (Human) can be influenced by Procedures / 03.11.2011 Selection / TrainingSaMSys / Time (Fuel) Risk Calculation

P Engine Failure on final: 10-7

Acceptable Failure rate (SIM): 10-3 (1 out of 1.000 exercises)

Safety Target (2nd Sub-Level): 10-10

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 46 Deutsche Lufthansa AG Limit: 10E - (Safety Target + 2) SaMSys

10e-10

10e-8 10e-9

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 48 Deutsche Lufthansa AG

10e-9

10e-10 10e-8 10e-9

10e-10 10e-8 Accident-Category: Suicide by Airline-Pilot

Date Airline Fatalities

21.08.1994 Royal Air Maroc 44 Flight 630 19.12.1997 Silk Air 104 Flight 185 31.10.1999 Egypt Air 217 Flight 990 29.11.2013 LAM Mocambique 33 Flight 470 24.03.2015 German Wings 150 Flight 9525

PT Flight / Pilot Suicide: ~0,8 x 10E-8

Approach during a thunderstorm

TP :~10E-5

Traffic flow is not interrupted: ALoSP: 10E-5 (ICAO)

If 20% (+/-10%) of the flights continue to land: IATA-average (80% do not land)

EASA-ALoSP: ~2 (+/-1)% land (99% do NOT land)

LH-ALoSP: ~0,2 (+/- 0,1)% land (99,9% do NOT land)

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 55 Deutsche Lufthansa AG Challenge for Flight Operations in the 21st Century

Defining an acceptable level of Safety ALoSP (Limit)

measuring the actual Identifying risk level Deviations (Δ)

Corrective action(s) Lehrstuhl für Ergonomie Technische Universität München

Simulatorstudie zu manuellen Flugfertigkeiten

Flight data recording traffic scenario

Eye-tracking and communication recording simulator control performance analysis Lehrstuhl für Ergonomie Technische Universität München

The Team • 11 Simulator Technicians and Engineers (Samsys I + II) • 8 Scheduling • 7 Fleet & Trainings Management • 192 Pilots (Captains and First Officers A346 and A320) • 12 Specialists for Simulator Ops and Data Recording • 4 Union • 5 DLR-Specialists • 5 Simulator Scheduling • 6 Lufthansa Flight Training • 14 Meeting-Preparations / Catering Sum: 262 Persons

SAMSYS i + II Simulator Studies Lehrstuhl für Ergonomie Technische Universität München

NO volunteers

Randomised Study

Simulatorstudie zu manuallen Flugfertigkeiten Traffic Scenario „Eurotrans „Lufthansa zero seven“ seven two seven“

„Quality nine three kilo“

„Continental „Whitestar one ooh six“ one eight“

„Lufthansa seven three one“

„Lufthansa one seven eight niner“

„Singapore realistic three two eight“ ATC environment Task Related Probability Of Errors MTBFs Prof. Bubb TU-Munich

CategoryKategorie ErrorFehler probability MTBFMTBF

Simple and regularely 1 . 10-3 performed tasks at a low ~30 min stress level.

Complex, regularely 1 ∙ 10-2 ~5 min performed tasks in a well known working environment at a low stress level. Complex tasks in unusual 1 ∙ 10-1 ~30 sec situations at a high stress level and / or time pressure. Flight Safety Research looked at the following aspects

Pilot Selection concept Validation Basic Piloting Skills Raw data approaches Landing technique Go-around handling Side stick handling Handling of non-standard / complex Abnormals in a realistic environment CRM / Team Interaction Fuel Status Flight Safety Research

Basic Piloting Skills

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 66 Deutsche Lufthansa AG Lehrstuhl für Ergonomie Technische Universität München 2. manual Flying

Capt Jacques Drappier, Airbus vice-president training, 17. September 2009:

“We need to refocus on basic handling…” actually < 2h / year “…long-haul pilots typically log 800-900 flying hours a year, although this could include less than 3h of stick time, the majority of which is accumulated on final approach and flare.”

“We put people into our training today who have forgotten how to fly, basically,”

But (same presentation):

Airbus expects pilot type conversion training to take five days between the A380 and the A350, compared with 10 days from the A330/A340 family to the A350 and 11 days from the A320.

Simulatorstudie zu manuallen Flugfertigkeiten Lehrstuhl für Ergonomie Technische Universität München manual Flying

Many accidents occure during flight phases requiring manual flying skills (2010):

Simulatorstudie zu manuallen Flugfertigkeiten

Technische Universität Berlin Datarecording (A340 & A320 FFS)

4 x 10 Topansicht 20

XERUM Raw data approaches

15 BURAM

A340-600 A320

10 WLD

MIQ Y [ft] Y

DM429

0 DM431 MAGAT EDDM 26R GUDEG

-5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 X [ft] 5 71x 10 MaximaleMaximum Abweichungen Localizer (LOC)vom Localizer Deviation (LOC) betweenzwischen 3000ft 3.000ft und and1000ft 1.000ft AGL AGL

Ablage LOC in Dots -2.0 -1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0

CPTs

F/Os MaximaleMaximum Abweichungen Glideslope vom (GS) Glideslope Deviation (GS) betweenzwischen 3000ft3.000ft und and1000ft 1.000ft AGL AGL 2.0 CPTs F/Os

1.5

1.0

0.5

0.0

Ablage in Ablage GS Dots -0.5

-1.0

-1.5

-2.0 Lehrstuhl für Ergonomie Technische Universität München

Manual Flying Skills

recurrent training and basic flying skills 74 Lehrstuhl für Ergonomie Technische Universität München

Manual Flying Skills

Risk > 10E-8

FO A320 > CPT A320 > FO A340 > CPT A340

recurrent training and basic flying skills 75 Lehrstuhl für Ergonomie Technische Universität München Correlation SIM-Study DLR-Test (CP A346) 54% 9 (8) 47% (5) Stanine 8 56% Psychomotoric 39% (4) 55% 7 53% 6 42% 47% 41% DLR 5 German (6) Air and Space 37% Center 4 (6)

1 6 5 4 3 2 grade 1 Risk: >10-6SimulatorstudieSimulator Study: zu manuellen manual flyingFlugfertigkeiten skills <10-7 Lehrstuhl für Ergonomie Technische Universität München Correlation SIM-Study DLR-Test (CP A346) 54% 9 (8) 47% (5) Stanine 8 56% Psychomotoric 39% (4) 55% 7 53% Correlation6 r = -0,87 42% 47% 41% DLR 5 German (6) Air and Space 37% Center 4 (6)

(4) 33% 3

2 (ne) 33% (ne) 1 31% (ne) (ne) 6 5 4 3 2 grade 1 Risk: >10-6SimulatorstudieSimulator Study: zu manuellen manual flyingFlugfertigkeiten skills <10-7 Lehrstuhl für Ergonomie Technische Universität München Correlation SIM-Study DLR-Test (CP A346) 9 54% Training ! (8) 35% (6) 47% (5) Stanine 8 56% (5) Psychomotoric Training ! 39% (4) 55% 7 53% 6 50% 42% 47% Training ! 41% DLR 5 German (6) Air and Space 37% Center 4 (6)

(4) 33% 3

2 (ne) 33% (ne) 1 31% (ne) (ne) 6 5 4 3 2 grade 1 Risk: >10-6SimulatorstudieSimulator Study: zu manuellen manual flyingFlugfertigkeiten skills <10-7

8 8 8

Speed-Checks

Speed / Speed/vert.SpeedVertical Speed- ChecksChecks unter below 100ft 100ft RA 100% CPT FO 90%

80% der Piloten der 70% F/O A320 Anteil Anteil 60% F/O A320

50%

40%

64.0% 30% 60.0%

20% CPT A346

10% 19.0% CPT A346 9.5% 0% Speed überprüft zwischen 100ft und TD vert. Speed überprüft zwischen 100ft und TD Speed checked below 100ft and TD V/S checked below 100ft and TD SaMSys Simulator Experiments

Measuring… Economical Flight Performance FMS Mode Awareness / Automation Handling unclear situations (Warsawa) Redundancy and team-interaction Go-Around-decision and -performance Basic Flying Skills (AI and Boeing) Predictability of pilot performance (DLR-test)

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 87 Deutsche Lufthansa AG

Statistical Evaluation of Flight Operation 89 Statistics – Calculating Probabilities Runway Overrun Probabilities

Combinations / Path Probability Rank Conditional Overrun Total Overrun Probability Probability Rank Speed not in limits AND Go- 2.61 E-2 1 4.0 E-5 Lognormal 1.04 E-6 1 Around failed Tailwind not in limits AND Go- 2.07 E-5 5 6.0 E-4 GEV 1.2 E-8 2 Around failed Tailwind not in limits AND 6.00 E-7 8 6.0 E-3 GEV 3.6 E-9 3 Speed not in limits AND Go-Around failed A/SKID Fault AND 2.8 E-4 2 <1.0 E-16 GEV 2.8 E-20 6 Recognition failed Dual BSCU Fault AND 3.4 E-4 3 <1.0 E-16 GEV 3.4 E-20 7 Recognition failed (Dual BSCU Fault OR 6.2 E-4 4 <1.0 E-16 GEV 6.2 E- 20 8 A/SKID Fault) AND Recognition failed All Brakes Inoperative 6.4 E-9 7 <1.0 E- 16 GEV 6.4 E-25 9 Spoiler not deployed 1.0 E-7 6 2.4 E-4 Gamma 2.4 E-11 4 (ARP 4761 PSSA) Spoiler not deployed AND 1.0 E-7 * 5.0E-3* 9 1.5 E-4 GEV 1.71 E-17 5 Reverser Fault AND A/BRK 2.27 E- 4 Fault (Normal Wheel Brake = 1.14E-13 Fault)

Simulation of Flight Operation Test Scenario 2

• The tower reported wind is within limits. However the actual surface wind is beyond the limit (18Kts instead of max. 10Kts)

• A Go-Around should be performed!

• In 70ft a Go-Around is triggered by the PM

• Runway change after the Go-Around.

Simulator Study: Manual Pilot Skills Technische Universität Berlin Decision Making Analysis

100 93,3% 90 85,2% 80 70 60 50 A340 40 A320 30 relative Häufigkeit [%] 20 14,8% 10 6,7% 0 yesJa Neinno

(%) of Pilots observing the TW during approach above 1000ft AGL

Masterarbeit Alexander Schmidt -91- Technische Universität Berlin Decision Making Analysis

60 51,9% 51,9% 48,1% 48,1% 50

40 A340 30 A320 20 relative Häufigkeit [%] 10

0 Jayes Neinno

(%) ofBeachten Pilots observing die VPen theden TW RW during im Anflug approach unter 1000ftbelow AGL?1000ft AGL

Masterarbeit Alexander Schmidt Technische Universität Berlin Decision Making Analysis

80 70,4% 70 59,3% 60 50 40,7% 40 A340 29,6% A320 30 20 relative Häufigkeit [%] 10 0 yesJa noNein

(%) of PilotsVP wäre who ohne would Windhinweis have landed des without PM gelandet the PM callout

Masterarbeit Alexander Schmidt Technische Universität Berlin Go-Around Anslysis (MSc of Alexander Schmidt)

39% of all pilots discussed the tailwind situation during the approach briefing.

89% of all pilots talked about the strong tailwind during approach above 1000ft AGL. 50% of all pilots checked the wind indication during final approach below 1000ft AGL. 64% of all pilots would have landed without the wind information given by the PM (TW of 18 Kts).

14% of all pilots decided to land, knowing the tail wind was 18Kts. 44% of all A340 pilots performed the Go-Around only after the PM gave the command in ~70ft AGL.

94 Calculations: R ~10E-8 Is this result acceptable ? …according: ICAO IATA EASA LH Calculations: R ~10E-8 Is this result acceptable ? …according: ICAO OK IATA OK EASA CRITICAL LH NOT ACCEPTABLE SaMSys Simulator Experiments

Measuring… Economical Flight Performance FMS Mode Awareness / Automation Handling unclear situations (Warsawa) Redundancy and team-interaction Handling the Go-Around Basic Flying Skills (AI and Boeing) Predictability of pilot performance (DLR-test)

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 97 Deutsche Lufthansa AG Technische Universität Berlin Go-Around Analysis (MSc of Alexander Schmidt)

a) 1 A340-600 captain (3,7%) performs the Go-Around correctly. 10 A320-F/O (33%) perform the Go-Around correctly. b) In general more than one SOP-violation per Go-Around Average: A340-600 Cpt.: 2, A320 F/O: 1,3. c) Common SOP violations: • „Missed Appr. Proc.“ not correct („NAV-Mode“ not activated). • Flaps not operated according SOP. • Thrust Reduction to late and/or MCT is selected instead of CLB thrust. • Vertical speed is not reduced during the last 1000ft before reaching Go-Aroud Altitude.

98 Fligth Recorder Data Analysis

. Only 5% of all Go-Arounds are flown correctly.

. In average 2,7 SOP-violations can be observed per Go-Around.

. Analysis of 150 Go-Arounds showed 415 SOP-violations.

. Maximum number of violations per Go-Around: 10.

Go Around Campaign FRA CF, Annina Müller 23. Februar 2010 Page 99 SPI calculation for a specific aircraft Aviation Data Go-Around Aircraft Manufacturer Provider Accident Database series and event (Airbus) (OAG, Innovata, (ECCAIRS, ASN etc.) Ascend)

IW = IA x (FW / FA) x ASF x UCF Source External

Transition Airline Safety Probability-6 Factor ~10 ~1 PT ASF

Unreported Cases Factor Incidents in Flights in Accidents in Airline UCF Airline Airline Lufthansa(theoretical) IA FA AA = IA x PT

Probability Probability Incident~10 in Airline-2 Accident~10 in -Airline8 Airline ( Lufthansa ) Airline PIA = IA / FA PAA = AA / FA (Lufthansa)

Go Around Campaign FRA CF, Annina Müller © Daniel Schoolmann 23. Februar 2010 Page 100 100 Calculations: R ~10E-8 Is this result acceptable ? …according: ICAO IATA EASA LH Calculations: R ~10E-8 Is this result acceptable ? …according: ICAO OK IATA OK EASA CRITICAL LH NOT ACCEPTABLE …2014: B767 Go Around: Technische Universität Berlin

16 130

14 110

12 90

10 70 Pitch[˚] 8 50

6 Pitch-Up Illusion 30 Geschwindigkeitszunahme [kt] Geschwindigkeitszunahme

4 10

2 -10 0 5 10 15 20 25 30 Zeit [sec]

VP A VP B VP C VP D Pitch Referenz im GA

Pitch Geschwindigkeitszunahme

Pitch / speed second go-around (A340)

Masterarbeit Alexander Schmidt -104- Exceeding max. flap speed causing flaps locked

Flight to the Alternate

Landing with less than < 5 min. remaining flight time Flight Safety Research

Handling of non-standard / complex Abnormals in a realistic environment

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 106 Deutsche Lufthansa AG 54 ECAM Warnings

November 4th, 2010 Qantas Flight 32

Multiple Failures B 747- 400 SIN-FRA EICAS Caution and Advisory Messages:

DET FIRE/OVHT 1 HYD OVHT SYS 1 BLEED 1 OVHT BLEED HP ENG 1 BLD 1 OVHT/PRV ENG 1 OVHT LP A ENG 1 OVHT LP B ENG 1 FIRE LP A ENG 1 FIRE LP B OVHT ENG 1 NAC FUEL SPAR VLV 1 ENG 1 VIB ..... 4 Pilots in the Cockpit Time versus Decision-Making

Start of the scanario: Consequence: Airports: New York or remaining flight time ~40min. Nice (~80% fuel flow increse) 1h remaining fuel

Malfunction Probability: ~1 in 10 Million (10E-7)

Event: Hydraulic fluid loss (>10-5) => landing gear problem NOSEGEAR UNLOCKED During the second Approach Go Around SLATS LOCKED

SAMSYS II Versuchsreihe | E. Schubert | Institut für Luft- und Raumfahrt | Fachgebiet Flugführung und Luftverkehr Seite 114 Simulator Study – Set up No. 2 (A340-600)

Nose Landing Gear unlocked Slats locked GREEN HYD lost abort the approach, go around, determine landing conf. gravity gear extension calculate LDR

JFK Carnarsie-VOR Appr.: Arrival Fuel >7,500 kg ~1h+

IATA OPS-Conference | Kuala Lumpur April 1st, 2014 FRA CF/J, SaMSys A320 A320 Traffic Scenario „Eurotrans „Lufthansa zero seven“ seven two seven“

„Quality nine three kilo“

„Continental „Whitestar one ooh six“ one eight“

„Lufthansa seven three one“

„Lufthansa one seven eight niner“

„Singapore realistic three two eight“ ATC environment Technische Universität Berlin Datarecording (A340 & A320 FFS)

4 x 10 Topansicht 20

XERUM Raw data approaches

15 BURAM

A340-600 A320

10 WLD

MIQ Y [ft] Y

DM429

0 DM431 MAGAT EDDM 26R GUDEG

-5 -2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 X [ft] 5 119x 10 30x A340 Crews – Flight Tracks KJFK

Simulator Study II A346 30x A320 Crews – Flight Tracks LFMN

Simulator Study II A320 Workload versus Performance

1 4

• mean rating: medium workload condition of M = 3.04, SD = .82 high workload condition (time pressure) M = 2.71, SD = .85 • Time pressure leads to significant reduced performance, t(58) = 5.6, p ~ .001.

Error indicator: standard error

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 122 Deutsche Lufthansa AG Performance under Stress

performance

Σ PM

workload

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 123 Deutsche Lufthansa AG During G/A: Inefficient and disturbing callout: “SPEED“ Supporting callout: “INCREASE PITCH + SET CLB THR“ Findings regarding „10E-8“

10% (6) of these experiments ended in an accident. (NO accident in this scenario during normal recurrent training with >1.000 pilots!)

3% (2) of these experiments ended in a fatal accident.

All accidents were triggered by wrong PM-inputs.

PScenario >10E-8 x Error Rate ~10E-1 =>

accident probability RScenario >10E-9 (Limit exceeded!)

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 125 Deutsche Lufthansa AG Calculations: R >10E-9 Is this result acceptable ? …according: ICAO IATA EASA LH Calculations: R >10E-9 Is this result acceptable ? …according: ICAO OK IATA OK EASA CRITICAL LH NOT ACCEPTABLE Challenge for Flight Operations in the 21st Century

Defining an acceptable level of Safety ALoSP (Limit)

measuring the actual Identifying risk level Deviations (Δ)

Corrective action(s) Risk

Errors

Crew

Threats Threats Threats

Ground Technic Flight Training Ops Ops Selection

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 129 Deutsche Lufthansa AG Findings regarding „10E-8“

Standard Airbus (or Boeing) Type-Rating Courses are not sufficient for an ALoSP of „10E-8“. Manual / Basic Flying Skills have to be improved:  Sidestick Handling  Raw Data Flying  A346-Landing Technique  X-Wind Handling  Go Around Handling

Complex abnormals are not handled sufficiently well under time pressure. PMs showed deficiencies in supporting / controlling the PF.

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 130 Deutsche Lufthansa AG Recommendations to achieve „10E-8“

Additional training is required (Type Rating): Sidestick-handling & manual flying skills / raw data / landing technique / x-wind / go around.

More than 4 x 4 hours Simulator Training per year is required (basic piloting skills including APP & LDG / GA / failure analysis).

Pilot Monitoring Performance has to be improved (support & controlling).

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 131 Deutsche Lufthansa AG Recommendations to achieve „10E-8“

Ambigious / complex malfuctions have to be trained in the simulator under realistic conditions (ATC).

Fallback Strategies for complex malfuctions are required.

A procedure TIME CRITICAL LANDING is required.

Minimum fuel at landing 45 minutes.

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 132 Deutsche Lufthansa AG 30x A340 Crews – Flapsetting und Malfunctions A320 A320

133 SAMSYS II Remaining Fuel A346 (600kg for Final APP, additional fuel calculation with 180kg/min)

Run VPId Min[kg] Time[min] Run VPId Min[kg] Time[min] Additional 7 4C10 4410 26 7 4C10 2810 17 Pattern 33 4C29 2649 14 33 4C29 1049 7 3 4C08 2380 14 3 4C08 680 5 possible 54 4C53 2161 13 54 4C53 561 4 13 4C16 1986 12 13 4C16 386 3 56 4C54 Additional1918 12 56 4C54 318 2 17 4C18 1809 12 17 4C18 209 1 59 4F58 1738Pattern11 59 4F58 138 1 21 4F23 1719 11 21 4F23 119 1 23 4C24 possible1460 10 23 4C24 0 0 Est. Fuel at T/D 5 4F09 1445 10 5 4F09 0 0 Est. Fuel at T/D 39 4C44 1408 10 39 4C44 0 0 KJFK: 49 4F48 1383 10 49 4F48 0 0 KJFK: 9 4C11 1380 10 9 4C11 0 0 15 4F17 1224 9 15 4F17 0 0 45min. 25 4F25 1189 7 25 4F25 0 0 30min. 31 4C28 1172 7 31 4C28 0 0 27 4F26 1131 7 27 4F26 0 0 61 4F59 1111 7 61 4F59 0 0 29 4F27 1037 7 29 4F27 0 0 46 4C47 930 6 46 4C47 0 0 41 4F45 747 5 41 4F45 0 0 19 4C22 711 5 19 4C22 0 0 1 4F07 677 5 1 4F07 0 0 11 4C12 612 3 11 4C12 0 0 52 4C52 528 3 52 4C52 0 0 43 4C46 320 2 43 4C46 0 0 64 4C60 169 1 64 4C60 0 0 35 4F30 130 1 35 4F30 0 0 37 4C43 0 0 37 4C43 0 0

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 134 Deutsche Lufthansa AG Recommendations to achieve „10E-8“

Prerequisite for the additonal training is a positive, relaxed and friendly learning atmosphere.

Mistakes and errors have to be expected, they are „normal“ and an important fundament for performance improvement.

A simulator session with complex and demanding content cannot be failed!

…required for an ALoSP beyond „10E-7“

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 135 Deutsche Lufthansa AG …there are still „low hanging fruits“: Evidence Based Risk Management

Diagnostically conclusive Pilot selection should be mandatory

Improved initial training and type rating Increased and realistic recurrent training

Min. 45 minutes fuel at Landing

25.07.2016 Bitte diesen Text mit dem Präsentationstitel überschreiben Seite 137 Deutsche Lufthansa AG