ARP 4761 and STPA (Using the Wheel Brake Example in ARP 4761)

SERL

ARP 4761 and STPA (using the Wheel Brake Example in ARP 4761)

Cody Fleming

March 27, 2014

SYSTEMS ENGINEERING RESEARCH LABORATORY Goals of this Talk SERL 1. How does regulation work in aviation? – ARP 4761, others

2. What are the objectives of 4761? – What methods, outputs, processes does it require?

3. Can STPA satisfy the 4761 objectives?

4. What is necessary for #3 to happen? – Do we have to re-write 4761, do we have to modify STPA, are they already compatible?

1 Agenda SERL • ARP4761 Process • ARP4761 Application • STPA Results • 4761 and STPA • Future

2 ARP 4761 SERL

• What is ARP 4761???

• “Describes guidelines and methods of performing safety assessment for certification of civil aircraft” [SAE 1996]

3 Development & Certification Process SERL

Development and Design Guidance [ARP-4754A, 2010]

Guidelines and methods for Safety Assessment [ARP-4761, 1996]

Software Design Guidance Considerations Electronic Hardware [DO-178, 2011] Integrated Modular [DO-254, 2000] Avionics [DO-297, 2005] 4 Safety Assessment Elements SERL • Functions, Design Constraints, Reqs, …

• Functional Hazard Assessment • Identify failure, error conditions according to FHA severity • Aircraft level & System level

• Preliminary System Safety Assessment PSSA • Complete failure conditions list • Generate safety requirements

• System Safety Assessment SSA • Comprehensive analysis of implementation 5 Development Assurance Levels SERL

Per flight Hour Probability 1.0 1.0E-3 1.0E-5 1.0E-7 1.0E-9 (Quantitative) Probability Frequent Reasonably Remote Extremely Extremely (Descriptive) Probable Remote Improbable Failure Condition Minor Major Sever Major Catastrophic

Severity Minor Major Hazardous Catastrophic Classification Failure Cond. - slight reduction in - significant - large - all failure Effect safety margins reduction in reduction in conditions - slight increase in crew safety safety which workload margins or margins or prevent - some inconvenience to functional functional continued occupants capabilities capabilities safe flight - … - … Development Level D Level C Level B Level A Assurance Level

6 4761 Basics SERL • PRA – Some requirements leveled in terms of probabilities – Not all requirements are leveled in terms of Pe • E.g. software assumed as Pe=0 • Level A failures cannot be argued probabilistically

• Methods – FTA, FMEA – Zonal, CCA, DD, MA

7 Agenda SERL • ARP4761 Process • ARP4761 Application • STPA Results • 4761 and STPA • Future

8 4761 – Wheel Brake System SERL

Pedal Pwr Pedal Green Blue Pos. 1 Pos. 2 Pump Pump

Shut Off BSCU Selector Isolation Valve Valve

A Brake System N L O T Annunciation R E M R A N L A Selector T Valve E

AS Shut Off Accumulator Anti Skid Valve CMD/ Meter Anti Skid Valve Mech. Pedal Position Meter Valve

Wheel

9 Aircraft FHA SERL

1 2 Failure Condition 3 4 Effect of Failure Condition on 5 V&V Function (Haz Description) Phase Aircraft/Crew Classificat’n Decelerate Loss of Deceleration Landing/ See Below Aircraft on Capability RTO/… the a. Unannuciated loss Landing/ Crew is unable to decelerate the Catastrophic Aircraft Ground of deceleration RTO aircraft resulting In a high speed Fault capability overrun Tree b. Annunciated loss of Landing Crew selects a more suitable airport, Hazardous Aircraft deceleration notifies emergency ground support Fault capability and prepares occupants for landing Tree overrun. c. Unannunciated loss Taxi Crew is unable to stop the aircraft on Major of deceleration the taxi way or gate resulting In low capability speed contact with terminal, aircraft, or vehicles d. Annunciated loss of Taxi Crew steers the aircraft clear of any No Safety deceleration obstacles and calls for a tug or Effect portable stairs Inadvertent Takeoff Crew is unable to takeoff due to Catastrophic Aircraft Deceleration after Vl application of brakes at the same time Fault (Takeoff/RTO as high thrust settings resulting in a Tree decision speed) high speed overrun 10 Aircraft FHA SERL

1 Function 2 Failure Condition 3 4 Effect of Failure 5 (Haz Description) Phase Condition on Aircraft/Crew Classificat’n

Decelerate d. Annunciated Taxi Crew steers the No Safety Aircraft on loss of aircraft clear of any Effect the Ground deceleration obstacles and calls for a tug or portable stairs

11 PSSA SERL • Continuation of FHA on “systems”

• Refined requirements, refined failure assessments, …

12 PSSA SERL

Loss of Aircraft Preliminary Fault Tree LOSSOFAC

Unannunciated Loss Inadvertent Other Failure of Deceleration Deceleration Conditions Capability After V1 UNANLSSDEC INADDEC+V1 OTHERFAILS 5.00E-09

Unannunciated Inadvertent Inadvertent Loss of Effective Inadvertent Spoiler Loss of Thrust Thrust Reverse Wheel Braking Wheel Braking Deployment After V1 Reversers After V1 After V1 UNANLSST/R UNLSSEFFWB INADTR+V1 INADSPL+V1 INADW/B1 PROB: 5.00E- 1.00E PROB: PROB: PROB: 03 (0) -05 5.00E-09 (0) 5.00E-09 (0) 5.00E-09 0)

Unannunciated Loss of Unannunciated All Speedbrakes on Loss of All Wheel Contaminated Runway Braking

UNLSSPDBR UNANLSSWB PROB: PROB: 5.00E-07 (0) 5.00E-07 (0) 13 Derived Safety Requirements SERL

Safety Requirement Design Decisions Remarks 1. Loss of all wheel braking More than one hydraulic The overall wheel brake (unannunciated or system required to achieve system availability can annunciated) during landing the objective (service reasonably satisfy this or RTO shall be less than experience). Dual channel requirement. See PSSA FTA 5E-7 per flight. BSCU and multimode brake below. operations. 2. Asymmetrical loss of Separate the rudder and The wheel braking system wheel braking coupled with nose wheel steering system will be shown to be loss of rudder or nose wheel from the wheel braking sufficiently independent steering during landing system. Balance hydraulic from the rudder and nose shall be less than 5E-7 per supply to each side of the wheel steering systems. flight. wheel braking system. System separation between these systems will be shown in the zonal safety analysis and particular risk analysis

14 Derived Safety Requirements SERL

Safety Requirement Design Decisions Remarks 1. The primary and Install hydraulic supply to Compliance will be shown secondary system shall be the brakes in front and by ZSA and PRA. (Editor's designed to preclude any behind the main gear leg. Note: In this example only common threats (tire burst, for the main gear bay zone tire shred, flailing tread, and the tire burst structural deflection). particular risk.). 2. The primary and Choose two different Compliance will be shown secondary system shall be hydraulic systems to by CMA. designed to preclude any supply the brakes, common mode failures emergency braking without (hydraulic system, electrical power. electrical system, maintenance, servicing, operations, design, manufacturing, etc.).

15 SSA SERL • Continuation from PSSA

• Based on final designs and implementations

16 SSA SERL Detectable BSCU 1 Failure Causes Bad Data Which is Provided to Normal System MV BSCU1DETD Computer-related FTA

2.00E-09

BSCU 1 P/S Failure/ BSCU 1 I/O or CPU Error Causes Bad Data Failure/Error Causes Which is Provided to Bad Data to be Sent to Normal System MV Normal System MV BSCU1PSIND BSCU1CDIND 1.00E-09 1.00E-09

BSCU 1 Power Supply BSCU 1 Power Supply BSCU 1 Monitor BSCU 1 Command Channel Motor Stuck Valid Failure Causes Bad Channel Always CPU or I/O Failure/Error Data Reports Valid Causes Bad Data BS1PSMOFV BSCU1PSF BSCU1MORV BSCU1CDF PROB: 2.00E-02 (1) PROB: 5.00E-08 (1) 2.00E-02 RATE: 2.00E-07 /H RATE: 1.20E-05 /H 5.00E-08 EXPO: 1.00000 H EXPO: 0.004167 H Budgeted F.R. Budgeted F.R.

BSCU 1 Validity Monitor BSCU 1 Monitor BSCU 1 Failure Causes BSCU 1 Command Failed Valid Due to Channel Design Error Loss of Braking Channel I/O Failure Hardware Failure Commands Causes Bad Data BSCU1MOFV BSCU1CMDE BSCU1CPUBD BSCU1I/OF PROB: 2.00E-02 (1) PROB: 0.00E+00 (0) 3.12E-08 PROB: 1.88E-08 (1) RATE: 2.00E-07 /H Level B RATE: 4.50E-06 /H EXPO: 1.00000 H EXPO: 0.4167 H Budgeted F.R. Budgeted F.R. BSCU 1 Command BSCU 1 CPU Function Channel CPU Design Error Causes Hardware Failure Bad Data Causes Bad Data BSCU1FAILS BSCU2FAILS PROB: 3.13E-08 (1) PROB: 0.00E+00 (0) RATE: 7.50E-06 /H Level A EXPO: 0.4167 H Budgeted F.R. 17 Computer-related FTA SERL

BSCU 1 Monitor Channel Always Reports Valid BSCU1MORV 2.00E-02

BSCU 1 Validity Monitor BSCU 1 Monitor Failed Valid Due to Channel Design Error Hardware Failure BSCU1MOFV BSCU1CMDE PROB: 2.00E-02 (1) PROB: 0.00E+00 (0) RATE: 2.00E-07 /H Level B EXPO: 1.00000 H Budgeted F.R.

18 Safety Assessment Elements SERL

Probability 1.0 1.0E-3 1.0E-5 1.0E-7 1.0E-9 (Quantitative) FHA Development Level D Level C Level B Level A Assurance Level

PSSA

SSA • Flow into DO-178 (software) and DO- 254 (hardware) • Those documents provide guidance in how to achieve the different levels (a discussion for another time) 19 Agenda SERL • Motivation • ARP4761 Process • ARP4761 Application • STPA Results • 4761 and STPA • Future

20 Control Structure SERL

Other Aircraft Crew, left/right Sensory inputs Systems

 Anti-skid on/off switch Annunciations/  Autobrake arm/rate switch  Spoiler/air brake position displays  Brake pedal demand  Ground/air status External  Nosewheel rotation speed Brake Sensors  Ground speed Hydraulic pressure Controller  Altitude  Flight mode

 Main wheel rotational speed Hydraulic Valve positions Meter Valves Bogie Sensors  Disc temperature Pressure  Tire pressure

Main Wheel Bogie Assy

21 Unsafe Control Actions SERL

Control Action Not providing Providing causes Too soon, too late, Stopped too soon, Flight Crew: causes hazard hazard out of sequence applied too long CREW.1 CREW.1a1 CREW.1b1 CREW.1c1 CREW.1d1 Manual braking Not providing Manual braking Manual braking Manual braking manual braking provided with applied before stopped too soon via brake pedals during landing or insufficient pedal touchdown causes before safe taxi RTO while autobrake pressure, resulting wheel lockup, loss of speed reached, not providing inadequate control, tire burst resulting in over- braking (or deceleration during speed or overshoot insufficient braking), landing leading to overshoot CREW.1b2 CREW.1c1 CREW.1d2 Manual braking Manual braking Manual braking provided with applied too late to applied too long, excessive pedal avoid collision or resulting in stopped pressure, resulting in conflict with another aircraft on runway loss of control, object or active taxiway passenger/crew injury, brake overheating, brake fade or tire burst during landing

22 Unsafe Control Actions SERL

Control Action Not providing Providing causes Too soon, too late, Stopped too soon, BSCU: causes hazard hazard out of sequence applied too long BSCU.1 BSCU.1a1 BSCU.1b1 BSCU.1c1 BSCU.1d1 Command Braking pressure Braking pressure Braking pressure Reduced not provided during commanded applied before deceleration if Braking RTO (to V1), excessively, touchdown, brake pressure is Pressure resulting in inability resulting in rapid resulting in tire released during to stop within deceleration and burst, loss of landing roll before available runway injury in pushback control, injury, TBD taxi speed length other damage attained BSCU.1a2 BSCU.1b2 BSCU.1c2 BSCU.1d2 Brake pressure not Braking pressure Braking pressure Stop on runway if provided during applied applied too long brake pressure not landing roll, inappropriately after touchdown, released during resulting in during takeoff, resulting in landing roll after insufficient resulting in insufficient TBD taxi speed deceleration and inadequate deceleration and attained potential overshoot acceleration potential loss of control, overshoot … …

23 BSCU (Brake Comp.) Analysis SERL

Sensor: Caliper and WOW Sensors Actuator: Meter valves * Meter valve position Hydraulic pressure measurement error Signal to release brakes mis-read/not incorrectly read Weight on wheels incorrect or misread received Aircraft pitches (e.g. due to winds) Meter valve stuck open upon initiation of resulting in inconsistent weight-on- takeoff wheels measurements Meter valve incompletely closed Controlled Process: Brake Caliper Sensors degrade or lose calibration Sensors effected by weather Corroded brake piston causes caliper to stay engaged; Contaminated brake fluid causes Caliper pressure given caliper to remain engaged, even Caliper pressure not detected Hydraulic pressure higher after valves are correctly closed; than valve capability Fluid not released, or spring mechanism cannot overcome latent hydraulic pressure 24 BSCU (Brake Comp.) Analysis SERL

V1 speed Sensor: Caliper and WOW Sensors Actuator: Meter valves * Meter valve position Hydraulic pressure measurement error Signal to releaseProc brakes misModel-read/not Variablesincorrectly: Groundspeed read Weight on wheels incorrect or misread received Aircraft pitches (e.g. due to winds) Meter valveBSCU stuck open upon incorrectly initiation of assesses that aircraftresulting in inconsistent weight-on- takeoff wheels measurements Meter valve incompletely closed Controlled Process: Brake Caliper Sensors degrade or lose calibration has left ground Sensors effected by weather Corroded brake piston causes Proc Model Variablescaliper to: stayAltitude engaged; , WOW Contaminated brake fluid causes Caliper pressure given caliper to remain engaged, even Caliper pressure not detected Hydraulic pressure higher after valves are correctly closed; than valve capability Fluid not released, or spring mechanism cannot overcome latent hydraulic pressure 25 Crew Analysis SERL

[Thornberry 2014] 26 Agenda SERL • ARP4761 Process • ARP4761 Application • STPA Results • 4761 and STPA • Future

27 A Note on “Annunciation” SERL • Air France 447 had plenty of annunciations prior to crash

[Telegraph/Getty Images 2012]

28 STPA and 4761 FMEA SERL • Where is the pilot in 4761? – The only relevant thing to pilot is “unannunciated”,

– What about when and why it is annunciated,

– Assumes that pilot will be able to account for and react to brake failures

• Why? – It is not just because FTAs and other methods

29 STPA and 4761 SERL • What about software?

– Software often “ends” with a failure node in a FTA

– There are other tools in the suite of tools allowed by 4761 that we need to assess

– Software development is (somewhat) out-of-scope for 4761

– But STPA can help here!!!

30 STPA and 4761 SERL • Can STPA find the things about hardware that are already in existing techniques?

– How does it compare with FMEA & FTA?

– Does it find things beyond what they find?

– Does STPA help to achieve 4761 objectives?

31 Agenda SERL • ARP4761 Process • ARP4761 Application • STPA Results • 4761 and STPA • Future

32 Next Few Months SERL

• STPA Analysis is ongoing – Fidelity of STPA analysis ≈ fidelity of ARP analyses, examples

• More thorough analysis – of how STPA compares to existing techniques – of how STPA fits into (or doesn’t) ARP4761

33 Longer Term SERL

• Can we get STPA into ARP4761?

• What will ARP4761A look like?

• Does STPA help to achieve 4761 (and 4754A) objectives? Does the FAA want this?

34 References SERL 1. NTSB Case Number: DCA13IA037, Interim Factual Report Boeing 787-8, JA829J, Japan Airlines (Boston, Massachusetts, January 7, 2013), National Transportation Safety Board, Office of Aviation Safety, March 7, 2013 2. Boeing 787 Program Information “About the Dreamliner” (accessed 20 March 2014) http://www.boeing.com/boeing/commercial/787family/background.pag e? 3. Boeing 787 Wikipedia page (accessed 20 March 2014) http://en.wikipedia.org/wiki/Boeing_787_Dreamliner 4. ARP 4761: Guidelines and Methods for Conducting the Safety Assessment Process on Civil Airborne Systems and Equipment. Warrensdale: SAE International, 1996. Print. 5. 787 picture 6. The Unwanted Blog, http://up-ship.com/blog/?p=6045, May 01, 2010 7. Ross & Tweedie, The Telegraph, UK http://www.telegraph.co.uk/technology/9231855/Air-France-Flight- 447-Damn-it-were-going-to-crash.html, April 28, 2012, Getty Images

35 BSCU (Brake Comp.) Analysis SERL

Flawed Control Unsafe Control Action: BSCU.1b2 Algorithm: Braking pressure applied inappropriately during takeoff, resulting in inadequate acceleration Algorithm for determining that aircraft Manual braking has lifted off is incorrect pressure incorrectly Due to… applied or misread Algorithm for determining pilot braking... Controller: BSCU Incomplete or inaccurate process model: Flawed ControlSensor : Caliper BSCU is unaware and that aircraft WOW is in SensorsV1 speed incorrect or misread Algorithm: takeoff phase (but may still have correct Ground/airspeed incorrect or misread Brake apply signal Algorithm for WOW and air/groundspeed readings and Flight mode incorrect or misread given during takeoff determining when it is too thus applies brakes) late to apply brakes Proc Model Variables: Flight Mode Hydraulic pressure measurementMismatch error in data format; Command to close during takeoff is  BSCU is unaware that aircraft has passed Mismatch in units (Eng v valve (and release incorrect, i.e. incorrect V speed Weight on1 wheels incorrect or misreadMet) brake) is given to computation of V1 speed Proc Model Variables: Groundspeed Sensor readings are too incorrect valve; e.g.  BSCU incorrectly assesses that aircraft slow (BSCU computes at to Green sys. during Aircraft pitcheshas left ground (e.g. due to winds) Flawed Control 100Hz but sensor alternate mode Proc Model Variables: Altitude, WOW Algorithm: resulting in inconsistent weight-updateson- at 1Hz) wheels measurements Sensor: Caliper and WOW Sensors Actuator: Meter valves * Meter valve position Hydraulic pressure measurement error Signal to release brakes mis-read/not Sensorsincorrectly degrade read or lose calibrationWeight on wheels incorrect or misread received Aircraft pitches (e.g. due to winds) Meter valve stuck open upon initiation of resulting in inconsistent weight-on- takeoff Sensors effected by weatherwheels measurements Meter valve incompletely closed Controlled Process: Brake Caliper Sensors degrade or lose calibration Sensors effected by weather Corroded brake piston causes caliper to stay engaged; Contaminated brake fluid causes Caliper pressure given caliper to remain engaged, even Caliper pressure not detected Hydraulic pressure higher after valves are correctly closed; than valve capability Fluid not released, or spring mechanism cannot overcome latent hydraulic pressure 36 BSCU (Brake Comp.) Analysis SERL

Flawed Control Unsafe Control Action: BSCU.1b2 Algorithm: Braking pressure applied inappropriately during takeoff, resulting in inadequate acceleration Algorithm for determining that aircraft Manual braking has lifted off is incorrect pressure incorrectly Due to… applied or misread Algorithm for determining pilot braking... Controller: BSCU Incomplete or inaccurate process model: Flawed Control  BSCU is unaware that aircraft is in V1 speed incorrect or misread Algorithm: takeoff phase (but may still have correct Ground/airspeed incorrect or misread Brake apply signal Algorithm for WOW and air/groundspeed readings and Flight mode incorrectSensor or misread: Caliper and WOW Sensors given during takeoff determining when it is too thus applies brakes) Actuator: Meter valveslate to apply brakes Proc Model Variables: Flight Mode Mismatch in data format; Command to close during takeoff is  BSCU is unaware that aircraft has passed Hydraulic pressure measurement error * Meter valve positionMismatch in units (Eng v valve (and release incorrect, i.e. incorrect V speed Signal to release brakes mis-read/not 1 Met) Weight on wheels incorrect or misread brake) is given to computation of V1 speed Proc Model Variables: Groundspeed incorrectly read Sensor readings are too incorrect valve; e.g.  BSCU incorrectly assesses that aircraft received slow (BSCU computesAircraft at pitches (e.g. due to winds) to Green sys. during has left ground Flawed Control 100Hz but sensor alternate mode Proc Model Variables: Altitude, WOW Meter valve stuck openAlgorithm upon: initiation of updates at 1Hz) resulting in inconsistent weight-on- takeoff wheels measurements Sensor: Caliper and WOW Sensors MeterActuator valve: Meter valves incompletely closed Sensors degrade or lose calibration * Meter valve position Hydraulic pressure measurement error Signal to release brakes mis-read/not incorrectly read Weight on wheels incorrect or misreadSensors effected by weather received Aircraft pitches (e.g. due to winds) Meter valve stuck open upon initiation of resulting in inconsistent weight-on- takeoff wheels measurements Meter valve incompletely closed Controlled Process: Brake Caliper Sensors degrade or lose calibration Sensors effected by weather Corroded brake piston causes caliper to stay engaged; Contaminated brake fluid causes Caliper pressure given caliper to remain engaged, even Caliper pressure not detected Hydraulic pressure higher after valves are correctly closed; than valve capability Fluid not released, or spring mechanism cannot overcome latent hydraulic pressure 37 Definitions SERL

• Functions = intended behavior of a product based on a defined set of requirements regardless of implementation

38 Definitions SERL

• Failures = loss of function or a malfunction of a system or a part thereof (different than 4754)

39 Definitions SERL

• Errors = (1) an occurrence arising as a result of an incorrect action or decision by personnel operating or maintaining a system, (2) a mistake in specification, design, or implementation

40 Definitions SERL

• Hazards = potentially unsafe condition resulting from failures, malfunctions, external events, errors, or a combination thereof

41 Definitions SERL

• These definitions present some hurdles in terms of communication

• But STPA can help with ARP4761…especially with identifying ‘errors’, why they might occur, how to generate requirements

42 4761 Objectives SERL • FHA Outputs – FHA input function list – Environmental and Emergency Configuration List – Derived safety requirements for the design at each level – FHA Report • Functions, failure conditions, phase of ops, …

43 4761 Objectives SERL • PSSA Outputs – Planned compliance with FHA requirements – Updated FHAs – Material supporting classification list – Failure condition list – Lower level safety requirements (including DALs) – Qualitative FTAs – Preliminary CCAs – Operational requirements

44 4761 Objectives SERL • SSA Outputs – Updated failure condition list or FHA which includes rationale showing compliance with safety requirements (qual and quant) – Documentation showing how req’s for the design of the system items’ installation have been incorporated (segregation, protection, etc.) – Materials used to validate the failure condition classification – Maintenance tasks – Documentation showing how system has been developed according to DAL 45 Background SERL • Aircraft are VERY safe

• Development & Certification process has been very successful

• This is due at least in part to ARP4761

• Accidents due to mechanical failure have decreased dramatically over the years…

46 Background SERL

• Why has approach been so successful?

• Will the assumptions hold in the future?

47 Past, Present, and Future SERL

• What do we see in the aircraft of ‘yesterday’?

• What do we see in the aircraft of ‘today’?

• What will we see in the aircraft of ‘tomorrow’?

48 Past SERL

[UWB, 2010] 49 Present SERL

[Bicheno-Brown, 2012] 50 Present SERL

Fuselage – CFRP composite HUD Electric power (vs bleedless and hydraulic) LiCo batteries IMA, AFDX (ethernet comm) Self monitoring & Reporting Increasing global manufacturing

[Bicheno-Brown, 2012] 51 Future SERL

52 A Note on PRA SERL • Boeing 787 LiCo Batteries

• Prediction/Certification: – No fires within 107 flight hours – Followed 4761 certification paradigm

• Actual experience: – Within 52,000 flight hours – 2 such events – 2.6 x 104 flight hours [NTSB 2013]

[http://upload.wikimedia.org/wikipedia/commons/9/95/Boeing_Dreamliner_battery_original_and_damaged.jpg] 53 A Note on PRA SERL • I love the 787 and I continue to cheer it on!

• LiCo technology a fairly significant departure from yesteryear’s battery technology – More energy density, requiring more complexity to control it

• This is a battery – what will happen if/when we drastically change the role of software & humans?