JSC S&MA Analysis Branch

Analytical forProcess Recognizing Probabilistic Probabilistic

Chief, Safety & Mission Safety& Mission Assurance (S&MA) andOperational NASA Johnson Space Center Risk& Reliability Analysis BSEE PRAIndustryDay Roger L. L. Roger Boyer,CRE Risk Assessment(PRA March 1, March 2018 Prepared for Prepared Prepared byPrepared

R

isks Branch

D esign ) :

1

JSC S&MA Analysis Branch

Using differentviews in analysis Pop Pop Quiz:

2

JSC S&MA Analysis Branch

A sphere with a hole through the center? A circle with dot in a the center? What What does this look like?

3

JSC S&MA Analysis Branch

It could It could this… be

4

JSC S&MA Analysis Branch

A single can view mislead you… Or it could be this…

5

JSC S&MA Analysis Branch

looking at systems and operations in a and operations at systemslooking i we be smart must we s a tool toyou the bys atoolassess risk help different view bothquantitativelydifferentand view Probabilistic Given our available budgetour available Given and what we do. That’s where we PRA That’sdo. where and what can make a difference. can make qualitatively. qualitatively. Conclusion Risk Assessment (PRA)Risk Assessment and

efficient in how how efficientin

and

time, 6

JSC S&MA Analysis Branch

Introduction

7

JSC S&MA Analysis Branch

• • • • addition to its traditional human space programs. programs. space to human traditional its addition to respectPRA with &Gas companies Oil in several and (BSEE) Enforcement Environmental and Safetyof Bureau the is activelysupporting (JSC) Center Space Johnson The transportation power,nuclear as commercial such aerospace, industries,PRAin several used successfullybeen has efficient. smarterand more to we have less,thus be S&MA public. the with to from itsjob do continue must NASA with expectations getbudgets high to continues facilities,vessels, andmissions.vehicles,systems, width in evaluating and both depth PRA/PSA provides Safety(PSA). as Probabilisticto referred Assessmentalso NASA’s It’sMission toolbox. & (S&MA)Safety Assurance Risk Probabilistic ( Assessment , oil & gas, andmedical., oil & Introduction PRA) is one of of in tools is the one PRA)

chemical,

8

JSC S&MA Analysis Branch

• • • • tree models, and simulation models. PRAsareoften characterized by from how designers think (e.g. success space). The models aredeveloped in “failure space”. This usually is different Thereare other thatseeks answers threeto basic questions: philosophically affectthe system or process being studied. is It inherently and take into account all possible eventsinfluencesor that could reasonably attemptsIt toquantify rare event probabilities offailures. It attemptsto identifyingand analyzing risk in engineered systems and/or processes. PRAcomprehensive,a is structured, and disciplined approach to    Containment, Reactor CoreDamage Frequency)? (e.g., Loss of What consequences could result these from eventsor scenarios or scenarios? What arethe likelihoods and associated uncertainties ofthe events wrong)? What kinds kinds eventsof or scenarios can occurwhat (i.e., cango

definitions and questions thatcanit help answer. a Bayesianmethodology. Crew/Life, Loss

What is What PRA (but not limited to) limited not (but of Mission, Mission, Loss ofHydrocarbon

In In general, PRAaprocessis

? event treemodels, fault

9

JSC S&MA Analysis Branch • • •

process/design. For example: For process/design. proposed and currentstudies between risktrade Focused Level System Assessment RiskFacility Level – – – – – – – – – – Evaluating operational workarounds given an initiating event, such as bolt failure. Comparing different BOP ram driversand sealing. STD the Well ControlRule fiveafor year time frame vs. the existingsystem in API Evaluate MudSystems Dynamic Positioning System (DPS) Blowout Rigs and Platforms Subsea Shallow Water DeepwaterDrilling Operation - 53. Oil Preventer(

the proposed requirement foradditional subsea accumulator bottles in Production

Risk AssessmentRisk Drilling Oil & GasOil Examples

BOP)

Operation

10

JSC S&MA Analysis Branch

increasing increasing productivity. changes and associated with PRA can be applied to EXISTING SYSTEMS throughout to The NEWDEVELOPMENTS plans plans can In of the event INCIDENTRESPONSE incorporate incorporate the ideal ideal to time conduct assess assess the cause of the during during the cycle, life including design and operations. In nutshell, a PRA can applied be from concept decommissioning to to minimize to minimize the probability When can PRA be Performed?PRAWhen canbe the development the development phase minimal at cost. help avoid compromises avoid help compromises unexpected unexpected downtime operations. operations. PRAs evaluate can the necessary necessary safety and

existing systems existing systems

a PRAa failure failure is at the beginning is at the beginning of of comparable of comparable and or an accident, or accident, an in quality in quality

to identify identify to and avoidance risk develop appropriate develop appropriate or reliability or reliability while events events in the future

impact impact of system a good good a PRA team the design the design prioritize prioritize risks measures measures

mitigation mitigation process process .

11

JSC S&MA Analysis Branch • • • •

been been developed by the industry and the Nuclear Regulatory Commission (NRC large amount data,of PRA isnow practiced allby commercial nuclear plants the inUnited States and a WASH Nuclear risk estimated risk was believed to be too high, so it was abandoned and quantitative Risk Assessment (PRA) in NASA experimented with Fault Trees and some early attempts doto Probabilistic analysis began. launch systems for nuclear In late fifties early / sixties Boeing and Bell Labs developed Fault Trees evaluate to – – – – – assessment was offthe table until post some oil companies now. now. companies oilsome andNASA by used also U.S. NRC and the for Lab NationalIdaho theby developedtool software (Systems SAPHIRE approach informed” a “ using industrynuclear thecommercialof responsibilityoversightits NRC practices The concepts. In Depth” “Defense as well as PRA on based plants theirlicense must NuclearPlants newAll easily gained by any other means determinedthat the WASH Was shelved until This is considered the first modern - 1400 (Reactor Safety Study) in power

industry picked up the technology in early seventies and created Some Background Some Analysis Programs for Handsfor Programs Analysis the Three that is heavily dependent on PRA dependentheavilyis that methodology, the sixtiesthe

weapons and early approaches humanto reliability - 1400 1400 study gave insights to the incidentthat could not be Mile IslandMile (TMI) incident happened 1979. in It was .

PRA. (most notably on the ApolloProgram and documentation forPRA technology has the the mid

- on Integrated Reliability Integrated on - Challenger. seventies. .

Evaluations) is a PRA PRA is a Evaluations) ). The Risk 12 ).

- JSC S&MA Analysis Branch

PRAOverview

13

JSC S&MA Analysis Branch

Engineering Analysis response response time,etc. success support support criteria, criteria, used used to

is is

PRA Process PRA and independent independent review process Documentation

long supports supports a successful

- term PRA application

of of the PRA

14

JSC S&MA Analysis Branch • • • •

Building or developing a PRA involves: a PRAdeveloping or Building or need. can use that modelers the events basic the for uncertainty their associatedand probabilities generate to Good facility.of failure or vehicle the logic the in fail. This developing work and is essentialsystems how to in order understand backgrounds design and/or operations of Amajority management and communication experts domain systemboth team includes analysissystem APRA

– – – – trouble shooting it (nothing works the time) first populating it withthe desired failure logic andprobabilities, and designing how it will serve that purpose, understanding its purpose data analystsdata and

PRA analysts. The The analysts.PRA PRA analysts PRA

between the PRA analysts, domain experts, domain analysts, PRAthe between .

The PRA Team The PRA understand how data available to the how take understand and

theappropriate modeling techniques,

have engineering degrees with with degrees engineering have key to success is to keysuccess

multi

- way 15

JSC S&MA Analysis Branch

The PRA Team The PRA

16

JSC S&MA Analysis Branch

PRADevelopmentProcess 17

JSC S&MA Analysis Branch

Failure Failure Logic

18

JSC S&MA Analysis Branch • • • •

eventsequences. event and the linked trees are get together to the potential appropriate Multiple trees event are order used in model to complete a mission, or a event failure during the event. mission with branch points that generally represent a successful The trees eventare timelines critical of that events occur during a successful well closure or not. starting with an initiating as such event, a kick, well ending and with Event trees are tools that the the model mission overall or operation phase, element, system,etc. failure scenarios for the entire mission that categorized can be by The results the of event tree analysis is a of list ranked “cutsets” or – – – example An of Shuttle a event tree is shown the following on page. end state. branch Each of the event tree is followed in deductive a fashion to its associated such with as it remote operated vehicles (ROVs). directly a loss hydrocarbon in of event, it or may mitigations have A failed branch in an event tree is the start of scenario a that may end

Event Tree Analysis

19

JSC S&MA Analysis Branch

ENTRY ENTRY_INIT Entry initiated -

End End of Deorbitto Burn Wheelstop

RCS failures RCSfailures on entry RCS - ENT

structural failure structural due Loss Loss of or control to to TPSdamage THERMAL Example Example EventTree

Success Success

Orbiter structural Orbiter failure on failure entry STRUCT Events - ENT

cooling on cooling entry Life Life support & ECLSS - ENT

-

Up Aerosurface Aerosurface failures AEROSURFACES on on entry

Landing gear Landing gear deploy LGEAR

Touchdown and rollout failures rollout ROLLOUT Failure Failure

2006/03/02 8 7 6 5 4 3 2 1 #

LOCV LOCV LOCV LOCV LOCV LOCV SUCCESS

LOCV - END NAMES

-

STATE

Down

-

Page 2 Page

LOCV LOCV LOCV LOCV LOCV LOCV LOCV A SEQUENCE

NAMES - - STRUCT AEROSURFACES - - - - - RCS TPS LIFE LGEAR ROLLOUT

- - ENT - ENT ENT

End States End States End End State 20 Success

LOCV

– –

JSC S&MA Analysis Branch • • • • • •

Each Each fault produces tree “ existing data. Fault tree developed logic is a downward to compatible level with require both failure to a a occur recover. failure and to Recovery maybe actions included the in logic of fault tree, the that a in total loss the of systemor function. fault trees for must account partial losses multiple in phases resulting Manysystems used are in multiple mission phases, e.g. power, so occurring eventtop and developing logic that will result in top the event The fault developed trees are a in starting deductive fashion, with a trees. Fault trees are tools that the the model individual events the in event integrated mission level results. event. fault trees are The the input trees eventinto to develop overall – Typically failure aof system orfunction

Fault Fault Tree Analysis cutsets

” ” or failure scenarios for that top

21

JSC S&MA Analysis Branch SRBSEP - SRB separation failures separation - SRB SRBSEP

Left SRB Separation A-BL-SEP-CAT Catastrophic Failures 314 Right SRB Separation Failures SRBs) (2 A-B-SEP-CAT Catastrophic Right SRB Separation A-BR-SEP-CAT Catastrophic Failures 315 Example Example Fault Tree Analysis A-BL-ELE-MF-SEP-FLRS A-B-ELE-PR'MEA-FLRS Measurement Signals Left SRB Separation Electrical Command Functional Failures of RSRM Pressure Left SRB Loses Function 172 313 SRB Separation Failure LOCV SRBSEP A-BL-SEP-FUNC-FLR A-BL-SEP-BLT-FLRS Left SRB Separation Left SRB Separation Functional Failures Functional Failures Bolt Circuit 176 Top Event A-BL-SEP-BSM'FUNC-FLR Functional Failure Left SRB BSM Right SRB Separation Functional Failures 185 A-B-SEP-FUNC (2 SRBs) (2

A-BR-ELE-MF-SEP-FLRS Separation Electrical Command Function Right SRB Loses of Right SRB 280 or an typically “Or” events is Logic to combine

“And” “And” gate A-BR-SEP-FUNC-FLR A-BR-SEP-BLT-FLRS Right SRB Separation Right SRB Separation Functional Failures Functional Failures Bolt Circuit 283 2008/12/01 A-BR-SEP-BSM'FUNC-FLR

Functional Failure Right SRB BSM

310 Page 4 Page 22

datapermits as levelslower toproceeds Logic

JSC S&MA Analysis Branch C o

n

t r C o l N l e r

f a i l s Fault Trees AreAttachedto the EventTree C LD o t d L f r m a a e e i n m l

u t a P s e o r O d k e n P R c u

o c t n c f e e a

o P r u d s t s e t r a P n r s e f P a d s i u s 1 i l d s s u c L o o e r e w e l r a

a n 1 t i k s o D

t n s r L e o

v a u a m r l c v

e e o s logic f t r a P n r s e f P a d s i u s 2 l s u c I s e r o e r t

c o

2 v o

a c m V l l o v m e s 1 e a

1

n o

d f n a i Hydrazine leaks Hydrazine l s L i s e IE o a C

l O k o a R n LI

t t r n C e

o l o N l d e

r t

f a i l s detected notLeak I LD s o t c o

v o

a c m V l l o v

m e s 2 e a

2

n o

d f n a i l s Leak not isolated notLeak u p L s e LI t a r v

e k a U a

l s v m L o e

u s o r f c

i e s

o damage to flight to flight damage othertechniques critical avionics critical integrated integrated structure inand physical, etc.) an simulation reliability,models various(logic, PRA of embodies collection a model A

damage to damage equipment scientific scientific S

23

Loss of spacecraft of Loss spacecraft of Loss spacecraft of Loss

Loss of science of Loss science of Loss science of Loss End state End OK OK OK

JSC S&MA Analysis Branch

Basic EventBasicDevelopment Data AnalysisData o r

24

JSC S&MA Analysis Branch •

5 th Percentile For each EventBasic – – – 5 distributionprobability of failure probability th

and 95 and

Mean th

percentiles 95 th Percentile Basic EventBasic Input

SAPHIREScreenshot

25

JSC S&MA Analysis Branch

• • • • • Common Common Cause initiating eventinteractions. pre Human situations. over pressurization, ascent debris, structural failure, and othersimilar scenarios, including leaks offlammable/explosive fluids, engine burn through, or events.Phenomenological events cancover broad a range failureof interactions between systemsand their environment or otherexternal factors thatarenot solelybased on equipment performance but on complex Phenomenological update asShuttle specificdata becomes available. typicallyinto fall two categories, time Replaceable Unit(LRU) or Shop Replaceable Unit(SRU). Functional failures failures aregenerally defined atthe majorcomponent level such asLine byfailurea mode forthe component type (e.g., fails start).to Functional Functional failures arespecified byacomponent type (e.g., motor pump) and component type,such asa valve or pump, performto intendedits function. Functional eventsfailwill thatanevent hasalready happened what theis probability successivethat Conditional due toa shared cause. similar components within systema that occur within specifieda period time of - initiating initiating event,initiating event(orhuman

–

Threetypes of human errorsaregenerally included in faulttrees: Typesof Data that Exist in theModels

–

–

Afunctional failure event is generally defined asfailure aof

Aprobabilitythat is conditional upon another event,giveni.e.

–

Common Common Cause Failures (CCFs) aremultiple failures of –

Phenomenological events include non

- based and demand - induced andinitiators), post

- based. Bayesian - functional events

26

- JSC S&MA Analysis Branch

• • • • • • • • • SINTEF OREDA Contractor/vendor NASA’s PRACA databases are sources Shuttle for specific failure data Miscellaneous Miscellaneous references ExpertOpinion (NUCLARR) generic data is a source for demand on failures Nuclear Computerized Library for Assessing Reactor Reliability failure time data electrical for components Electric Parts Reliability Data is a (EPRD) generic data source run for for time run failure data for mechanical components Non - electric electric . Offshore Offshore . and Onshore Reliability Data 6 Reliability Reliability Data Safety for Instrumented Edition. 2013 System, Part Reliability Part Reliability Database (NPRD) is a generic data source Functional Data Sources

data, data, when

available

th

Edition. Edition. 2015

27

.

JSC S&MA Analysis Branch • • • •

significant contributors. significant the on with onlya performed errorsdetailed analysis bulk of the human on the is performed analysisScreening design. equipment and procedural training, including areasimprovement, for potential on information valuable can provide qualityHRAAhigh contributions. riskof riskand the picture complete a more provides PRA in a failure their corresponding with actions human Modeling reliability. affectavailability and that of machines complex operation failures inthe of human occurrence the quantitatively, and qualitatively describe, usedto method is a HRA

Human ReliabiltyAnalysis(HRA)

28

JSC S&MA Analysis Branch

Case 3 Case 2 Case Case 1 Case For available available For data, CREAMcompared well with

– Comparison ofResults HRAto Training Data

Addedto credibilitythe of analysis

CREAM CREAM CREAM SIM SIM SIM 1.E-06 1.E-05 1.E-04

simulator simulator 1.E-03 data

1.E-02 29

JSC S&MA Analysis Branch • •

computers out Common within independence specific more In

PRA,

multiple - - -

components, a

single Single Sometimes error, May

Common event Cause,

or

be

or components

common

point

Common Cause Definition at

mission

the due which

the as

called result failures,

Cause to

used subsystems, same

some

fluid

bypassed of a

generic

such a

in

adverse Failures time,

such

design

header PRA,

as

failure or

these,

as

error, common is

or or

in to

(CCFs)

a not

.

a

structures multiple

invalidated are

installation common relatively

a

environment modeled

single are

pumps

failures

error,

short

power

failure explicitly due redundancy

or .

to interval

30

maintenance

that supply of

in a

a two

single PRA takes

like

or or to

JSC S&MA Analysis Branch

WithoutConsidering VALVE_A_FAILS VALVE A VALVE Common CauseCommon FAILS 1.0E

FAILURE OF FAILURE Results in a in a ~ 4.7E Results TWO PATHS TWO - 3

Times Cause the CommonRiskWithout Considering

VALVE_B_FAILS VALVE B VALVE FAILS Common Cause Common Cause Modeling 1.0E

A system oftwo consisting trains: 1E - 3

- 6

- 05 of UnderestimateRisk05 is48 Which Beta ( Beta = 0.047 b )

FAILURE OF TWO OF FAILURE COMMON CAUSECOMMON EVENT ConsideringCommon PATHS - 4 - 0 4.7E

FAILURE OF TWO OF FAILURE CAUSECOMMON VALVE_A_FAILS VALVE AFAILS VALVE - 5

PATHS Cause

FAILURE OF TWO OF FAILURE

1.0E

- (2) PATHS 3

VALVE_B_FAILS VALVE B FAILSB VALVE

4.8E 1.0E

- 5

-

3

31

JSC S&MA Analysis Branch VALVE_A_FAILS VALVE A VALVE FAILS

1.0E WithoutConsidering

- 3

Common CauseCommon Results in a a 4.7E ~ in Results Note: VALVE_B_FAILS PATHS THREE OF FAILURE Times the Risk Without Considering Common Cause Common Timesthe RiskWithout Considering B VALVE FAILS

Using a MGL a Using Model Wouldto Result Reduce 1.0E

-

3

Common Cause Common Cause Modeling

VALVE_C_FAILS Aoftrains:threesystem consisting 1E FAILS C VALVE - - 9

05 Underestimate of Risk Which is Riskof Which Underestimate05 1.0E

- 3

VALVE_A_FAILS VALVE A VALVE FAILS 1.0E

- 3

THREE PATHS THREE OF FAILURE VALVE_B_FAILS B VALVE FAILS

1.0E

ConsideringCommon

- 3

Cause (Beta (Beta CauseModel)

VALVE_C_FAILS C VALVE FAILS (3)

FAILURE OF FAILURE THREE PATHS

1.0E

2.6E -

3

47,000 47,000

- 05

AUSE COMMON 4.7E FAILURE CCF

32

- 4.7E 5

- 5

JSC S&MA Analysis Branch

• • experience. 0.1 0.9) to and are usuallyderivedfrom expert opinion or direct Conditional probabilities are typically relatively large (e.g. values like successive events will fail. Given an has eventthat already happened is what the probability that - strut fails strut fails orcrew looses control vehicle). of andnose geartouch downwhat is the probability that the Orbitercrashes (i.e. Example: Given blowna tire thein time interval between main touch gear down

Conditional Probability

33

JSC S&MA Analysis Branch

Keep Mind in

34

JSC S&MA Analysis Branch • • •

may have little to do with the intentionally engineered functional engineered intentionallythewith to littledo mayhave intra complex involvingfrequently from that result theyare typicallyHowever, model. thein risk representedsystem that theare of dependencies functionalto system duethe thethrough propagate that failures of thatfrom component resultcombinations failures These practiced. traditionally as PRA as such analysisof methods risk for simulated challenge completeness model Risk relationships of systemof the relationships Unknown Unknown andUnderappreciated Risks methods are generally effective generally are methods unknown or underappreciated (UU or underappreciated unknown

. ineffective

has long been recognized as a as recognizedbeen longhas -

and interand

at identifying system failures system failures at identifying

at identifying system at identifying - system interactions thatinteractions system

)

risks , 35

JSC S&MA Analysis Branch •

contributory contributory factors: in Earlier 2009, the NASA Advisory Council the noted following

– – – – – – – – – – – – – – assumptions. lack of document formal recordingfor keytechnical, schedule “go as youcan afford”approach lack of probabilistic estimating Reserveposition adequacy poor tracking contractor of requirements against plans eroding annual funding instability Inadequate assessment of impacts schedule of changes oncost changes scope in (design/content) higher technical complexity projects of than anticipated Inadequate risk assessments inability execute to initial schedule baseline optimistic estimates/estimatingcost errors commitments Inadequate definitions prioragencyto budgetdecision andto external Unknown Unknown andUnderappreciated Risks in - housetechnicalexpertise

(Cont’d)

, , and

set set programmatic programmatic of of 36

JSC S&MA Analysis Branch

Examplesof Results

37

JSC S&MA Analysis Branch

1.0E 1.0E 1.0E

- - - 12 06 0.01 02 = = 0.000000000001 = = 0.000001

  

1:100 (Probable) (Probable) 1:100 1:1,000,000 (Improbable) First, First, 1:1,000,000,000,000 the Math ( ridiculous

  )

 ~ShuttleMission Risk on tails simulaneously landing havingcoins 20

Built Dinosaurs?by

38

JSC S&MA Analysis Branch

~14 billion years agobillionyears~14 ~46,000 years ago years ~46,000 1.2 1.2 4 x 10 x 4 x

10 8 14 hours ago hours

hours ago hours

Time Perspective

2.1 x 10 x 2.1 ~240 years ago years~240 ~4.5 billion years ago yearsbillion ~4.5 4 x

10 6

13 hours ago hours

hours ago hours

2 ~228 x

10 6.3 6.3 12 – ~72 years agoyears~72

x – 80 million years ago yearsmillion80

10 7 x 5

10 hours ago hours 11

hours agohours

39

JSC S&MA Analysis Branch

Rank

6 5 4 3 2 1

of Total of %age %age 10 15 30 2 5 5

PRATop ContributorsRisk Scenarios / Cumulative Total 45 30 67 65 60 55

Probability Estimate (1:4400) (1:1500) (1:1200) 2.3E 6.5E 8.2E 1.1E 1.5E 3.3E (1:940) (1:670) (1:300) Point (1:n) ------03 03 04 04 04 03

catastrophic failure catastrophic Engine Rocket or entry orbit on LOC to leading orbit strikes on vehicle Debris failure during ascent during failure catastrophic2 System failure catastrophic1 System entry duringCrew error or entry orbit LOC on to leadingvehicle strikes debris Ascent Failure Scenario Scenario Failure

Description Notional

Numerous Numerous Numerous Reports HA HA HA HA HA HA HA Related Hazard Others HA ------

- 059 217 192 079 007 009 007 21

evaluating this risk. this evaluating be spent willeffort continued therefore and field, developing planned. activities upgrade significantNo Assigned) RiskProgram (No planned. development PRA additional planned. activities upgrade significantNo Assigned) RiskProgram (No planned. development PRA additional significant planned. activities upgrade significant No sufficient. is level trainingcurrent that communitythe with Concurrence (IRMA 4068) improvement. further for monitor to reviewed benefits. significant provide crewrescue capabilities and repair, Inspection, improvement. for further implemented been or have evaluation undereither plansprocessing and design Additional 2682) 2681, 2679, RisksDebris (IRMA Ascent planned.are improvements major other No . failure catastrophic mitigates which upgradesystem for the credit takesx Iteration iteration. this SSME 2723) (IRMA concern risk the reduce could inspection late for inspection late through applicable are crewrescue capabilities and repair, Inspection, risk. profile reduce to attitudeflight missionvehicle Optimizing risks) child numerous (IRMA 2530, - induced catastrophic failure is still a top contributor in top contributor a still is failure catastrophic induced

(Open/Accepted

Program Program

Action / Action Mission data will continue to be be to continue data will Mission

SIRMA risks)SIRMA . A xx” discernment or use use or discernment xx” A .

Status .

System 2 is a a is 2 System significant No

No No 40

Rank New Previous Iteration Iteration Previous (For Comparison) (For 5 4 3 2 1

Probability (1:4400) (1:1500) (1:1200) 2.3E 6.5E 8.2E 1.1E 1.5E 3.3E (1:940) (1:670) (1:300) ------04 04 04 03 03 03

JSC S&MA Analysis Branch • • • •

of the estimated risk lies between. The The This distribution is a representation of the uncertainty associated with a PRA’s results the value or metric used to verify LOCrequirements. The 95 5 Median Mean th th

percentile mean 5 median

percentile th –

and 95

1.1E –

1.1E is a common measure of risk that accounts for uncertainty or this distribution, thus

- is also referred to as the 50 02 (1:90) 02 – - th 02 (1:95) 02

– 7.7E

1.7E percentile - 03 (1:130) 03 - 02 (1:60) 02

UncertaintyDistribution

are are common points on a distribution to show the range that 90%

th

percentile

41

JSC S&MA Analysis Branch

Mission Conditional System 2 LOC 2 System LOC 1 System Estimated Estimated Risk vsRequirements/Goals LOM IDAC

Failure

- 2 PRA Results Results PRA 2

(Mean Values)(Mean Requirements (I.MPCV (R (R LOC/LOM - Current 1 in 1,400in 1 - (CA5913) CV0978)16, 5.00E 1.18E 1.82E 7.14E 16, SLS.16)16, 550in 1 1 in 20in1 85 in 1 - SLS.3132) Summary - - - - 02 02 03 04

Percentile vs Requirements vsRequirements 1 in 1,800in 1 2,500in1 3.49E 4.65E 5.55E 3.88E 1 in 210in 1 1 in 29 in 1 NotionalPRA(Estimates)Results 5 th

- - - -

02 03 04 04

1 in 1,600in 1 1 in 1000in 1 5.55E 7.14E 1.00E 6.12E 1 in 140 in 1 Mean 1 in 18 in 1 - - - - 02 03 03 04

Percentile 1 in 1,000in1 8.28E 1.10E 1.84E 9.81E 1 in 540in 1 42 1 in 12 in 1 90 in 1 95

th - - - - 02 02 03 04

JSC S&MA Analysis Branch MPCV Program MPCV Program (Conditional) Conditional Conditional

SLS Program SLS Program

Abort LOC System 2 System 1 Failure LOM LOC LOM LOC

1/10000 Showing Showing UncertaintyRequirements wrt 2,500 Red Green 1 in

1,800 1 in

Bar Bar 1 in 1,600

Bar shows Requirement shows Bar Requirement Value is met shows Requirement shows Requirement Value is 1/1000 1 in 1000 Notional

1,000 1 in

1 in 500 1 in 200

1 in 150

1/100

not 1 in 100

met

1 in 30

43

1 in 18

1/10 1 in 10

JSC S&MA Analysis Branch

They assume Theyassume completeness System Failure System MPCV Abort LOC Due Due LOC to MPCV Direct MPCVDirect GSDO Direct GSDO Direct 7% No PieCharts, GN&C Failure MPCV Abort

LOC Due Due LOC to LOC 10% LOC 3% Egress LOC Emergency What What Not toDo 14%

16%

and

we we know we that know don’t all. it LOC DIRECT 50% - SLS

-

44

JSC S&MA Analysis Branch

Subsystems Subsystems and A Pareto chart like this can made be for each project, mission phase, etc. Scenarios Various Notional

(e.g. Top Risk)80% Calculated(e.g. Top of

Risk DriversRisk via Pareto % of of % Risk

1 in in 1 xxxTotal Risk

45

JSC S&MA Analysis Branch

nice exponential curve, but only after something fails end and undoing changes). doesn’t Note that risk decrease according to a changes over a This • Design#1 Change

chart chart shows Probability 0.02 0.04 0.06 0.08 0.12 0.1 0 1:12

1 1:10

5

1:10 Risk RegressionExample 10 • Design Change

30 year program year 30 program by peeling back the “onion” (starting at the 15

how 20

1:10 # 2

25 1:36 1:17

calculated calculated changed risk following 30

35 •

Design Change • 40

Design Change •

• Design Change Design Change 45 1:37 •

Design Change Flight Sequence # Flight #6 50

# • 4 Design Change

# 55 # 3 5

#7 60

65 #

8

70

1:38 75

80 •

Design Change 1:21 • Design Change 85

1:21 90

and 1:47 #10 95

• #9 Design# Change

100 design design and ops •

Design# Change

it gets gets it “fixed”. 105 1:47

13 110 •

Design# Change

1:73

115 11

120

125 12

46 130 1:90

JSC S&MA Analysis Branch • • • •

it work. to make programs and disciplines acrosssupport management …., needs thus available, when evidence historical (as applicable), &performance health input, human operational analysis, engineering It on system.the be, based is, or should is PRA Training Risk) vs (e.g.HRAdrivers risk rank and identify help to industries in multiple been demonstrated a tool has is that PRA useful arebut wrong, some are all models "Essentially, probabilistic. probabilistic. are thus happen, that never hopefully events address of PRAs data / experience.amounts largeaddressStatistics not

just a math model, but an integrated assessment of of assessment anintegrated but model, a just math

Today’s“Take Aways”

.“ 47

JSC S&MA Analysis Branch • • •

tryingareto decide: you questions when right the today. to inorderask you insight give was to This presentation PRA about know to more seen what you’vethan is much There Roger.L.Boyer@nasa.gov questions have me if Call you drivers. risk top the on focusing by problem the solve help can ReliabilityEngineers and analysts & Effect Mode the Failure riskcontributors, the (FMEA)Analysis PRAidentifies analyses) deterministic helpof (with the o o o

is it answering the question(s) you is it being performed properly and by qualified analysts, whether you a need PRA not, or In Closing

at

(281) 483 need

answered. answered.

- 6070 or e or 6070

- mail meat and 48

ranks JSC S&MA Analysis Branch

Questions?

49

JSC S&MA Analysis Branch

Backup Charts Backup

50

JSC S&MA Analysis Branch • • •

performed in“failure space”). is PRAthe failure (since a of probability having to is assessed inaPRAevent Each ranking”. relative the just risk estimate, absolute the believe “Don’t heard, mayhave You working on the top risk drivers firstdrivers risk top the on working it to by to how reduce want others know while risk, overall the know want takersor riskto makers decision a result, some As – – – their their pay help to minimize the difference. so their may rankings. where is This experienced PRA analysts earn needed in scope full PRAs), then the absolutes be can challenged and different If approaches and methods are used sometimes (which are added produce to the overall risk. probability of each scenario thus ranks the scenarios as well as being the failure probabilities of event each are used establish to the determine these failures are combined via the failure logic used is which to how they are combined they how are combined AbsoluteRelativevsRisk?

and the resulting scenarios. .

51

JSC S&MA Analysis Branch • • • •

to implementation” or “concept to closeout” throughout program the life cycle from “formulation To support Risk Informed Decision Making (RIDM) or the agencycan live with either todue it being crewed When therisk of losing theproject is greater than Design (RID minimal cost affect the design and corresponding risk with earlyAsin the design as youprocess can in to order Order Flying Nuclear Payloads in support of Presidential

for financial for financial reasons

When When Should You PRA? Do a ))

impact impact (i.e.

to support Risk Informed

52

JSC S&MA Analysis Branch • •

being assessed being assessed the and mission life cycle desired as well as the size/complexity of the item The of cost is PRAa a function oflevel the of detail do PRA?”a you As can also ask, “How will much it to cost

– – mission duration and operational phases. phases. operationaland durationmission to duetheir quitecomplex becan Mars and landers probes space deep However, satellite. communications Earth an scalethan a different is on Shuttle Space andISS the Modeling that time. at assessment reliabilitybut afurther. It nota PRA, belevel may thatto studyyouneedyouthat is telling PRAyour then sublevel, at a exists risk that significantfurther. may Youidentifynoand data youhaveof that detail tolevel the model only shouldYou

How How much does a PRAcost?

not 53