NASANASA ActivitiesActivities inin RiskRisk AssessmentAssessment NASANASA ProjectProject ManagementManagement ConferenceConference MarchMarch 30-31,30-31, 20042004 Michael G. Stamatelatos, Ph.D.,Director Safety and Assurance Requirements Division Office of Safety and Mission Assurance NASA Headquarters 1 NASA is a Pioneer and a Leader in Space; Therefore Its Business Is Inherently Risky International Space Station Safe assembly and operation Space Transportation Space Shuttle Orbital Space Plane 2 Our Goal z Improve risk awareness in the Agency ¾ Conduct PRA training for line and project managers and for personnel z Develop a corps of in-house PRA experts z Transition PRA from a curiosity object to baseline method for integrated system safety, reliability and risk assessment z Adopt organization-wide risk informed culture ¾ PRA to become a way of life for safety and technical performance improvement and for cost reduction ¾ Implement risk-informed management process 3 Probabilistic Risk Assessment (PRA) Answers Three Basic Questions Risk is a set of triplets that answer the questions: 1) What can go wrong? (accident scenarios) 2) How likely is it? (probabilities) 3) What are the consequences? (adverse effects) Kaplan & Garrick, Risk Analysis, 1981 Event Event Sequence Sequence Frequency Modeling Evaluation 2. How frequently does it happen? (Scenario frequency quantification) Event Initiating Sequence Risk Event PRA Insights Logic Integration Selection Development 1. What can go wrong? Risk statement Decision Support (Definition of scenarios) 3. What are the consequences? (Scenario consequence quantification) Consequence Modeling PRA is generally used for low-probability and high- consequence events 4 Relationship Between Risk Management and Probabilistic Risk Assessment (PRA) Continuous Risk Management Method Technique Application QualitativeQualitative FMEA. RiskRisk FMEA. MLD, AssessmenAssessmentt MLD, ESD,ESD, Technical ETA, Technical ETA, RiskRisk FTA,FTA, RBDRBD and/or ProbabilisticProbabilistic and/or Risk Risk Program Assessment ActuActuarial/arial/ Program Assessment Risk StatisticalStatistical Risk AnalyAnalysesses Legend: Decision Decision FMEA - Failure Modes & Effects Analysis Analysis Analysis MLD - Master Logic Diagram ManagementManagement ESD - Event Sequence Diagram SySystemstem ETA - Event Tree Analysis FTA - Fault Tree Analysis RBD - Reliability Block Diagram 5 NASA Risk Management and Assessment Requirements • NPG 7120.5A, NASA Program and Project Management Processes and Requirements ¾ The program or project manager shall apply risk management principles as a decision-making tool which enables programmatic and technical success. ¾ Program and project decisions shall be made on the basis of an orderly risk management effort. ¾ Risk management includes identification, assessment, mitigation, and disposition of risk throughout the PAPAC (Provide Aerospace Products And Capabilities) process. • NPG 8000.4, Risk Management Procedures and Guidelines ¾ Provides additional information for applying risk management as required by NPG 7120.5A. • NPG 8705.x (draft) PRA Application Procedures and Guidelines ¾ Provides guidelines on how to apply PRA to NASA’s diversified programs and projects 6 How Does PRA Help Safety? Provides a basis for risk reduction through: 1. Accident/Mishap Prevention (best) 2. Accident/Mishap Consequence Mitigation Adverse Event in a Sequence Prevention Mitigation 7 The Concept of an Accident Scenario PIVOTAL EVENTS Pivotal Pivotal End State IE Event 1 Event n Detrimental The perturbation Mitigative consequence of Aggrevative interest (Quantity of intereset to decision-maker) Risk Scenario is a string of events that (if they occur) will lead to an undesired end state. 8 Exact vs. Uncertain Probabilities Uncertainty distribution of probability values Exactly known probability ability ability b value b Pro Pro A narrower range means a more precise knowledge of the distribution, or less uncertainty 9 Quantification of Uncertainty Probability density function, e.g., probability of LOCV Uncertainty Distribution: P(x) is the probability (or xth percentile ρ(x) confidence) that the result value is x P(x) median is the 50th percentile is area under 5% x 50% 95% curve between Median 0 and x Uncertainty Range: Uncertainty range 5th percentile 95th percentile (spread) from the 5th to the 95th percentile Uncertainty (confidence) range 10 Event- and Fault-Tree Scenario Modeling No damage to No damage to Hydrazine leaks Leak detected Leak isolated flight critical scientific End state Leak not detected avionics equipment IE LD LI A S OK Loss of science Loss of Spacecraft Presure Presure Controller fails transducer 1 transducer 2 OK fails fails Loss of science CN P1 P2 common cause Loss of failure of P. Spacecraft transducers OK PP distribution Loss of science Loss of Spacecraft Fault tree Event tree 11 PRA Methodology Synopsis Inputs to Decision Making Process Master Logic Diagram (Hierarchical Logic) Event Sequence Diagram (Logic) IE End State: ES1 A B End State: OK EnEndd S Stattate:e: ES2 ES2 End State: ES2 C D E End State: ES1 End State: ES2 LING DE C MO Event Tree (Inductive Logic) Fault Tree (Logic) One to Many Mapping of an ET-defined Scenario I End Not A IE A B C D E NEW STRUCTURE State LOG Logic Gate 1: OK Basic Event Internal initiating events One of these events External initiating events 2: ES1 Hardware components AND 3: ES2 Human error 4: ES2 Software error one or more Common cause of these 5: ES2 elementary Environmental conditions events 6: ES2 Other Link to another fault tree Probabilistic Treatment of Basic Events Model Integration and Quantification of Risk Scenarios Risk Results and Insights 30 50 60 25 50 40 End State: ES2 Integration and quantification of 20 40 30 100 logic structures (ETs and FTs) 15 IC 30 Displaying the results in tabular and graphical forms 10 20 80 and propagation of epistemic 20 uncertainties to obtain Ranking of risk scenarios 10 5 10 End State: ES1 60 ST 0.02 0.04 0.06 0.08 Ranking of individual events (e.g., hardware failure, I 0.01 0.02 0.03 0.04 0.02 0.04 0.06 0.08 minimal cutsets (risk 40 human errors, etc.) scenarios in terms of Examples (from left to right): 20 basic events) Insights into how various systems interact Probability that the hardware x fails when needed likelihood of risk Tabulation of all the assumptions ABIL Probability that the crew fail to perform a task 0.01 0.02 0.03 0.04 0.05 scenarios Identification of key parameters that greatly B Probability that there would be a windy condition at the time of landing uncertainty in the likelihood estimates influence the results The uncertainty in occurrence of an event is Presenting results of sensitivity studies characterized by a probability distribution PRO 12 What Decision Types Can PRA Support? z Safety improvement in design, operation, maintenance and upgrade (throughout life cycle); z Mission success enhancement; z Performance improvement; and z Cost reduction for design, operation and maintenance For all these areas of application, PRA can help: z Identify leading risk contributors and their relative values z Indicate priorities for resource allocation z Optimize results for given resource availability 13 Areas of PRA Application at NASA z In Design and Conceptual Design (e.g., Crew Exploration Vehicle, Mars missions, Project Prometheus) z For Upgrades (Space Shuttle) z For Development/construction/assembly (e.g., International Space Station) z When there are requirements for Safety Compliance (e.g., nuclear missions like Mars ’03; Project Prometheus, Mars Sample Return) 14 NASA Procedural Requirement NPR 8705 (Draft) CONSEQUENCE NASA PROGRAM/PROJECT CRITERIA / SPECIFICS PRA SCOPE CATEGORY (Classes and/or Examples) Planetary Protection Program Mars Sample Return Missions F Requirement White House Approval Nuclear Payloads Public Safety F (PD/NSC-25) (e.g., Cassini, Ulysses, Mars 2003) Human Safety and Space Missions with Flight Launch Vehicles F Health Termination Systems International Space Station F Human Space Flight Space Shuttle F Orbital Space Plane/Space Launch Initiative F High Strategic Importance Mars Program F Launch Window High Schedule Criticality F (e.g., planetary missions) Mission Success Earth Science Missions L/S (for non-human (e.g., EOS, QUICKSCAT) rated missions) Space Science Missions All Other Missions L/S (e.g., SIM, HESSI) Technology Demonstration/Validation (e.g., L/S EO-1, Deep Space 1) F = Full scope; L/S = Limited or Simplified PRA Scope Legend: F = Full scope; L = Limited scope; S = Simplified PRA 15 NASA Special PRA Methodology Needs z Broad range of programs: Conceptual non-human rated science projects; Multi-stage design and construction of the International Space Station; Upgrades of the Space Shuttle z Risk initiators that vary drastically with type of program z Unique design and operating environments (e.g., microgravity effects on equipment and humans) z Multi-phase approach in some scenario developments z Unique external events (e.g., micro-meteoroids and orbital debris) z Unique types of adverse consequences (e.g., fatigue and illness in space) and associated databases z Different quantitative methods for human reliability (e.g., astronauts vs. other operating personnel) z Quantitative methods for software reliability 16 Space Shuttle Probabilistic Risk Assessment 17 STS Nominal Mission Profile ASCENT completes ORBIT completes APU Shutdown TAL TIG-5 t ~ 150kft) SSME start ENTRY completes APU start 18 Current Shuttle PRA Results for LOCV (provisional) 1/10025 median = 1 in 123 y t i s n e on i D mean = 1 in 76 t y t i l i 5th percentile = 1 in 617 Func ab b o r 95th percentile = 1 in 23 P 1/250 0.1/00500 1/0.10001 0.1/0250 1/0.3303 1/0.2504 1/0.2005 truncated Number of Missions 19 Summary of Shuttle PRA Historical Results 1993 PRA 1988 PRA update the First PRA 2003 PRA 1998 PRA Galileo study conducted on Integrated Unpublished 1995 PRA results to reflect the Space PRA with all analysis using First major the current Shuttle for 0.1 elements. QRAS. No study of (April 1993) test 1996 PRA ascent only, 18 Orbiter Integration of 1997 PRA Shuttle PRA.