Overview of Risk Assessment Methods & Applications
Overview of Risk Assessment Methods & Applications
Dan Weese Executive Director, Corporate Quality Engineering Objectives
• Know the most commonly used risk assessment methods • Understand that the effectiveness of risk assessments are dependent on: • The ability of the assessment facilitator • Participation of appropriate Subject Matter Experts in the assessment • Establishment of useful scoring criteria • Establishment of a threshold for action • Know that effective risk management integrates risk assessment methods into a larger framework
2 Risk Assessment is only one part of Risk Management
Initiate Quality Risk Management Process
Risk Assessment Risk Identification
Risk Analysis
Risk Evaluation
Unacceptable Risk Control Risk Reduction
Risk Acceptance Risk Management Tools Management Risk Risk Communication Risk
Output / Result of the Quality Risk Management Process
Risk Review Review Events
Source: ICH Q9, June 2006, Figure 1
3 ICH Q9 Annex I provides an overview of commonly used risk management (assessment) methods & tools
• Failure Mode Effects Analysis (FMEA)
• Failure Mode, Effects and Criticality Analysis (FMECA)
• Fault Tree Analysis (FTA)
• Hazard Analysis and Critical Control Points (HACCP)
• Hazard Operability Analysis (HAZOP)
• Preliminary Hazard Analysis (PHA)
• Risk Ranking and Filtering
4 The methods & tools can be separated into different levels of detail & complexity
Methods & Tools
Simpler Methods • Preliminary Hazard Analysis (PHA) Process/product knowledge limited • Risk Ranking and Filtering High level view of potential risks
Detailed Methods • Failure Mode Effects Analysis (FMEA) Process/product knowledge necessary • Hazard Analysis and Critical Control Points (HACCP)
More Detailed Methods or • Customized Assessment Methods knowledge of methods Process/product knowledge • Fault Tree Analysis (FTA) much more important • Hazard Operability Analysis (HAZOP) Most Detailed Methods • Failure Mode, Effects and Criticality Extensive process/ product Analysis (FMECA) knowledge required
5 Preliminary Hazard Analysis (PHA)
Description: High-level analysis method used to prioritize product or process hazards
Deliverable: Risk Ranking (RR) of each identified hazard = Severity of Likelihood of Consequence X Hazard Occurring (Harm) Score Score (See note)
Note: ISO 14971:2007(E) computes the likelihood of occurrence in a hazard analysis as the Probability of a hazardous situation occurring X Probability of a hazardous situation leading to harm. However, this definition is never directly associated with the name PHA.
Benefits: Limitations: • Does not require detailed knowledge of product or process • Results are very high-level • Typically used to identify product or process hazards early • Usually requires a follow-on analysis in development when mitigation costs are lowest; however, • Subjective assessment dependent on involvement of can be used anytime appropriate SME’s • Screens & prioritizes hazards – precursor to detailed risk assessments 6 Risk Ranking & Filtering
Description: High-level analysis method that combines qualitative & quantitative factors to compare & rank risks
Deliverable: Relative Risk Score =
Example
Benefits: •Simple • Business-focused
Limitations: • High-level • Subjective assessment dependent on involvement of appropriate SME’s 7 Failure Mode & Effects Analysis (Product)
Description: “Bottoms-up” analysis method used to identify potential product failures and evaluate the effect of the failure.
Deliverable: Risk Priority Number (RPN) of Severity of Likelihood of Detectability of each identified potential failure/cause combination = Effect of Cause Failure or Failure X Occurring X Cause Score Score Score
Benefits: Limitations: • Screens & prioritizes failures • Product functions must be defined • Most commonly used qualitative risk assessment • Tedious for complex processes method • Does not directly capture interactions (e.g. effect of multiple failures) • Subjective assessment dependent on involvement of appropriate SME’s
8 Failure Mode & Effects Analysis (Process)
Description: “Bottoms-up” analysis method used to identify potential process failures and evaluate the effect of the failure.
Deliverable: Risk Priority Number (RPN) of Severity of Likelihood of Detectability of each identified potential failure/cause combination = Effect of Cause Failure or Failure X Occurring X Cause Score Score Score
Benefits: • Screens & prioritizes failures • Most commonly used qualitative risk assessment method
Limitations: • Process must be defined • Tedious for complex processes • Does not directly capture interactions/multiple failures • Subjective assessment dependent on involvement of appropriate SME’s
9 Hazard Analysis Critical Control Points (HACCP)
Description: Critical control points are identified based on a completed Hazard Analysis
Deliverable: Critical Control Points, Limits & Monitoring =
Benefits: • Typically used to identify & manage risks associated with physical, chemical & biological hazards
Limitations: • Hazard analysis must be completed first • Comprehensive product/process understanding required
10 Customized Assessments (Example)
Description: Customize risk assessment method to fit different situations
Deliverable: Risk Rating (RR) related to narrowly defined effects such as contamination events or specific Quality Attributes Example
Benefits: • More focused assessments • Less time-consuming
Limitations: • Narrow focus – may not uncover other effects related to a failure mode • Specific expertise must participate in the assessment
11 Fault Tree Analysis (FTA)
Description: Pictorial analysis method that uses “logic” gates to identify root causes and/or combinations of root causes that could lead to the single, identified failure. Failure
Deliverable: Fault Tree
Benefits: AND • Typically used to complete complaint or deviation root cause analysis • Effective in identifying multiple causes that Contributory Event Contributory Event could lead to a failure A B • Can be used to identify causes in an FMEA
Limitations: OR • Narrow focus – single failure • Requires significant process information/expertise • Can be time-consuming Basic Basic • Requires more experienced Event B1 Event B2 facilitator/analyst
12 Hazard Operability Analysis (HAZOP)
Description: Rigorous analysis method that uses guide words to identify potential deviations from design or operating intentions.
Deliverable: Risk Ranking (RR) of Severity of Likelihood of each identified potential consequence/cause Consequence X Cause combination for particular “nodes” = Score Occurring Score
Benefits: Limitations: • Systematically identifies hazards • Time-consuming • Thoroughly and proactively examines potential • System or process must be well-defined consequences of deviations • Hazards may be missed due to focus on equipment
13 Failure Modes, Effects & Criticality Analysis (FMECA)
Description: Extension of an FMEA from a qualitative to a quantitative analysis method
Deliverable: Criticality Conditional Fraction of Number (C ) probability part failure Part m Operating for each that effect rate resulting failure X X X Time identified failure will result in in failure rate mode = severity mode rating
Then, sum Cm for each item or function
Benefits: • Quantitative vs qualitative assessment
Limitations: • Requires extensive failure information
14 Effectiveness of qualitative risk assessment methods depends on …
• Having an individual well-trained in facilitation and in the method lead the assessment – Scoring is consensus-based
• Ensuring the appropriate Subject Matter Experts (SME’s)participate in the assessment − Value of every assessment is directly related to input from SME’s
• Establishing useful definitions for scoring criteria
• Having a pre-established threshold for action
15 Effective Risk Management integrates assessment methods into a larger framework
Initiate Quality Risk Management Process
Risk Assessment Risk Identification
Risk Analysis
Risk Evaluation
Unacceptable Risk Control Risk Reduction
Risk Acceptance Risk Management Tools Management Risk Risk Communication Risk
Output / Result of the Quality Risk Management Process
Risk Review Review Events
Source: ICH Q9, June 2006, Figure 1
16 Summary
• Reviewed the most commonly used risk assessment methods • Discussed the dependencies of an effective risk assessment: • Ability of the assessment facilitator • Participation of appropriate Subject Matter Experts • Establishment of useful scoring criteria • Establishment of a threshold for action • Effective risk management integrates risk assessment methods into a larger framework
17 One last risk scenario…
You are a South African bush pilot
Working for Blue Sky Aviation.
You fly in some critical medical supplies,
Enjoy a quick lunch at the hospital.
It's a stifling 100 degrees in the shade and
You're eager to get back aloft to the cooler upper atmosphere.
On the way back to your aircraft,
You discover that the only bit of shade within 1 mile
Has become very popular.
You start estimating the distance to the aircraft door and wonder...
18 'Do I feel lucky today?‘ Well do you??? One last risk scenario…
19