Root Cause Analysis and the Audit Life Cycle

ROOT CAUSE ANALYSIS AND THE AUDIT LIFE CYCLE

DEREK BUTLER SENIOR MANAGER GRANT THORNTON LLP +1 443.414.3257 [email protected]

AHIA 35th Annual Conference – September 11-14, 2016 www.ahia.org

Agenda

 Define what is Root Cause Analysis and discuss some key features

 Identify common uses of Root Cause Analysis

 Identify tools and techniques for conducting Root Cause Analysis

 Highlight benefits and challenges associated with conducting Root Cause Analysis

2 What is Root Cause Analysis? Root Cause Analysis - Definitions and Terms

 Root Cause Analysis (RCA) is a:

− Systematic, formalized approach used for reviewing an adverse event and identifying root causes

− Tool designed to help identify not only what and how an event occurred, but also why it happened using a structured format

− A process for analyzing the possible causes to a problem and related effect(s)

− It not only aims to identify the true cause of a problem, but the actions necessary to eliminate it and prevent recurrence

 Root Cause Analysis is also referred to as:

− Ishikawa diagrams

− Fishbone diagrams

− Cause-and-effect diagrams

What is Root Cause Analysis? Root Cause Analysis - Point of View

When done correctly, a RCA is a powerful tool that deserves a prominent place in your problem-solving toolkit as internal auditors.

− Provides fact-based statements that describe the deepest underlying cause, or causes, of operational or performance challenges

− Force us to focus on things that can be changed and need to change. Efforts are redirect from areas where you have no control over (e.g., weather)

− Provides a healthy forum for exploring multiple possibilities. It is a tool that allows multiple points of views and voices to be integrated into the conversation and our thoughts for consideration

− Requires that a consensus to be reached around the factors that lead to the problem so that effective correction action plans can be developed

What is Root Cause Analysis? Root Cause Analysis - Point of View

What I’ve observed and learned over the years:

− RCA is sometimes not used appropriately (or at the right time)

• Solution-centered mindsets renders the RCA irrelevant

• Potential root causes are not validated

• Not used as intended and ends up as merely interesting artwork in a presentation to management

− RCA are sometimes a rushed production versus a methodical and thoughtful approach to finding potential – and ultimately – the root cause.

− Lack a focused problem statement which produces a whale versus a simple fish

When is Root Cause Analysis not the best tool? Root Cause Analysis - Sample Problem Descriptions

To use – or not to use - that is the question . . .

 When a RCA is not the best tool:

• The company wants to know if the number of emergency room complaints have increased over the last two years.

• A hospital is receiving numerous complaints from patients that are being admitted and who are upset because the admission process was cumbersome and required patients to endure a lengthy process. To better understand what is going on, the hospital wants to investigate the time it takes to admit a patient and what opportunities exist to streamline the process.

• Given the size and nature of the operations for many healthcare systems and the degree of sensitivity that now exists around patient access and information, managing the physical access and providing the appropriate degree of security to ensure a safe environment for patients, visitors, employees, and volunteer staff continues to be a major issue. As a result, the company is interested in benchmarking its security protocols and standards that are currently in place against those that are deemed to be leading practices and more widely used in the healthcare industry today.

No focused problem statement (yet).

When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

“A problem well defined is a problem half solved”

7 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

 Understanding the severity of the problem is a key component to conducting an effective RCA. There are several standards that must be utilized when developing and writing an appropriate problem description.

 The fundamental elements to defining a problem addresses: 3W:

− What was affected?

− When was the problem discovered?

− Why did the problem happen? 2H:

− How much was affected?

− How often has the problem occurred? 1C:

− What is the consequence?

 Less focus on "where the problem took place" and "who discovered it"

8 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

Developing a complete problem description statements requires that we focus on what, when, where, and the how aspects of a problem. “Ask What, How, and Why?” Determine Root Cause

What Happened? How it Happened? Why It Happened? Drive & Anchor the Improvement

. Trace the problem to its origins . Categorize the causes of events (causal factor, direct cause, contributing cause, root cause)

9 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

 There are often several causes to a problem:

− Causal Factor – a condition or event that results in an effect

− Direct Cause – cause that directly resulted in the occurrence

− Contributing Cause – a cause that contributed to the occurrence, but by itself would not have caused the occurrence

− Root Cause – cause that if corrected, would prevent recurrence of a non- conformity and similar occurrences both within the department and more widely across the organization

 Understanding not only the problem, but the causes to the problem is key to ensuring you do not have a poor RCA. Don't assume the problem is known and understood well enough to determine the root cause without understanding the underlying causes.

 Examples of traditional root causes tend to focus on lack of a procedure, training of stakeholders in applicable policy/procedures, lack of an effective tool or system to perform a specific task, etc.

10 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

Common questions that can stimulate the thought process and get you on the track of identifying the various causes to a problem.

Objective Getting to the Cause – Questions to Consider What are the risks? • What could this problem impact? • Does results indicate a theme or pattern? Determine Significance • Is there a larger business issue? • How widespread is the issue? • How material is the issue to financial results or to the organization future and strategic plans? Implications / Impact • How critical is the area? Is this mission critical or non-critical? • Who is affected by this problem – customers, employees, vendors, etc.? • What is the reputational risk? So What? / Why Should • Would this issue be embarrassing on the front page of the newspaper? On I Care? television news? • Is there a regulatory or legal compliance concern? • Could there be fraud or illegal act committed? Professional Judgment • Should This Observation Be Rated High, Moderate or Low Risk? Why? • How does this compare with previous experience?

11 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

There are certain "Rules of the Road" to developing an appropriate problem statement that will allow for an effective RCA to be conducted.

 Write complete sentences:

− The problem description needs to be written clearly

− Stick to the facts. Don’t make it a story.

− Be precise in defining the requirement and deficiency

− Avoid abbreviations, acronyms…

 Problems / issues (Major, Minor) tend to “violate” some requirement or may result from the absence or insufficient performance of a process

− Include the violated requirement in the problem description

 Evaluate the sufficiency of problem description

− Identify What, Where, When, as appropriate

− There needs to be enough information from the problem statement to conduct RCA and identify corrective action 12 When is Root Cause Analysis the right way to go? Root Cause Analysis - Problem Description Requirements

 Seek to understand the severity of the problem

 Self-check the problem description

− Is the statement complete and self-sufficient?

− Ask yourself … So What?

Root Cause Analysis - Types

Process Capability Barrier Analysis

5 Why’s Control Plans Pareto Charts Fishbone Tree Diagrams Process Maps Risk Tree Analysis Apollo Interrelationship Digraph Pie Charts Run Charts Histograms Change Analysis Process FMEA Brainstorming Flowcharts Affinity Diagrams

Control Charts Design FMEA Events & Causal Analysis Scatter Diagram Force Field Analysis

Root Cause Analysis – Overall Process

Regardless as to the RCA model that is used, every option will require the same fundamental steps to be performed:

. Step 1 – Problem identification . Step 2 – Data collection . Step 3 – Identify possible causal factors . Step 4 – Root Cause identification and validation . Step 5 – Corrective action recommendations and implementation

“Every problem is an opportunity. Every defect is a treasure, if the company can uncover its cause and work to prevent it across the corporation.” Kilchiro Toyoda, founder of Toyota

Root Cause Analysis – Data Collection

Root Cause Analysis - 5 Why's

A more methodical approach to conducting a root cause analysis is to use the "Five Why" approach and diagramming…

 One of the simplest tools

 Easy to perform without statistical analysis

 Determines the relationships between different root causes of a problem by asking 5 times the question “Why?”

 The tool allows one to get down to the underlying causes without being deceived by appearances

 Provides a test for logic when examining actions or problems

 Can be used in an informal setting to elicit the best responses from your audience

Root Cause Analysis - 5 Why's

Root Cause Analysis – The Fishbone Diagram

Fishbone diagram, also referred as Ishikawa diagram or cause-and-effect diagram, is a common tool used for a cause and effect analysis, where one tries to identify possible causes for a certain problem or event.

Root Cause Analysis – The Fishbone Diagram

Employees System

The Supervisor is not performing The system is not notifying the a review of the information Supervisor that the review was provided by the application completed. owners. Employees not following established The system is not saving the recertification procedures information submitted. The re-certification process did not include supporting evidence for ten Recertification procedures are not The current training schedule for employees. clear on what evidence is required application owners does not Procedures are not include training on the application communicated or easily recertification review process accessible to the employees responsible for performing the A training covering the user recertification re-certification process has not been developed.

Policy & Training Procedures 20

Root Cause Analysis – Flowcharting

Provides a tool for understanding and analyzing processes from start to finish

Flowcharts illustrate the direct causes linked to their effects

Breaks down process into individual events to display these in shorthand form showing the logical relationships

Develops causal chains to aid in identifying root causes

Root Cause Analysis – Flowcharting

There are various forms of flowchart diagrams that can be used to conduct a RCA. Some of the more common and frequently used flowchart types are described below:

Basic Flowchart is a simple diagram of a step-by-step execution of a process. This type of flowchart is very simple and provides a good visual for creating and understanding a process. Basic flowcharts are widely used in the different areas of knowledge or documenting processes.

Business Process Modeling Diagram is a standardized graphical notation that allows the creation of a graphical view of the business processes in a workflow. This flowchart diagram was developed with the purpose of helping all business users to easily understand each other. It is a simple and visual tool.

Root Cause Analysis – Flowcharting

Cross Functional Flowchart is a one that is used to show the processes within a business model. This type of flowchart uses swim lanes. Cross functional flowcharts are very simple and useful, but demand a certain degree of detail.

Data Flow Diagrams is one of the main tools for structural analysis and design of the information systems. Data flow diagram graphically represents how the information system work and develop, the interactions between the system and external data sources in terms of data flows, the data flows that the information system receives from the outside.

Root Cause Analysis – Flowcharting

Workflow Diagram is a graphical description of the information flows, the relationships between the processes of information processing and the objects that are the part of these processes. Workflow Diagram describes the stages of the business processes from the initial to the final stage that allows the organization to trace the progress of the processes quickly and clearly.

Decision Tree Diagram is a decision support tool that uses a tree-like graph or model of decisions and their possible consequences, including chance event outcomes, resource costs, and other aspects. Decision trees are commonly used in operations and research, specifically in decision analysis, to help identify a strategy most likely to reach a goal.

Root Cause Analysis and Components of Professional Skepticism

Regardless of the RCA that is being used (5 Why's, Fishbone, or flowcharts), the relationship between the problem and possible causes must be clearly understood. Be prepared to use: • Knowledge and reasoning “Sixth – Technical and auditing skills Sense” – Industry knowledge and experience – Objective and rational thinking “Gut • Interpersonal skills Instinct” – Keep an open-mind – Gain an understanding of how other people approach their jobs – Communicate effectively with others, regardless of their position or “Smell power Test” – Be prepared to probe or question others – Maintain composure if conflict arises

RCA helps you think beyond your obvious gut feelings to arrive at the true root cause(s). 25

Root Cause Analysis - Validating Root Causes

Validating the root causes is essential to performing an effective RCA and to developing the appropriate correction actions.

Fishbone Diagram Problem Statement: The IT re-certification process did not include supporting evidence for ten employees.

Possible Root Causes Questions to Explore Data Sources Validation

IT has implemented a • What strategy is • Re-certification • New strategy new strategy for re- currently being used? strategy/policy effective 9/1/16 certification purposes, • When will staff receive • Training schedule • Staff training but staff training has training? scheduled in 2017 not occurred. • Staff interviews Less time is being • Are there open IT • Staff interviews No evidence exists devoted to the re- positions? that less time is • Listing of open job certification process devoted to the re- • How often does the re- requisitions due to staffing certification process. certification process shortages. occur?

26 What makes for a good Root Cause Analysis? Root Cause Analysis – Things to Consider

 Ensuring that the appropriate data collection and analysis is performed

 Focusing on evidence-based risk factors and causes of the problem

 Involving business owners and subject matter experts who have an understanding of the problem

 Narrowing and prioritizing to eliminate explanations over which you have no control. Eliminate explanations that are:

− Not logical based on the data that has been gathered

− Opinion-based

− Not able to be tested or verified

 Incorporating the 5 Why's concept into the process in order to continuously drive to the root cause. Keep asking....WHY?…at least 5 times!!!!!

 CONNECTING THE DOTS!!

− Determining whether the root cause fits the problem statement

− Validating the root cause

27 What are some benefits to conducting a Root Cause Analysis? Root Cause Analysis – Benefits

 Saves time and money

 Provides internal audit departments with a logical approach to problem solving, using data that already exists

 It can be used as an approach to deal with any problem – operational, compliance or financial

 Helps to identify underlying causes of problems, so that permanent solutions can be found

 Establishes a repeatable, step-by-step processes, in which one process can confirm the results of another

 Provides process owners with a long-term perspective for the improvement within the business and avoids encountering repeat findings

 Identifies systemic, rather than one-time solutions to problems

 Helps to ensure biases are minimized, assumptions are challenged, and evidence is fully evaluated before recommending corrective actions

 Helps develop a culture of collaboration across different departments or business units in order to find solutions from different perspectives

28 What are challenges to conducting a Root Cause Analysis? Root Cause Analysis – Challenges

Business Management Support Lack of Data/Subjectivity • Reluctant to support Internal • Determining root cause may be Audit's role difficult even when significant data is • Chief audit executive and auditors available may need to work with • Need input from multiple data management to demonstrate the sources to get to the root cause (e.g., value stakeholders, documentation, and system information) Time and Cost Considerations • Time and resource commitment Internal Audit Specific may not be available • Too many Internal Audit departments • Many focus on a short-term fix are often suspected of only recommend that management fix the • Long term fixes may not be readily issue and not the root cause identified if resources are not

available 29 HAVE QUESTIONS?

We've got nswers! Save the Date August 27-30, 2017

36th AHIA Annual Conference