Journal of Industrial Technology • Volume 20, Number 2 • February 2004 to April 2004 • www.nait.org Volume 20, Number 2 - February 2004 to April 2004 A Statistical Comparison of Three Root Cause Analysis Tools By Dr. Anthony Mark Doggett Peer-Refereed Article KEYWORD SEARCH Leadership Management Quality Research Sociology The Official Electronic Publication of the National Association of Industrial Technology • www.nait.org © 2004 1 Journal of Industrial Technology • Volume 20, Number 2 • February 2004 to April 2004 • www.nait.org A Statistical Comparison of Three Root Cause Analysis Tools By Dr. Anthony Mark Doggett To solve a problem, one must first Spencer, 1998; Dettmer; 1997; Lepore recognize and understand what is & Cohen, 1999; Moran et al., 1990; Dr. Mark Doggett is a post-doctoral fellow and causing the problem. According to Robson, 1993; Scheinkopf, 1999; instructor at Colorado State University and is an adjunct faculty member at Aims Community Col- Wilson et al. (1993), a root cause is the Smith, 2000) lege. He is currently working on grants for the National Science Foundation in medical technol- most basic reason for an undesirable For example, Ishikawa (1982) ogy and the Department of Education in career condition or problem. If the real cause advocated the CED as a tool for and technical education. He also teaches courses in process control, leadership, and project man- of the problem is not identified, then one breaking down potential causes into agement. His research interests are decision-mak- is merely addressing the symptoms and more detailed categories so they can be ing and problem-solving strategies, technical management, theory of constraints, and opera- the problem will continue to exist. For organized and related into factors that tions system design. this reason, identifying and eliminating help identify the root cause. In contrast, root causes of problems is of utmost Mizuno (1979/1988) supported the ID importance (Dew, 1991; Sproull, 2001). as a tool to quantify the relationships Root cause analysis is the process of between factors and thereby classify identifying causal factors using a potential causal issues or drivers. structured approach with techniques Finally, Goldratt (1994) championed designed to provide a focus for identify- the CRT as a tool to find logical ing and resolving problems. Tools that interdependent chains of relationships assist groups and individuals in identify- between undesirable effects leading to ing the root causes of problems are the identification of the core cause. known as root cause analysis tools. A fundamental problem for these tools is that individuals and organiza- Purpose tions have little information to compare Three root cause analysis tools them to each other. The perception is have emerged from the literature as that one tool is as good as another tool. generic standards for identifying root While the literature was quite complete causes. They are the cause-and-effect on each tool as a stand-alone applica- diagram (CED), the interrelationship tion and their relationship with other diagram (ID), and the current reality problem solving methods, the literature tree (CRT). There is no shortage of is deficient on how these three tools information available about these tools. directly compare to each other. In fact, The literature provided detailed there are only two studies that com- descriptions, recommendations, and pared them and the comparisons were instructions for their construction and qualitative. Fredendall et al. (2002) use. The literature documented pro- compared the CED and the CRT using cesses and structured variations for previously published examples of their each tool. Furthermore, the literature is separate effectiveness while quite detailed in providing colorful and Pasquarella et al. (1997) compared all illustrative examples for each of the three tools using a one-group post-test tools so practitioners can quickly learn design with qualitative responses. and apply them. In summary, the There is little published research that literature confirmed that these three quantitatively measures and compares tools do, in fact, have the capacity to the CED, ID, and CRT. This study find root causes with varying degrees attempted to address those deficiencies. of accuracy, efficiency, and quality The purpose of this study was to (Anderson & Fagerhaug, 2000; Arcaro, compare the perceived differences 1997; Brown, 1994; Brassard, 1996; between the independent variables: the Brassard & Ritter, 1994; Cox & 2 Journal of Industrial Technology • Volume 20, Number 2 • February 2004 to April 2004 • www.nait.org cause-and-effect diagram (CED), the participant groups and did not affect process. The activity of the facilitators interrelationship diagram (ID), and the the overall perceptions or results. Also, was intended to help control the current reality tree (CRT) with regard the sample problem scenarios used in potential diffusion of treatment across to causality, factor relationships, the study were considered as having the groups. usability, and participation. The first equal complexity. To ensure consistency, each dependent variable was the perceived treatment packet was similarly format- ability of the tool to find root causes Methodology ted with the steps for tool construction and the interdependencies between The specific design was a within- and a graphical example based on causes. The second dependent variable subjects single factor repeated mea- published material. Each treatment was the perceived ability of the tool to sures with three levels. The indepen- group also received the same supplies find relationships between factors or dent variables were counterbalanced as for constructing the tools. The depen- categories of factors. Factors may shown in Table 1, where T represents dent variables were measured using a include causes, effects, or both. The the treatment, M represents the mea- twelve-question self-report question- third dependent variable was the sure, and the group observations are naire with a five-point Likert scale and overall perception of the tool’s usabil- indicated by O. The rationale for this semantic differential phrases. ity to produce outputs that were design is that it compares the treat- logical, productive, and readable. The ments to each other in a relative Procedure fourth dependent variable was the fashion using the judgments of the Participants and facilitators were perception of participation resulting in participants. In this type of comparative randomly assigned to one of three constructive discussion or dialogue. In situation, each participant serves as his groups. The researcher provided addition, the secondary interests of the or her own control making the use of simultaneous instructions about the study were to determine the average independent groups unnecessary experiment, problem scenarios, and process times required to construct (Girden, 1992). The advantage of this materials. Five minutes were allowed each tool, the types of questions or design is that it required fewer partici- for the participants to review their statements raised by participants during pants while reducing the variability respective scenarios and instructions, and after the process, and the nature of among them, which decreased the error followed by a ten minute question the tool outputs. term and the possibility of making a period. The participants were then asked Type I error. The disadvantage was that to analyze and find the perceived root Delimitations, Assumptions, and it reduced the degrees of freedom cause of the problem. The facilitators Limitations (Anderson, 2001; Gliner & Morgan, were available for help throughout the The delimitations of the study were 2000; Gravetter & Wallnau, 1992; treatment. Upon completion of the that the tools were limited to the CED, Stevens, 1999). treatment, the participants completed the ID, and CRT while participants were self-report instrument. This process was limited to small groups representing an Measures and Instrument repeated until all groups applied all authentic application of use. The Three facilitators were trained in three analysis tools to three randomized limitations of the study were grounded the tools, processes, and procedures problems. Each subsequent treatment in the statistical requirements for the before the experiment. They were was administered every seven days. General Linear Model. The experimen- instructed to be available to answer tal results reflected the efficacy of the questions from the participants about Reliability and Validity tools in the given context. While the the tools, goals, purpose, methods, or Content validity of the instrument researcher attempted to control obvious instructions. The facilitators did not was deemed adequate by group of extraneous variables during the study, provide information about the problem graduate students familiar with the participant and organizational cultural scenarios. They were also trained in research. Cronbach’s alpha was .82 for attributes, politics, and social climate observational techniques and instructed a pilot study of 42 participants. The remained outside the scope and control to intervene in the treatment process if dependent variables were also congru- of the analysis. a group was having difficulty con- ent with an exploratory principle The assumptions of the study were structing the tool or managing their component analysis. that (a) root cause analysis techniques are useful in finding root causes, (b) the identification