1 You have performance data! Now, what do you do with it? 2 “If I had to reduce my message for management to just a few words, I’d say it all had to do with reducing variation.” W. Edwards Deming The Problem! 3 Aggregated data presented in tabular formats or with summary statistics, will not help you measure the impact of process improvement efforts. Aggregated data can only lead to judgment, not to improvement. Percent of A&E patients Seen by a Physician within 10 min Did we improve? What will happen next? Should we do something? Source: R. Lloyd Percent of A&E patients Seen by a Physician within 10 min 100% 95% 90% 85% 80% 75% 70% 65% 60% Change 55% made here 50% 1/9/2008 2/6/2008 3/5/2008 10/3/2007 1/23/2008 2/20/2008 3/19/2008 10/17/2007 10/31/2007 11/14/2007 11/28/2007 12/12/2007 12/26/2007 Did we improve? What will happen next? Should we do something? Source: R. Lloyd Change made between week 7 and week 8 Was the change an improvement? 10 9 8 8 7 6 5 Delay Time (hrs) Time Delay 4 3 3 2 1 0 1 2 Before Change After Change (measure on Week 4) (measure on week 11) Was the change an improvement? Case1 12 10 8 6 Delay Time (hrs) Time Delay 4 Make Change 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 2 10 Delay Delay Time (hrs) 8 Make 6 Change 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 3 10 8 Delay Delay Time (hrs) 6 4 Make Change 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Case 4 12 10 8 Delay Delay Time (hrs) 6 Make 4 Change 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 5 10 8 Make Delay Delay Time (hrs) 6 Change 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 6 10 8 Make Delay Delay Time (hrs) 6 Change 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 2 10 Delay Delay Time (hrs) 8 Random Variation Make 6 Change 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 3 10 Headed down before change. 8 Delay Delay Time (hrs) 6 Where begin? 4 Make Change 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Case 4 12 10 Change did not hold 8 Delay Delay Time (hrs) 6 Make 4 Change 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 5 10 Improvement: before change 8 Make Delay Delay Time (hrs) 6 Change (between week 4 & 5) 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 12 Case 6 10 Week 4 not typical of process 8 Make Delay Delay Time (hrs) 6 Change 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 People unclear of the concept! Percent Hand Hygiene “…and then another decrease in the percent compliance with hand hygiene this month. But, I have a really good feeling about next month! The average of a set of numbers can be 10 created by many different distributions Average Measure Time If you don’t understand the variation that lives in11 your data, you will be tempted to ... • Deny the data (It doesn’t fit my view of reality!) • See trends where there are no trends • Try to explain natural variation as special events • Blame and give credit to people for things over which they have no control • Distort the process that produced the data • Kill the messenger! Distorting the Data! “You'll be happy to see that I’ve finally managed to turn things around!” The 13 to understanding quality performance, therefore, lies in understanding variation over time not in preparing aggregated data and calculating summary statistics! “A phenomenon will be said to be controlled when, through the use of past experience, we can predict, at least within limits, how the Dr. Walter A Shewhart phenomenon may be W. Shewhart. Economic Control of expected to vary in Quality of Manufactured Product, 1931 the future” 15 “What is the variation in one system over time?” Walter A. Shewhart - early 1920’s, Bell Laboratories Dynamic View UCL Static View time Every process displays variation: LCL • Controlled variation stable, consistent pattern of variation “chance”, constant causes • Special cause variation Static View “assignable” pattern changes over time Types of Variation Common Cause Variation Special Cause Variation • Is inherent in the design of the process • Is due to irregular or unnatural causes that are not inherent in the design of the process • Is due to regular, natural or ordinary causes • Affect some, but not necessarily all aspects of the process • Affects all the outcomes of a process • Results in an “unstable” process • Results in a “stable” process that that is not predictable is predictable • Also known as non-random or • Also known as random or assignable variation unassignable variation Common Cause Variation 100 90 80 70 60 50 40 30 20 10 0 3/1/2008 3/8/2008 4/5/2008 5/3/2008 6/7/2008 3/15/2008 3/22/2008 3/29/2008 4/12/2008 4/19/2008 4/26/2008 5/10/2008 5/17/2008 5/24/2008 5/31/2008 • Points equally likely above or below center line • There will be a high data point and a low, but this is expected • No trends or shifts or other patterns Courtesy of Richard Scoville, PhD, IHI Improvement Advisor A Stable Process is Predictable ! Thus you can confidently: • Counsel patients about what to expect • Plan for the future • Inform management • Use PDSA testing to improve it! Courtesy of Richard Scoville, PhD, IHI Improvement Advisor Two Types of Special Causes Unintentional When the system is out of control and unstable Holding the Gain: Isolated Femur Fractures Intentional 1200 1000 800 When we’re trying 600 Patient to change the 400 200 Minutes ED to per OR system 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 Sequential Patients Courtesy of Richard Scoville, PhD, IHI Improvement Advisor Point …Variation exists! Common Cause does not mean “Good Variation.” It only means that the process is stable and predictable. For example, if a patient’s systolic blood pressure averaged around 165 and was usually between 160 and 170 mmHg, this might be stable and predictable but completely unacceptable. Similarly Special Cause variation should not be viewed as “Bad Variation.” You could have a special cause that represents a very good result (e.g., a low turnaround time), which you would want to emulate. Special Cause merely means that the process is unstable and unpredictable. 21 Common Cause Variation Special Cause Variation Normal Sinus Rhythm (a.k.a. Atrial Flutter Rhythm (a.k.a. Common Cause Variation) SpecialHolding the Gain:Cause Isolated Femur Variation) Fractures 1200 1000 800 600 Patient 400 200 Minutes ED to per OR 0 1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64 Sequential Patients Appropriate Management Response to 22 Common & Special Causes of Variation Is the process stable? YES NO Type of variation Only Common Special + Common Change the Investigate the origin of the Right Choice process special cause Treat normal variation as a Change the process Wrong Choice special cause (tampering) Wasted Consequences of Increased making the wrong resources! choice variation! (time, effort, morale, money) Source: Carey, R. and Lloyd, R. Measuring Quality Improvement in Healthcare: A Guide to Statistical Process Control Applications. ASQ Press, Milwaukee, WI, 2001, page 153. 2 Questions … 1. Is the process stable? If so, it is predictable. 2. Is the process capable? The chart will tell you if the process is stable and predictable. You have to decide if the output of the process is capable of meeting the target or goal you have set! 23 Understanding Variation Statistically Unplanned Returns to Ed w/in 72 Hours Month M A M J J A S O N D J F M A M J J A S ED/100 41.78 43.89 39.86 40.03 38.01 43.43 39.21 41.90 41.78 43.00 39.66 40.03 48.21 43.89 39.86 36.21 41.78 43.89 31.45 Returns 17 26 13 16 24 27 19 14 33 20 17 22 29 17 36 19 22 24 22 u chart 1.2 1.0 UCL = 0.88 0.8 0.6 Mean = 0.54 Rate per 100 ED100Patients per Rate 0.4 0.2 LCL = 0.19 0.0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 STATIC VIEW DYNAMIC VIEW Run Chart Descriptive Statistics Mean, Median & Mode Control Chart Minimum/Maximum/Range (plot data over time) Standard Deviation Bar graphs/Pie charts Statistical Process Control (SPC) 24 The SPC Pioneers Walter W.