6/12/2012
Objectives • Participants will be able to:
Using Data for Improvement in Healthcare: The Essential Toolkit
- Identify fundamental differences between data when used for improvement, accountability and research - Appreciate the value of viewing data graphically and over time p data on tools - Learn when to use and how to interpret fundamental to improvement: • • • • •
Run chart to identify statistically significant signals of change Shewhart Chart (Introduction only) Pareto chart Histogram (Frequency Plot) Scatter Plot
- Select the appropriate tool for the question being asked
Sandra K. Murray
[email protected]
Copyright © 2012
References Books: 1. The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011. 2. Total Quality Tools For Health Care. Productivity-Quality Systems, Inc. Miamisburg Ohio. ISBN: 1-882683-04-8 Tel. 1-800-777-2255. 3. The Improvement Guide. Gerald J. Langley, Kevin M. Nolan, Thomas W. Nolan, Clifford L. Norman, Lloyd P. Provost, Jossey-Bass, 2009. Video: 1. Making Sense Out of Control Charts. NAHQ. 1-800-966-9392 Software Used to Produce Charts: 1. ChartRunner. PQ Systems. 1-800-777-3020. 2. QI Charts. API, 1-512-708-0131 3. Minitab,1-814-238-3280 Articles: 1. The run chart: a simple analytical tool for learning from variation in healthcare processes. Rocco J Perla, Lloyd P Provost and Sandra K Murray. BMJ Qual Saf 2011 20: 46-51.
Purpose of Measurement • Measurement for Improvement • Measurement for Accountability • Measurement for Research The Three Faces of Performance Measurement: Improvement, Accountability and Research. Journal on Quality Improvement, Volume 23, Number 3, March, 1997.
I
Copyright © 2012
Copyright © 2012
Data for Improvement, Accountability and Research in Health Care Aspect Aim:
Methods: Bias: Sample Size:
Flexibility of Hypothesis:
Improvement
Accountability or Judgment
Research
Improvement of care processes, systems and outcomes
Comparison for judgment, choice, reassurance, spur for change
New generalizable knowledge
Test observable
No test, evaluate current performance
Test blinded
Accept consistent bias
Measure and adjust to reduce bias
Design to eliminate bias
“Just enough” data, small sequential samples
Obtain 100% of available, relevant data
“Just in case” data
Hypothesis flexible, changes as learning takes place
No hypothesis
Fixed hypothesis
Testing Strategy:
Sequential tests
No tests
One large test
Determining if a Change is an Improvement:
Run charts or Shewhart control charts
No focus on change Shewhart charts for monitoring
Hypothesis, statistical tests (ttest, F-test, chi square, pvalues)
Confidentiality of the Data:
Data used only by those involved with improvement
Data available for public consumption
Research subjects’ identities protected
Daily, weekly, monthly
Quarterly, annually
At end of project
Frequency of Use:
Copyright © 2012 Source: The Health Care Data Guide: Learning from Data for Improvement. Developed from Solberg, Leif I., Mosser, Gordon and McDonald, Susan. “The Three Faces of Performance Measurement: Improvement, Accountability and Research.” Journal on Quality Improvement. March 1997, Vol.23, No. 3.
Graphical Display of Data • Effective visual presentations of data, instead of tabular displays, provide the most opportunity from variation
Copyright © 2012
1
6/12/2012
Graphical Display of Data • Effective visual presentations of data, instead of tabular displays, provide the most opportunity from variation • Viewing variation over time enhances learning
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011. Copyright © 2012
Unit 1
Cycle Time Results for Units 1, 2 and 3
Copyright © 2012
What’s the Question You’d Ask Here? Run Chart of Measure
Unit 2
100 95
Goal = 90
90 Median = 84
%
85 80 75
Unit 3
70 65 60 Jan
The run chart: a simple analytical tool for learning from variation in healthcare processes. Rocco J Perla, Lloyd P Provost and Sandra K Murray. BMJ Qual Saf 2011 20: 46-51.
Copyright © 2012
Feb
Mar
Apr
May
Jun
Jul
Aug
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Sep
Oct
Nov
Copyright © 2012
Repeated Use of the PDSA Cycle Model for Improvement Pareto Charts
What are we trying to accomplish?
Run or Shewhart Charts Run and Shewhart Charts, Pareto charts, Frequency Plots, Scatter Pl t Plots
How will we know that a change is an improvement? What change can we make that will result in improvement?
What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement?
Reduce Per-op harm by 30% •% Pts with Peri-op harm •Peri-op Harm Rate •Unplanned returns OR
Changes That Result in Improvement
--DVT Prophylaxis --Beta Blocker Prophy --SSI interventions
Implementation of Change
Act Study Run or Shewhart Charts AND Qualitative Data
Model for Improvement
Plan Do Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Use clippers Instead of Shaving site
A P
S D Followup Tests Hunches Theories Very Small Ideas Scale Test
Wide-Scale Tests of Change
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
2
6/12/2012
Tools for Understanding Variation
Tools to Learn from Variation in Data
• Run Chart: Study variation in data over time; understand the impact of changes, detect signals of improvement. • Shewhart Chart: Distinguish between special and common causes of variation. Is process stable, predictable? • Pareto Chart: Where should we focus? Focus improvement on area with greatest potential impact. • Frequency Plot: Understand distribution of data (e,g, central location, spread, shape, and patterns). • Scatter Plot: Analyze potential relationship between two variables. Frequency Plot Copyright © 2012
Pareto Chart
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Scatter Plot Copyright © 2012
Run Chart
Fundamental Uses of Run Charts
• Graphical display of data plotted in some type of order. Also has been called a time series or a trend chart.
• How much variation do we have? - Display data to make process performance visible • Have our changes yielded improvement? - Determine whether a change resulted in evidence of improvement • Are the gains we made slipping away? - Determine whether we are holding the gain made by our improvement
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
How Do We Tell a Change is an Improvement?
• Run charts speak for themselves…or..
• Analyze with probability-based rules
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
3
6/12/2012
Unplanned Returns to OR: Pilot Population
6
(N~200/Mo.) Pre-Procedural Briefings Prophylactic ABX Timing
5
Razors to Clippers
4
Bleeding risk assessment, DVT Proph Beta Blocker use, Normothermia
%
3 2
1 0 1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Months
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Looking at Timeliness
How Do We Tell a Change is an Improvement?
Month
% Timely
Month
% Timely
• Run chart may speak for itself
1‐ 2007
32
1‐2008
23
• If run chart does not speak for itself we can analyze it further using probability-based rules - Can detect signal of change ( a non-random pattern tt in i the th data) d t )
2
23
2
32
3
32
3
36
4
38
4
29
5
35
5
38
6
35
6
42
7
40
7
39
8
21
8
36
9
38
9
50
10
26
10
48
11
22
11
39
12
27
12
44
Copyright © 2012
MEDIAN
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Why Median Rather Than Mean?
MEDIAN: In a series of numbers, the median is physically the middle number . It has the same number of points equal to it or above it as it has equal to it or below it. MEAN: The average.
• 8,10,11,14,16,18,20
Mean= 13.8 Median=14
• 8,10,11,14,16,18,95
Mean= 24.5 Median=14
• 1,10,11,14,16,18,20
Mean= 12.8 Median=14 Mean = arithmetic average of data Median = middle value of ordered data
Copyright © 2012
Copyright © 2012
4
6/12/2012
50 48 44 42 40 39 39 38 38 38 36 36 35 35 32 32 32 29 27 26 23 23 22 21
Finding the Median: Reordering the Data
• To find the median reorder the numbers from high to low and find the number physically in the middle. If you have two numbers left in the middle, add them together and divide by two. • Excel: place cursor in i blank cell and type=MEDIAN(A2:A21) where A2 is the first cell you want to include and A21 the last)
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Rule 1: Shift
Why Bother..What Do we Do With A Signal?
• Signals can be evidence of improvement
• Six or more consecutive POINTS either all above or all below the median. Skip values on the median and continue counting points. Values on the median DO NOT make or break a shift.
- That changes are adding up to improvement
Rule 1
• Our job when seeing a signal - Go learn from signal and take appropriate - action Copyright © 2012
Measure or Characte ristic
• Signals can be evidence that things got worse - Changes caused unexpected degradation of process or outcome - Something else entered the process - resulting in a signal
Copyright © 2012
25 20 15 10 5 Median=11 Median=10
0 1
2
3
4
Median 10 5 6 7 8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
The Health Care Data Guide: Learning from Data for Improvement. L loyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Rule 2: Trend •Five points all going up or all going down. If the value of two or more
successive points is the same count the first one then ignore the identical points when counting; like values do not make or break a trend. Rule 1 YES
Rule 2
2 1
5 6 3 4
7
8
M e a sure or C ha raa c te ris tic
25
20
15
10
5
Median 11
0 1
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
5
6/12/2012
Rule 3: Runs
Document
To Determine The Number of Runs Above and Below the Median: - A run is a series of points in a row on one side of the median. Some points fall right on the median, which makes it hard to decide which run these points b l belong to. t - So, an easy way to determine the number of runs is to count the number of times the data line crosses the median and add one. - Statistically significant change signaled by too few or too many runs.
Rule 1-YES Rule 2-NO
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Microsoft Word
Copyright © 2012
Copyright © 2012
Rule 3: # of Runs Table for Checking for Too Many or Too Few Runs on a Run Chart
Rule 3: NUMBER OF RUNS • Steps
Microsoft Wo Documen
points on the run chart
Measure or Characeristtic
Upper limit for the number of runs
(< than this number of runs is “too few”)
(> than this number of runs is “too many”)
median 10
Rule 3 Data line crosses once Too few runs: total 2 runs
20 Median 11.4
15
Lower limit for the number of runs
that do not fall on the
- Count the # of data points not falling on the median (in this case 10) - Count the # of runs (# times data line crosses the median + 1) (in this case 2) - Go to table and find out if you have too few or too many runs
25
Total number of data
10
3
9
11
3
10
12
3
11
13
4
11
14
4
12
15
5
12
16
5
13
17
5
13
18
6
14
19
6
15
20
6
16
21
7
16
5
22
7
17
0
23
7
17
24
8
18
25
8
18
1
2
3
4
5
6
7
8
9
10
Table is based on about a 5% risk of failing the run test for random patterns of data. Frieda S. Swed and Churchill Eisenhart, The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
(1943). “Tables for Testing Randomness of Grouping in a Sequence of Alternatives. Annals of Mathematical Copyright © 2012 Statistics. Vol. XIV, pp.66 and 87, Tables II and III
Copyright © 2012
Rule 3: # of Runs
Rule 3
Table for Checking for Too Many or Too Few Runs on a Run Chart
• To Determine The Number of Runs Above and Below the Median: Microsoft Word
-
A run is a series of points in a row on one side of the median. Some points fall right on the Document median, which makes it hard to decide which run these points belong to. So, an easy way to determine the number of runs is to count the number of times the data line crosses the median and add one. Statistically significant change signaled by too few or too many runs.
-
Total number of data points on the run chart
Lower limit for the number of runs
Upper limit for the number of runs
(< than this number of runs is “too few”)
(> than this number of runs is “too many”)
that do not fall on the median 10
3
9
11
3
10
12
3
11
13
4
11
14
4
12
6
15
5
12
5
16
5
13
17
5
13
-
Rule 3
8 7
Median 3.66
4
18
6
14
3
19
6
15
2
20
6
16
21
7
16
22
7
17
23
7
17
24
8
18
25
8
18
20 data points not on median. 18 crossings +1= 19 Runs= Too many runs
1 0 J03
F
M
A
M
J
J
A
S
O
N
D
J04
F
M
A
M
J
J
A
S
Table is based on about a 5% risk of failing the run test for random patterns of data. Frieda S. Swed and Churchill Eisenhart, The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
(1943). “Tables for Testing Randomness of Grouping in a Sequence of Alternatives. of Mathematical Copyright © Annals 2012 Statistics. Vol. XIV, pp.66 and 87, Tables II and III
6
6/12/2012
Rule 3: NUMBER OF RUNS
Microsoft Wo Documen
• To Determine The Number of Runs - A run is a series of points in a row on one side of the median.
Some points fall right on the median,
-
which makes it hard to decide which run these points belong to. So, an easy way to determine the number of runs is to count the number of times the data line crosses the median and add one.
-
A signal is evidenced by too few, or too many runs.
Rule 1-YES Rule 2-NO
• Steps -
7 + 1 + 8 Runs
Count the # of data points not falling on the median (in this case 10) Count the # of runs ((# times data line crosses the median + 1)) ((in this case 2)) Go to table and find out if you have too few or too many runs ( in this case should have 3-9 runs. Only have 2, so too few runs.)
• What does it mean? - Too few runs with data going in our desired direction is signal of improvement - Too few runs if data going in undesirable direction is signal of degradation The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
Rule 3: # of Runs Table for Checking for Too Many or Too Few Runs on a Run Chart Total number of data points on the run chart
Lower limit for the number of runs
Upper limit for the number of runs
(< than this number of runs is “too few”)
(> than this number of runs is “too many”)
that do not fall on the median 10
3
9
11
3
10
12
3
11
13
4
11
14
4
12
15
5
12
16
5
13
17
5
13
18
6
14
19
6
15
20
6
16
21
7
16
22
7
17
23
7
17
24
8
18
25
8
18
Rule 1-YES Rule 2-NO Rule 3-NO
Table is based on about a 5% risk of failing the run test for random patterns of data. Frieda S. Swed and Churchill Eisenhart,
(1943). “Tables for Testing Randomness of Grouping in a Sequence of Alternatives. of Mathematical Copyright © Annals 2012 Statistics. Vol. XIV, pp.66 and 87, Tables II and III
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
RULE 4: Astronomical For detecting unusually large or small numbers: • Data that is Blatantly Obvious as a different value • Everyone studying the chart agrees that it is unusual • Remember:
Rule 1-YES Rule 2-NO Rule 3-NO Rule 4-NO
– Every data set will have a high and a low - this does not mean the high or low are astronomical
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
7
6/12/2012
How Do We Tell a Change is an Improvement?
Let’s Practice
• Run chart may speak for itself
• Please work in pairs • Evaluate the following run charts to determine :
• If run chart does not speak for itself we can analyze it further using probability-based rules - Can detect signal of change ( a non-random pattern tt in i the th data) d t ) - Signal could be improvement or degradation
Does the chart show a signal? If signal noted which of the four rules did you use to find it?
-
Copyright © 2012
Copyright © 2012
Rules for Indentifying Non-Random Signals of Change
Behavioral Health: Crisis Hours Provided In-Network Hours 625 556 492 699 435 553 526 675 611 700 727 647 664 695 602 789 710 761 710 723 722 712 743 729
Run chart
1,000 900
Desired Direction
800
H ours
700 Median line = 625 600 500 400 Chg. 2
300
Chg. 1
Chg. 4 Chg. 3
200 D The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
J
F
M
A
M
J
J
A
S
O
N
D
J
Copyright © 2012
F
M
A
M
J
J
A
S
O
N
Copyright © 2012
Percent Ventilator Associated Pneumonia Bundle Compliance %
71.0
68.2
84.9
89.9
81.0
62.0
92.3
91.2
95.4
94.1
96.0
Run chart
100
Desired Direction
95
90 Median = 89.9
P e rc e n t
85 80 75 70 65 60 Change 1
55
Change 2
Change 3
50 J
A
S
O
N
D
J-12
F
M Copyright © 2012
A
M Copyright © 2012
8
6/12/2012
Why Bother..What Do we Do With A Signal?
Some Keys to Good Graphical Display with Run Charts
• When do we begin a run chart? - As soon as we have a data point
• Signals can be evidence of improvement - That changes are adding up to improvement
• Signals can be evidence that things got worse - Changes caused unexpected degradation of process or outcome - Something else entered the process resulting in a signal
• Action when seeing a signal - Go learn from signal and take appropriate action
• If testing change and see no signal: - Changes not strong enough - Changes really made? - Testing on such small scale--not impacting system yet - Measure not sensitive Copyright © 2012
When Do We Start a Run Chart?
Copyright © 2012
Proper Use of the Median • When should we apply a median? - Will depend on your situation • If very little data baseline median may be only a few data
points • If want to apply probability-based rules for analysis of run
chart need 10 data points for median
- If graph shows no signals (shift, trend, runs astronomical) and median made from 10 or more data points freeze and extend median into the future • This will result in earliest possible detection of signals
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
If median not frozen and extended will result in delayed detection of signals
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
9
6/12/2012
If a signal is detected and sustained a new median may be created for the new process performance
A signal is detected utilizing both original and extended median
• When analyzing run chart with two separate medians rules
are must be applied separately to the data surrounding each median
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Plotting Rare Events • Results in too many zeros • Makes interpretation difficult and chart of little value • Useful alternative is to chart time or workload between undesirable events - Up is always good for these charts
Copyright © 2012
Days Between MRSA Run chart
40
D ay s Be tw e e n Ca s es of M R SA
35 30 25 20 15 10 5
Median line = 7 Extra line
Chg 2 Impl Chg 1
3/
2/
11 3/ 6 3/ 3/ 7 1 3/ 5 22 4 4/ /1 1 4/ 1 1 4/ 4 26 5/ 3 5/ 5/ 3 1 5/ 3 19 5/ 28 6/ 6/ 4 1 6/ 0 1 6/ 4 2 6/ 1 30 7/ 3 7/ 7/ 7 1 7/ 8 2 7/ 3 25 8/ 2 8/ 8/ 8 21 9/ 9/ 5 2 1 1 100/8 / 11 31 /2 1 0 1 2/5 1/ 2/2 14 6 /1 2 2 2/ /1 1 2/ 8 2 3/ 8 19
0
Chg 3 Impl Chg 2 Chg 3 Chg 1 Impl
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
10
6/12/2012
Judgment Vs. Improvement
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Nifty Things You Can Do With Run Charts
Copyright © 2012
Improvement Projects Require a Family of Measures
Copyright © 2012
Improvement Projects Require a Family of Measures • 2-8 measures typically -Each on a graph -All viewed on one page
Fig 3.6: Improvement Evident Using a Set of Run Charts Viewed on One Page The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Small Multiples • Multiple run charts viewed on one page • All these run charts are about the same measure but for a different location, provider or segment of the population • Each has the same scale vertically and horizontially • Allows for rapid comparison
Copyright © 2012
11
6/12/2012
May Display More Than One Measure on a Graph
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
May Use Different Measure for Each Axis
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Stratification or Disaggregation
Sometimes We Don’t Have Much Data • May not be rich in data but that data may still lead to a high degree of belief in the change(s) tested • Characterize the change by describing the before and after medians • Minimizes point-to-point variation
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Cautions with Graphing Raw Data • Plotting raw data can be misleading if a useful denominator would lead to another conclusion • Use of ratio minimizes confusion from changes in denominator volume
Number of Falls 12
10
• Ratio = numerator for key measure #F Falls
8
denominator (for unit of production or volume related to key measures)
6
Key Measure (Numerator)
Possible Denominator
Ratio
# ADEs
# Doses Dispensed
ADE/Dose
OR Costs
# Surgeries
OR Cost/Surgery
2
# Peri‐operative Adverse Events
# Admissions
POAE/Admission
0
Patients LWBS
# Patients Registering in ED
Patients LWBS/# Patients Registered
# Falls
# Patient days
Falls/Patient Day
4
Copyright © 2012
M- A M J J A S O N D J- F M A M J J A S O N 07 08
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
12
6/12/2012
Tools to Learn from Variation in Data
Run Chart • • • •
A line graph of data plotted over time Data is kept in time order Can see flow of data Helps p answer qquestions: -
What is our baseline variation? How much variation do we have? How is process changing over time? Has our change resulted in an improvement? Did I hold the improvement? Frequency Plot Copyright © 2012
• Shewhart Chart: Is my process stable; predictable? Distinguish between special and common causes of variation. • Pareto Chart: Focus improvement on with greatest potential impact.
Scatter Plot Copyright © 2012
Shewhart Control Charts: What Am I Looking At and Why Bother!
Tools for Understanding Variation • Run Chart: Study variation in data over time; understand the impact of changes.
Pareto Chart
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
• • • • • •
What is Shewhart chart? Special and common cause variation How to interpret one Uses of Shewhart charts Why bother? There are different kinds of Shewhart charts
• Frequency Plot: Understand distribution of data (e,g, central location, spread, shape, and patterns). • Scatter Plot: Analyze potential relationship between two variables. Copyright © 2012
Copyright © 2012
Introduction to Shewhart Chart
Shewhart Chart: What Is It?
• Statistical tool used to distinguish special from common cause variation
• A tool to differentiate special from common cause variation • Data is usually displayed over time • Most often in time order
Shewhart chart will include:
Straight limits indicate equal subgroup size
•Center line (usually mean) •Data points for measure •Statistically calculated upper and lower 3 sigma limits (Limits typically created with 20 or more subgroups) The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
13
6/12/2012
Types of Variation: Common Cause • The variation is due to the process or system design • It is produced by interactions of inherent variables in the process • The causes affect everyone working in the process and all outcomes of the process • Process having only common cause affecting the outcome is called stable
- Performance is predictable
Smaller subgroup = wider limits Larger subgroup – tighter limits Varying limits indicate unequal subgroup size The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Management Strategy:
Copyright © 2012
Common Cause System
STRATEGY TO TAKE:
• Process Study and Redesign!! - Understand that process performance will not change unless process design is fundamentally altered - Identify process variables contributing to common cause variation - Determine which aspect of the process to change - PDSA the process change ACTIONS TO AVOID:
• Doing nothing at all! • Tampering • Trying to attach specific meaning to fluctuations in the data (i.e. explain the difference between points that are high vs... low) The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Management Strategy: Special Cause System
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Types of Variation: Special Cause • Variation in the process assignable to a specific cause or causes - not part of the usual process • This variation due to specific p circumstances • Process not stable - Is not predictable
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Distinguishing Special from Common Cause Variation
IMPROVEMENT STRATEGY:
• Investigate, learn and standardize the process!! - Immediately try to understand when Special Cause occurred - Study what was different when Special Cause occurred - Identify ways to prevent or use it, if understandable, to standardize the process • either i h standardize d di back b k to where h the h process was • or standardize in a new better place ACTIONS TO AVOID:
• Doing nothing at all • Failing to involve the people who work in the process in identifying special causes The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
14
6/12/2012
Understanding Variation • We can make two mistakes - Mistake 1: thinking an outcome is due to a special cause when it was really due to common causes - Mistake 2: thinkingg an outcome is due to common causes when it was really due to a special cause • Shewhart charts help minimize these two mistakes
Rules or detecting a special cause
Note: A point exactly on a control limit is not considered outside the limit When there is not a lower or upper control limit pp y to the side missing g limit Rule 1 does not apply
Note: A point exactly on the centerline does not cancel or count towards a shift
Note: Ties between two consecutive points do not cancel or add to a trend.
When there is not a lower or upper control limit Rule 4 does not apply to the side missing limit
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Percent Handwashing Compliance
Let’s Analyze One Together
UCL
90
80
CTL
%
• We always apply all 5 rules to each chart - Any one rule “activated” indicates special cause in that area - Common cause is determined by “ruling out” special cause (none of 5 rules activated) • Let’s consider appropriate action based on your analysis - Special cause action? - Common cause action?
p c hart
100
70 LCL
60
50 J 08 F
M
A
M
J
J
A
S
O
N
D JAN 09 F
Copyright © 2012
A
M
J
J
A
Copyright © 2012
Percent Parent Satisfaction in Top Box
Let’s Practice
p chart
30
25 UCL = 23.95
20
%
Please analyze one of these charts Apply all 5 rules to each chart Circle special cause if you find it What action would you take based on your analysis? l i? - Special cause action? - What would you do if it is solely common cause?
15 CTL = 11 11.73 73
10
5
Copyright © 2012
/2 00 8
6/ 1
/2 00 8 /2 00 8 5/ 1
4/ 1
/2 00 8 /2 00 8 3/ 1
2/ 1
00 7 /2 00 8
/2 /1 12
1/ 1
00 7
00 7 /2
/1 11
/2 00 7
/2
/1
/2 00 7
9/ 1
10
8/ 1
/2 00 7 /2 00 7 7/ 1
/2 00 7
6/ 1
/2 00 7
5/ 1
/2 00 7
4/ 1
3/ 1
/2 00 7
2/ 1
/2 00 7
0
1/ 1
• • • •
M
Copyright © 2012
15
6/12/2012
Average Time to Acknowledge Referrals
Why Distinguish Special From Common Cause Variation?
Individuals 34 32
• When monitoring key processes - Can tell if they have remained the same, degraded or improved • When working specifically to improve: - Special p cause:
30 UCL = 29.05 28 26
H o u rs
24 22 20 18 16 14 12
• may be evidence of improvement • or… an unintended consequence such as degradation
10 Mean = 9.44 8
in the results
6 4
2
8
8/
8/
18
25 7/
23
7
7/
3
7/
7/
30
- Common cause:
Chg 4
7/
6/
14
21 6/
6/
4
10
6/
19
Chg 3
Chg 2
6/
5/
5
5/
3
13
5/
5/
26
14
1
11
5/
4/
4/
22
4/
4/
15
8
3/
3/
3/
6 3/
28
Chg 1
2 0
• indicates that the changes have not resulted in
improvement
Weeks
Copyright © 2012
Copyright © 2012
4
6
4
8
3
6
7
6
7
8
7
7
8
6
8
9
6
7
8
6
6
7
8
6
Individuals
14
Stable process Predictable
12 UCL=11.1
10
LOS in Days
• Learn how much variation exists in process - If stable are predictable. Can use info in planning, communicate with staff, patients, family • Assess stability and determine improvement strategy (common or special cause strategy) • Monitor performance and correct as needed • Find Fi d and d evaluate l causes off variation i i • Tell if our changes yielded improvements • See if improvements are “sticking”
5
24
5
23
7
22
Using a Control Chart
6
21
Are Our LOSs for DRG XXX Stable? Data4
8
Mean-6.2
6
4
2
LCL=1.7
25
20
19
18
17
16
15
14
13
12
11
9
10
8
7
6
5
4
3
2
1
0
Sequential Cases Copyright © 2012
Copyright © 2012
Are Our LOSs for DRG XXX Stable?
Using a Shewhart Chart
Data4
• Learn how much variation exists in process • Assess stability and determine improvement strategy (common or special cause strategy)
Monitor performance and correct as needed Find and evaluate causes of variation Tell if our changes yielded improvements See if improvements are “sticking”
5
4
6
4
8
3
6
7
5
6
7
8
7
7
8
6
8
9
Individuals
Stable not process Stable but perhaps good enough Requires process redesign to improve Predictable
UCL=11.1
10
8
Mean-6.2
6
4
2
LCL=1.7
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
2
3
0 1
• • • •
7
12
LOS S in Days
- When sponsoring improvement effort it’s helpful, if data readily available, to determine if process has only common cause or if special cause also present
6
14
Sequential Cases Copyright © 2012
Copyright © 2012
16
6/12/2012
Coding Errors per Groups of 20 Records c c hart
30
Special Cause variation What is our action here?
# Cod iin g E rro rs
25
20
UCL = 19.37
15
10
Mean = 9.92
Using a Shewhart Chart • Learn how much variation exists in process • Assess stability and determine improvement strategy (common or special cause strategy) • Monitor performance and correct as needed • Find and evaluate causes of variation • Tell if our changes g yielded y improvements p • See if improvements are “sticking”
5
0
LCL = 0.47
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Sequential Subroups of 20 Records Copyright © 2012
Copyright © 2012
Using a Shewhart Chart • Assess stability and determine improvement strategy (common or special cause strategy) • Monitor performance and correct as needed • Find and evaluate causes of variation • Tell if our changes g yielded y improvements p • See if improvements are “sticking”
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
Using a Shewhart Chart • Learn how much variation exists in process • Assess stability and determine improvement strategy (common or special cause strategy) • Monitor performance and correct as needed • Find and evaluate causes of variation • Tell T ll if our changes h yielded i ld d improvements i t - When you intend to improve process you are on the lookout for special cause indicative of improvement
• See if improvements are “sticking”
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
17
6/12/2012
Percent Unplanned Returns to OR P chart # Surgeries # Pts Return
984 982 27
20
996
998 1070 1031
25
23
31
17
886 21
964 1128 960 1193 28
24
22
998 1070 895
19
30
22
852
963
15
18
956 1001 12
22
956
995
8
2
987 943 9
6
965
980
20
6
923 1106 2
6
p c hart
4.0 3.5
24
Good
UCL = 3.54
P Percent
3.0 2.5 CTL = 2.16
2.0 1.5 1.0 LCL = 0.78
0.5
Goal = 0.5
Using a Shewhart Chart
Chg 2 & 3 Chg 14 Chg 10 & 11 Chg 7 & 8 Chg 4 & 5Chg 9 Chg 12 & 13 Implement
Chg 1
• Learn how much variation exists in process • Assess stability and determine improvement strategy (common or special cause strategy) • Monitor performance and correct as needed • Find and evaluate causes of variation • Tell if our changes yielded improvements • See if improvements are “sticking”
0.0 F 04M A M J J A S O N DJ 05F M A M J J A S O N DJ 06F M A M Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Run Vs. Shewhart Chart
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Technique matters! -Obtain baseline mean/limits from stable period and freeze them -Minimum baseline 12, preferred 20-30
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
18
6/12/2012
Tools to Learn from Variation in Data
Selecting the Appropriate Shewhart Chart Type of Data Count or Classification (Attribute Data)
Count (Nonconformities) Equal Area of Opportunity
Classification (Nonconforming)
Unequal or Equal Subgroup Size
Subgroup Size of 1
Unequal Area of Opportunity
Unequal or Equal Subgroup Size
U Chart
P Chart
Nonconformities Per Unit
Percent
X‐Bar and S chart
I Chart (X chart)
C Chart Number of Nonconformities
Continuous (Variable Data)
Individual Measures
Nonconforming
Average and Standard Deviation
Other types of control charts for attribute data:
Other types of control charts for continuous data:
1. NP (for classification data) 2. T-chart [time between rare events] 3. Cumulative sum (CUSUM) 4. Exponentially weighted moving average (EWMA) 5 G chart (number of opportunities between rare events) 6. Standardized control chart
7. X‐bar and Range 8. Moving average 9. Median and range 10. Cumulative sum (CUSUM) 11. Exponentially weighted moving average (EWMA) 12. Standardized control chart Copyright © 2012
Frequency Plot
Pareto Chart
Scatter Plot
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Source: The Health Care Data Guide. Provost and Murray Jossey-Bass, 2011
Fall Rate per 1000 Resident Days
Copyright © 2012
Factors Associated with Resident Falls N=254
# Days/1000 3.357 3.012 3.718 2.983 3.108 2.948 2.721 2.690 2.567 2.667 2.824 2.882 3.429 2.829 3.092 2.605 2.610 2.531 2.502 2.615 2.662 2.806 2.591 2.403 # Falls 8 13 11 7 11 18 10 15 9 9 5 16 9 9 8 4 7 12 15 9 12 13 9 15 104 40.94%
u chart
R a te p e r 1 0 0 0 R e s id e n t D a y s
7 UCL = 7.16
3
46 18.11%
21 8.27%
17 6.69%
Bumped Bum ped
Trip
Copyright © 2012
Us Using ing Rest Res t Room
Ju l Au g Se p Oc t No v De c
Ju l Au g Se p O ct No v De Ja c n 11 Fe b M ar Ap r M ay Ju n
10 Fe b M ar Ap r M ay Ju n
n Ja
Getting Out of Bed
LCL = 0.29
0
Bending Over
1
No Glas G lasses ses
6 2.36%
2
4 1.57%
2 0.79%
2 0.79%
W et Floor
4 CTL = 3.73
52 20.47%
O Other ther
5
Missed Chair
# oof Events
6
Copyright © 2012
Number of Falls by Time of Day Histogram
Pareto Chart
30
25
• Bar chart with bars in rank order • Each bar represents a different variable, factor or problem • Becomes useful with 30-50 p pieces of data • Looking for 20% of bars representing 80% of opportunity • Want to know where to focus our efforts
# F a lls
20
15
10
5
0 0
2
4
6
8
10
12
14
16
18
20
22
- Which are the vital few areas we should concentrate on? - Which variables out of many are occurring most?
Time of Day (24 Hour Clock) Copyright © 2012
Copyright © 2012
19
6/12/2012
Pareto Chart: What Does One Look Like?
Pareto Chart: What Does One Look Like?
Reasons Cited for Lack of Childhood Immunizations: Group A Count
Reasons Cited for Lack of Childhood Immunizations: Group A
1,503
1,400 80%
1,200 1,000
45 2.99%
43 2.86%
40% 264 17.56%
400 200
Other
No Info
No Time
Cost of Imm.
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
134 8.92%
128 8.52%
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
When Is It Used?
20%
98 6.52%
45 2.99%
43 2.86% 0%
Childcare Needs
0
Childcare Needs
No Transport
0
98 6.52%
600
Other
100
Don't Believe
200
128 8.52%
Don't Believe
134 8.92%
60%
No Info
264 17.56%
300
791 52.63%
No Transport
400
800
No Time
500
# Respo onses
600
Cost of Imm.
700
Copyright © 2012
Model for Improvement What are we trying to accomplish?
• When data can be arranged into categories • When the rank of each category is important • When we need to focus on the most important problems or causes of variation
How will we know that a change is an improvement? What change can we make that will result in improvement?
Act
Plan
Check
Do
Copyright © 2012
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Needlesticks By Location (n=224)
Frequency Table: Where Are Needlesticks Occurring? Variable 3W
Wk 2
Wk1 17
Wk 3 14
Wk 4 16
Total 59
12
ED
4
3
4
6
17
4
7
2
3
16
7.14
2N
1
3
5
3
12
5.36
3N
7
2
3
5
17
7.59
2W
4
6
4
6
20
8.93
2S
6
2
3
3
14
6.25
10
13
12
13
48
21.43
3
2
3
4
12
5.36
Other
3
3
3
0
9
4.02
224
25% 48 21.43%
50
7.59
Lab Grand Total
Percent 59 26.34%
26.34
ICU
Allergy/Imm
Count 60
%
20%
C Count
# Respo onses
Percent 100%
1,503
Count
791 52.63%
40 15% 30 20 8.93%
20
17 7.59%
16 7.14%
14 6.25%
12 5.36%
12 5.36%
10
9 4.02%
5%
100 % 3W
Copyright © 2012
10% 17 7.59%
2S
2W
3N
ED
ICU
Allergy/Imm.
Lab
2N
Other
Copyright © 2012
20
6/12/2012
Factors Related To Severely Mentally Disabled Adult Recidivism (Readmission)
How Is It Interpreted?
196
Count
Percent
103 52.55%
100
50%
• Look for the Pareto effect - Is entire chart speaking to us? re-stratify? stratify? - Can we re - Last choice is selecting a column and tackling it!
C ount
• We won’t always find it!
80
40%
60
30%
38 19.39%
40
20%
20 10.20%
20
Non-Comp.w/Meds ETOH/Oth Sub.
14 7.14%
8 4.08%
Instablity Housing Non-Comp other TX. Lack Fam. Supt.
Copyright © 2012
2 1.02%
Other Psyhosoc.
Other
Copyright © 2012
Factors Related to Pediatric Head Injury
How Is It Interpreted?
Count
Percent 20 29.41%
20
30%
16 23.53%
25% 14 20.59%
15
20%
C ount
• Look for the Pareto effect • We won’t always find it! - Is entire chart speaking to us? - Can we re re-stratify? stratify? - Last choice is selecting a column and tackling it!
10%
11 5.61%
15%
10
6 8.82%
10% 4 5.88%
5
2 2.94%
Rollerblade Skateboard
Bike
Motor Veh.
Fall
Struck
2 2.94%
2 2.94%
Pedestrian Motorcycle
Copyright © 2012
5% 1 1.47%
1 1.47%
Other
Fight
Copyright © 2012
Location of Resident Falls
How Is It Interpreted?
Count
# Falls
53 20.87%
50 43 16.93%
42 16.54% 37 14.57%
40
# o f F a lls
29 11.42%
30
19 7 48% 7.48%
20
14 5.51%
12 4.72%
10
Copyright © 2012
Library
Lounge
Trips
Gardens
Dining Area
Halls
Room
0
Rest Room Roomss
5 1.97%
Other
• Look for the Pareto effect • We won’t always find it! - Is entire chart speaking to us? - Can C we re-stratify? t tif ?
Copyright © 2012
21
6/12/2012
Factors Associated with Resident Falls N=254
Other Ways To Use Pareto
104 40.94%
46 18.11%
17 6.69%
No G Glas las ses s es
Using Us ing Res Restt Room
Bum ped
Trip
G etting Out O ut of Bed
Bending O ver v er
6 2.36%
4 1.57%
2 0.79%
2 0.79%
W et Floor
21 8.27%
O ther
52 20.47%
M is isss ed Chair
# o f Events
• Stratification
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Pareto Chart • Bar chart with bars in rank order • Each bar represents a different variable, factor or problem • Looking for 20% of bars representing 80% of opportunity • Want to know where to focus our efforts - Which are the vital few areas we should concentrate on? - Which variables out of many are occurring most?
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
22
6/12/2012
Tools to Learn from Variation in Data
Frequency Plot (Histogram): What Is It? • A bar chart for one variable only • Most often used with time, money, throughput or a scaled measurement (i.e. dollars, weight, age, height) • Used to visualize central location, shape and spread of the data • Each bar equal, equal each distinct • Becomes useful with 30-50 pieces of data • Frequency Plot does little good for interpretation if process not stable • Doesn’t show stability
Frequency Plot
Pareto Chart
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Scatter Plot Copyright © 2012
Copyright © 2012
The Tool List
What Does a Histogram Look Like? Age of People with Diabetes Who Have HbA1C> 8
Count
• Frequency Plot:
140 131
- How is this one variable distributed (what is the spread of LOS, Cost, HA1C, etc. in our population)?
?
120
100
80
60 52 40
40
51 43
40
20
16 7 0-9
10-19
20-29
30-39
40-49
50-59
60-69
Copyright © 2012
When Is It Used? 1. Have a set of values related to your question (i.e. arrival times in ED) 2. Want to see central location, shape, spread of data to learn about system - Any patterns that bear looking into? - Does all of process fit within needs? (Our standards)
Copyright © 2012
40
29
70-79
80-89
90-99
Copyright © 2012
Model for Improvement What are we trying to accomplish? How will we know that a change is an improvement? What change can we make that will result in improvement?
Act
Plan
Study
Do
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
23
6/12/2012
Jun 2011 ED Patient Arrival Times (1 Week Weekdays, n=524) 80
How Is It Interpreted?
70
• Evaluate central location • Evaluate spread • Learn from shape
60
# P ts
50
40
30
20
10
0 0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Time of Day 24 Hour Clock Copyright © 2012
g
30
Copyright © 2012
j
How Is It Interpreted?
24
25
# of H Head Injuries
• Evaluate central location • Evaluate spread • Learn from shape
20
15
8
10
8
7
6 5
5
2
3 1
0
Age in Years Copyright © 2012
Copyright © 2012
How Long do Patients Wait In Our Clinic? 45
# of Times a Patient Waited in This Time Range
40
35
30
25
20
15
10
5
0
Minutes The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Copyright © 2012
24
6/12/2012
Common Frequency Plot Shapes
How Is It Interpreted? • Evaluate central location • Evaluate spread • Learn from shape
Symmetrical •normal distribution
Bimodal •two peaks •data from two processes •separate and analyze each
Copyright © 2012
How is Age Distributed Among Patients Who Fell in our Care?
Common Frequency Plot Shapes
# of Falls in That Age Category
30
Copyright © 2012
25
20
15
Uniform •provides little info •check to see if multiple sources variation combined •if so, re-stratify and graph •may mean not enough bars •if so, change bar width and graph
10
5
0
Random •provides little info •check to see if multiple sources variation combined •if so, re-stratify and graph •May mean too many bars •if so, change bar width and graph
Age in Years Copyright © 2012
What Time Do People Call the Crisis Hotline?
30
# Times Hotline C Called In This Timeframe
# Times Hotline C Called In This Timeframe
40
Copyright © 2012
35
What Time Do People Call the Crisis Hotline?
25
30
20
25
20
15
15
10
10
5
0
5
0
Hours (24 Hour Clock)
Hours (24 Hour Clock) Copyright © 2012
Copyright © 2012
25
6/12/2012
Shewhart’s Rules
Are They the Same?
• When average, range or histogram used to summarize data: - Summary should not mislead user into taking any action user would not take if data were presented in a time series (graph) - Averages, etc.. are useful, but seeing the sequence and variation in data is most meaningful
Clinic
Avg. Annual Sat
Capitated Cost
(1-5 Scale)
Annual
A
3.9
$980 $
B
3.9
$940
C
3.9
$945
Copyright © 2012
Copyright © 2012
# M o n t h s F a llin g in E a c h C a t e g o ry
# M o n t h s F a llin g in E a c h C a t e g o ry
# M o n t h s F a llin g in E a c h C a t e g o ry
Comparison of Averages, Frequency Plots and Run Charts A ver age Client Satis f ac tion-Clinic A
Stratification with Frequency Plot
Client Satisf ac tion -Clinic A
7
Individuals
5.0 4.8
6
4.6
5
UCL = 4.47
4.4
4 3
4.2
Mean = 4.17
4.0
LCL = 3.88
3.8 3.6
2
3.4
1
3.2 3.0
0 3.4
3.5
3.6
3.7
3.8
3.9
4.0
4.1
4.2
1
4.3
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
16
17
18
19
20
21
22
23
24
17
18
19
20
21
22
23
24
Month
Aver ag e Sati sfac ti on Scor e
A ver age Client Satis f ac tion-Clinic B
Client Satisf ac tion -Clinic B
8
Individuals
5.0
7
4.8
6
4.6
UCL = 4.69
4.4
5
4.2
4
4.0
3
3.8
Mean = 4.17
LCL = 3.66
3.6
2
3.4
1
3.2 3.0
0 3.4
3.5
3.6
3.7
3.8
3.9
4.0
4.1
4.2
4.3
1
4.4
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Month
Aver ag e Sati sfac ti on Scor e
Averag e Clien t Satis factio n-Clinic C
Client Satis faction -Clinic C
7
Individuals
5.0 4.8
6
UCL = 4.70
4.6
5
4.4
Mean = 4.17
4.2
4
4.0
3
3.8
LCL = 3.65
3.6
2
3.4
1
3.2 3.0
0 3.4
3.5
3.6
3.7
3.8
3.9
4.0
4.1
4.2
4.3
4.4
Aver ag e Sati sfac ti on Scor e
1
2
3
4
5
6
7
Copyright 9 10 11 ©122012 13 Mo nth
8
14
15
16
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Stratification with Frequency Plot
Copyright © 2012
Another View of Stratification Average Satisfaction with Clinic (1-5 Scale) Indiv iduals
5.0 4.8 UCL = 4.68 4.6
Average Sattisfaction
4.4 4.2 4.0 3.8
Mean = 3.81
3.6 3.4 3.2 3.0 LCL = 2.94 2.8 2.6 1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
Sequential Weeks
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
26
6/12/2012
Stratification
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Various Formats
Copyright © 2012
Frequency Plot (Histogram): What Is It?
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
Tools to Learn from Variation in Data
• A bar chart for one variable • Used to visualize central location, shape and spread of the data • Each bar equal, each distinct • Most often used with time, money, throughput or a scaled measurement (i.e. dollars, weight, age, height,) - Frequency Plot does little good for interpretation if process not stable - Doesn’t show stability or capability in and of itself Frequency Plot Copyright © 2012
SCATTER PLOT: What Is It?
SCATTER PLOT: What Does It Look Like? Does Customer Waiting Time Affect Customer Satisfaction? High
Low Copyright © 2012
Copyright © 2012
Y
Negative Correlation
Customerr Satisfaction Ra atings
• Each dot on the chart represents a pair of measures • Becomes useful between 30-50 data points
Scatter Plot
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
• Graph to evaluate theory about relationship between one variable and another - Test for possible cause and effect - Does not prove a C & E relationship exists - A cause and effect relationship will be verified only when the improvement is tested and results studied using a control chart
Pareto Chart
Customer Waiting Time
High
X
Copyright © 2012
27
6/12/2012
Data for Scatter Plot : Does wait time impact satisfaction with clinic?
Model for Improvement What are we trying to accomplish?
Min Wait Sat Score
49 78 3 55 15 28 96 47 15 82 24 68 64
How will we know that a change is an improvement? What change can we make that will result in improvement?
Act
Plan
Study
Do
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Min Wait Sat Score
3.5 1 5 2.5 4 3 1.5 3 3.5 1 4 3 1
42 51 76 46 83 31 60 85 70 5 50 74 21
Min Wait Sat Score
4 3.5 3 5 2 5 2 2.5 1.5 5 3 2 4.5
Copyright © 2012
Wait Time and Satisfaction Scattergram
•Values higher as go up on graph •Start scale with actual lowest value in your data set
S a tis fa c tio n ( 1 - 5 S c a le ) D e p e n d e n t V a r ia b le
5.0
Wait Time and Satisfaction-Blank
4.5
Sc attergram
S a tis fa c tio n (1 -5 S c a le ) D e p e n d e n t V a ria b le
(49,3.5)
4.5
2 1.5 5 3.5 4 2 4.5 5 4 1.5 5 3 2.5
Copyright © 2012
Draw Graph •Independent Variable on X Axis (Horizontal) •Dependent Variable on Y Axis (Vertical)
5.0
74 72 15 64 17 91 10 5 9 71 7 55 74
Negative Correlation
4.0 3.5
4.0
3.0
3.5
2.5
3.0 2.5
2.0
2.0
1.5
1.5
1.0
1.0 10
20
30
40
50
60
70
80
Wait time (Min) Independent Variable
90
10
20
30
40
50
60
70
80
90
Wait time (Min) Independent Variable
Copyright © 2012
Copyright © 2012
How Is It Interpreted? • Look for patterns in the scatter plot - A narrow band of dots - A circular pattern - Peaks or troughs
Copyright © 2012
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
28
6/12/2012
Outliers
How Is It Interpreted?
Case Load Related to Sick Leave
• Outliers - Points that do not fall into the pattern of the others - Do not cluster with other points
16
Days of Sick k Leave Used
• • • •
18
Should investigate why appear May be a measurement error Possible may be a signal of a process change Possible may be change in relationship between the factors
14 12 10 8 6 4 2 0 30
35
40
45
50
55
60
65
70
Case Load
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Copyright © 2012
All Departments: Does Case Load Impact Sick Leave Use?
Department A 14
12
12
D a y s S ic k ( Y )
16
14
D a y s S ic k ( Y )
16
10
Stratification Using Symbols to Distinguish Each Department
10
8 6 4
8 6 4
2
2
0
0 30
35
40
45
50
55
60
65
30
35
40
45
50
Case Load (X)
Case Load (X)
Department B
Department C 16
14
14
12
12
D a y s S ic k ( Y )
16
D a y s S ic k ( Y )
Copyright © 2012
10
55
60
65
55
60
65
10
8 6
8 6
4
4
2
2
0
0 30
35
40
45
50
55
60
65
30
35
40 Copyright 45 © 201250
Case Load (X)
Case Load (X)
Acuity vs Cost-Total
Scattergram
8
6
6
What Did We Address?
$ in T h o u s a n d s
7
$ in T h o u s a n d s
7
5
5
4
4
3
3
2
2
1
1
0
0 3.0
3.5
4.0
4.5
5.0
5.5
6.0
6.5
3.0
3.5
4.0
Acuity
4.5
5.0
5.5
6.0
6.5
6.0
6.5
Acuity
Acuity vs Cost-Department B
Acuity vs Cost-Department C
Scattergram
8
Copyright © 2012
Acuity vs Cost-Department A
Scattergram
8
The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011.
Scattergram
8
6
$ in T h o u s a n d s
7
6
$ in T h o u s a n d s
7
5
5
4
4
3
3
2
2
1
• The value of displaying data graphically vs. table of numbers or summary statistics • The differences between data used for improvement, accountability and research • The value of displaying data over time: - when working to determine impact of changes being tested - To see if are sustaining gains • The Model for Improvement
1
0
0 3.0
3.5
4.0
4.5
5.0
Acuity
5.5
6.0
6.5
3.0
3.5
4.0
4.5 © 2012 5.0 Copyright Acuity
5.5
Copyright © 2012
29
6/12/2012
What Did We Address?
What Did We Address?
• Run charts: what they are, when used, how interpreted
• Introduction to Shewhart charts: what they are, what they are used for, how interpreted
- Median vs. mean: median used as center line - Rules for analysis to detect signals of improvement or degradation
-
• Ways to use
• Importance of good technique with limits • There are different types of Shewhart charts • How purpose of run chart differs from purpose of Shewhart chart
- Family of measures for improvement project - Small multiples - Stratification with
• Importance of good technique with median Copyright © 2012
What Did We Address? • Pareto charts, histograms and scatter plots: - what each looks like - what each is used for - how h eachh is i interpreted i t t d - stratification using these tools • Matching each of the 5 fundamental tools to the question being asked
Common and Special cause variation Different approaches to improvement with two types of variation What are upper pp and lower limits and where come from 5 rules for analysis to detect special cause
Copyright © 2012
References Books: 1. The Health Care Data Guide: Learning from Data for Improvement. Lloyd Provost and Sandra Murray, Jossey-Bass, 2011. 2. Total Quality Tools For Health Care. Productivity-Quality Systems, Inc. Miamisburg Ohio. ISBN: 1-882683-04-8 Tel. 1-800-777-2255. 3. The Improvement Guide. Gerald J. Langley, Kevin M. Nolan, Thomas W. Nolan, Clifford L. Norman, Lloyd P. Provost, Jossey-Bass, 2009. Video: 1. Making Sense Out of Control Charts. NAHQ. 1-800-966-9392 Software Used: 1. ChartRunner. PQ Systems. 1-800-777-3020. 1 800 777 3020. 2. QI Charts. API, 1-512-708-0131 3. Minitab,1-814-238-3280 Articles: 1. The run chart: a simple analytical tool for learning from variation in healthcare processes. Rocco J Perla, Lloyd P Provost and Sandra K Murray. BMJ Qual Saf 2011 20: 46-51. I
Copyright © 2012
Copyright © 2012
30
