Upload
aaliyah-thorpe
View
219
Download
2
Tags:
Embed Size (px)
Citation preview
Exploring the Medical Laboratory Quality ToolBox-A
Michael A Noble MD FRCPCClinical Microbiology Proficiency Testing program
Program Office for Laboratory Quality ManagementUniversity of British Columbia
Vancouver BC
Management
by theMoment
Management
by theMomentYou can try
to fix them…
Management
by theMomentBut th
ey never go away!
So the better approach is…
An systemic approach of organization, plan, review, and
action gives you the best chance of success.
The Quality Toolbox
The Quality Toolbox
Lean
RISKFMEA
Six Sigma
Metrics
Surveys
Statistics
The Quality Toolbox
January 2009 7
What are quality tools?
ISO 9001: 2000
Six Sigma
LEAN
A quality tool is a supplement or component part of a quality program that usually will not stand alone
but can enhance the total quality system
Tools in the Toolbox
• Mega-Tools– Lean– Six Sigma – Priority Matrices and
Risk Assessment
• Tools/Techniques– Balanced Scorecards– Brainstorming– Control Charting– Flow Charting– Quality Indicators– Surveys
Quality Indicators are Metrics(measured process information)
• Determine quality of services.• Highlight potential quality concerns, • Identify areas that need further study and
investigation, and • Track changes over time.
A really good, inexpensive
reference book
Amazon.com$30.00
Keeping ScoreUsing the Right Metrics to Drive World Class Performance
1996
Many organizations spend thousands of hours collecting and interpreting data. However many of these hours are nothing more than wasted time because they analyze the wrong measurements, leading to inaccurate decision making.
– Mark Graham Brown.
Characteristics of Good Metrics
Timedshort and long term
Timedshort and long term
Engagingall levels
Engagingall levels
Balancedfull cycle
Balancedfull cycle
Actionableaction oriented
Actionableaction oriented
Interpretablespecific
Interpretablespecific
Achievablecontained
Achievablecontained
Measurableobjective
Measurableobjective
GoodMetricsGood
Metrics
Seven Steps to Successful Indicators
Do not start data collection until these are addressed
1. Objective
2. Methodology
3. Limits
4. Interpretation
5. Limitations
6. Presentation
7. Action plan
Developing IndicatorsObjective What are you trying to measure?
1. Why am I collecting this information? Be specific
Methodology How to capture the data1. What data needs to be captured2. Who (or what) to capture the data3. How often to capture the data4. Is it achievable (time, resources, revenue)?
Limits Can I preset levels for:1. Acceptable, Concern, Unacceptable, Critical
Presentation Graphic or Text
Interpretation 1. What does it mean?2. Does it reflect on YOUR quality?3. Can I compare it?4. Can I trend it?
Limitations 1. Unintended variables
2. What does it not mean?
Action Plan 1. What will I do if it indicates acceptable performance?
2. What will I do if it does not?
Developing IndicatorsObjective What are you trying to measure.
1. Why am I collecting this information? Be specific
Methodology How to capture the data1. What data needs to be captured2. Who (or what) to capture the data3. How often to capture the data4. Is it achievable (time, resources, revenue)?
Limits Can I preset levels for:1. Acceptable, Concern, Unacceptable, Critical
Presentation Graphic or Text
Interpretation 1. What does it mean?2. Does it reflect on YOUR quality?3. Can I compare it?4. Can I trend it?
Limitations 1. Unintended variables
2. What does it not mean?
Action Plan 1. What will I do if it indicates acceptable performance?
2. What will I do if it does not?
Setting Relevant Limits and Ranges
• Set Objectively• Validate by Study• Clinical Relevancy• Customer Expectation• Matched Benchmarks• Regulation
60 minutesRelevant or Easy?
Assessing Quality Indicators• Importance Potential for Improvement
• Scientific Acceptability Reliability and Validity
• Feasibility Implementation and cost
• Usefulness Comprehensive
Having Quality Quality Indicators
IQLM Indicator List
1. Diabetes monitoring (system) 2. Hyperlipidemia screening (system)
• Test Order Accuracy and Appropriateness (pre-analytic)
• Patient Identification (pre-analytic)
• Adequacy and Accuracy of Specimen Information (pre-analytic)
• Blood Culture Contamination (pre-analytic)
• Accuracy of point-of-care testing (analytic) • Cervical cytology/biopsy correlation (analytic) • Critical Values Reporting (post-analytic) • Turnaround time (post-analytic) • Clinician satisfaction (post-analytic) • Clinician follow-up (post-analytic)
CLMA Survey Pre-examination Phase Indicator List
•Ordered test is appropriate for patient care
•Patient consent appropriately collected
•Test utilization by clinician for best patient care
•Physician written order with every specimen
•Cost/benefit assessment for laboratory test menu
•Patient identification and its accuracy
• Preparation of patient for specimen collection
•Appropriate specimen container•Timing of specimen collection•Phlebotomy success•Specimen integrity •Specimen quantity•Specimen transportation•Accuracy of specimen identification
•Condition for specimen storage
CLMA Survey Examination Phase Indicator List
• Quality Control• EQA-external quality
assessment • Time to first result
availability • Specimen
contamination
• Laboratory injuries or accidents
• Competency of testing personnel
• Vacancy of technical staff
CLMA Survey Post-examination Phase Indicator List
• Result reporting accuracy• Adequacy of information
for interpretation of laboratory tests
• Report delivery turnaround time
• Consistency of critical values reporting
• Result interpretation by physician
• Patient’s satisfaction with laboratory services
• Patient’s satisfaction specifically with phlebotomy services
• Physician’s satisfaction with laboratory services
Contmination Rate: Blood Culture Sets
0.0%
1.0%
2.0%
3.0%
4.0%
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Time Period
Per
cent
2002-2003 2003-2004 2004-2005
Contmination Rate: Blood Culture Sets
0.0%
1.0%
2.0%
3.0%
4.0%
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Time Period
Perc
ent
2002-2003 2003-2004 2004-2005
Limits: Below 2%
Interpretation: Meeting accepted limits all the time
Limitations Definition may include some true infections and may miss others
Presentation: Linear time graph
Action plan: Identify and educate blood collector group.If stable for 2 years, then consider dropping from routine
Objective: to ensure that blood culture results reflect sepsis.
Methodology: Count single bottle positives of common skin flora/ total sets
Quality Indicators and Timing
Use an indicator only as long as
it providesyou with
useful information. Don’t get tied to
your indicators
Underfill Blood Collections (As a percent of collections per site)
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
RF2
RF1
ER
ICU
Phlebotomists
Limits: Below 2% (?)
Interpretation: Wards with inexperienced collectors have problems
Limitations Some frail and elder people have very weak veins and may be impossible to collect
Presentation: Linear time graph
Action plan: Identify and educate blood collector group.
Objective: to ensure that blood culture are properly
filled.
Methodology: Count underfilled bottles / total bottles collected
Underfill Blood Collections (As a percent of collections per site)
0.0
2.0
4.0
6.0
8.0
10.0
12.0
14.0
16.0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39
RF2
RF1
ER
ICU
Phlebotomists
Caution about patient outcome indicators
Theoretically, outcomes best assess quality, but they are the most difficult to measure – too many confusing variables
• Age, underlying conditions, therapy, circumstance
– require high volumes of detailed data – need long collection periods
David HsiaMedicare Quality Improvement Bad Apples or Bad Systems? JAMA. 2003;289:354-356.
Quality Indicators, Done Well, Will Consume More Time Than
You Have
FACT:
•Set Priority•Set Limits•Drop Non-Productive Activity•Target: 10-12
Don’t be an
Indicators Glutton
Computer Nonsense Metrics[urine culture] * [glucose] * [INR]
[NUPA hr] * [Telephone minutes] X100
Just because a computer can
calculate a value, doesn’t mean that
it should.
The BIG SECRET for Quality Indicator Team
Engage the folks who do the work, because they know what they do!
Risk management is activity directed towards assessing,
mitigating (to an acceptable level) and monitoring of risks to an enterprise.
Risk Managementhelps definePRIORITIES
and helpsPREVENT ERROR
Tough Decisions 1
• A group of 10 laboratories over 300 square miles considers centralizing all tests to a single facility.
Improved cost efficienciesSimplified process controlCloser oversightImproved utilization managementImproved Patient Care
Reduced clinical-lab interfaceExtended pre-examination phase
Test delaysImpeded Patient care
Tough Decision 2
• A laboratory considers laboratory redesign based on LEAN analysis.
Improved workflowImproved time efficienciesSolve ergonomic challengesImproved cost efficienciesImproved Patient Care
Expensive renovationsConstruction interruptions
Staff retrainingEquipment specificityTest menu specificity Impeded Patient care
Tough Decision 3
• Laboratory considers implementing a quality management team.
Better information for better decisionsBetter process controlContinual Improvement ProcessImproved Patient Care
Staff ReallocationIncreased immediate costs
Quality Commitment Impeded Patient care
Tough Decision 4
• A laboratory proposes the need for an on-site level 3 (increased biosafety level) microbiology laboratory.
Improved biosafetyReduced laboratory infectionsFaster diagnosisImproved Patient Care
Construction costsOperating costs
Potential Bioterror targetImpeded Patient care
Risk and Priority and Prevention are Relative Terms
• Risk tolerance• Risk transfer
• Risk reduction• Risk prevention• Risk avoidance
• Risk aversion
CO
MF
OR
TACTION
PR
IOR
ITY
Risk Management Documents for the Medical Laboratory
• ISO 14971:2007Medical devices -- Application of risk management to medical devices
• ISO 20993:2006 Biological evaluation of medical devices -- Guidance on a risk-management process
• ISO WD TS22367:2007Medical laboratories -- Reduction of error through risk management and continual improvement
Linking Quality and Risk Managements
Introduction to ISO TS 22367
“Risk management framework has been described as being of
steps, planning for risk, identifying risk and its impacts,
developing risk-handling strategies, and monitoring for risk
control. These steps are consistent with management
requirements as described in ISO 15189:2003 including
identification and control of non-conformities, establishment of
preventive and corrective actions, performance of internal
audit and management review and continual improvement.
It is the objective of this technical report to link these in the
context of the medical laboratory.”
Linking Prevention and RiskISO TS 22367
• Assessment of risk of deviations from standard• The quality manager should establish and maintain a process
for identifying incidents associated with deviations from standards requirements, estimating and evaluating the associated risks to patient care, and controlling these risks and monitoring effectiveness of the control.
• The process should include a prospective risk assessment method for processes considered as high risk.
• Assessment of potentially high risk processes may be based upon previous audit, survey, experience, or evidence-based literature on procedures where a failure may lead to significant safety risk to patients.
Linking Prevention and RiskISO TS 22367 (Continued)
– The quality manager should identify a team of people to study the selected process.Note 1: The team should have personal knowledge of the process.Note 2: The team should be comprised of people with different levels and types of knowledge.
– The team should organize a thorough investigation of the process to include:a) each step of the processb) how each step of the process may failc) how each failure at each step of the process may affect patient safetyd) the rank of severity of each failure mode effecte) the most critical failure mode effects f) potential root causes of the most critical failure mode effectsg) procedures to address the potential root causes.
– The analysis of this Failure Mode Effects Analysis should form the basis of a Prevention Action Plan.
Occurrence – Severity Grid(Qualitative)
Remote Rare Common Frequent
Insignificant
Mild
Moderate
Severe UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
DEBATABLERISK
UNACCEPTABLERISK
UNACCEPTABLERISK
DEBATABLERISK
UNACCEPTABLERISK
UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
DEBATABLERISK
ACCEPTABLERISK
DEBATABLERISK
DEBATABLERISK
Occurrence – Severity Grid(Semi-Qualitative)
Remote (0.2)
Rare (0.4)
Common(0.8)
Frequent (0.9)
Insignificant(0.1)
Mild(0.2)
Moderate (0.8)
Severe(0.9)
UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
DEBATABLERISK
UNACCEPTABLERISK
UNACCEPTABLERISK
DEBATABLERISK
UNACCEPTABLERISK
UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
DEBATABLERISK
ACCEPTABLERISK
DEBATABLERISK
DEBATABLERISK
Severity – Occurrence Grid
• Numerical Values– Literature Benchmarks– Consensus– Experience
• Regardless of source, values should be both verified, and validated to fit the FMEA being performed.
HIV Testing Risk EvaluationCanadian versus American Risk Analysis
Remote (0.2)
Rare (0.4)
Common(0.8)
Frequent (0.9)
Insignificant(0.1)
Mild(0.2)
Moderate (0.8)
Severe(0.9)
UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
? UNACCEPTABLERISK
UNACCEPTABLERISK
DEBATABLERISK
UNACCEPTABLERISK
UNACCEPTABLERISK
ACCEPTABLERISK
ACCEPTABLERISK
DEBATABLERISK
ACCEPTABLERISK
DEBATABLERISK
DEBATABLERISK
Quality Approaches Look Alike
Strategy Event Steps
Corrective Action Plan
PLAN MONITOR EXAMINEREMEDIATE
AND CORRECT
CONTROL
Preventive Action Plan
PLAN IDENTIFY EXAMINEDEVELOP STRATEGY
MONITOR
Risk Management
PLAN IDENTIFY EXAMINEDEVELOPSTRATEGY
MONITOR
Six Sigma DEFINE MEASURE ANALYZE IMPROVE CONTROL
Failure Mode Effects Analysis
• FMEA is a systematic method of studying failure.
• 1940s US Armed Forces1960s Aerospace (Apollo Space program)1970s Ford Motor Company (post-Pinto)1990s semiconductors, food service, plastics,
software, and diagnostics equipment.• Documented FMEA is a REQUIREMENT in the
automotive, in-vitro diagnostics, and pharmaceutical industries.
October 1, 2008
FMEA: Failure Mode Effects Analysis
In a process:– where are points that the process is likely to
break down?– what is the probable occurrence of the
different possible break-downs?– what are the likely severity and consequences
of the different possible break-downs?– What is the priority list for addressing the
different possible break-downs?
FMEA: Failure Mode Effects Analysis
• FMEA– Failure Mode Effects Analysis
• FMECA– Failure Modes Effects and Criticality Analysis
• FTA– Fault Tree Analysis
• HACCP– Hazards Analysis and Critical Control Points
• PDPC– Process Design Program Chart
A Medical Laboratory FMEA
PhysicianRequisition
SampleCollection
Sample Set-up
SampleAnalysis
SampleTransport
A Medical Laboratory FMEA
PhysicianRequisition
SampleCollection
Sample Transport
Sample Set-up
SampleAnalysis
Good Sample
Proper ContainerSite Identification
Proper Packaging
Courier competent and available
Equipment and stainsworking properly
Workload
Time issues
Expertise
Labelling
A Medical Laboratory FMEA
PhysicianRequisition
SampleCollection1.5
Sample Transport0.5
Sample Set-up1.9
SampleAnalysis1.3
Good Sample
Proper ContainerSite Identification
Proper Packaging
Courier competent and available
Equipment and stainsworking properly
Workload
Time issues
Expertise
0.5
0.5 0.2
0.8
0.3
0.2
0.6
0.5
0.5
0.30.4
Labelling0.9
SEVERITY
A Medical Laboratory FMEA
PhysicianRequisition
SampleCollection0.45
Sample Transport0.8
Sample Set-up0.25
SampleAnalysis1.05
Good Sample
Proper ContainerSite Identification
Proper Packaging
Courier competent and available
Equipment and stainsworking properly
Workload
Time issues
Expertise
0.5
0.05 0.3
0.1
0.4
0.4
0.1
0.5
0.5
0.050.05
Labelling0.1
OCCURENCE
Pre-Examination Grid
October 1, 2008
Remote 0.2
Rare 0.5
Common 0.8
Frequent>1
Insignificant(0.2)
Mild(0.5)
Requisition Transport
Moderate (0.8)
Severe(>1)
Set-up Collection Analysis
A warning about FMEA
• If done as a structured “top-down” exercise, many critical break points will be missed.
• If done as an unstructured “bottom-up” exercise, the team will get bogged down in minutiae
FMEAs should ALWAYS be:✔Directed✔Team Efforts✔Planned✔Structured
The BIG SECRET for The FMEA Team
Engage the folks who do the work, because they know what they do!