Embed Size (px)
Citation preview
Ensuring Quality
of Health Care Data:
A Canadian Perspective
Data Quality Asia Pacific Congress 2011
Heather RichardsConsultant
Canadian Institute for Health Information (CIHI)
Tel:+1 250 220 2206
Email: [email protected]
1
> The Canadian Institute for Health Information
> Data Quality Challenges in Canada
> Strategies to Ensure Data Quality:
– CIHI’s Data Quality Framework
– Data Quality Reporting Tools and Studies
– Techniques for Communicating Data
Quality
Agenda
2
The Canadian Institute for
Health Information
3
Canadian Institute for Health Information
> National, independent, not-for-profit agency,
established in 1994
> One of Canada’s leading sources of high-quality, reliable and timely health information
> 27 health databases
and registries
> 7 offices
4
CIHI’s Mandate
> Coordinate, develop, maintain and disseminate
health information on Canada’s health system
and the health of Canadians
5
CIHI's Mandate Con't
> Provide accurate and
timely information
required for:
– Sound health policy
– Effective management
of the health system
– Public awareness about
factors affecting
good health
6
Data Quality Challenges
in Canada
7
Data Challenge: Variety of Partners
> Accommodating different coding standards at
provincial/territorial level versus national level;
> Recognizing different uses of the data and different
focus on data quality;
> Adjusting for differing data collection methods
8
CIHI PartnersRegional
health authorities
Health
Canada
Professional
associations
Private sector
organizations
Researchers
Non-governmental
organizations
Ministries
of health
Statistics
Canada
Health
facilities
CIHI
9
Data Challenge: Secondary Data Collector
> CIHI does not collect data directly
> Our data comes from:
– provincial governments;
– hospitals; and
– professional associations
10
… this means that
we cannot affect first hand
how that data is captured and collected.
> CIHI relies on data providers (some are voluntary
data providers) to report accurate information
> Poor quality data often result from difficulties in
collection standards, coding standards and
chart documentation – and lack of training
11
Data Challenge: Secondary Data Collector
Data Challenges: Other
> Variety of databases and usability
> Data flow and timeliness
> Coding and comparability
> Hospital practices and data completeness
12
End-stage renal failure
13www.vancouversun.com
Question: Are Risk Factors Completely
Captured at all Facilities?
14
Prevalence of Pulmonary Edema
0%
10%
20%
30%
40%
50%
60%
70%
Inter-Quartile Range: 13-27%
Questionnaire Reveals a Correlation of
Data Completeness to Hospital Practices
15
Reviews select documentation
Reviews all documentation
Prevalence of Pulmonary Edema
0%
10%
20%
30%
40%
50%
60%
70%
IQR: 16-29%IQR: 8-21%
Chart Review Confirms Under-Reporting
16
Prevalence (%)
Data Captured
by Dialysis
Clinic Staff
Data Captured
by CIHI coder
during Chart
Review
Pulmonary edema 22 27
Sensitivity=62%
Specificity=93%
PPV=77%
NPV=87%
Epidemiologists and clinical
researchers prefer seeing
these statistics…
Strategies to Ensure Data Quality
17
• CIHI’s Data Quality Framework
• Data quality reports and studies
• Techniques for communicating DQ
Strategies to Ensure Data Quality
18
CIHI’s Data Quality Framework
CIHI’s Data Quality Framework
> Objective approach to
assessing data quality
and producing standard
documentation
> Three parts
1. Work Cycle
2. Assessment Tool
3. Documentation
19
1. Data Quality Work Cycle
20
Plan
ImplementAssess
2. Data Quality Assessment Tool
> Provides a consistent
approach for defining
data quality
> Five dimensions
– Accuracy
– Comparability
– Timeliness
– Usability
– Relevance
5
19
61
Dimensions
Characteristics
Criteria
21
2. Data Quality Assessment Tool
22
AccuracyComparability
Timeliness
Usability
Relevance
CoverageCapture and collection
Unit non-response
Item (partial) non-response
Measurement error
Edit and imputation
Processing and estimation
Population of reference explicitly stated
Coverage issues are documented
Frame validated
Under or over-coverage rate
Assessment Tool: Educational Component
23
3. Data Quality Documentation
> Details the data quality
documentation required
for each data holding
24
Metadata Documentation
Retain
knowledge
about the
management
of a
database
with the
database.
25
Strategies to Ensure Data Quality
26
Data Quality Reporting Tools
and Studies
Deputy Minister Data Quality Reports
> Bird’s eye view
> Broad DQ scope:
assess accuracy,
timeliness, comparability
and usability
> Specific audience:
Deputy Ministers of
Health
27
Features of the Deputy Minister
Data Quality Reports
> Each indicator is important to the success
of a database and has a defined action to
improve performance
– Snapshot of results across all jurisdictions
– Trending over time
> 11 databases
– 8 from CIHI
– 3 from Statistics Canada
28
Components of the Deputy Minister
Data Quality Reports
29
Database-
specific reports
Technical
documents
P/T indicator
tables
Trending
results
Flags table
Each DM package
Trending: Discharge Abstract Database
2003-04
2004-05 2005-06
2006-07
2007-08
2008-09
2009-10
0.0
0.5
1.0
1.5
2.0
2.5
Optimal Value = 0
Indicator 1: Total Outstanding Hard Error Rate, per 1,000 Abstracts
30
Response to Reports
Positive:
> Highlights to DM the value of a
database; increases coverage of
data holdings
> Reveals systemic problems
causing DQ issues; helps Deputy
Ministers prioritize and reallocate
resources
> Congratulates on past DQ
improvements; facilitates creation
of DQ improvement action plans
31
Reabstraction Studies
> Detailed review
> Narrow DQ scope: assess
coding consistency,
correctness, completeness
> Wide audience
32
Reabstractorrecodes from chart
Application reveals original data
Application compares data
Reabstractorassigns reasons for differences
Study Methods
> A chart review to
recapture the data and
compare
33
Overview
Determine study
method
Develop data
collection tool
Train coders,
collect data
Process and
analyze data
Share results
34
Study Objectives
Reabstraction Study Example
DAD: Discharge Abstract Database
> Data on acute-care hospital activity
> Data supports:
– funding and system planning decisions at government level
– management decisions at the facility level
– clinical research at the academic level
35
Strategies to Ensure Data Quality
36
Techniques for Communicating
Data Quality
Communicating Data Quality Using
Different Lenses
37
Statistics for OECD
international
comparisons
Health
indicators
Categorizing
hospitalizations
for hospital
management purposes
Isolating determinants
of good health
Assessing
quality of care
Clinical
research
purposes
such as
survival analysis
Health
Indicators
> Assess population health and
health system performance
> Will look at one indicator:
ACSC hospitalizations
38
Health Indicator: ACSC Hospitalizations
39
2001-022002-03 2003-04 2004-05 2005-06
2006-072007-08
0
100
200
300
400
500
600
Age-Standardized Rate of ACSC Hospitalizations per 100,000 Population
459
326
2007-08 DAD Study: ACSC Hospitalizations
> Question: Is the decrease in ACSC
hospitalizations real or is it due to changes in
coding quality?
> Answer: The observed decrease is real
– National rates are indeed decreasing
– Reabstraction studies found that certain patient
populations had lower quality data
40
2007-08 DAD Study: ACSC Hospitalizations
Sensitivity
Grand mal status, epileptic convulsions 81%
Chronic obstructive pulmonary diseases 91%
Asthma 90%
Diabetes 95%
Heart failure and pulmonary edema 84%
Hypertension 100%
Angina 94%
Any ACSC hospitalization 90%
41
Data Quality Challenges that Lie Ahead
> The health sector is a changing landscape
– Electronic health record
– Health care funding
– New technologies
– New modes of delivering
health care
> New data will bring new quality challenges
42
43
“Taking health information further”
