Embed Size (px)
Citation preview
1
Assessing the Quality of Administrative Data
2012 FCSM Statistical Policy SeminarSession 7: New Perspectives on the Quality of
Administrative Data
December 5, 2012
Bill Iwig, USDA/NASS
(on detail to OMB/SSP)
2
Development of the Data Quality Assessment Tool• Data Quality ~ Fitness for Use• Intended to facilitate decisions by a potential data user
about the suitability of the data for identified purpose.• Intended to prompt additional questions.• Based on the information gathered, the data user may
decide to:• Use the data as planned• Alter plans• Accommodate some quality weakness in the data• Not use the data
3
Development of the Data Quality Assessment Tool• Multi-Dimensional
• Relevance• Accessibility• Interpretability• Coherence• Accuracy• Institutional Environment
• Questions within each dimension are designed to address quality characteristics of micro-level administrative data.
• Example answers provided based on characteristics of a Supplemental Nutrition Assistance Program (SNAP) data set.
4
Development of the Data Quality Assessment Tool
Discovery Phase
• Evaluation period leading to approval for developing a data-sharing MOU.
Initial Acquisition Phase
• Evaluation period from MOU development approval to first-time receipt of data.
Repeated Acquisition Phase
• Continuing periodic receipt of data.
5
Putting the Phases and Dimensions Together• Discovery Phase: 12 questions covering Relevance,
Accessibility, and Interpretability, including a request for a Data Dictionary
• Initial Acquisition Phase: 29 questions covering Accessibility, Interpretability, Coherence, Accuracy, and Institutional Environment
• Repeated Acquisition: 11 questions covering Interpretability (updated Data Dictionary), Coherence, Accuracy, and Institutional Environment
6
What it is and What it isn’t• Evaluates quality characteristics of an
administrative data file.• Doesn’t address quality characteristics of linked
data.• Doesn’t provide guidance for making fitness for
use decisions.• Doesn’t provide quality improvement
recommendations.• Not specifically designed to address quality of
commercial data.
7
Status of Pilot Testing
• Obtained feedback from three federal/state reviewers prior to pilot testing.
• Pilot tested on:• Child Care data files from the Office of Child Care Information
Systems at DHHS – Initial Acquisition Phase• Wisconsin Child Care Payment System files – All Phases
• Expecting a response for two additional files.• Planning to conduct follow-up interview after completion
of assessment tool.
8
General Comments
Completing the Data Quality Assessment Tool is perceived to be burdensome to the administrative agency.
Some agencies may be concerned about data quality implications.
Many administrative data files are compiled from various data sources.
Right scope and content to be incorporated into agency administrative record procedures?
9
Next Steps
• Update based on feedback received.• Make available to the public.• Pursue opportunities to promote use.
10
Thank You
Bill Iwig
Detailee to the Statistical and Science Policy Office, OMB
202-395-2293
