Embed Size (px)
Citation preview
Quality Management of Statistical Processes Using
Quality Gates
Narrisa Gilbert
Australian Bureau of Statistics
May 2011
Overview
• Background• What are Quality Gates• Attributes of Quality Gates• Six Components of Quality Gates • Outcomes of Quality Gates• Further information
Different perspectives on quality
ABS approach to quality management
• What?– Reinforcing a high degree of credibility for the ABS
and its outputs– Maintaining the relevance of ABS outputs
• How?– Effective relationships with respondents– Processes that produce high quality outputs– Regular review and evaluation of statistical activities– Staff skilled and motivated to assure the quality of
ABS outputs
Background• The Australian Bureau of Statistics produces a
lot of statistics
• On occasion the quality (accuracy) of data has been called into question at the last minute (right before release to the public)
• Sometimes no issues are identified• Sometimes errors are discovered
• Sometimes errors in data are identified after release
What are quality gates?
"checkpoints or decision points at various strategic places in a statistical
process at which the quality of the process at or up to that point is
explicitly assessed"
Attributes of Quality Gates
• Planned in advance of issues occurring
• Designed to facilitate the detection, discussion and resolution of issues and problems with processes earlier
• Constantly reviewed and revised
Attributes of Quality Gates
• Collaborative
• Communicate openly the progress/issues associated with the collections
• Provide a model of accountability and responsibility for survey processes
Attributes of Quality Gates
• Knowledge Management
• Create a store of corporate knowledge
• Document and monitor issues and actions throughout the collection cycle
Six components of quality gates
Placement Quality measures Roles Tolerance Actions Evaluation
1. Placement
Assigning priorities Related to risk profile of collection cycle Want to detect errors 'upstream‘
What can go wrong?
When can this problem occur?
What impact can it have?
Placement of Quality Gates
• For example
Within
Handover
UNECE (United Nations Economic Commission for Europe) Secretariat 2009, Generic Statistical Business Process Model, Version 4, April 2009, UNECE, Geneva.
Areas of risk changes to processes, systems,
methodologies
data transformations
overlap / coordination / integration with other areas
knowledge management
2. Quality measures Choose good indicators of potential
problems Prioritise! Definitions May need to drill down Time series useful
How would we know if this problem occurred?
3. Roles Owner area Sign-off person Operational person / Gate Keeper Stakeholders Gate definition Gate assessment / sign-off
Who is responsible?
Who will this affect?
4. Tolerance Driven by user requirements Consider size of natural variation, sampling
errors, level of detail, importance of outputs Need to form expectations Historical data may help Predetermined!
What is an acceptable level of quality?
5. Actions reflect extent of the problem reflect consequences "traffic lights" concept
What will we do (if there is a problem)?
Who needs to be informed?
6. Evaluation evaluation of processes evaluation of quality gates
What has this information told us about our quality?
How can we improve in the future?
Outcomes from Quality Gates
Business processes well-defined and known by staff
Responsibilities for processes and quality clearly defined
Statistical risk explicitly identified & assessed
Outcomes from Quality Gates
Explicit sign-off occurs at Quality Gates
Quality Measures clearly defined and understood
• Errors identified earlier (and fixed)
Further information
• Quality Management of Statistical Processes Using Quality Gates, Dec 2010, (cat.no. 1540.0) on the ABS website.
• http://www.abs.gov.au/ausstats/[email protected]/mf/1540.0
