Data, Metadata and Quality Management
Framework
(Quality and Information Management Framework at the Vietnamese Ministry of
Planning and Investment)
Michael Colledge and Bryan Fitzpatrick, Consultants
European Technical Assistance Programme for Vietnam (ETV2)
2
Content of Presentation
1. Context – Vietnamese Ministry of Planning and Investment
2. Problems
3. Solutions
4. Quality Concepts and Management
5. Information, Data and Metadata Concepts and Mgt
6. Quality and Information Management Framework
7. Conclusions
3
1. Context: Vietnamese Ministry of Planning and Investment (VMPI)
• Core business functions
• Organisation and working environment
• Inputs and outputs
4
Core business functions as defined in regulations• Developing strategies and plans for national socio-
economic development− includes analyzing and forecasting economic
performance
• Developing mechanisms and policies for economic management
• Issuing decisions, instructions, and circulars on planning and investment
• Managing official development assistance− including developing strategies for attracting,
coordinating and reporting
5
Core business functions as defined in regulations (continued)
• Managing procurement and tendering
• Managing enterprise and business registration
• Developing plans for renewal and development of state enterprises
• Promotion of small and medium size businesses
• Appraising and monitoring individual investments
6
Core business functions
• Briefly summarised−planning, monitoring, analysis, forecasting,
decision making
7
Organisation and Environment
• 28 Departments and Centres− fragmented, overlapping responsibilities
• Rapid evolution− Transition for market economy −No such ministry in western developed
countries−Move to role as Ministry of Economy
• General Statistics Office recently added
8
Information and Data Inputs and Outputs
• Characteristics of inputs−diverse range − relatively low volume
• Characteristics of outputs−mostly information rather than data
9
2. Quality Problems and Issues
• Information and data acquired are poor or unknown quality and/or lacking in timeliness
• Procedures and facilities for acquiring, processing, analysing and sharing information and data are poor
• Information and data often acquired and maintained on paper
• Data sources not fully exploited – duplication of data collected by GSO
• There are differing versions of what are nominally the same data
10
Quality Problems and Issues (continued)
• Limited metadata accompanying information and data− data are not well understood
• Departments/centres receive funding directly from donors and undertake uncoordinated developments− No organisational unit with mandate or
sufficient resources to take coordination or leadership role.
• There is no definitive repository of information and data
• MPI website is poorly populated and data are error prone
11
Underlying Problems• Division of functions amongst departments has
resulted in semi-autonomous fiefdoms with little motivation to collaborate
• No organizational unit responsible for developing, facilitating and monitoring general quality management policies, guidelines and procedures for the MPI as a whole
• No organizational unit responsible for developing, facilitating and monitoring general information, data and metadata management policies, guidelines and procedures
12
3. Solutions
• Introduction of the Quality and Information Management Framework
• Development principles−Simplicity−Use of Standards, Guidelines and
Recommended Practices−Harmonisation and Integration
13
Harmonisation and Integration
14
Transformation Process
Inputs Outputs
Figure 1:
15
Transformation ProcessInputs OutputsAcquisition
ProcessDistribution
Process
Repository
Figure 2
16
Figure 3
17
Transformation Processes
Inpu Acquisition
ProcessDistribution
Process
Repository
Figure 4
18
Transformation Processes
Inpu
Acquisition Processes
Distribution Processes
Repository 1
Figure 5
Repository 2
19
4. Quality Concepts and Management
• Evolution of quality concepts− Inspection−Quality control −Quality assurance− Total quality management
• International quality standards− ISO 9000 series−European Foundation for Quality
Management (EFQM) Excellence Model
20
ISO 9000 Series
• ISO 9000: 2005 Quality Management Systems - Fundamentals and Vocabulary− To provide concepts and terminology
• ISO 9001:2000 Quality Management Systems - Requirements − Basis for certification
• ISO 9004:2000 Quality Management Systems – Guidelines− for performance improvements in mature
system
21
ISO 9000 Quality Concepts
• Customer focus
• Leadership
• Involvement of people
• Process approach
• System approach to management
• Continuous improvement
• Fact based decision making
• Mutually beneficial supplier relationships
22
ISO 9001: 2000 Quality Management Systems
• Basis for certification
• Comprises five parts: − Quality Management System− Management Responsibility− Resource Management− Product Realization− Measurement, Analysis and Improvement
• Does not provide implementation guidelines
• Needs to be interpreted/adapted to particular circumstances
23
Interpretation/Adaptation to VMPI Context
• ISO Standards primarily designed in context of enterprises selling to market
• Need to be interpreted/adapted to particular situation
• Government organisation− not profit based − “user” rather than “customer”− mostly internal users− user cannot influence quality through purchase
decisions
• Whose products are information and data − with structural and reference metadata
• structural – needed to access product• reference – needed to understand its quality
24
Interpretation/Adaptation to VMPI Context (cont)
• Statistical offices are specialists in information/data
• Have developed several standards− Eurostat Quality Framework− OECD Quality Framework− IMF Data Quality Assessment Framework− Statistics Canada Quality Framework− Statistics Sweden…
• Frameworks differ only slightly
25
General Definition of Product Quality
• Starting point for quality management
• From customer/user perspective− “Totality of features and characteristics of a
product or service that bear on its ability to satisfy stated or implied needs” (ISO 8402 – 1986)
− “Degree to which set of inherent characteristics (of product) fulfill requirements (ISO 9000)
− “Fitness for Use”
• Need to specialise to government information agency − by identifying and defining characteristics of quality
26
Characteristics of Information/Data Product Quality
• Relevance− Degree to which products meet current and potential
user needs− Are required information/data available?− Are information/data available required?
• Accuracy/Reliability− Degree to which information/data correctly describe
event/phenomenon/situation they are designed to measure/represent
− Many aspects, no single overall measure
• Timeliness− Length of time between information/data availability and
events/ phenomena they describes− Measured relative to time period for which product is
likely to be useful
27
Characteristics of Information/Data Product Quality
• Accessibility− Extent to which users informed of information/data
availability− Suitability of format and medium by which product
accessed− Cost of access
• Interpretability− Ease with which user may understand and use
informatin/data− Definitions of concepts, coverage and data elements
• Coherence− Degree to which information/data products are
logically consistent and complete
28
Other Quality Considerations
• Quality dimensions are overlapping and interrelated
− cannot be combined into a single indicator
• Achieving acceptable level of quality is matter of trade-off
− for example accuracy against timeliness
• Cost must also be considered as constraint on quality
−quality cost trade-off
• Product quality achieved through process quality and performance management
29
Process Quality and Performance Management
• Two aspects to management of processes−Effectiveness – degree to which
processes generates products of high quality
−Efficiency – minimal use of resources in generating products
30
Figure 6. Process Quality and Performance Management
Quality and Performance Characteristics
Methodology Management
Information /Data
Management
Technology Management
Human Resource
Management
Effectiveness(Product Quality)
Relevance ** *
Accuracy ** *
Timeliness * * * *
Accessibility ** * *
Interpretability ** *
Coherence * * *
Efficiency(performance)
Resource Usage
** ** *
Key: ** primary responsibility; * shared responsibility
31
Quality Concepts - Conclusion
• In organisation whose primary inputs and outputs and information/data
information, data and metadata management is key to quality and performance management
32
5. Information, Data, Metadata Concepts and Management
33
Information and data management concepts and management
Terminology
• Data refers to numerically structured information− typically tables of numbers, graphs and charts
• Data has to be accompanied by metadata− describing to what the numbers refer and how
they have been produced
• Used in narrow sense information refers material that are not structured− reports, correspondence, pictures, images,
etc.
• Used in broad sense information includes data and metadata
34
Information Metadata Standards
• Dublin Core ISO 1588-2003
• Comprises set of descriptors for any information resource:− title, creator, subject−description, publisher, contributor−date, type, format, identifier− source, language, relation− coverage, and rights
• Others?
35
(Data) Metadata Standard
ISO/IEC 11179: Information Technology Metadata Registries
Standard aims to:
• provide a common understanding of data elements across organizational units and between organizations
• support re-use and integration of data over time, space, and applications
• harmonise and standardise data within an organization and across organizations
36
(Data) Metadata Standard
ISO/IEC 11179: Information Technology Metadata Registries (continued)
Comments:
• Part 3: Registry metamodel and basic attributes – generalised, complex difficult to understand
• Part 6: Registration – good starting point, will use simplified version
37
Data and Metadata Standards
ISO/TS 17369:2005 SDMX V1.0
• To support business practices by enabling efficient exchange and storage of data and metadata
• Provides format for structuring and reporting metadata, including methodology
• Provides an agreed structure fordata/metadata flows
SDMX V2.0 (not yet ISO standard)
• Being reviewed by WG2 of ISO TC 154
• Enhanced treatment of metadata
38
Figure 7: SDMX Top Level Model
39
SDMX Top Level ModelThis is a description of an data or metadata “flow” – an
abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by
MPI from various sources)
40
SDMX Top Level Model This is a description of an data or metadata “flow” – an
abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by
MPI from various sources)
This is an instance data or metadata set from a particular
provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period
41
SDMX Top Level Model This is a description of an data or metadata “flow” – an
abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by
MPI from various sources)
This is an instance data or metadata set from a particular
provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period
Provision Agreements indicate what Providers will provide what subset, when,
how often, and how
42
SDMX Top Level Model This is a description of an data or metadata “flow” – an
abstracted data or metadata set that will potentially occur for many periods and from many providers (eg a regular table received by
MPI from various sources)
This is an instance data or metadata set from a particular
provider at a particular time, eg, a particular table from Ninh Binh province, for a particular period
Provision Agreements indicate what Providers will provide what subset, when,
how often, and how
This identifies the Data Providers, giving indicative and contact information and linking to Provision Agreements and actual data and metadata sets
43
SDMX Top Level Model
This describes the structureof the data or metadata flow –
all the metadata needed to request and understand an instance of the flow (an actual data or metadata set). Links to all other structural
metadata.
This categorises all the defined data and metadata flows, providing
a structuring framework and a basis for searching. Links to
other structural metadata.
44
SDMX Top Level Model
45
SDMX Application at VMPI
Envisage SDMX Registry/Repository with:
• Code sets and classifications− environment for standardising and harmonising
• Data structure definitions− for all regular data sets, i.e., data flows
• Categorisation schemes− to index data flows
• Data storage environment for data sets− initially simple file store− possibly a database store− possibly a star-schema store
• with star schema, design generated automatically from structural metadata
• provides options for different “cuts” through data flows
46
6. Elements of Quality and Information Management Framework at VMPI
• A: More Strategic and Integrated Perspective of VMPI Role, Functions and Information:− A1: Review and Revision of Role and Functions− A2: Comprehensive Understanding of Inputs,
Processes and Outputs
• B: Fully Functional Quality and Information Management Programmes:− B1: Establish Quality Management System− B2: Establish Information, Data and Metadata
Management Programme
47
Elements of Quality and information management framework (continued)
• C: Improved Quality and Information Management Infrastructure:− C1: Develop Corporate Quality Management
Facilities− C2: Develop Corporate Metadata Management
Facilities− C3: Develop Corporate Facilities for
Acquisition, Capture, Storage, Access and Dissemination of Data
− C4: Develop Corporate Facilities for Acquisition, Storage, Access and Dissemination of Information
− C5: Enhance Corporate Planning, Monitoring, Analysis and Forecasting Facilities
48
Elements of Quality and information management framework (continued)
• D: Continuous Improvement and Reengineering of Business Processes:− D1: Continuous Improvement of Core Business
Processes− D2: Reengineering of Core Business Processes
• E: Comprehensive Quality and Information Management Training Programme:− E1: Develop and Conduct Quality Awareness and
Management Training− E2: Develop and Conduct Information, Data and
Metadata Management Training− E3. Develop and Conduct Training in Planning,
Monitoring, Analysis and Forecasting
49
Conclusions• In an organisation whose main inputs
and outputs are information and data− Quality management and information/data
management go hand in hand− Information/data management can be
viewed as component of quality management
− Quality management is good umbrella for achieving information/data management
• Need for recipe describing:− (parts of) international standards that are
readily applicable and useful− additional best practices that should be
considered