Peter Benson Fima 2009

Managing Your Enterprise–Wide Data To Minimize RiskManaging Your Enterprise–Wide Data To Minimize Risk

Day 1 - Data Quality and MetricsDay 1 - Data Quality and Metrics

4:15-4:504:15-4:50

Bringing your data up to Bringing your data up to

ISO 8000 Data Quality StandardsISO 8000 Data Quality Standards

FIMA 2009 Monday March 16, 2009

New York, NY

Mr. Peter BensonISO 8000 Project Leader Executive Director and Chief Technical OfficerElectronic Commerce Code Management Association (ECCMA)

Standards Hierarchy

STRATEGIC

TACTICAL

Market Acceptance

Market Access

Social Responsibility

Standards of Convenience

Customer Requirements

Government Requirements

Societal Expectations

Industry and CompanyPreferences

Mandatory Voluntary

FIMA 2009, March 16, 2009, New York, New York

If

Quality information comes from Quality Data

and

Enterprise-wide data is Master Data

What is ISO 8000-110:2008 Master Data Quality? and

What are the costs and benefits of bringing data up to the new international standard?


Wisdom

Knowledge

Information

Data

Information vs. Data

“… the quality of the data element values is a major determinant of the quality of the information and consequently the accuracy and reliability of the knowledge that can be derived from the information.” ISO 8000-110:2008


Information* representation of something and what can be logically inferred from it NOTE 1 The term "representation" is used in its most general sense to include not only symbolic representations, but also images and what is held in the mind.

NOTE 2 The phrase "logically inferred" means statements that can be deduced by applying the laws of logic.

Data* symbolic representation of something that depends, in part, on its metadata for its meaningNOTE An image could be a representation, but not a symbolic representation.

EXAMPLE The written description of a chair is a symbolic representation. This is contrasted with a picture of a chair, which is a non-symbolic representation.

* Represents international consensus of ISO TC 184/SC 4 subject to ballot

ISO 8000 Definitions


ISO 8000 – Data Quality

• ISO 8000 addresses data quality. ISO 8000 is concerned with:

– the principles of data quality;

– the characteristics of data that determine its quality;

– the processes to ensure data quality.


data

transactional data

master data describes entities that are independent and fundamental for the organization;

needs to be referenced in order to perform transactions

identification datadefined by reference to another organization’s

master data

descriptive datadefined by the

characteristics of the entity being described

Master Data

Data that identifies and describes individuals, organizations, locations, goods, services, rules

and regulations.

• Vendor master

• Customer master

• HR master

• Asset master• Material master• Service master• etc.


ISO 8000 Data Quality

100 Series covers Master Data QualityISO 8000-100:2009* Master Data Quality – Introduction and terminologyProvides an overview of master data and the terms and definitions used to define it.

ISO 8000-110:2008 Master Data Quality - Syntax, semantics, and conformance to data requirementsSpecifies requirements that can be checked by computer for the exchange, between organizations and systems, of master data that consists of characteristic data.

ISO 8000-120:2009* Master Data Quality - ProvenanceSpecifies requirements for representation and exchange of information about provenance of property value pairs.

ISO 8000-130:2009* Master Data Quality - AccuracySpecifies requirements for representation and exchange of information about accuracy of property value pairs.

* Completed ISO ballot cycles, awaiting publication


ISO 8000-110:2008

Syntax

Each data set shall contain a reference to the syntax to which the data set complies....The reference shall be resolvable to the specification of the syntax through a mechanism that is publicly available.

Semantic encoding

Each data element value shall reference all concepts necessary to unambiguously define its meaning. Each reference shall be to a concept dictionary entry contained in a concept dictionary that supports an interface for resolution of a concept identifier.

Syntax and semantic resolution shall be available at no charge unless the data carries a “fee based encoding” warning label.

Conformance to requirements

Each data set shall contain a reference to the data requirements statement to which the data set complies. The reference shall be a globally unambiguous identifier that was used to encode the data set. The reference shall be resolvable to the data requirements statement. The data requirements statement shall be publicly available.


All identifiers are copyright. They belong to the organization that issued them and their use is subject to the terms and conditions imposed by the issuer.

Unless identifiers have been declared available for public use without a licence, they should never be used to retrieve data that was not supplied by the owner of the identifier unless you have specific permission to do so.

In order to protect your data from claims of “joint work” you should not use proprietary identifiers as metadata.

Data quality and Intellectual Property (IP)


A syntax must exist

Semantic encoding must be explicit

“The great thing about standards is that there are so many to chose from.” Anon.

ISO 8000-110:2008 is standards neutral

ISO 9362 Bank Identifier Code (BIC / SWIFT Code / )

ISO 10383:2003 Codes for exchanges and market identification (MIC)

ISO 6166:2001 International Securities Identification Number (ISIN)

ISO 9735 Electronic Data Interchange

ISO 15022 Securities -- Scheme for messages

ISO 20022 Universal financial industry message scheme

ISO 19312 Market Data Model - under development

Country code + Tax IDor DNA?

Ultimate Personal Identification Number

Country code + Tax ID International Organization Identification Number

GPS? International real estate parcel identification number


Structured Data

Data requestor

Data

Application

Application

Graphics Documents

Data provider

Data

Traditional data transfer


ISO 8000-110 is about automating the data supply chain

Data requestor

Data provider

Sub

eOTD-i-xml(data requirements statement) ISO 22745-30

eOTD-q-xml(query)

ISO 22745-35

Sub-TiereOTD-q-xml

Sub-TiereOTD-r-xml

eOTD-r-xml(data exchange)ISO 22745-40


Meeting the requirements of ISO 8000-110:2008 Master Data Quality

• Certified Software applications and data cleaning services

– Know how to use an open technical dictionary for metadata encoding and mapping

– Know how to read and write master data requirement specifications

– Know how to generate requests for master data

– Know how to read and write encoded master data messages

• Certified Master Data Quality Managers (data requestors)

– Know how to specify their master data requirements

– Know how to ask for they data they need to validate or complete their master data

• Certified Quality Master Data Providers

– Know how to respond to a request for master data


NEXT STEPS1. ISO 8000-110:2008 Master data quality; Syntax, semantic

encoding and conformance to data specification.

a) Develop your metadata registry (map it to the eOTD*)

b) Develop your data requirement statements (as eOTD-i-xml Identification guides*)

c) Measure the quality of your data - perform a gap analysis

d) Develop a plan to source missing data

e) Develop a plan to validate data

2. ISO 800-120:2009 Master data quality; Provenance

a) Add provenance data to imported and exported data

* optional

Technology

Peter Benson Fima 2009