B A C K G R O U N D B R I E F I N G

A N D

N E X T S T E P S

METIS

Geneva, February 2004

Statistical Data and Metadata Exchange Initiative

2

Challenges and Opportunities

What is SDMX?

Progress

Next steps

Overview

3

Exchange of statistical data is still complex, resource intensive, and expensive

Various international organisations have individual approaches for their constituencies

Uncertainties about how to proceed with new technologies (XML, web services …)

Challenges

4

International OrganisationsRegional Organisations

accountsstatistics

Banks, CorporatesIndividual Households

trans-actions

accounts

National Statistical Organisations

accountsstatistics

180

+ C

ount

ries

180

+ C

ount

ries

Inte

rnet

, S

earc

h, N

avig

atio

nIn

tern

et,

Sea

rch,

Nav

igat

ionwww.z.org

www.hub.org

www.x.org

www.y.org

Statistical information building blocks

5

Constituencies of international organisations have asked for common solutions

International organisations are in a position to expand their cooperation

IT technology seems to offer some ways forward that we should not ignore

Opportunities

6

Initiative of BIS, ECB, EUROSTAT, IMF, OECD, World Bank and UN

“ ... to focus on business practices in the field of statistical information that would allow more efficient processes for exchange and sharing of data and metadata within the current scope of our collective activities”

Global workshop at IMF, reports to UN Statistical Commission, liaison with international standards community and private sector initiatives

www.sdmx.org

What is SDMX? (1)

7

8

9

10

Projects (shared funding and resources) launched in June 2002

batch data exchange

metadata common vocabulary

metadata repositories

a practical case study on emerging e-standards

What is SDMX? (2)

11

Overall goal: reality check on emerging standards by building prototype.

Closely coordinated work with other projects – exploring ways to validate practical impact of project deliverables.

Builds on experimentation taking place in different SDMX institutions.

Support from other institutions and central banks.

Case Study

12

Case Study: general approach

Selected the BIS-IMF-OECD-World Bank hub on external debt for the demo example

Reviewed available and emerging standards

Established experimental XML standards based on existing statistical exchange standards (i.e. GESMES/TS metadata “key family” concepts)

Built a metadata registry and populated it with links to where the data can be found

Built web services (registration, search, transformation, comparison)

13

Case Study: key findings

Technology could be made to work better for statistical data exchange

Common information model underlies all statistical data

Common standards cut across different patterns of data exchange: bilateral, gateway, data sharing

Potential of (new) web technology is significant

XML is widely supported by software vendors, making it easier to use off-the-shelf tools to develop applications

registry-based architecture for metadata can facilitate navigation (i.e. data google)

use of web services enables automated data exchange

Possible to build on existing infrastructures, technologies and standards

14

Example of information model: metadata “key family”

CountryStock/Flow

Unit Multiplier

Unit

Topic

Time/Frequency

15

Simple tool for creating a key family (Demo)

16

Spreadsheet input to XML output (Demo)

17

Patterns of Data Exchange

Bilateral individual provider-to-consumer links could be batch or web dissemination

variety of format standards

Gateway many bilateral-to-central authority then

on to others (bilaterally) still many bilateral formats typical in data bank exercises

Data sharing statistical hubs through common

standards and formats data/metadata useable without prior

agreement dissemination could replace reporting

18

GESMES/TS Reporting & Registry-Based Dissemination (1)

Database EDIGateway

EDIGateway

Database

GESMES/TSData

(1) Today, the sender renders the data out of the database as GESMES/TS EDIFACT, and sends it to a counter-party accordingto prior agreement. When received by the counter-party, it goes through a gateway and is passed into their internal systems.

19

GESMES/TS Reporting & Registry-Based Dissemination (2)

Database

EDI & SDMX-MLGateway

EDI & SDMX-MLGateway

Database

Transmission of GESMES/TS & SDMX-ML Data

SDMX-MLGateway

Website

Registration

SDMX-MLGateway

Notification

Retrieval of SDMX-ML Data

Registry

20

Possible registry-based infrastructure

Registry

ClassificationScheme

ClassificationSchemeKey familyKey family DatasetDataset

Registry Services - RegistrationRegistry Services - Registration

StatisticsUser

ConsumerConsumerPublisherPublisher

IMF Website

World Bank Website

OECDWebsite

BIS Website

Central BankWebsite

Key family metadata

Key family metadata

Search, Retrieve, TransformSearch, Retrieve, Transform

DatasetDataset

21

Web Services for statistical information exchange and sharing

Web

Service

Service/Data Provider

Registry

Web

Service

Service/Data ConsumerService/Data Locator

22

23

24

25

26

27

Selecting Data Category to Search (Demo)

28

Results of Registry Search – Comparison View (Demo)

29

Reduced reporting burden – via common formats adopted by international organisations for data and metadata exchange

More user-friendly access when publishing national dataand metadata on the web - via global standards for data formats, catalogs/registries and associated services

Enhanced opportunity for navigation, discovery and comparative data analysis – via global guidelines for metadata vocabularies and repositories linked to statistical information in common formats

Can replicate these models and tools for statistical information systems at national levels

Anticipated benefits

30

Transition from projects to SDMX deliverables, on a learn-by-doing basis

metadata standards

technical standards

Pilot for Joint External Debt Hub (involving cost sharing by participating international organisations)

to implement initial version 1.0 standards

to implement re-usable and freely available tools and registries to provide the basis for capacity building, fostering the adoption of SDMX standards

to implement a pilot for creditor and debtor statistics based on these standards and tools

Envisaged next steps

SDMX

StandardsTools

RegistriesRepositories

Conformance TestsMonitor Projects / ActivitiesTraining / Capacity BuildingCopyright and Intellectual

PropertyContacts/Liaison N

atio

nal

Acc

ou

nts

SDMXMetadata Standards

Met

adat

a C

om

mo

n V

oca

bu

lary

Bal

ance

of

Pay

men

ts

Ext

ern

al D

ebt

Info

rmat

ion

Mo

del

Arc

hit

ectu

re

International

Sponsoring

Institutions

Fo

rmat

s (E

DI,

XM

L)

Aligned Subject-Matter Activities

(institutions and other fora adopting SDMX standards)

www.sdmx.org

Mo

net

ary

and

Fin

anci

al

Aligned Implementation

Activities

(institutions and other fora adopting SDMX standards)

Co

re S

ub

ject

-Mat

ter

Do

mai

ns

use

input inputinput

SDMXTechnical Standards

examplesexamplesdeliverables

deliverables

Co

re S

tati

stic

al C

on

cep

ts

Join

t E

xter

nal

Deb

t H

ub

input

OE

CD

.Sta

t

BIS

Dat

a B

ank

Co

mtr

ade

NA

WW

E

examples

StandardsTools

RegistriesRepositories

Conformance TestsMonitor Projects / ActivitiesTraining / Capacity BuildingCopyright and Intellectual

PropertyContacts/Liaison

SDMXMetadata Standards

Met

adat

a C

om

mo

n V

oca

bu

lary

Ext

ern

al D

ebt

Info

rmat

ion

Mo

del

Arc

hit

ectu

re

Fo

rmat

s (E

DI,

XM

L)www.sdmx.org

External Debt Joint Hub

Co

re S

ub

ject

-Mat

ter

Do

mai

ns

input inputinput

SDMXTechnical Standards

deliverables

Co

re S

tati

stic

al C

on

cep

ts

input / use

input / use

input / use

use

/ inp

ut

input / use

Aligned Subject-Matter Activity

UN Statistical Commission: Inter-Agency Task Force on Finance Statistics

(institutions and other fora adopting SDMX standards)

Aligned Implementation Activity

BIS-IMF-OECD-World Bank

(institutions and other fora adopting SDMX standards)

SDMX Standards-setting (Version 1.0) - Joint External Debt Hub Project: Overview

deliverables

use

/ inp

utus

e / i

nput

33

More about SDMX to follow in Working Paper 7 :

Inter-agency cooperation for the dissemiantion and exchange of standard metadata

www.sdmx.org