National Accounts implementation stories

7th Meeting of the Expert Group on SDMX 27-30 October 2014, Seoul, Korea

Daniel SURANYI, Eurostat Daniel.Suranyi@ec.europa.eu

National Accounts implementation stories

What you will hear today…

• Implementation of SDMX for National Accounts

• Business Case

• Governance

• Use case national implementation

• Use case international data cooperation

• Summary: state of play SDMX-NA

National Accounts implementation stories

• Implementation of SDMX for National Accounts

• Business Case

• Governance

• Use case national implementation

• Use case international data cooperation

• Summary: state of play SDMX-NA

Why SDMX for National Accounts?

Within the organisation

Across organisations

Internationally

Why SDMX for National Accounts?

Within the organisation

Across organisations

Internationally

Within the organisation

Across organisations

Internationally

National Accounts business needs

Within the organisation

Across organisations

Internationally

• Main trigger: new methodology • SNA 93 SNA 2008

• ESA 95 ESA 2010

• Business process reengineering • Corporate production

systems

• Integration of production chains

• Harmonised coding

National Accounts business needs

Within the organisation

Across organisations

Internationally

• Cross-domain integration

• Balance of Payments

• Foreign Direct Investment

• Reengineering of data exchange

• Standardisation

• Improved validation

• Reuse of tools

National Accounts business needs

Within the organisation

Across organisations

Internationally

• Data quality • 1 country 1 year 1 GDP

• Shared validation rules

• Shared responsibilities

• Lowering reporting burden • 1 data flow from

national to international

• International cooperation • Inter-Agency Group on

Economic and Financial Statistics = SDMX Sponsors

• Example: PGI Website http://www.principalglobalindicators.org

BOP National Accounts

Main Aggregates

Sector Accounts

Supply & Use Input / Output

GFS

FDI

BOP National Accounts

Main Aggregates

Sector Accounts

Supply & Use Input / Output

GFS

FDI

BOP National Accounts

Main Aggregates

Sector Accounts

Supply & Use Input / Output

GFS

FDI

SDMX

National Accounts

•SNA 2008

•ESA 2010

•Other NA related domains

FDI statistics

Other domains

Balance of Payments

Statistics

Cross Domain Concepts and

Code Lists Shared

Concepts and Code Lists

• 2 global SDMX implementation projects

• …for National Accounts (SNA2008 / ESA 2010)

• …for Balance of Payments (BPM6) and Foreign Direct Investment (BMD4)

• 2 project steering groups and 2 technical groups

• Running from 2011 until 2013

• Publication of V1.0 DSDs in September 2013

• Steering groups Ownership Group

Background

• Mandated by SDMX Sponsors in January 2014 • As a single OG for NA & GFS, BOP and FDI

• Representation of management from content and SDMX side

• Oversees maintenance of SDMX artefacts • Mandated technical groups

• 1 on BOP and FDI; 1 on NA and GFS

• Close cooperation with IAG Task Force

international data cooperation (TFIDC)

Ownership Group (OG)

Maintenance Agreement

• MA will only apply changes agreed upon by OG

• Change requests may lead to…

• Structural changes: major release (e.g. SNA2025 )

• Annual maintenence release (regular case)

• Fast-track maintenance release (only for small changes limited to selected dataflows) (subject to review during next annual release)

State of Play

• V1.1 published in Jan 2014 • V1.2 in April 2014 in SDMX Global Registry

• V1.3 fast-track release for NA Sector Accounts (to fix a mistake in one code list)

• V1.4 regular release currently in preparation • Support for coding Euro Area 19 • Inclusion of IMF GFSM coding • Support for SNA93 in the context of international

data cooperation • Support for additional dataflows (e.g. GNI, EDP) • Minor additions to existing code lists

• Next OG meeting: November 2014

SDMX-NA Governance

• Maintenance Agency: Eurostat

• Ownership Group:

• Active: ECB, Eurostat, IMF, OECD, UNSD

• Passive: BIS, World Bank

• Published in the global registry:

• http://registry.sdmx.org

• Annual maintenance cycle (April-April)

• SDMX-NA maintenance page:

• http://sdmx.org/?page_id=1498

Data Structure Definitions (DSDs)

DSD ID

NA_...

DSD Title Tables in transmission programs from

2014

MAIN Main aggregates ESA / OECD: 1, 3, 5, 20, 22

SEC Sector accounts

and government

finance

ESA / OECD: 2, 6, 7, 8, 9, 11, 26, 27, 28

ECB: MUFA, GFS

EDP: 1, 2, 3, 4

OECD: 119, 7HH, 7II, PSD

G20: Recommendation 15

SU Supply / use and

input / output

ESA : 15, 16, 17, 18, 19

REG Regional statistics ESA: 10, 12, 13

PENS Pension schemes ESA: 29

National Accounts implementation stories

• Implementation of SDMX for National Accounts

• Business Case

• Governance

• Use case national implementation

• Use case international data cooperation

• Summary: state of play SDMX-NA

Who?

What?

When? Who?

Where? How?

What?

ESA 2

010 T

P

SDMX-NA

Technical Integration

National statistical organisations

International organisations

National databases

(Mapping)

Source

Source

Source

SDMX Data Structure

Defintions & Data Flows

National data sources

Metadata driven process based on SDMX standards and tools Full automation of the data exchange possible

Data provider view: One goal – different possibilities

Webforms Online data collection tools

SDMX-ML file

Database export

SDMX Converter SDMX-ML file

Database SDMX Reference Infrastructure

SDMX-ML file

SDMX Web Service

Same DSD

Same Format

Excel sheets

SDMX Converter SDMX-ML file

SDMX process flow and supporting tools

Non-SDMX

local data

NSI Map /

Recode Extract Transform SDMX file

Disseminate / Transmit

NSI software

NSI software

SDMX Converter

Processing for sending

Push

SDMX Converter

SDMX-RI

Mapping Assistant

Test client Web client

Processing for sending

NSI Web service

Push

Pull / Hub

NSI development

NSI software Push / Pull

/ Hub

NSI developed software Eurostat tools

SDMX Reference Infrastructure https://webgate.ec.europa.eu/fpfis/mwikis/sdmx/index.php/SDMX_Reference_Infrastructure_SDMX-RI

Data Provider Data Collector

Non-SDMX local database

SDMX data set

SDMX Registry

DSD

SDMX-RI

Mapping Assistant

Web Svc

Test Client

Web Client

Data Validation Process

Before/During Transmission (“First Level”)

- Covered by SDMX today

- Format Check (SDMX-ML)

- Code Check (SDMX DSD)

After Transmission ( “Second Level”)

- Not yet covered by SDMX

- Detailed value check

- Mirror check

- …

National Accounts implementation stories

• Implementation of SDMX for National Accounts

• Business Case

• Governance

• Use case national implementation

• Use case international data cooperation

• Summary: state of play SDMX-NA

International Data Cooperation Background

General objectives • Reduce reporting burden on NSIs

• More efficient use of resources in

International Organisation (IO)

• Ensure high quality and consistency

of data of official statistics

• Improve global data exchange and dissemination

SDMX provides • technical/statistical standards

• new exchange modes (hubs)

• clear rules and responsibilities

SDMX Web Services

A possible data sharing model

EU OECD

other than EU World other than OECD

Eurostat - ECB

OECD

IMF, UN, WB, BIS, other IOs

U S E R S

The necessary pre-conditions

• SDMX maintenance agreements Ownership and maintaining agency to be defined (normally between the international organisations)

• Internationally agreed data validation

Data validation rules and SDMX data validation standards (beyond file structure validation)

• Streamlined data exchange processes Frequency and timeliness for data dissemination, data quality management etc.

International Data Cooperation in National Accounts

• IAG: Interagency Group on Economic and Financial Statistics

• BIS, ECB, Eurostat, IMF, OECD, World Bank

• same organisations as SDMX Sponsors

• IAG Task Force International Data Cooperation

• TFIDC, co-chaired by Eurostat and IMF

• Pilot 1 on GDP and population

• Pilot 2 on Sector Accounts

TFIDC – State of Play

• Pilot 1 on GDP and population

• Data flow template agreed, based on NA_MAIN DSD

• Monthly data exchange, senders: Eurostat, OECD, IMF

• Some minor technical issues: e.g. SDMX header compatibility SDMX TWG

• Go live planned for Q1/2015

• Pilot 2 on Sector Accounts

• First tests successful, based on NA_SEC DSD

• Lessons learned from pilot 1

• Pilot to kick off Q1/2015

TFIDC – Next steps

• Agreement on detailled validation rules • Eurostat / OECD agreement as basis

• Support for SNA 93 data exchange • Change request pending in Ownership Group

• Improvement on data exchange frequency • Monthly PUSH not enough to meet objectives

• Discussion on future topics • Plans for dissemination using global DSDs

• PULL mode, improved data coverage…

National Accounts implementation stories

• Implementation of SDMX for National Accounts

• Business Case

• Governance

• Use case national implementation

• Use case international data cooperation

• Summary: state of play SDMX-NA

SDMX-NA : Here & Now

• Governance operational Ownership Group • Harmonised development with BOP/FDI ensured

• National implementation projects on-going • Various tools and possibilities available

• International Data Cooperation • Promising projects for improving official statistics

globally

• SDMX-NA as showcase implementation project