Upload
edna
View
53
Download
0
Tags:
Embed Size (px)
DESCRIPTION
IMF Approach to Storing Metadata with Macroeconomic Statistics. UNECE Workshop on the Common Metadata Framework (Vienna, Austria, 4-6 July 2007). Dissemination Standards Bulletin Board (DSBB). data standards initiative (SDDS/GDDS countries’ dissemination practices - PowerPoint PPT Presentation
Citation preview
IMF Approach to Storing Metadata with Macroeconomic Statistics
UNECE Workshop on the Common Metadata Framework
(Vienna, Austria, 4-6 July 2007)
Dissemination Standards Bulletin Board (DSBB)
• data standards initiative (SDDS/GDDS• countries’ dissemination practices• information that SDDS countries provide the
IMF on their dissemination practices• direct links to the economic and financial data
that countries disseminate under the SDDS• information that GDDS countries make
available to the IMF on their statistical practices• http://dsbb.imf.org
Collaboration with OECD
Dec 2006 - Agreement to use Dotstat and MetaStore to form the basis of the IMF data warehouse
Jan 07 – software available on joint Team Foundation Server (TFS)
Feb 07 IMF.Stat installed with the assistance of OECD
May 07 have loaded: International Financial Statistics (IFS), World Economic Outlook (WEO), and Sub Saharan Africa Regional Economic Outlook (REO)
June 07 – signed an MOU which supports a collaboration approach to future enhancements for the mutual benefit of both organizations
Data Fact table
CouGrpID
ConceptID
DataSourceID
UnitOfMeasID
TimeFreqID
StatusID
Observation
Flag
100
250
3000
10
25
2
158.1
E
Country Group CouGrpID
ParentID
Code
100
Null
156
Label Canada
Concept ConceptID
ParentID
Code
250
200
NGDP
Label Gross ...
DataSource
DatSrceID
ParentID
Code
3000
Null
WEO
Label World ...
Unit Of Measure
UofMID
ParentID
Code
10
Null
N
Label Nat Curr
Time & Frequency TimeFreqID
ParentID
Code
25
Null
200401
Label 2004 Q1
Status StatusID
ParentID
Code
2
Null
SHARE
Label Shareable
Metadata Fact table
CouGrpID
ConceptID
DataSourceID
UnitOfMeasID
TimeFreqID
StatusID
MetadataID
100
250
3000
10
25
2
5487
Country Group CouGrpID
ParentID
Code
100
Null
156
Label Canada
Concept ConceptID
ParentID
Code
250
200
NGDP
Label Gross ...
DataSource
DatSrceID
ParentID
Code
3000
Null
WEO
Label World ...
Unit Of Measure
UofMID
ParentID
Code
10
Null
N
Label Nat Curr
Time & Frequency TimeFreqID
ParentID
Code
25
Null
200401
Label 2004 Q1
Status StatusID
ParentID
Code
2
Null
SHARE
Label Shareable
Data Referential Metadata
Metadata
TextChain-linked GDP volume measures are expressed in ...
MetadataID 5487
IMF.Stat Data ModelIMF.Stat Data Model
Structural metadata
• Economic Concepts -mapped as many time series as possible to the Catalogue of Time Series and loaded them to IMF.stat
• Countries and groups – used IFS version of Country names and codes as the authoritative source for codes and labels
• Unit – chose to combine unit and scale e.g. Millions of US dollars
• Storing data in native units i.e. not trying to convert observations to a common unit.
• Status, Source and Time and Frequency reasonably straight forward so far. Will become more problematic when we introduce versioning.
• Working through existing metadata from IFS publications and production system
• Where necessary/possible cleaning it up, standardizing it and loading it to MetaStore
• WEO – metadata sourced from the external web site, reformatted and stored in MetaStore then exported to IMF.stat
• All referential metadata loaded to MetaStore and then exported to IMF.Stat
Referential Metadata
Data- IFS
All time series which were able to be mapped to the Catalogue of Time Series (CTS)
• Includes – Exchange rates– Balance of Payments– International Investment Position– Real Sector Statistics– International Liquidity– Money and Banking non-SRF data
• Excludes– Government Finance– Money and Banking SRF data– Fund Accounts
» 191 concepts» 233 countries» 39 groups» 7.6 million observations
WEO• Two most recent editions • Includes series published externally as well as other
series available internally• Concept - generally consistent with the CTS• Country and group – some differences in codes used
so mapped where possible. Some groups added.• Unit - limited number of units used and mainly
consistent across countries
Data-WEO
Sub Saharan Africa REO–Structural Metadata
• Concepts – virtually no codes or labels in common with the CTS
• Able to map those series published in the REO but the supporting series too difficult. Are now working through them on a case by case basis to determine which if any map to the CTS
• Country and group – country codes and labels mainly consistent with WEO. Groups all different even though sometimes have the same label.
• Units - mainly ratios which were added to the authoritative list.
Sub Saharan Africa REO Referential Metadata
• Have sourced top level referential metadata only. Will work with the Africa Department after the data are loaded to identify any usable referential metadata.
Data-REO
MetaStore• Some modifications with assistance from OECD
– Now includes • structural metadata • mappings to authoritative lists• referential metadata
SchemaLogic– In future may integrate structural metadata in
MetaStore or replaceAlignment with SDMX
– Have used 42 ‘types’ to categorize our referential metadata
– Added one to the OECD set, which are consistent with SDMX
Managing metadata within the IMF
• Locate relevant sources of metadata• Locate potential warehouse content• Central repositories for data and metadata• Harmonizing and mapping to a preferred term• Authoritative lists• Working with Information Services Division (ISD) to
ensure information management best practice• Assigning data stewards to manage metadata
Governance
• Establishing groups and individuals with certain roles and responsibilities for management of metadata– Economic Data Advisory Group
• Representation from departments across the Fund• Includes several working groups with specific focus
– Information Services Division • Responsible for provision of metadata
– Metadata and Standards team• New group in the Statistics Department currently focusing on
metadata used in the data warehouse
Next Steps
• Changes to work practices across the Fund• Identify a data steward for each dimension in IMF.Stat• Standardization, authoritative sources• Reuse of metadata across systems• Raise awareness of the value of quality metadata• Tie together basic schemas
IFS
WEO
Internal
External
Data sources
DataStream
ETL
IMF.stat
MetaStore
User interface
End-users
User interface
Referential and structural
metadata
Haver
111 USA112 UK273 MEX...2005;USA,GDP,548.25
...2004;USA,GDP,526.25
Data flow
Referential metadata
Structural metadata
EDW Top Level Diagram
ConceptCountryGroup
Data Source
Time& Freq