Data Archiving at the U.S. Central Bank

Linda F. PowellBoard of Governors of the Federal Reserve System

Research and Statistics Division

IASSIST Conference May 2005

Challenges

Collection/AcquisitionQualityStorage

DocumentationCataloguing Dissemination

Data Collection / Acquisition

SurveysUniform storage and retrievalUniform documentationExpensive collection process

Purchase data from vendors or acquire data from other sourcesVarious formats and technologyUnique documentationLicensing agreements

Data Collection through Storage – Micro Data Surveys

Edit data at FRS

DataTransmission

DataTransmission

FRS FinclData Repos.

FRS Struct DataRepos.

Reporter (e.g. bank)

Process Data at FRS

Display Data

Reports

Data Collection through Storage – Micro Data Purchased from Vendors

Comes in a variety of formats and modes of access

Some data are processed and stored in SAS datasets

Other data are only available through proprietary software

Licensing agreements vary, affecting who has access to data and how the data can be stored

Data Collection through Storage – Macro Data

Acquired from other public sourcesA limited amount is purchased from

vendorsMicro data are aggregated to create

macro dataMost production macro data are stored

in FAME databases (time series data)

Data Collection through Storage – Data Quality

Not all data are created equalMacro data are edited for

reasonableness and consistencySurvey micro data are edited for

reasonableness, consistency, and compliance with business rules

Vendor data are generally reviewed for accuracy by the end users. Resolution of problems varies by product

Data Documentation – Micro Data

Purchased data have individual documentation

Survey data have a uniform set of documentation compiled in the Micro Data Reference Manual (MDRM)

MDRM

Each accounting or economic concept gets a numberThe same number is used across surveys

for the same conceptEach survey gets a mnemonic

Surveys can have sub mnemonics to show distinctions between the series within a survey

MDRM Example

Total Assets of an institution is collected on many surveys

Total assets always has the number 2170For finance companies the variable name

is DFCR2170Commercial banks have:

RCON2170 for domestic total assetsRCFD2170 for foreign total assets

MDRM Information

Data Documentation – Macro Data

The FAME database can be self documentingLocation, contacts, periodicity, units, …

Data stored in FAME use a hierarchical nomenclatureE.g. Disposable Outlays of Personal Income

data are labeled Y.P.D.O

What Data are Available?

Survey Micro Data are recorded in MDRM Macro Data are listed in FAME Other data are recorded in Data & News

Catalogue All purchased data Some databases created in-house (e.g. merger

adjusted financial data and supervisory data) Surveys performed by external organizations There is some overlap with MDRM and FAME

Data Dissemination

Data are stored on various platforms in different applications

Survey micro data are stored on mainframe and replicated on SQL server and Unix/Linux Users write programs against databases Some in-house end-user applications

Macro data stored in FAME users write programs against FAME database

Vendor purchased data are accessed through the vendor software

Data Dissemination –Data and News Catalogue Division:   Monetary Affairs

Database:   TINYVendor:  Federal Reserve BoardData Contact:  John DoeLicense Contact:  Jane DoeLicense Information:  Board Unix userForm of Access:  Unix Network: www.rsma.frb.gov/cbp/Description:  Merger adjusted Call Report data. Data are from the quarterly Reports of Condition and Income (Call Reports) for insured domestic commercial banks and nondeposit trust companies. The data consolidate information from foreign and domestic offices and have been adjusted to take account of mergers.Status:  Active ContractKeyword(s):  Accounting , Bank Mergers ,Bank Profits, Banking Structure, Banks and Banking , Call Report , Deposits , Financial Institutions , Financial Statements , Interest Rates    

Link to database or documentation

Link to vendor website

Data Dissemination

Important factors to make data dissemination easier

Uniform storage facilitiesHigh quality meta dataUsing transmission protocols and

standardsXBRL (financial micro data)SDMX (economic macro data)

In Summary

As the volume and diversity of data increase the need for organized, flexible, and well documented data archives becomes more critical.