www.eia.gov U.S. Energy Information Administration Independent Statistics & Analysis

Adding Relational Metadata to Statistical

Data APIs

April 27, 2015 | Washington, D.C.

Mark Elbert

Director, Office of Web Management

mark.elbert@eia.gov

Evolution of EIA Data Publishing

Mark Elbert, Washington, D.C.,

April 27, 2015 2

web

Application Programming Interface

Data Browser Electricity (Apr 2012)

Coal (Feb 2014)

Electricity (Apr 2012)

State Energy Data System (Sept 2013)

Petroleum and Natural Gas (Oct 2013)

Annual Energy Outlook (May 2014)

STEO (Sept 2015)

Crude Oil Imports (Nov 2015)

Short Term Energy Outlook (May 2014)

Crude Oil Imports (Nov 2015)

People wanting

quick summaries

People wanting

deep data dives

Systems Applications &

websites

Bulk Data File System (Jan 2014)

Coal (Feb 2014)

Program office databases create products to post to the web

Leveraging EIA’s data API

Mark Elbert, Washington, D.C.,

April 27, 2015 3

Chart widget:

Fetches data and create charts from series

IDs. Created Winter Heating Fuels page with

408 interactive charts in record time.

Excel add-in:

Game changer for analysts who live in

Excel.

Mark Elbert, Washington, D.C.,

April 27, 2015 4

geoset

time series

relation

Population

hypercube

example:

Mark Elbert, Washington, D.C.,

April 27, 2015 5

prime vertex

secondary vertex

tertiary vertex

lowest level

aggregate

male

female

nativ

e A

meric

an

age

race

sex

white

bla

ck

Asia

n

oth

er

multi ra

cia

l

Population hypercube example (cont.):

Mark Elbert, Washington, D.C.,

April 27, 2015 6

1 prime vertex

13 secondary vertices

52 tertiary vertices

60 lowest level

aggregate

126 geo-sets (= 𝑙𝑒𝑛𝑔𝑡ℎ 𝑜𝑓 𝑑𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑖 )

Each geo-set (cell) may contains thousands of data series each. The

use of data series and geo-sets structures simplifies the task of

provided relational metadata for the hyper-cube.

1D relationships and visualization:

Mark Elbert, Washington, D.C.,

April 27, 2015 7

nativ

e A

meric

an

age

race

sex

white

bla

ck

Asia

n

oth

er

multi ra

cia

l

2D relationship and visualization:

Mark Elbert, Washington, D.C.,

April 27, 2015 8

male

female

age

sex

nativ

e A

meric

an

age

race

sex

white

bla

ck

Asia

n

oth

er

multi ra

cia

l

Non-prime vertices have relationships too:

Mark Elbert, Washington, D.C.,

April 27, 2015 9

It can get a little complicated…

Mark Elbert, Washington, D.C.,

April 27, 2015 10

Dimensions can be

hierarchical, creating

cubes within cubes…

Capturing this information allows government

data APIs to drive data visualizations with

dynamic drill-downs and data context

…but

EIA data API visualization library

Mark Elbert, Washington, D.C.,

April 27, 2015 11

• Open source JavaScript library to be released on

github.com/eiagov

• Pre-release examples

– SEDS Fossil Fuel relation:

• Chart

• Map

• Relation

• Map-relation interactive

– Electricity by fuel

– Map supported

• US states

• US electricity regions

• PADD and sub-PADD

• world

Effort and Release Dates

Mark Elbert, Washington, D.C., April

27, 2015 12

• Surprisingly simple to add to an existing API of time-series

– Less than 4 developer weeks in total

• Very few additional records needed (for EIA’s data)

– less than 1 geoset record for every 50 time series

– Less than 1 relation for every 100 time series

• Creates a very close relation between statistical provider and

consumer

• Expected launch date: May 2015

Office of Communications

April 10, 2015 13

Questions?