Data Portals in National Statistics Offices: Case of Developing Countries

Data Portals in National Statistics Offices (NSOs)

- A view on deployment, design and technology -

@rajiv_r_in

Context

Country-led data revolution

NSO processes

Focusing on dissemination

Data portals

Operating environment

Emerging themes

Conclusion

AGENDA

Context

3

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Demand for data

Demand for data

Not new & on the rise

2015: MDG report: In sub-Saharan Africa, where poverty is most severe, 61 per cent of countries have no adequate data to monitor poverty trends.

- The Millennium Development Goals Report 2015

2000: Setting out the Millennium Development Goals (MDGs): Large gaps in data at the forefront.

http://www.un.org/millenniumgoals/2015_MDG_Report/pdf/MDG%202015%20Summary%20web_english.pdf

Demand for data

Not new & on the rise

Aid effectiveness

MDGs

National development plans

Traditional key drivers

Demand for data

Not new & on the rise

Marrakech Action Plan for Statistics (2004)

Monterrey Consensus (2002) [MfDR]

Paris Declaration (2005)

Efforts to meet the demand

African Charter on Statistics & Strategy for Harmonization of Statistics in Africa (2009)

Dakar Declaration on the Development of Statistics (2009)

Busan Action Plan for Statistics (2011)

Traditional key drivers

Demand for data

Not new & on the rise

Efforts to meet the demand

New demand from civil society

National Open Data Initiatives

Open Government Partnership Traditional key drivers

http://ckan.org/

Demand for data

Not new & on the rise

Efforts to meet the demand

New demand from civil society

New sources and new players

Big Data

Private sector Traditional key drivers

https://www.premise.com/

Sustainable Development Goals

Sustainable Development Goals

No excuse not to have data “No one must be left behind”

http://sdgcompass.org/sdgs/

Sustainable Development Goals

No excuse not to have data

Expression

Sustainable Development Goals

Data revolution

http://www.uwtsd.ac.uk/ba-ethical-political-studies/

Sustainable Development Goals

Expression

No excuse not to have data Data revolution

Call for improved availability and accessibility of data

http://www.undatarevolution.org/report/

Country-led data revolution

14

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

National Statistics Offices (NSOs)

National Statistics Offices (NSOs) Any effort to implement the data revolution at the country level will need to address the role of the NSO…

- Rachel Quint, Program Fellow, Global Development and Population Programme, Hewlett Foundation.

http://post2015.org/2015/05/01/toward-an-african-data-revolution/

As stewards of official data, NSOs should be at the heart of each country’s data revolution.

- Shannon Kindornay , adjunct research professor at Carleton University, Canada

http://www.scidev.net/global/data/opinion/flashy-innovation-fuel-data-revolution-post-2015.html

Center stage

National Statistics Offices (NSOs)

Center stage

Fit for the purpose?

http://www.oecd-ilibrary.org/development/a-road-map-for-a-country-led-data-revolution_9789264234703-en

National Statistics Offices (NSOs)

Center stage

Fit for the purpose?

There is too little investment in people and skills.

National statistical offices have only limited powers and status within national statistical systems.

Data are not adequately disseminated and used.

The design and management of statistical processes is not satisfactory.

The potential of Information Technology is not fully harnessed.

Technical and financial aid is not well-aligned with national priorities.

Countries face significant costs in managing aid projects.

The overall coordination of national statistical systems is a concern.

Challenges: cross-country survey and in-depth country studies

National Statistics Offices (NSOs)

Center stage

Fit for the purpose?

Cross-country survey and in-depth country studies

Going forward

Risks in execution without (well) capacitated NSOs.

http://demonstrations.wolfram.com/TheMysticRose/

NSO processes

20

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

GSBPM*

Lot going on

Specify needs

Design

Build

Collect

Process

Analyse

Disseminate

Evaluate

Fee

db

ack

*Generic Statistical Business Process Model

Lot going on

Cross-country survey and in-depth country studies

Specify needs

Design

Build

Collect

Process

Analyse

Disseminate

Evaluate

Study focus: Dissemination

Fee

db

ack

GSBPM

Lot going on

Cross-country survey and in-depth country studies

Study focus: Dissemination High visibility from outside

Signs of the preceding processes

Much technology infusion

GSBPM

Focusing on dissemination

24

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Going digital

Going digital

Centralized

Dig

ital

Distributed

No

n-d

igit

al

Delivery

Form

The shift From print (non-digital and centralised) to digital (digital and distributed)

Going digital

The shift Crowded space at the NSOs

Typical user interfaces NSO website NADA Redatam/IMIS Dataportal (Prognoz)

OpenDataForAfrica (Knoema)

DevInfo CensusInfo CountryStat SDMX Registry SMS/Mobile Social Media

Dat

a p

latf

orm

s

Going digital

The shift Data dissemination is prejudicially becoming ‘IT department’ centric

Influence on NSO structure

Putting pressure to acquire new skills and more staff; traditional roles are changing at the NSOs Typical user interfaces

Data portals

29

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Data portals in NSOs

Data Portals

Data demands

Internet

Mac

hin

e r

ead

abili

ty/D

b d

rive

n

Dat

a e

ntr

y b

y N

SOs

Positioning

Data portals in NSOs

Positioning

Definition

Data portals are NSO specific adaptations of generic but distinct web-based interactive data (and metadata) platforms - directly populated by the NSOs. • Data portals vs data platforms

• Tanzania NSO specific: TNADA; Generic:

NADA

Specificity of data platforms for statistical data and conceptual modeling of statistical data (‘cube’)

Distinct from ckan, dkan and Socarata etc.

• Data portals vs data platforms

• E.g. Tanzania NSO specific: TNADA; Generic: NADA

Data portals in NSOs

Positioning

Definition

Drawing attention

UNECA hand book Link

http://documents.worldbank.org/curated/en/2014/07/20467305/open-data-challenges-opportunities-national-statistical-offices

http://documents.worldbank.org/curated/en/2014/10/20451797/technical-assessment-open-data-platforms-national-statistical-organisations

Operating environment

33

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Insights and challenges

Capacity (skills and numbers of personnel)

Infrastructure (hardware and connectivity) Large numbers of deployments

Role of development partners

Insights and challenges

Viability, feasibility, usability and desirability

Business process alignment

Context oblivious deployments

Large numbers of deployments

Insights and challenges

Multiplicity of data portals in same data types in same NSOs

Data types and platforms

Redundancy

Context oblivious deployments

Large numbers of deployments

Delivery

Micro data

Aggregate data

Geo-spatial data

Geo-spatial data

NADA

IMIS

DevInfo

Prognoz

AGOL*

*ArcGISOnline

Insights and challenges

Search functionality

NSO website (look and feel + navigation)

Terms of use and copyrights

Redundancy

Lack of design integration

Context oblivious deployments

Large numbers of deployments

Insights and challenges

Delayed data wrt print (and other platforms)

Few and selected data

Inconsistent data across platforms and medium

Redundancy

Lack of design integration

Context oblivious deployment

Manual data entry as a result of tech limits

Large numbers of deployments

Insights and challenges

Redundancy

Lack of design integration

Context oblivious deployment

Manual data entry as a result of tech limits

Large numbers of deployments

Ineffectively facilitated consumption

Inadequate mechanisms of data sharing with national open data portals - which are gradually becoming windows of national data.

Distinct reporting to international institutions continues; burden not eased by the data portals

Emerging themes

40

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Themes

Governance Stakeholder's roles range & decision making process and criteria (preventing chaos)

NSO

People

Feasibility (Can we do this?)

https://dschool.stanford.edu/our-point-of-view/

Themes

Alignment

Governance Making ‘business analysis’ part of the deployment approach for IT in general; may lead to process adaptations too!

Processes Systems

Themes

Alignment

Governance Implementation keeping the needs of both the data managers (at the NSOs) and end-users

Users centered design

User experience

https://dschool.stanford.edu/our-point-of-view/

Themes

Alignment

Governance Linking production system to the data portals through automated data-entry & OD sharing

Users centered design

Automation

http://rajivranjan.org/what-will-it-take-to-improve-statistical-data-dissemination-in-the-digital-realm/

Conclusion

45

Context

Country-led

data rev

NSO

Processes

Focusing on

dissemination

Data

Portals

Operating

Environment

Emerging

Themes Conclusion

Transformation roadmap

Building on ‘better’ practices

SDMX -Statistical Data and Metadata eXchange

Examples

UNSD-DfID supported project

Used SDMX standards for data sharing

Used SDMX Registry for automation

http://data.un.org/countryData/

Transformation roadmap

Building on ‘better’ practices

http://rajivranjan.org/disseminating-aggregate-data-and-associated-metadata-using-sdmx/

API - Application Program Interface

Examples

UNICEF using DevInfo API for dataviz

Positive impact on data quality

User enabled interface

http://dashboards.devinfo.org/

Transformation roadmap

Building on ‘better’ practices

DDI-CKAN bridge module

Examples

World Bank initiative

Bridging the Open Data divide

Import data in DDI format directly into a CKAN

http://ckan-ddi.clients.liip.ch/dataset

Transformation roadmap

Building on ‘better’ practices

Transformation roadmap

Building on ‘better’ practices

Conscious of information structure Distinct considerations for different data types

Data workflow at NSOs driven by data types

Modular (but interoperable) solutions are the way forward

Transformation roadmap

Building on ‘better’ practices

Data portals maturity model

Conscious of information structure

Dat

a d

isse

min

atio

n e

ffic

ien

cy

Time

Fragmented

Interoperable

Integrated

User-focused

User-enabled

Automation Transformation

Data portals maturity model

Transformation roadmap

Building on ‘better’ practices

Data portals maturity model

Conscious of information structure A complex system that works, is invariably found

to have evolved from a simple system that

worked

- John Gall

A complex system?

Thank you!