The views expressed herein are those of the author and should not be attributed to the IMF, its Executive Board, or its management.Reproductions of this material, or any parts of it, should refer to the IMF Statistics Department as the source.

IMF Statistics Department

Andreas Hake

April 14, 2014

Streamline, Standardize and Automate Statistical Data Processing - Case Study

IMF Statistics Department

2

Business Context In 2009 the international community identified important

data gaps that needed to be addressed by the IMF and other international organizations

A report on these information gaps was prepared by the FSB and IMF staff, and endorsed by the G-20 Finance Ministers and CB Governors in Nov 2009 (G-20 Data Gaps Initiative)

As a result, it is anticipated that the volume of data processed by the IMF Statistics Department will increase by a factor of four over the next five years

To cope with this significant increase, the IMF Statistics Department needs to redesign its business processes and extend the capabilities, scalability, accuracy, reliability and timeliness of strategic business operations

IMF Statistics Department

3

Goal: standardized business processes

Strategy strongly recommends a generalized, flexible and scalable approach that could be reused across statistical products

An exercise of this magnitude will span across multiple years, impacting people, processes and tools with a significant investment, which makes it critical for achieving the desired results

Design principles:• business process change and not IT tools implementation• Based on an enterprise data and metadata model• Reduction of manual steps through automation and

standardization• Preparation of IT tools

IMF Statistics Department

4

Organizational Specialization and Operational Independence

Collection

Production Dissemination

Standards, Processes and Technology

Inte

rface

Inte

rface

IMF Statistics Department

Process Automation and Resource allocation (To-be)

Collection Production Dissemi-nation

Standards, Processes and Technology

Inte

rfac

e

Inte

rfac

e

IMF Statistics Department

6

Standard Production Process Template

IMF Statistics Department

7

Goal: support increased demands and improve timeliness of data delivery To validate the approach for streamlining, it was

recommended to implement the new processes on a pilot dataset.

Two possible options for the pilot• Parallel run to compare and validate an existing dataset; or • New dataset.

First approach would be safer but could result in possible delays due to conflicting priorities, while the later approach poses high risks by relying solely on new processes and tools.

IMF Statistics Department

8

Pilot selection

The Coordinated Portfolio Investment Survey (CPIS) dataset required a major change due to the expanded coverage, which almost doubled its size

The change was impacting all full data process including collection, processing and dissemination

Analysis confirmed that the existing tools and processes would not be able to deliver the desired results in the expected timeframe

Hence the expanded CPIS was taken up as the pilot implementation for the new streamlining exercise

IMF Statistics Department

9

Pilot implementation

Support for increased data demand (data coverage expanded to approx. 34,000 series per country from 17,000)• Reduce the size of report collection forms• Just in time processing• Readiness to disseminate data real-time

Automation and easy data validation: The new implementation has eliminated most of the manual steps by implementation of automated workflows

Transparent workflow through data workflow dashboards Performance improvements and access to business user

tools

IMF Statistics Department

10

Pilot results - Collection

Before AfterTime series Collected 17,000 34,000File Size 10-20 Mb 1-2MbICS Processing Time > 1 hour < 5 minutesCorrespondent Download speeds

2x faster

Download required before Upload

Yes No

Correspondent-side errors

Some No errors to date

File Formats supported

Report Form only Report Form, CSVFuture: ODP, SDMX

IMF Statistics Department

11

Pilot results - ProcessingProcessing Before AfterSubmission Processing

Manual Automated

Data Transformations Overnight Automatic upon receipt of submission

Submission Review Manual Semi-AutomatedOn-demand Management reports

No Yes

Validation Before AfterValidation Reports Static, Excel-based Dynamic, IntegratedOther data sources No IntegratedMetadata No IntegratedCharts No YesDiagnostic Summary No Yes

IMF Statistics Department

12

Submission status reports

IMF Statistics Department

13

Work in Production - ValidationCharts

Detailed Diagnostics

Cross-Database Comparisons

Diagnostic Summary

OLAP AnalyticsMetadata Integration

IMF Statistics Department

14

Pilot results – Dissemination

Before AfterPublication Review End of cycle In cycleData Publication Manual AutomatedEarly access for Fund Staff

No Available upon departmental approval

Future: Need to Re-review Outputs in EDD Staging

Yes No

Future: Portals No YesFuture:Report/Portal Creation

Double work One time creation

IMF Statistics Department

15

Success indicators

Business ownership at end user level Strategic buy-in from Senior management Allocation of budgets for capital investment

Sentiments in the business community

“Can we be next?”

IMF Statistics Department

16

Critical areas identified

The successful pilot implementation demonstrated the benefits of standardization and automation

Key areas to be addressed for full implementation:• Organizational structure• People change management• Outreach and communication• Governance• Plan overall at high level, but detailed for next six months• Share success and celebrate

IMF Statistics Department

17

Next Steps

Establish steering group to oversee and govern the change process

Detail the overall high-level plan for the coming months Adjust the organizational structure Communicate, communicate, communicate…

IMF Statistics Department

18

Thank You

Andreas Hakeahake@imf.org+1 (202) 623 8130