1

{Open|Big|Linked} Data Enabling better policy, services and cost efficiency in Government Version 0.5

Pia Waugh Director of Coordination and Gov 2.0

Technology and Procurement Division

Office of the Australian Government CTO

2 2

Benefits to Government in Opening Data

Cuts red tape

• Improves efficiencies in sharing data across government and with public

• Proactive automated publishing rather than manual retrospective approach

Improves Government Operations

• Enables collaboration and consistency across government and with public

• Improves policy analysis, development, implementation and reporting

• Improves service delivery by enabling thematic personalised approach to info including mobile services that leverage cloud hosted data and automated APIs

• Improves data quality through enabling verifiable public contributions

Innovation

• Enables innovation and new opportunities in government, industry and research

• Enables greater capacity for public to contribute meaningfully to public policy

3 3

The Government Data Landscape (latest version online)

4 4

The APS Policy Landscape

Others:

• Publishing Public Sector Information & National Standards Framework

• Open Public Sector Information: From Principles to Practice Report

• Declaration of Open Government

• Gov 2.0 Taskforce Report

• Statement of IP Principles for Government (CC-BY)

• Ahead of the Game

• Digital Transition Policy (Archives) & Accessibility Policy

• Emerging Open Research Policies

• Open Government Partnership (TBD)

5 5

State and Territory Policies

6 6

Policies Components

APS:

• Permissive copyright – CC-BY as the default

• Open by default

• Support reuse and innovation

• More public engagement

• Better use of data for government policy and service development

States/Territories add:

• Procurement – open by design

• Reporting – dashboards

• Departmental strategies

7 7

Agencies pressures

Meeting outcomes and expectations, funding

challenges, efficiency dividends, organisational change, and

achieving ICT delivery targets

Australian Government Information Management Office

Technology advances

Mobile, broadband, cloud computing, virtualisation, big

data analytics and other emerging technologies

Productivity performance

Global economic impacts, increasing global competition,

reduced resource demand, demographic changes,

environmental constraints

Public expectations

Influenced by innovative digital private sector services, broadband availability,

smartphone take-up, social media and blogs

Online interaction Connected service delivery ICT investment framework Skills and capability

APS ICT Strategy 2012-2015

Operational efficiency ($1.8B) Improved agency capability Whole-of-govt approaches Coordinated procurement Benchmarking

Declaration of open govt Online engagement APS use of social media Open public sector info

Using ICT to increase public sector and national productivity by:

Enabling better service delivery

Improving the efficiency of government operations

Supporting open engagement to better inform decisions

APS ICT Strategy

promoting better government through the innovative and strategic use of ICT

addressing today’s challenges…

8 8

Big Data Strategy

Develop better practice guidance

Identify and report on barriers to adoption of big data analytics

Enhance skills and experience in big data analysis – initiation and support of pilot projects

Develop guide to responsible data analytics

Develop guidance to enable agencies to create information asset registers

Monitor technical advances

Data is a national asset

Privacy by design

Data integrity and the transparency of process

Skills, resources and capabilities will be shared

Collaboration with industry and academia

Enhancing open data

Enhanced services

New services and business partnership opportunities

Improved policy development

Protection of privacy

Leveraging Government’s investment in ICT

V I S I O N P R I N C I P L E S A C T I O N S

Australian Government Information Management Office

9 9

Privacy and confidentiality

• Custom API approach that confidentialises on the fly (eg, ABS)

• Deidentification of data – to appropriate level

• Aggregation

• Leveraging existing processes for researcher (unit level) access rather than conflating open data discussions

• Privacy Commissioner as point of reference and support

• Avoiding common identifiers across multiple datasets

10 10

Open by Design

Building proactive publishing into:

• Systems

• Processes

• Procurement

• Planning

• Records management

Leveraging open data through:

• Public APIs

• Analysis tools and datavis

• Internal processes looking for external sources

11 11

Data Portals

12 12

Other Data Projects

• Spatial

• Geosciences

• Research

• Sensor

• Realtime (eg Transport)

• Census/Statistics

• Cultural

• Data about government

• International:

Aid/Extractive Industries

13 13

Loads of Tools Available

• Publishing tools – CKAN, Socrata, bespoke

• Automation – FME, Kettle

• Data visualisation – Tableau, SuperDataHub, SpatialKey

• Analysis – R, domain specialist software, Palantir

• API development

• Application development

• Linked data tools

• Metadata tools

14 14

New and Old Skills Required

1. Publishing

2. Automation

3. Metadata/linked data

4. API development

5. Plumbing

6. Data visualisation

7. Analysis and statistics

8. Policy development

9. Public consultation

10. Online skills

15 15

Some Challenges

• Legislative

• Culture

• Systems

• Reactive vs proactive

• Metadata/semantic context

• Too much data

• Real time vs historic

• Definitions and common references

• Limited skills and over specialisation

16 16

data.gov.au

Free, cloud based, highly scalable platform for hosting government data.

Staged approach

1. Publishing (2013) Improving the functionality and ease of

publishing for agencies with training and

documentation

2. Value realisation (Early 2014) Providing useful front end tools for data.gov.au

including data visualisation and analysis tools

3. Data quality (Late 2014) Looking at ways to provide agencies the ability

to accept iterative data improvements in a

verifiable way

Features

• Federated search making data and data services easier to find

• Manual and automated publishing options

• API access to government data

• Easy to publish, download and interact with data online

• Basic data visualisation capability

17 17

All the pieces are in place,

we need people to put the puzzle together

18 18

{Open|Big|Linked}

Data

Without data, you only have open, big & linked.

What {public|private} data do you need?

Questions?