Lincoln ddj

A Tinkerer’s Toolbox: Data Driven Journalism

Tony HirstDept of Communication and Systems

The Open UniversityVisiting Senior Research Fellow, University of Lincoln

@psychemedia

blog.ouseful.info

#???

Where I situate myself…

Visualising data helps me make sense of the world around me

Do you know what’s

possible?

Ctrl-F

#ddj

Google Spreadsheets

Data DistributionsIBM Many Eyes

Outliers

Trends and (anti)correlations...

(Benford’s Law)

Natural Views

Explanatory visualizationData visualizations that are used to transmit information or a point of view from the designer to the reader. Explanatory visualizations typically have a specific “story” or information that they are intended to transmit.

Exploratory visualizationData visualizations that are used by the designer for self-informative purposes to discover patterns, trends, or sub-problems in a dataset. Exploratory visualizations typically don’t have an already-known story.

Exploiting Structure

Structure in data - h

ierarchies

Hierarchical data and treemaps - medals

Pivot tables

IBM Many Eyes

Templated data views

Macroscopes

Glanceability

Processing(.js)

Look for Differences

Question Discovery

Data Can Tell a Story

http://www.musik-therapie.at/PederHill/Structure&Plot.htm

Visual Data Summaries

ggplot2

ggplot() + geom_linerange(data = d1,aes(x= car, ymin = ymin,ymax = ymax)) + geom_point(data = d2,aes(x= car, y= value,shape = variable),size = 2) + opts(title="F1 2011 Korea \nRace Summary Chart", axis.text.x=theme_text(angle=-90, hjust=0)) + labs(x = NULL, y = "Position", shape = "")

Writing Diagrams

GraphViz

Data Clean(s)ing

Google Refine

(Inner) Joins & Reconciliation

Google Fusion Tables

Geocoding inside

Google Refine

OpenHeatMap

Geocoding inside

“Data Flow”

“Analog Synth Meeting”, Todd Huffman

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Find the data…

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Get the data as data…

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Transform the data…

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Enrich the data and transform again…

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Display the data…

Wikipedia HTMLGoogleSpreadsheet=importHTML

CSV Yahoo! PipeImport CSV

KMLGoogle MapEmbedded object <embed>

Publish the displayed data…

=importData(“CSV_URL”)

The online CSV filebecomes a spreadsheet

becomes A DATABASE

Finding data…

site:.gov.ukfiletype:xls

underspend

inurl:http://phx.corporate-ir.net/phoenix.zhtml? intitle:presssite:phx.corporate-ir.net

inurl:http://phx.corporate-ir.net/phoenix.zhtml? intitle:presssite:phx.corporate-ir.net

Tapping the Data Burden

Receiving bodyReporting body

Data tap

Data Burdens and FOI

Opening Data Up via FOI

Protection of Freedoms

“Public Data” & Social Media

Mapping

No truth, many truths

Emergent views of structural properties

My “journalism” is tracking down tools and working out recipes that

help datasets tell stories

http://delicious.com/stacks/view/CROBXt

Build lazy…

Electrical Safety 101

We get a lot of stuff from Asia, so it all comes with funny plugs, travelling just adds to the fun.

Left to right top to bottom we have:

Singapore wall socket UKAdapter UK -> NZ/AUDouble adapter NZ/AUMy cell charger NZ/AUAdapter NZ/AU -> everythingAndreas cell charger EuroCamera charger US

tolomea

“Hands Passing Baton at Sporting Event”, tableatny

@psychemedia

blog.ouseful.info