Bulletproofing the investigation JAWS 2014 Jennifer LaFleur, CIR (With artwork from young friends of...

Bulletproofing the investigation

JAWS 2014

Jennifer LaFleur, CIR

(With artwork from young friends of journalism)

Investigative reporting is a powerful tool.

Snapshot of big famous story.

It can make this

Into this

It gives us the power to check assumptions.

Know the extremes

First, we must face a fact.

But don’t be afraid…

Bulletproof your data and documents

Before ever reporting data or building an appDo integrity checks to find the flawsAdd caveats where necessaryDo your own analysis rather than relying on an agency’s analysis

External checks

Understand what the data and documents mean.Know how many records you should have.Check counts and totals against reports.Are all possibilities included?

Internal checksCompare fields to check for red flags• More teachers than students• More money going to vendors

than to contractors• What things just don’t make

sense

Integrity checks for every data set

Check for missing data/documents

Integrity checks for every data set

Check for missing dataCheck for duplicates

Duplicates may not always be obvious

Integrity checks for every data set

Check for missing data, misplaced data or blank fields Check for duplicatesTake margins of error into account if necessary

Median HH Income by Metro Area

Integrity checks for every data set

Check for missing data, misplaced data or blank fields Check for duplicatesTake margins of error into account if necessary (important if you’re using Census data).Check for outliers and extreme ups and downs

Truck accidents by year and agency

Beyond the basics

Keep a notes fileIf you’re working with data or documents – keep a copy of your originalKnow the sourceCheck against summary reports As you begin to write, footnote your documents and data

Texas test score data official results versus district

Duncanville district reported 4th grade writing

Official report for Duncanville 4th grade writing

Courtesy Holly Hacker, The Dallas Morning News

Beyond the basics

Keep a notes fileIf you’re working with data or documents – keep a copy of your originalKnow the sourceCheck against summary reportsUse the right tool

H/T Matt Waite

Disparities in water usage

“Water use highest in poor areas of the city”Mapping and statistical analysis

Beyond the basics

Check with expertsKnow the standardsFind out what others have done Gut check – does it just seem wrong?

Beyond the basics

Check with expertsKnow the standardsFind out what others have done Gut checkGo physically see a record or spot check against documents

If you’re matching or backgrounding people…

A name is not enough. Lots of people have the same name

Get dates of birth and other information to make sure you have the correct person.

Even people with seemingly unique names aren’t so unique

Be transparent: Bounce your findings off targets and tell users/readers/viewers what you did.

From the data source

Get the questionnaire and methodologyBeware of nonscientific methods: Web surveys, man on the streetKnow the sample sizeAccount for margin of error and non-responseRun statistical tests on the data if possible

Erroneous government databases can often be a story themselves

123456789 compared to 123-456-789

What appears to be an error may be the story.

Why data?

Contrasts are in the dataYour most powerful figures are in the dataYou can make connections you might not be able to make otherwise

Why data?

Contrasts are in the dataYour most powerful figures are in the dataYou can make connections you might not be able to make otherwiseTest assumptions

When there is no data, you may have to make your own.

From the data source

Get the questionnaire and methodologyBeware of nonscientific methods: Web surveys, man on the street

@j_la28 jlafleur@cironline.org

A special thanks to: Tracy, Olivia, Iris, Emmeline, Thomas, Silas, Carson, Harper and Erin