Gimme back my data! government transformation by data visualization

Personal Democracy Forum 2008June 24, 2008

W. David StephensonStephenson Strategies

1

Remember the end of “Raiders of the Lost Ark,” when the Ark of the Covenant was moved to a government warehouse? You knew it would never be seen again. That’s what seems to happen with a lot of government data. We pay taxes to collect them. Our activities and lives are their raw material. They determine

whether many of us get more government benefits and which states and communities get grants. But once they’re collected, most citizens -- and a lot of government employees for that matter -- don’t have a clue where government data are stored or how they’re used. Even worse, that robs us of important tools that could improve government’s performance and cut its operating costs.

2

Fast forward to 2008. Lo and behold, in the latest Indiana Jones sequel, Indy retrieves the Ark! In my book, that’s an omen that you can’t keep things hidden forever! Similarly, closely-controlled and long-lost government data are being liberated by the growing demand for transparency by watchdog groups, the media --

and us.

3

Beyond shedding light on how government operates, far-reaching and unprecedented change can result when we make reams of data available, plus tools to

portray them visually.Generally acknowledged as the leading thinker on data graphics, Edward Tufte says that even the most skilled statisticians often find representing data

visually is the most insightful way of making sense of them:"Modern data graphics can do much more than simply substitute for small statistical tables. At their best, graphics are instruments

for reasoning about quantitative information. Often the most effective way to describe, explore and summarize a set of numbers -- even a

very large set -- is to look at pictures of those numbers. Furthermore, of all methods for analyzing and communicating statistical information, well-designed data graphics are usually the simplest and at the same time the most powerful.”

This example is a Google mashup Jon Udell whipped up quickly to highlight pothole complaints to the DC Department of Public Works, and track -- on a real-time basis (because the city releases that data automatically) -- the repairs’ status.

Sure, you might find that information in a chart, but who’d sift through pages of records in hopes of possibly finding the one or two that applied to their

neighborhood? By contrast, if you saw this map, and lived near one of the pointers, wouldn’t curiosity compel you to click on it? Wouldn’t the fact that it includes not only information about where the pothole is and when the complaint was made, but also the repair status TODAY, both fascinate you -- and provoke you to call the DPW if it’s now 3 months later and the map shows the repair still hasn’t been made?

Thus, a simple map can be the impetus for citizen awareness – and greater agency accountability. Incidentally, this example also illustrates an important aspect of data visualizations: while many are done by organizations, many are done by individuals

with a passion for a specific issue, such as..

4

… Rami Tabello’s illegalsigns.ca, documenting illegal billboards in Toronto ….

5

…. Adrian Holvotny’s ChicagoCrime ….

6

…. and Jacqueline DuPree’s documentation of neighborhood issues in Southeast D.C.

7

Some visualizations combine various data bases to illustrate convergence, contrasts or possible causality.

This example is Neighborhood Knowledge Los Angeles, a collaboration between UCLA and community activists. Their motto: “neighborhood improvement

and recovery is not just for the experts.” This is an great example of data visualization’s impact, because it combines and maps data on 7 “problem indicators” (including code violations, property tax delinquencies, and fire records, etc.) that might have otherwise remained isolated within city government. When you see so many danger signs are repeated on a map of a single block, that’s a red flag to city officials to intervene NOW with coordinated services to halt the decline.

8

Still other data graphics give context to global issues, which can seem so massive and complex that many of us shy away from trying to understand, let alone

to influence them. None are as eye-catching, and informative as the visualizations on issues facing developing nations created by the Gapminder Foundation using its

innovative, animated Trendalyzer software. Goggle now offers Trendalyzer for general use under the Motion Chart name, so perhaps they will become commonplace.

This static screengrab can't do justice to the powerful additional understanding gained when you view one of Gapminder’s animated trend diagrams!

“… put together big enough and diverse

enough groups of people & ask them to

make decisions affecting [the] general

interest, [and] that group's decisions will,

over time, be intellectually superior to the

isolated individual, no matter how smart or

well-informed he is.

”-- The Wisdom of Crowds

9

Equally important, web-based data visualization sites often include a variety of community-building Web 2.0 such as topic hubs, tags, and discussion areas.

They make it easy to focus many individuals’ and groups’ attention on a policy issue, increasing the chance that new insights will emerge precisely because of the interplay of so many perspectives.

As James Surowiecki wrote in “The Wisdom of Crowds,” “… put together big enough and diverse enough groups of people & ask them to make decisions affecting matters of general interest, [and] that group's decisions will, over time, be intellectually superior to the isolated individual, no matter how smart or well-informed he is."

TextTextText

1st: release the dataText

10

Successful governmental data visualization projects include two components. The first is to release data -- plenty of it -- in easy-to use, easy-to-find, ways.

Sure, some motivated, technologically- sophisticated individuals can create informative data visualizations the hard way, by “scrapping” data from governmental web sites.

However, now that it is so simple to create data feeds such as RSS that are generated automatically as new data are added, there’s little rationale not to do so. In fact, Princeton researchers recently released a paper making a startling assertion: “Rather than struggling, as it currently does, to design sites that meet each end-user’s need, we argue that the executive branch should focus on creating a

simple, reliable and publicly accessible infrastructure that exposes the underlying data”

11

Several federal and state agencies now publish a variety of data feeds.

The most exciting model is the District of Columbia’s Citywide Data Warehouse. It provides real-time numerical and geospatial feeds, drawn from more than 150 data sets, ranging from crime reports to to building permits to those pothole complaints.

2nd: visualize data

12

The second major component of a public data project is to help people find simple-to-use ways to portray the data visually. A growing range of new Web 2.0-based visualization tools are readily available. Several of the commercial sites now offer secure versions making it simple for agencies to also add internal visualization sites.

The creators of IBM’s Many Eyes say:“Our goal is to ‘democratize’ visualization and to enable a new social kind of data analysis …. All of us ... are passionate about the potential of data

visualization to spark insight. It is that magical moment we live for: an unwieldy, unyielding data set is transformed into an image on the screen, and suddenly the user can perceive an unexpected pattern.

“As visualization designers we have witnessed and experienced many of those wondrous sparks. But in recent years, we have become acutely aware that

the visualizations and the sparks they generate, take on new value in a social setting. Visualization is a catalyst for discussion and collective insight about data .... When we share it and discuss it, we understand it in new ways.”

This particular visualization was the first one that I personally created, to help understand patterns in DHS's disbursement of funds for one of its programs. The simple-to-understand directions allowed me to upload the data and create the visualization in a matter of minutes.

13Swivel operates a very

similar site.

Transparency begins at home

14

Here’s a great way for government agencies to ease into public data feeds and data visualization: follow the District of Columbia's lead, and apply the same strategy behind the firewall first.

After all, agencies’ employees may be struggling with incompatible data bases, may need to reach across agency “silos” to see if there might be synergies

between programs, and employees from another agency may be able to provide new insights simply because of their differing life experiences and expertise. Also, as more young workers, who have never known life without the Web, join governmental workforces, they’ll naturally ask why tools they’ve used can’t be

used in government. A data graphics project can empower them and tap their expertise.

Let 1,000 mashups bloom!

15

Once an agency has done this behind-the-scenes work and realized value from an internal data visualization program, the prospect of a parallel set of public

data feeds and a data visualization site is less worrisome.As the public becomes more at ease with readily-available data visualization tools and sees the benefits of ad hoc projects such as illegalsigns.ca, they will do

more and more data visualizations whether or not agencies facilitate them. If that’s the case, why shouldn’t government also reap the benefits of growing public understanding and insight?

The payoff: transformation!

16

The potential benefits are many, and varied:

• more informed policy debate, grounded in fact, rather than rhetoric

• consensus building

• better legislation

• greater transparency and less corruption: greater accountability

• optimizing program efficiency and reducing costs:

• new perspectives, especially when “the wisdom of crowds” emerges.

Who would have believed that dry data -- with a healthy doses of Web 2.0 magic -- could become the engine to involve the public in governmental transformation!

Stephenson Strategies335 Main Street, Medfield, MA 02052

(617) 314-7858

17

To learn more about transparent government and how to create the processes and policies to make it a reality, contact:

Stephenson Strategies 335 Main Street, Medfield, MA 02052 (617) 314-7858 D.Stephenson@stephensonstrategies.com