Hacking dashboards: Tips and Tricks for Data Mining in Excel

Hacking Dashboards

Tips and tricks for Data Mining in Excel

Hacking Dashboards

Tips and tricks for Data Mining in Excel

david@rubiconproject.comsummer 2008

Today’s Lesson

Today’s Lesson: Unpivoting

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with

data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with

data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted,

or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of

someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality,

analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

Synopsis

Whereas PivotTables and Analysis Cubes are two of the most powerful tools for working with data, their use is predicated on the possession of clean, columnar, and normalized data. What if your data is bound to a homegrown spreadsheet ‘ecosystem’ that cannot be pivoted, unpivoted, or repivoted? What if you’d like to harvest usable data from a large, heterogeneous volume of someone else's spreadsheets? Well, unpivoting is a fiercely practical way to recover high quality, analyzable data from the vagaries of preformatted spreadsheets. Unpivoting is a black art, and

definitely not a science. This presentation contains step-by-step instructions on how to unpivot data using Excel.

End of the Introduction

The Problem

The Problem

"Many companies depend on homegrown, spreadsheet-based applications to support complex business processes such as financial planning, budgeting and reporting, despite the fact that they are largely unsuitable for such purposes.”

The Problem

"Many companies depend on homegrown, spreadsheet-based applications to support complex business processes such as financial planning, budgeting and reporting, despite the fact that they are largely unsuitable for such purposes.”

Paul HamermanVice President, Forrester ResearchJanuary 31, 2003

The Solution

The Solution: None

The Solution: None

Appears to be mostly psychological.

The Solution: None

However –

The Solution:

It is possible to recover high-quality, business grade data from the vagaries of preformatted spreadsheets.

The Solution:

It is possible to recover high-quality, business grade data from the vagaries of preformatted spreadsheets.

Doing so takes a judicious use of unpivoting and the text-to-columns feature of Excel.

The Solution:

It is possible to recover high-quality, business grade data from the vagaries of preformatted spreadsheets.

Doing so takes a judicious use of unpivoting and the text-to-columns feature of Excel.

Even more radical results can be achieved by using a sufficiently advanced programming language, such as Perl.

An Example

An Example

An Example

An Example

An Example

From the Wild

From the Wild

From the Wild

From the Wild

From the Wild

From the Wild

How do you unpivot homebrew data?

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Steps

The Result

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

Intermediate Techniques

The Result

Iterations

Iterations

Iterations

Iterations

Iterations

Iterations

Iterations

Design Patterns

Design Patterns

There are 5:

Design Patterns

There are 5:

Rotating

Design Patterns

There are 5:

RotatingFlattening

Design Patterns

There are 5:

RotatingFlatteningSplitting

Design Patterns

There are 5:

RotatingFlatteningSplittingJoining

Design Patterns

There are 5:

RotatingFlatteningSplittingJoiningFiltering

Design Patterns

There are 5:

RotatingFlatteningSplittingJoiningFiltering

Any data can be made equidimensional, normalized, and columned.

Applications

Applications: Data Harvesting

Next time

Next time: Analysis Cubes

and Web Spiders

Thank you