Best practice using Power pivot.

Henk Vlootman, Excel MVP

Thanks to our platinum sponsors :

PASS SQL Saturday – Holland - 20152 |

Thanks to our silver and gold sponsors :

PASS SQL Saturday – Holland - 20153 |

Agenda

Introduction

Need to know about Power pivot

Easy tips

Tips for the Pro’s.

9/28/2015 | Agenda4 |

The flavors of Power pivot

9/28/2015 | Need to know about Power pivot5 |

Flavors and differences

9/28/2015 | Need to know about Power pivot6 |

SQLPower pivot

Excel Power pivot

The Structure of SQL Power pivot

9/28/2015 | Need to know about Power pivot7 |

Different kind of tables in Power pivot

9/28/2015 | Need to know about Power pivot8 |

Power pivot works with tables. Be aware there that there are

different kind of tables. I find the next classification handy:

1. Dim(ension) tables

2. Fact tables

3. Measure tables

4. Variable tables

Use empty tables to create Measure tables

9/28/2015 | Need to know about Power pivot9 |

How to create empty tables?

In Excel:

• Select in a sheet any random blank cell and press Copy

• Switch to Power pivot. Beware that the calculation section should not be

active.

• Choose for Paste and follow the instructions.

In SQL:

• If you have a SQL database, choose Existing connections | Open

• Use a dummy SQL script, give a (unique) friendly name.

• The script could be something like:

Select “” as Column1

Or:

• Make a text file with only the header and import this one.

Relations: that is all about Power pivot

9/28/2015 | Need to know about Power pivot10 |

Relations

seen

through

the eyes

of the

Excel user

Relations

seen

through

the eyes

of the BI

specialist

A little more on compression…

9/28/2015 | Need to know about Power pivot11 |

Power pivot uses two sorts of compression:

Hash Encoding:

Used for tables of unique value (read: Dim tables)

Value Encoding:

Used for numeric value

Mathematical patterns and CPU operations (read: Fact tables)

Fact tables only have columns with the datatype Integer.

Dim tables can have columns with datatype Text for slicing.

Key columns in Dim and Fact tables should always be of the datatype Integer.

Narrow your tables.

9/28/2015 | Easy tips12 |

Excel users are addicted to two things. One of them is data. They look at

data as “I can get it, so it is mine…”.

Bi specialist tends to be better in importing data, but...

Here big gains can be found.

Be aware that extra columns can also drastically affect compression.

Columns can always be added later on when needed.

If possible don’t import columns with unique values in Fact tables.

Shorten your tables.

9/28/2015 | Easy tips13 |

Compression algorithms also works on rows in Columns. The more rows…

Filter out the rows you don’t need by importing the data. You can always

add later on, if necessary.

Narrow your calculations

9/28/2015 | Easy tips14 |

As said: Excel users are addicted to two things. The first one is grabbing

data when ever possible, the second one is calculations. They look at

calculations as “I don’t need it, but I can calculate this, so lets do it…”.

Here also big gains can be experienced here.

Affected measures are always recalculated if you use slicers and timelines.

Be aware that filter actions (by a slicer for instance) can affect multiple

tables and therefore calculations on more then one table..

Excel is calculation-driven

Power pivot is filter-driven

Do not think in calculations, think in filters!

Calculated Columns or Measures?

9/28/2015 | Easy tips15 |

Calculated Columns are recalculate by opening the data connection

once. That means more Calculated Columns takes time at the initial

opening of the Power pivot model. Usually the user of the model can live

with that (he / she gets a cup of coffee).

Measures are recalculate when using filters (read: Slicers and

timelines). That means: more measure more time required to reflect in

the dashboard. Usually the user of the dashboard gets quite annoyed if

he has to wait to see the results of his action.

Be sure to optimize your Measures towards filters. That will fasten your

response time.

Although Calculated Columns are better in performance then Measures,

they still take valuable memory. Can you avoid them, don’t hesitate.

Import of Dim tables

9/28/2015 | Easy tips16 |

Fact tables are based on databases, which can come from different

sources. Dim tables for connecting the two together are therefore often

not available. Excel can be a get tool to produce such tables. Create all

dimension tables in one Excel model, for it is easier to manage them.

Files on a windows OS uses lock files to indicate that a file is in use.

That means if your Excel file is locked, refresh will result in an error. You

can prevent this by exporting the results as text files and then import

those.

Text files don’t have an data type marking in it. So Power pivot decides

on import the data type for every imported column. Unfortunately this

data type can be changed, but sometimes the results are catastrophic.

Use Schema.ini file at the first import to keep control of the data types

within a text file.

Slicers. Fancy but killing the speed

9/28/2015 | Tips for the pro’s17 |

As said slicers increases refresh (and thereby waiting) time.

But slicer settings are also not optimized for performance.

As said slicers increases refresh (and thereby waiting) time.

But slicer settings are not optimized for performance.

Slicer setting Refresh time

Hide items with no data 7,3sec

Visually indicate items with no data 4,8sec

Show items with no data last 5,2sec

Visually indicate and show items last 5,2sec

* Based on a model with 5 tables, 70k rows of data and 47 slicers.

* Many thanks for Ken Puls for this table.

Precision is priceless

9/28/2015 | Tips for the pro’s18 |

Precision comes with a cost, namely memory. Decimals (which includes

the time section of dates) will cost more memory. If you can split them in

integers, you’ll be better off.

Split for instance columns in Euro’s and cents of the data type Integer

and re-aggregate the columns by Measures:

TotalEuro:=SUM([Euro])+DIVIDE(SUM([EuroCents]),100)

Memory (KB) in use

Decimal Integer Difference

Raw DateTimes 44.723,02 -

Rounded and split

Date - 2.302,96

Hour - 1,42

Minute - 295,51

Second - 488,86

Total 44.723,02 3.088,75 41.634,27

• Memory stats based om ~1M rows of random datetimes

spanning 17 years and holding precision to the second.

• Many thanks for Ken Puls for this table.

References

9/28/2015 | Tips for the pro’s19 |

Checklist for Memory Optimizations in PowerPivot and Tabular Modelshttp://www.sqlbi.com/articles/checklist-for-memory-optimizations-in-powerpivot-and-tabular-models/

Memory Considerations about PowerPivot for Excelhttp://sqlblog.com/blogs/marco_russo/archive/2010/01/26/memory-considerations-about-powerpivot-

for-excel.aspx

Creating a Memory Efficient Data Model Using Excel 2013 and Power

Pivothttps://support.office.com/en-us/article/Create-a-memory-efficient-Data-Model-using-Excel-2013-and-

the-Power-

Pivot-add-in-951c73a9-21c4-46ab-9f5e-14a2833b6a70?CorrelationId=3569f6ee-c419-4b59-95ec-

2c726900e016&ui=en-US&rs=en-US&ad=US

How Does Power Pivot Store and Compress Data?http://tinylizard.com/how-does-power-pivot-store-and-compress-data/

What is Eating Up My Memory in Power Pivot?http://tinylizard.com/script-update-what-is-eating-up-my-memory-in-power-pivot/

Please review the event and sessions

http://www.sqlsaturday.com/434/eventeval.aspx

Y

http://www.sqlsaturday.com/434/sessions/sessionevaluation.aspx

PASS SQL Saturday – Holland - 201520 |

Henk Vlootman

Excel MVP

https://nl.linkedin.com/in/vlootman

www.vlootman.nl

You -> awesome

Me -> grateful