Download pdf - everything you need to know about essbase load rules

Who are you to tell me about rules!? oDoug Pearce

oGeorge Cooper

Format – Lightning fast at first then slower as we go

Agenda oUse Cases Overview

oGetting Started

oUse Case Demonstrations

Questions and maybe answers

EVERYTHING YOU NEED TO KNOW ABOUT ESSBASE LOAD RULES

Essbase Load Rules

Doug Pearce

• Principal Consultant, Analysis Team Inc.

• 20+ years experience with Essbase

• Design/Develop for flexibility

• Automation Fanatic

• Four years as a Dodeca developer

George Cooper

• Essbase Wizard of Highest Order

• Pool Shark

• Astrophysicist

• Walks faster than anybody I know

About Analysis Team, Inc.

Full-cycle solution implementation o Analytics / Business Intelligence

o Financial & Management Reporting, Data Visualization

o Enterprise Planning

Oracle Gold Partner o Essbase

oOracle EPM Suite / Hyperion

oOn-prem and cloud

Applied OLAP Partner (Dodeca) o Dodeca SMS, Excel Add-In for Essbase, Drillbridge

www.AnalysisTeam.com

Preface • Dimensions can be built (and rebuilt) entirely using load rules • Dimensions can be built using a combination of manual outline edits and load rules

Disclaimer • Some Rules features we will show you are not applicable for all situations • Other Rules features are almost never applicable

Commitments We will make every effort in the next hour to show you: • What the features of Rules do • What works well • What to watch out for

Essbase Load Rules

Use Cases Overview

Essbase Rules apply tabular data (rows/columns) to: oUpdate Essbase Outlines Build Methods

– Generation (Balanced hierarchies - top down)

– Level (Bottoms up – unbalanced)

– Parent – Child (Most versatile and also great for shared member roll-ups)

– Others …

o Load data values to Essbase

– Single value per row or Multiple values per row

– Clearing data values

Essbase Load Rules

Getting Started

oNew Rule file

oOpen Data Source • File vs SQL

oView Field selection

oData Source Properties • Delimiters, Header options and lines to skip

oGeneration build – Products • Create rule

• Apply rule to update outline’s dimension(s)

• Review/recover from errors

Essbase Load Rules

• Create a new Rule

• Open data file for Products

• Data Source Properties o Delimiter and Header

• Set Field Properties o Dimension Build Properties

o Global Properties

• Test

• Improve

Generation Dimension Build

Essbase Load Rules – Lightning Round

Login to Essbase Administration Services (EAS)

File->New->Scripts->Rules File – 0pens the Data Prep Editor

Getting Started – New Rule

Open Data File

o Text file from a folder or Essbase Server/Application/Database

o EAS supports .txt and .xls extensions only

o Txt files can be numerous standard formats (e.g. CSV)

o Excel files must be in the older format and can only load the first tab. Rarely used anymore, txt files are safer.

Essbase Load Rules

File System – Select local or network folder Essbase Server - Select Server, Application & Database

Generation Dimension Build Open a Data File

Essbase Load Rules

Open SQL Data Source

Choose a data source

o SQL Data Sources – ODBC DSN set up on Essbase server

o Substitution Variables – with the name of an ODBC DSN

o OCI – very fast if you are querying an Oracle relational Instance

Fill in Connect properties for your specified Data Source

Fill in the SQL statement Fields • Select the names of the columns/expressions available in

the From (tables/views)

• Optionally add the Where clause to limit the rows returned

• Use Substitution Variables in the query to limit manual maintenance

Essbase Load Rules

View Field selection

Choose Dimension Build fields so the data prep editor shows your settings and validates appropriately

Selecting Data Load fields does the same if you are making a rule to load data to Essbase

A single rule can be used for both dimension builds and data loads.

You can also stick your thumb in your eye, but do you really want to?

Essbase Load Rules

Rules (Data Prep Editor) Toolbar


Regions of Data Prep Editor

• Toolbar

• Raw Data – use view menu to hide

• Resulting columns



Generation Build requires columns are ordered left to right with highest ancestor in the left-most position.


Data Source Properties • Delimiter - file uses Tab (Default) • Header – Skip one line to ignore header row.

1



Field Properties – Global

• Case options – nice but Essbase isn’t case sensitive by default

• Prefix/Suffix – more later

• Drop leading/trailing spaces – nice default since members and aliases can have neither

• Convert spaces to underscores – not sure we’ve ever needed this one

• Replace – here we’re changing Accessory to Accessories




Dimension Build Properties

Generation 1 is the dimension member




Validate the rule file and save

For The Record:

The validation will point out certain errors but does not ensure, in anyway, you will get the results you expected.

• Same Save as dialog – file or sever • Stick to the eight character limit



Open the outline in edit mode Menu->Update Outline

Don’t like what you see – Don’t Save

Select rule and data file Remember the overwrite check box


These steps apply to BSO databases (not ASO)




Using Field Edits to clean up the alias. Let’s combine the product number and description to keep aliases unique.



• Select the Product Number column

• Choose Create using join … • Ok

• Select the new column (Field2)

• Choose Move … Click down until it is at the bottom

• OK

• Choose Create using text… • Type “ – “ • OK



• Select the three columns • Choose Join … • Ok

• Use Data Source Properties • To view the Field Edits



• Change Dimension Build Settings

• Double click on Product

• Check Allow property changes – probably a good habit for any rule

• (and allow moves while we’re here)



• Style and bullet proofing options:

• Putting the identifier before the alias is often preferable

• Essbase has an 80 character limit for names and aliases. In this case, use Split field to get the first 72 characters of the description before joining to avoid errors

o 72 = 80 – 8

o 8 is for 5 digit code and 3 for the “ – “



All Locations - Level Dimension Build

• Header record for dimension building settings

• Level Dimension Build Settings • Left to Right – Bottom’s Up • Options for handling null/empty values • Member sorting options – case study

o Member Sorting vs. Remove unspecified

Alternate Location Roll-ups - Parent/Child Dimension Build

• Shared members for alternate rollups • Setting member properties (Consolidation)

Location Dimension

Essbase Load Rules

Header record for dimension building settings

Level Dimension Build

Essbase Load Rules

Double click Location Dimension first!!!

• Use level references

• Allow property changes – probably always a good idea

• Process null values

o Interesting but doesn’t work well in cases with null property columns

o Alternatives exist such as shifting the text file fields for level builds

• Save rule as Loc_1

Level Dimension Build

Essbase Load Rules

• Set the Field’s Dim Build settings to Parent, Child and Property

• Set field 4 to Ignore

Parent Child Build

Essbase Load Rules

Set Dimension Build Settings:

• Double click Location

• Check Allow property changes

• Use parent/child references

• Save rule as Loc_2

• Allow property changes is probably a good idea in 99%+ of all use cases

Parent Child Build

Essbase Load Rules

Using MaxL to make testing easier

Does not allow you to review changes before saving outline

Compound Dim Build

Essbase Load Rules

• All Locations has the unbalanced hierarchy from the Level Build

• Alternate Location Roll-ups is from our parent/child build. Shared Members are the nice default for parent/child

• Updating existing alternate hierarchies with load rules is risky if not dangerous – see us at the bar for scary tales

Compound Dimension Build

Essbase Load Rules

BSO Member Property Codes

Code Description

% Express as a percentage of the current total in a consolidation

* Multiply by the current total in a consolidation

+ Add to the current total in a consolidation

- Subtract from the current total in a consolidation

/ Divide by the current total in a consolidation

~ Exclude from the consolidation

^ Exclude from all consolidations in all dimensions

A Treat as an average time balance item (applies to accounts dimensions

only)

B Exclude data values of zero or #MISSING in the time balance (applies to

accounts dimensions only)

E Treat as an expense item (applies to accounts dimensions only)

F Treat as a first time balance item (applies to accounts dimensions only)

L Treat as a last time balance item (applies to accounts dimensions only)

M Exclude data values of #MISSING from the time balance (applies to

accounts dimensions only)

N Never allow data sharing

O Tag as label only (store no data)

S Set member as stored member (non-Dynamic Calc and not label only)

T Require a two-pass calculation (applies to accounts dimensions only)

V Create as Dynamic Calc and Store

X Create as Dynamic Calc

Z Exclude data values of zero from the time balance (applies to accounts

dimensions only)

Similar properties are available for ASO, Hybrid cubes and the new Smart View Cube Designer

Member Properties

Essbase Load Rules

Dimension Build Settings

Member Sorting • Default is None – probably your best

choice

• Ascending/Descending only considers member names

Sorting using the text and build order (and Remove Unspecified)

Let’s sort things out

Sorting Outline Members

Essbase Load Rules

Member order matters!

Can affect calculations • Correctness And performance

Can affect user experience • Might want alias sorts or even more

concise control over order

Can result in Outline validation Failures

Let’s sort things out

• Tried and proven solution patterns are available and will be presented shortly

Unfortunately, the Rules option “Member sorting” is probably as useless as a screen door on a submarine

Good News


Essbase Load Rules

Why sort member names option is not your friend

• Primary and alternate rollups are reordered :(

• Before member sorting


Essbase Load Rules

• Carlsbad is new but doesn’t come out in the correct order (according to Alicia)

Adding Additional Members

Essbase Load Rules

Start Remove unspecified – will remove all hierarchies in the dimension

• Requires master data for complete dimension build

• No manual outline maintenance allowed

• Decided to ignore field 2

Member Order from data record order and rule order

Essbase Load Rules

• Ignore Tokens will ignore rows that match

• Using Comma delimiter

• Quotes protect against in-line commas

• Rules don’t handle quotes within fields for text files

Data Source Properties

Essbase Load Rules

• Now Alicia is happy – the members follow the order of her location master.


Essbase Load Rules

• Change the file - #17 is paid off now

• Check Allow UDA changes will prevent this but may be a problem if you have other UDAS for those members

Adding UDA(s)

Essbase Load Rules

• Attribute Dimensions can take the place of both shared member roll-ups and UDA’s

• Can only be used for sparse dimensions • Use Print Preview to see a

Rule’s settings

Attribute Dimensions

Essbase Load Rules

Alias Table • An outline can have multiple alias

tables but a rule can only update one.

Data Configuration • Don’t use autoconfigure for

production

Arrange dimensions by size and type • Just as dangerous as Autoconfigure

Global select/reject • Need to understand field level

options before this make sense

More Dimension Build Settings

Essbase Load Rules

Dimensions from

• Default is “from Outline” – use it !

• Rules file o Dynamically adds a dimension to your cube. o Extremely unusual use-case with no other

significant value.


Essbase Load Rules

• Clearing member properties


Essbase Load Rules

• Don’t select a field type or you’ll have to start over.

Add as a Child of…New Members from Data Load

Essbase Load Rules

• Really useful when re-loading data that rejected – this is a case where you can use both data loading and dimension builds in the same rule

Essbase Outline - Alias Tables

Uses:

• Different languages

• Transition users to new aliases gently by using another alias table

• Mapping for data loading (1:1) o Load historical data with “retired”

codes (e.g. new chart of accounts)

o Source data with different “members”

Essbase Load rules - Alias Tables

There is no magic without rules

• An Alias can never be used by more than one member – Across ALL Alias tables

• Queries using an alias table will return the member name if there is no alias in that table. The more user-facing alias tables, the more maintenance you’ll have.

• A Rules file can only update one alias table

Essbase Alias Tables Export/Import

Two basic formats 1. One dimension across the columns 2. All dimensions on row’s and one *Data* column

• Note also the option to scale values (input is in thousands)

• Data values must be on the right • Sorting records on sparse dimensions will optimize

BSO data loads.

• Location loads to Alias, not ideal – but it works

Essbase Load Rules – Loading Data


• Flip Sign on UDA

• Revenue is negative – I GET IT! Now show me my Sales


• We don’t want to get into all of this now. You may never want to.

• Used sparingly and simply, Select/Reject works well.

• The more ways you use this feature, the more ways you’ll need to create test cases

Essbase Load Rules – Select / Reject

• Select /Reject using positive and/or negative criteria; Reject wins

• Field settings have two And/Or options – getting scared yet?

• Global And/Or setting

• Best to keep these to a bare minimum – more means more testing

• Alternatives include filtering the data source before it gets to the load rule or making multiple (special purpose rules).

Essbase Load Rules – Select / Reject

Reads header records specified in the data source. Then Essbase performs the following operations on each record of the data source during a data load or dimension build: Sets the file delimiters for all records. Applies field operations to the data in the order in which the operations are defined in the rules file. Field operations include joins, moves, splits, and creating fields using text and joins. Essbase applies all properties for each field, applying all properties to field1 before proceeding to field2. Essbase applies field properties in the following order:

o Ignores fields set to be ignored o Flags the data field o Applies field names o Applies field generations o Performs all replaces in the order in which they are defined in the rules file o Drops leading and trailing spaces, then Converts spaces to underscores o Applies suffix and prefix operations o Scales data values o Converts text to lowercase then Converts text to uppercase

Adds members, or member information, or both, to the outline If you chose to skip lines, Essbase skips the number of lines that you specified; otherwise, Essbase proceeds to the first record. Essbase performs selection or rejection criteria in the order in which the criteria are defined in the rules file. Essbase loads or rejects individual records of the data source based on the specified criteria.

Understanding How Essbase Processes a Rules File

Essbase Data Load Rules – Substitution Variables

• Reduce/eliminate periodic maintenance of data load rules

• Also uses a cross dimension in the column header

Essbase Load Rules – Clearing Data Values

Rules – native Data Clear is almost never useful

• Preferred: Consider a Pre-clear step before loading data. E.G. calculation script

• If a previous data file exists reliably, you can make a column from text = #Missing and load overwriting values. This makes your process more sensitive to hiccups.

• Warning, Subtracting from a previous load can result in tiny zeros 3.17E-17

• Two use cases for data records with null values in a particular field 1. Reject or skip the record

2. Assert a default value such as No Product or #Missing

• Create a field using text using highly unlikely value for the field - Zzyzx

• Join new field with one with nulls Record->Reject equal to the unlikely value

Field->Properties (order of replace steps matter) 1. Replace (match whole word) unlikely value with reject criteria or desired default

2. New Replace Criteria

3. Replace unlikely value (not whole word) with null nothing nada blank

Data Loads with Empty Columns

Essbase Load Rules

• Default is to show the first 50 records

• If the fifty first record has extra columns, the data prep editor won’t show them unless you change the View count or Start record (and data loads will have errors)

• Resist editing a rule without opening an appropriate data source.

Pitfalls and Possibilities

Essbase Load Rules – Record View Count

MaxL: Use load buffers – even if just loading one file Provides control over buffer usage, data values and slice management

Essbase Load Rules – ASO

Q: What was that thing they said about … A: find us at the conference or contact us [email protected] [email protected]

Essbase Load Rules – Questions & Answers