Oracle Business Intelligence 11g Tuning TimesTen for ... timesten for aggregate...• OBIEE Aggregate Persistence / Summary Advisor has also improved , making better decisions on when

Oracle Business Intelligence 11g

Tuning TimesTen for Aggregate Persistence

Alastair Burgess

May 2014

© Peak Indicators Limited 2

Agenda Tuning TimesTen for Aggregate Persistence

• Overview

• Building the Aggregates

• Test Results #1

• Tuning TimesTen

• Test Results #2

• Conclusion



Overview


Overview Aim of Presentation

• Since the release of TimesTen for Exalytics, many rounds of enhancements have been made to:

• Improve the performance of TimesTen “analytical” queries

• Enhance Oracle BI 11g to generate efficient TimesTen structures and queries

• The aim of this presentation is to:

• Demonstrate the query performance of TimesTen 11.2.2.6.3 on an Exalytics server

• Provide tuning tips that deliver significant gains in TimesTen performance

• Compare performance of TimesTen with an Oracle Database


Overview

Architecture


Overview

Building the Aggregates – Simple !


Overview

Implement Aggregates

nqcmd -d AnalyticsWeb -u weblogic -p welcome1 -s TTagg.sql



Test Results #1


Test Results 1 Demonstration

• We will now demonstrate how the TimesTen aggregates perform using the default settings

• Each test will compare against an Oracle Database running on the same Exalytics server (with Auto DOP – Typically 2 Px threads)

• A fact table with 24M records is used for the tests, each test has slightly different filters so that an increasing number of records are scanned during each test

• Do you think TimesTen will be faster?



Tuning TimesTen


Tuning TimesTen 1) RangeIndexType=0

• TimesTen 11.2.2.5 provides support for a new type of "in-memory B-Tree" style of index, it allows for much greater concurrency (as it does not require latches for reads etc).

• You can activate this new feature by setting the

• following parameter in the DSN attributes

[TT_HOME/info/sys.odbc.ini] :

• RangeIndexType = 0

• IMPORTANT: This parameter takes effect only when you connect to the database for the very first time - so you must apply the parameter when you first create the database (and before anyone connects to the database)

[TT_AGGR_3] Driver=..../lib/libtten.so DataStore=..../tt_aggr_3 LogDir=..../logs ConnectionCharacterSet=AL32UTF8 LogFileSize=1024 LogBufMB=1024 LogBufParallelism=16 Preallocate=0 PermSize=25000 TempSize=25000 MemoryLock=4 CkptFrequency=30 CkptLogVolume=0 CkptRate=20 PrivateCommands=1 RecoveryThreads=40 DatabaseCharacterSet=AL32UTF8 RangeIndexType=0


Tuning TimesTen 2) Use Most Appropriate Data Types

• TimeTen provides specific datatypes that can reduce data volumes when compared to Oracle DB equivalent

• Up to 300% reduction in the ‘Lab’

• Customers have seen reductions from 80GB to 30GB

• OBIEE 11.1.1.7 generated script is better at picking an efficient DT, however improvements can be made, adjust the CT script accordingly :

CREATE TABLE ag_Fact_Sales ( Customer_I00008CBF VARCHAR2(50 CHAR) NOT INLINE , Product_Id00008CC6 VARCHAR2(50 CHAR) NOT INLINE , Month00008CCD VARCHAR2(20 CHAR) NOT INLINE , Z_Orders00008CD1 BINARY_DOUBLE, TT_INTEGER Sales_Qty00008CD2 BINARY_DOUBLE, TT_SMALLINT Sales_Valu00008CD3 BINARY_DOUBLE NUMBER(15,2) )


Tuning TimesTen 3) RAM Policy

• By default TT may not always load your data into memory.

• For BI we are looking to utilize in-memory for performance

• We can force this with use of the ram policy configuration* : [Manual]

• $ ttadmin -ramload AGGREGATES

• We have found that “Always” instead of “Manual” also gives the desired effect

• $ ttadmin -rampolicy Always AGGREGATES

* requires a second command to be run following DB startup


Tuning TimesTen 4) Compression

• TimesTen compression operates as data-duplication

• No point on primary key columns,

• No point on numbers , (Fact Columns)

• OBIEE RPD Requires COMPRESS_COLUMNS feature enabled (on by default):


Tuning TimesTen 4) Compression

• Compression Pointer size defaults to 4 bytes

• This supports many distinct values!

• Where it is known, a smaller pointer size can be more efficient

• 1 byte <= 255 (distinct values)

• 2 Bytes <= 65,535,

• 4 Bytes <= 4,294,967,295

CREATE TABLE ag_Fact_Sales ( Customer_I00008CBF VARCHAR2(50 CHAR) NOT INLINE ,

Product_Id00008CC6 VARCHAR2(50 CHAR) NOT INLINE ,

Month00008CCD VARCHAR2(20 CHAR) NOT INLINE ,

Z_Orders00008CD1 TT_INTEGER,

Sales_Qty00008CD2 TT_SMALLINT, Sales_Valu00008CD3 NUMBER(15,2))

COMPRESS ( Customer_I00008CBF BY DICTIONARY maxvalues = 255, Product_Id00008CC6 BY DICTIONARY maxvalues = 255, Month00008CCD BY DICTIONARY maxvalues = 255 )

OPTIMIZED FOR READ;

OBIEE Automatically selects appropriate columns!


Tuning TimesTen 5) Index Advisor

• Timesten contains an easy to use index advisor

• From ttisql : call ttindexadvicecapturestart(0,31);

• Now provide the SQL to the advisor (obtained from BI Session logs) • WITH

SAWITH0 AS (select sum(T580.Sales_Valu00000060) as c1, sum(T580.Sales_Qty0000005F) as c2, count(T580.Z_Orders00000059) as c3, T550.Quarter0000004F as c4, T550.Year00000051 as c5 from ...

• Follow up with :

• all ttindexadvicecaptureend(0);

• call ttindexadvicecaptureoutput(0);

0 : just the current session 1 : entire database

Optional : advise only, no execution



• Output • < 4, create hash index AG_FACT_SALES_i1 on SALES_AGG.AG_FACT_SALES(MONTH00008CCD); >

• < 1, create hash index AG_FACT_SALES_i4 on SALES_AGG.AG_FACT_SALES(CUSTOMER_I00008CBF,MONTH00008CCD); >

• < 3, create hash index SA_DETAIL00008D07_i2 on SALES_AGG.SA_DETAIL00008D07(CUSTOMER_I00008CBF); >

• < 3, create hash index SA_DETAIL00008D01_i3 on SALES_AGG.SA_DETAIL00008D01(PRODUCT_ID00008CC6); >



• OBIEE (Agg Persistence) will only create RANGE indexes.

• Can be up to 40% slower for equality joins (=)

• TT Index advisor will advise HASH indexes

• It is important to review the DDL from OBIEE and switch to HASH indexes for any columns used in equality joins

• e.g Primary Key (Dimensions) and Foreign Key (Fact tables)

• Simply alter the syntax with additional keyword

• CREATE HASH INDEX



• Note – Be careful creating HASH indexes on empty tables ! • By default TT will size the index based on the current size of the table

• An empty table will therefore give a non-optimal index size

• Best to use the PAGES parameter as follows :

• CREATE HASH INDEX [name] ON [table] ([columns]) PAGES=[MAXROWS/256]

• Demonstration


Tuning TimesTen 6) Analyze Tables

• OBIEE by default just estimates statistics to 5%

• Use this command to gather stats on all tables

• call ttOptUpdateStats('', 1);

‘’ : All tables for current owner Force cached query plans to INVALID


Tuning TimesTen

7) Execute on Connect

• Oracle BI now automatically runs 4 statements to alter query optimisation settings to improve analytical performance:

• Those session hints are standard / mandatory on Exalytics. They are necessary in order for the special TimesTen optimisation options for star-joins to kick-in

• Fast Prepare (?)



Test Results #2


Test Results 2 Demonstration

• Now that we have applied at least 5 different optimisations to the aggregates in TimesTen, let’s see how it now performs compared to the Oracle Database

• For each query, which one do you think will be the fastest???


Test Results 3 Database Parallelism

• Up until now, the Oracle Database has had Parallel Query enabled with 2 parallel threads

• What happens if we increase this to 4 or 8 parallel threads?

• Will it make much difference???



Conclusion


Conclusion Interesting Results?

• Query performance with TimesTen (11.2.2.6.3 ) has greatly improved !

• OBIEE Aggregate Persistence / Summary Advisor has also improved , making better decisions on when to use compression and the choice of TT data types

• Applying 5-6 simple tuning techniques will further significantly improve response times (as shown in graph on next slide)

• Our tests show that TimesTen can compete with and even beat the Oracle Database with Parallel Query enabled (with low DOP)

• Remember Oracle Times Ten is 1/3 licence cost of Oracle EE ($350 v $950)


Helping Your Business Intelligence Journey

Documents

Oracle Business Intelligence 11g Tuning TimesTen for ... timesten for aggregate...• OBIEE Aggregate Persistence / Summary Advisor has also improved , making better decisions on when