“This presentation is for informational purposes only and may not be ... - Oracle · “This...

“This presentation is for informational purposes only and may not be incorporated into a contract or agreement.”

<Insert Picture Here>

Partitioning and Purging Best Practices for the E-Business SuiteAhmed Alomari & Mohsin SameenApplications Performance Group

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be

incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in

making purchasing decision. The development, release, and timing of any features or functionality described for Oracle’s products

remains at the sole discretion of Oracle.

Agenda

• Partitioning Best Practices• Partitioning Concepts• Supportability• Partitioning Methods• Customer Case Studies

• Purging• Business Processes• Purging Strategies• New Purge Portal

• Q & A

<Insert Picture Here>

Partitioning in the E-Business Suite

Partitioning Continues to Evolve

Database Release Partitioning Functionality

Oracle 8Range partitioned tablesLocal partitioned indexesGlobal range partitioned indexesHash and Composite; Range-hash partitioned tables, Range partitioned index-organized tablesList partitioned tables, Hash partitioned index-organized tablesComposite range-list partitioned tablesList partitioned index-organized tables

Global hash partitioned indexes

Oracle 8i

Oracle 9i

Oracle 9iR2

Oracle 10g

Partitioning Concepts

• Data Partitioning: • Process of logically and/or physically segmenting data (and

it’s associated storage, tables and indexes) into more manageable pieces, in order to improve manageability and accessibility.

• Example• Divide large Applications tables and their indexes such as

GL_BALANCES into smaller segments.

Partitioning in the eBusiness Suite

• Oracle Applications utilizes partitioning in the standard product in many modules:• Advanced Planning and Scheduling• Payables (Trial Balances)• Projects Resources• Workflow

• Directory Services• Runtime tables

• Daily Business Intelligence• HR (Employee Directory)• Engineering

Support Policy: Custom Partitioning

• Oracle Applications fully supports the use of custom partitioning of either Applications standard or custom tables.

• Definition of custom partitioning: Changing the partitioning definition of an existing Applications table as delivered (out-of-the-box).

• If custom partitioning is used, you must license the partitioning option.

• Refer to the write-up on the Technology Stack blog: http://blogs.oracle.com/schan/2006/09/06#a663

<Insert Picture Here>

Why Use Partitioning?

Partitioning Advantages

• Table availability• Significantly reduce recovery times of key transaction tables by

recovering the current partitions first.

• Table manageability• Backup, restore, and rebuild at the partition level.• Index rebuilds can be performed at the partition level.• Partition aware operations such as MOVE, EXCHANGE,

REBUILD can be used without affecting active partitions.

Partitioning Advantages

• Performance• Improves access path of most queries since the majority of the

access involves current data as opposed to historical data. • Optimizer automatically prunes unnecessary partitions.

• Analytical reports or period close jobs/reports improve by scanning the current partition as opposed to all the partitions.

• Improves purge performance.• Significantly improves upgrade performance

• Minimizes upgrade downtime.

Partitioning Advantages

• CBO is partition aware and employs partition pruning at runtime.• Eliminates partitions which are not needed by the SQL

statement.• Joins are also partition aware and benefit from pruning.

• Parallel operations can occur at the partition level.• Utilizes the Information Lifecycle Management (ILM)

infrastructure to allow partitions containing historical Applications data to be placed on lower cost storage devices.

<Insert Picture Here>

Partitioning Methods

Rangepartitioning

Hashpartitioning

Compositepartitioning

Listpartitioning

• The following partitioning methods are available:

Partitioning Methods

Partitioning Methods - Range

• Range partitioning requires that rows identified by a "partition key" fall into a predefined range of values.

• Value of the partition key determines the partition to which a row belongs.

• Range partitioning was introduced in Oracle8.• Range partitioning is useful for Applications tables for

which the partition key matches the access path key.

Partitioning Methods - Range

GL_BALANCES476,720,210 rows

JUN-044,886,548

rows

JUL-044,949,399

rows

AUG-043,833,931

rows

SEP-04464,417

rows

OCT-0434,212rows

Partition the GL runtime tables by period_name

CREATE TABLE GL_BALANCES(SET_OF_BOOKS_ID NUMBER(15) NOT NULL,CODE_COMBINATION_ID NUMBER(15) NOT NULL,CURRENCY_CODE VARCHAR2(15) NOT NULL,PERIOD_NAME VARCHAR2(15) NOT NULL,ACTUAL_FLAG VARCHAR2(1) NOT NULL,BUDGET_VERSION_ID NUMBER(15),LAST_UPDATE_DATE DATE NOT NULL,. . . . . . )PARTITION BY RANGE (PERIOD_NAME)(PARTITION jan04_per VALUES LESS THAN (‘JAN-2005’),PARTITION feb04_per VALUES LESS THAN (‘FEB-2005’). . . . . .);

• Range Partitioning Example:

Partitioning Methods

Partitioning Methods- List

• Partitions are mapped to an enumerated list. • Allows data to be mapped to partitions by specifying a

list of discrete values for the partitioning key.• List partitioning does not support multi-column

partition keys.• List partitioning was introduced in Oracle9i.

Partitioning Methods - List

OE_ORDER_LINES_ALL

(44,519,880 rows)

Open Orders(192,403 rows)

Closed Orders(40,169,899 rows)

Partition the OM runtime tables by open_flag

CREATE TABLE OE_ORDER_LINES_ALL(LINE_ID NUMBER NOT NULL,ORG_ID NUMBER,HEADER_ID NUMBER NOT NULL,LINE_TYPE_ID NUMBER NOT NULL,LINE_NUMBER NOT NULL NUMBERORDERED_ITEM VARCHAR2(2000),OPEN_FLAG VARCHAR2(1) NOT NULL,. . . . . . )PARTITION BY LIST (open_flag)(PARTITION open_par VALUES (‘Y’),PARTITION close_par VALUES (‘N’)

);

• List Partitioning Example:

Partitioning Methods

Partitioning Methods - Hash

Applies a hash function to the partitioning key to stripe data into partitions. Controls the physical placement of data across a fixed number of partitions.Hash partitioning was introduced in Oracle8i.Generally useful for Applications batch programs which use parallel workers for which block contention is significant.

Order ImportInvoice Import

Partitioning Methods - Hash

Hash15,045,430

rows

Hash25,045,402

rows

Hash45,042,929

rows

Partition the OM runtime tables by the primary key

OE_ORDER_LINES_ALL

(44,519,880 rows)

Hash35,043,712

rows

Hash55,046,608

rows

Hash65,045,024

rows

Hash75.048.923

rows

Hash85,044,312

rows

CREATE TABLE OE_ORDER_LINES_ALL(LINE_ID NUMBER NOT NULL,ORG_ID NUMBER,HEADER_ID NUMBER NOT NULL,LINE_TYPE_ID NUMBER NOT NULL,LINE_NUMBER NOT NULL NUMBERORDERED_ITEM VARCHAR2(2000),OPEN_FLAG VARCHAR2(1) NOT NULL,. . . . . . )PARTITION BY HASH (LINE_ID)PARTITIONS 8. . .. . .;

• Hash Partitioning Example:

Partitioning Methods

Partitioning Methods - Composite

Data is partitioned by using both the range method and then sub-partitioned by the hash method. Composite partitioning combines the best of both worlds (range and hash) by allowing logical groupings at the partition level and handling data skew within the sub-partitions.Composite partitioning was introduced in Oracle8i.

hash1 hash2 hash4

The Workflow Runtime tables use composite partitioning.

WF_ITEM_ACTIVITY_STATUSES

(82,585,860 rows)

hash3 hash5 hash6 hash7 hash8

Order Item Type(3,402,130 rows)

hash1 hash2 hash4hash3 hash5 hash6 hash7 hash8

Purchase OrderItem Type

(13,319,867 rows)

Item KeyHash

Partitioning Methods – Composite

create table wf_item_activity_statusespartition by range (item_type) subpartition by hash (item_key) subpartitions 8 (partition wf_item1 values less than ('A1'), partition wf_item2 values less than ('AM'), partition wf_item3 values less than ('AP'), partition wf_item4 values less than ('AR'), partition wf_item5 values less than ('AZ'), . . .partition wf_item48 values less than ('OE'), partition wf_item49 values less than ('OF'), partition wf_item50 values less than ('OK'), partition wf_item51 values less than ('OL'), . . .partition wf_item56 values less than ('PO'), partition wf_item57 values less than ('PQ'), partition wf_item58 values less than ('PR'), partition wf_item59 values less than ('QA'), . . .);

• Composite Partitioning Example:

Partitioning Methods

<Insert Picture Here>

Index Partitioning

Index Partitioning

• Indexes can be partitioned like tables• Partitioned or nonpartioned indexes can be used with

partitioned or nonpartitioned tables• Partioned indexes can be

• Global or local• Prefixed or nonprefixed

Normal Partitioned

Table Index Table Index

• Index Partitioning Methods• Global Non-partitioned (prefixed)• Global Partitioned (prefixed)• Local Partitioned Prefixed• Local Partitioned Non-Prefix

Index Partitioning

• Index Partitioning Methods• Global non-partitioned index• Global partitioned prefixed index• Local prefixed index• Local non-prefixed index

Global nonpartitioned index

Global partitioned Index

Table partitions

Local prefixed index

Local nonprefixed index

Global & Local Index Partitioning

• Index Partitioning Methods• Global Non-partitioned (prefixed)

• Use Global Non-partitioned indexes for all indexes which are not prefixed by the table partition key.

• Global Range Partitioned (prefixed)• Generally not useful for Applications.

• Global Hash Partitioned (prefixed) – new in 10g• Extremely useful for right growing indexes experiencing contention

due to high levels of concurrency.• Allows the index to be partitioned without affecting the table.

Global Indexes

• Index Partitioning Methods• Local Partitioned Prefixed

• Use in place of indexes which already contain the partition key as a prefix.

• Local Partitioned Non-Prefix• Generally not useful for Applications.• Should only be used when all the queries using the local

(non-prefixed) index always include the partition key filter.

Local Indexes

• Index Partitioning Example• GL_BALANCES table is partitioned by range on the

period_name

Existing IndexesNew

Index Name: Columns: Index Type:=======================================================================GL_BALANCES_N1 (CODE_COMBINATION_ID,PERIOD_NAME) GlobalGL_BALANCES_N2 (PERIOD_NAME) Not NeededGL_BALANCES_N3 (PERIOD_NUM, PERIOD_YEAR) GlobalGL_BALANCES_N4 (TEMPLATE_ID) Global

Global Index Partitioning

• Index Partitioning Example• WF_ITEM_ACTIVITY_STATUSES table is partitioned using

composite partitioning (range on item_type, sub-partitioned by hash on the item_key).

Existing IndexesNew

Index Name: Columns: Index Type:=============================================================================WF_ITEM_ACTIVITY_STATUSES_N1 (ACTIVITY_STATUS,ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_N2 (NOTIFICATION_ID) GlobalWF_ITEM_ACTIVITY_STATUSES_N3 (DUE_DATE, ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_N4 (ASSIGNED_USER, ITEM_TYPE) GlobalWF_ITEM_ACTIVITY_STATUSES_PK (ITEM_TYPE,ITEM_KEY,PROCESS_ACTIVITY) Local

Local & Global Indexes

Useful Data Dictionary Views Table Partitioning

View Name Purpose L

DBA_TABLES Table structure, Partition Y/N

Partition type, default values

Partitions detail

Partition keys

Histogram Statistics

Column and Histogram Stats.

DBA_PART_TABLES

T

T

P

P

P

DBA_TAB_PARTITIONS

DBA_PART_KEY_COLUMNS

DBA_PART_HISTOGRAMS

DBA_PART_COL_STATISTICS P

T = per Table P = per Partition

T = per Table I = per index P = per Partition

Useful Data Dictionary Views Index Partitioning

Name Purpose L

DBA_SUBPART_KEY_COLUMNSSub-partition key Information

Index Partition details

Index Sub-partition details

Partitioned Index Definition

DBA_IND_PARTITIONS

T

P

PDBA_IND_SUBPARTITIONS

DBA_PART_INDEXES I

<Insert Picture Here>

Customer Partitioning Case Studies

Partitioning & Table Compression

• Example OE_ORDER_LINES_ALL

• Before table compression

• After table compression

create table oe_order_lines_all_comp parallel compress tablespaceA_TXN_DATA_01 nologging;

select segment_name, blocks from dba_segments

where segment_name = 'OE_ORDER_LINES_ALL';

SEGMENT_NAME BLOCKS

---------------------------------------------------- -------

OE_ORDER_LINES_ALL 4322352

select segment_name, blocks from dba_segments

where segment_name = 'OE_ORDER_LINES_ALL_COMP';

SEGMENT_NAME BLOCKS

---------------------------------------------------- -------

OE_ORDER_LINES_ALL_COMP 1234957 (~3X)

• Customer Case Study• International Bank• Data Volumes

• GL_JE_LINES (1.1 Billion rows)• GL_CODE_COMBINATIONS (203 Million rows)• GL_BALANCES (1.3 Billion Rows)

• Partitioning Method: Range• Partition Key: set_of_books_id• # of Partitions: 34• Achieved Throughput:

• 11.4 million journal lines imported and posted per hour

Case Study 1: General Ledger (GL) Partitioning

• Customer Case Study• Australian Bank• Data Volumes

• GL_JE_LINES (650 million rows)• GL_CODE_COMBINATIONS (8.5 Million rows)• GL_BALANCES (200 million Rows)

• Partitioning Method: Range• Partition Key: period_name• # of Partitions: 109• Achieved Throughput:

• 7.5 million journal lines imported and posted per hour

Case Study 2: General Ledger (GL) Partitioning

• Partitioning Case Study• Payables Trial Balances• Data Volumes

• AP_LIABILITY_BALANCE (70,567,250 rows)• Partitioning Method: Hash• Partition Key: ORG_ID• # of Partitions: 32• Achieved Throughput:

• Trial Balance report runtime reduced from 2 hours to 10 minutes.

Case Study 3:Accounts Payable (AP) Partitioning

<Insert Picture Here>

Information Lifecycle Management (ILM)

The Data Lifecycle

• Tables continue to grow• Data comes from many sources such as

• Transactional (Fiscal/Non-Fiscal) (e.g. General Ledger)• Transient data (i.e. login, concurrent requests etc..)• Reference/seed data

Active Less Active Historical Archive

This Month This Year Previous Years

Is all this data needed or required to be available online? Will require more storage and memoryIncreased data will impact performance for both online and batch processing

Go-live

What is ILM?

• Information Lifecycle Management encompasses the following:• Policies which define how to manage the data• Processes which actually manage the data

• Including Archiving & Purging and Partitioning• Software which performs the policies & processes

• RDBMS Partitioning functionality • (ILM Advisor)• Purge & Archive concurrent programs

• Hardware where the data is stored• Partitioning, Purging and Archiving are all integral

parts of the ILM lifecycle

Implementing ILM

Active LessActive Historical

High PerformanceStorage Tier

Low CostStorage Tier

HistoricalStorage Tier

1. Define Data Classes

3. Manage Access and Migration of Data by Class2. Create Storage Tiers

for the Data Classes

D E M O N S T R A T I O N

ILM Assistant

<Insert Picture Here>

Archiving & Purgingin the E-Business Suite

Purging & Archiving

Archive

Periodically saving data (with reduced attributes) e.g. transaction data

Purge

Deleting data that you no longer need e.g. transient data or useless data such as dead leads or closed orders.

Improves Upgrade performance

(Less Data.. Less Work)

After Archive

Performance

Periodically

SpaceStatutory Requirements

HistoricalRequirements

Purging & Archiving

• Fiscal Data - Financial Data that can be used to represent a profit or a loss e.g. GL_JOURNALS

• Typically, you should purge and archive non-fiscal data such as sales leads, sales opportunities

Purging & Archiving

• Utilize the standard Oracle supplied programs• Purge/Archive programs provided at the module level in the

E-Business Suite• Many documented standard programs available

• ~214 purge programs in 11.5.10• ~260 purge programs in R12

• Financials• GL, AR, AP, • Costing, • Cash Management, • HR, FA, PA

• Manufacturing• Order Management, • INV, BOM, • WIP, QP, • Shipping, Purchasing

Purge and Archive Programs

Product Concurrent Program Name

Order Management Order Purge

AOL Purge Concurrent Request /Manager DataPurge "Sign on" Audit data

General Ledger GL Archive and PurgeAssets Mass Additions Purge Report

Purge accounting tablesPayables Payables PurgeReceivables Archive and Purge

Purchasing Purge and Archive POs

Customer Example:Typical Purge Programs run

Concurrent Program Name Concurrent Program

Purge Debug Log FNDLGPRG

Purge Cost Information CSTCSPCT

Purge Invoice Extract Output Table RAXINVPG

Catalog Data Purge POXCDXPG

Purge Concurrent Request/Manager Data FNDCPPUR

BEE Batch Process (Purge) PAYLINK(PURGE)

Purge Signon Audit data FNDSCPRG

Payables Open Interface Purge APXIIPRG

Purge System Saved Requisition POXSSPG

Purge Interface Tables PNVPURGE

Purge Obsolete Workflow Runtime Data FNDWFPR

Delete/Purge Timecards (OTL) DELPURTC

<Insert Picture Here>

DemonstrationPurge Portal

Navigation

• Accessing the Purge Portal• System Administrator > Oracle Applications Manager

>Purging/Critical Activities

• Purge Portal introduced in 11i10• Single purge/archive management console• Purge programs can be configured, initiated and monitored• Set the execution frequency as well view history of purge

programs.

• Further enhanced in Release 12• Purge programs tagged with the “purge” concurrent program

type.

Purge Portal Features

Purging – Purge Portal

More Purge Programs in Release 12

• Service Request Purge• Faster WF Purge

• Rewritten to use bulk processing• Email Fulfillment Purge• Export and Purge All GL Interface Data• Approvals Management Transaction Data Purge• Purge Secured Payment Data• Service Contracts Purge Credit Card• Shipping Purge• Task Purge Program• Web Analytics: Purge Tracking Data• Many others

Purging & Archiving

• Oracle is committed to providing Purge/Archive solutions for the key transactional entities

• Oracle continues to enhance its purge and archive offering

• The native E-Business Suite purge and archive programs are synchronized with the data model• Avoids having to re-apply the same patch to multiple schemas

when using non-native purge/archive solutions.

Further InformationMetaLink & Product User Guides

• Please refer to MetaLink (examples) • 138264.1 – General Ledger Archive/Purge FAQ• 144431.1 – Fixed Assets Archive/Purge FAQ• 136919.1 – General Ledger Archive/Purge Setup and Usage

GL 11.5.10User Guide Chapter: 8

Maintenance

AP 11.5.10User Guide Chapter: 10Resource Management

AR 11.5.10User Guide Chapter:1

Archive & Purge

<Insert Picture Here>

Summary

Options Available

Data StorageArchive

Purge

Partitioning Data Access

ILM

Partitioning in Oracle ApplicationsWhat you need to consider

• Functional use of the table (i.e. what exactly does the table store?).

• Which modules or business flows use this table?• Growth rates and access patterns of this table.

• Review the Statspack or AWR Repository for the relevant SQL statements.

• Determine the optimal table partitioning method based on your implementation and based on the data points above.

• Evaluate different indexing methods, global vs. local partitioned indexes.

• Leverage the ILM Assistant to implement partitioning.

Implementing Purging & Archiving

• Consider• Data Retention Requirements• Recovery Requirements• Data access methods and security• Storage topology and costs

• Define• Your archive and purge policy

<Insert Picture Here>

We would like to hear from you

If you have any questions or experiences you would like to share regarding partitioning and/or purging and archiving in the E-Business Suite, please send an e-mail:

mohsin.sameen@oracle.com

MetaLink References

Note #: Description:

138264.1 General Ledger Archive and Purge - Frequently Asked Questions

144431.1 Oracle Assets Archive, Purge and Restore Processes

136919.1 General Ledger Archive/Purge Setup and Usage

248857.1 Oracle Applications Tablespace Model Release 11i - Tablespace Migration Utility

<Insert Picture Here>

Questions & Answers