confidential

SAP NetWeaver BWImportant Housekeeping Principles for keepingyour SAP NetWeaver BW in good shape

Sarma Sishta, SAP Intelligence Platform and NetWeaver RIGRudolf Hennecke, SAP Intelligence Platform and NetWeaver RIG

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 2

Housekeeping activitiesOverview

House keeping activities must be an integral part of the operational concept of a

complex enterprise data warehouse. Planning the house keeping activities

regularly in the system will ensure optimum utilization of system resources and

there by increasing the overall system performance

© SAP 2009 / Page 3

Front-EndLayer

Housekeeping activitiesFocus areas

SAP Business Suite Customer & PartnerApplications

OrderMgmt. ...

Non SAP &Legacy

ETL

Layer SAP Business Objects Data Services

SAP Business Objects BW Tools

SAP NetWeaver BW & BW Accelerator

Met

adat

a M

anag

emen

t / B

usin

ess

Con

tent

Data WarehouseLayer

Ope

rati

on

Extract Transform Load

SAP NW BW ETL

DB LayerDatabase Platform

1. Business1. BusinessData

4. Database4. DatabasePlatform

3. MetaData

Statistics2. StatisticsData

© SAP 2009 / Page 4

confidential

Housekeeping activitiesMost important activities per focus area 1/3

1. Business DataPermanent Data

Definition of retention periods for data in BW InfoProvidersDefinition of needed granularityCompression of InfoCube dataArchiving of data:

Classical archiving (SARA)Near Line Storage (NLS)

(New) Master Data DeletionReorganisation of Master Data Attributes and Texts through process type in RSPC

Temporary DataDeletion of data from the Persistent Staging Area (PSA) DataDeletion of data from DataStore Object Change LogsDeletion of not used dimension entriesUsage and filling of OLAP Cache and aggregatesDeletion of temporary database objectsDeletion of DTP Temporary Storage

© SAP 2009 / Page 5

confidential

2. Statistics & Administrative DataDatabase Volume StatisticsArchiving / Deletion of BW Request Monitoring and Administration TablesFilling and Clean up of BW Statistics tablesDeletion of DTP error logsDeletion of BW Background Management (RSBATCH) informationBW and Database StatisticsDeletion of tRFC QueuesArchiving of IDocsDeletion of logs on the Application Server

Housekeeping activitiesMost important activities per focus area 2/3

© SAP 2009 / Page 6

confidential

3. Meta DataDeletion of (unused) AggregatesDeletion of (unused) queries / workbooks / bookmarksBW Content Analyzer Checks

4. Database PlatformDeletion of empty partitionsTable and Index fragmentationCompression on database level

Housekeeping activitiesMost important activities per focus area 3/3

© SAP 2009 / Page 7

confidential

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 8

confidential

Deletion of Persistent Staging Area (PSA)Overview

Large PSA tables / DataStore Object Change LogsImpact the data load performanceIncrease the downtime for maintenanceIncrease the cost of data storage

The process of PSA deletion has to be defined based onType of data involvedOrganization’s data upload strategyData retention policy

How to:From BW 3.x request deletion from PSA is integrated with in process chainsIn 3.x release the process type enables the deletion based on the PSA table name andthe change log table nameFrom BW 7.01, new process types have been introduced which will enable the PSAdeletion through DataStore Objects, DataSource and Source System combinationFor related information on checking consistency on PSA tables, please refer toSAP Note 1150724 Consolidated note on the check & repair reports for PSANote 706478 - Preventing Basis tables from increasing considerably

© SAP 2009 / Page 9

confidential

Deletion of Persistent Staging Area (PSA)New process type

Process type forrequest deletionfrom PSA

Process type fordeleting Changelog

The process type is delivered with SAP NetWeaver BW 7.0 EhP1 (7.01)

Upgrade to 7.01 will take care of automatic conversion of old process typedefinitions to new process type by replacing the InfoPackage, DTP and PSAtables with respective DataStore Object or DataSource / Source System.

© SAP 2009 / Page 10

confidential

Flexible Deletion of PSA Tables andChange Logs

By using selection patterns

Patterns support

DataStore Object and InfoArea(Change Log) or

DataSource and Source System(PSA)

Placeholders can be used for the fieldsDataStore Object and InfoArea orDataSource and Source system.

This has the advantage that allrequests for objects that are namedafter a specific pattern are alwaysselected (no fix table in transporting,only dynamic selection patterns).

Deletion of Persistent Staging Area (PSA)New process type – process maintenance

© SAP 2009 / Page 11

confidential

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 12

A new technical content with SAP NetWeaver BW 7.30 (0TCT_C25)provides information on data volumes of various BW objects

The Database Volume can be analyzed according to InfoArea,Application Component, BW Objects types and BW specific tableclassifications.

Use Case 1: Support of housekeeping activities like Archiving, periodicPSA and Change Log deletion, etc.

Use Case 2: Service Level Agreement Monitoring of implementedreporting scenarios for cross charging of cost to business departments

Database Volume StatisticsOverview and Motivation

© SAP 2009 / Page 13

Database Volume StatisticsExample Cockpit built with Technical Content

© SAP 2009 / Page 14

© SAP 2007 / Page 15

Database Volume StatisticsData Model

Aim of the data model is to analyze DB growth andsize from a BW perspective, using typical BWtaxonomy and dependencies

Hierarchical display of InfoArea (forInfoProviders) and Application Component (forDatasources).

Database table size (possibility of analyzing theDB size using the actual DB size after DBcompression) Logical DB Size(number of rows *record size)

Table Name e.g./BWC/F…

Table Type e.gInfoCube F Facttable

BW Object Typee.g. InfoCube

Deriving BW Object Information from the underlyingdatabase table name

Different queries are delivered as part of TechnicalContent

© SAP 2009 / Page 15

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 16

Data-Aging Strategies – Initial Steps

Online Database Near-LineStorage Classic Archive

Frequently read /changed data

Rarely read data

Very rarely readdata

Categorizing Information According to Importance:

© SAP 2009 / Page 17

Classic Archiving in BW vs. NLS in SAPNetWeaver 7.0 BW

Access Frequency/Possibility

Age of Data

Archiving (SAP BW 3.X)ADK-based (Archive DevelopmentKit) archiving solution for InfoCubesand ODS objectsCost-reduction due to storing dataon alternative storage mediaArchived data must be reloaded intothe SAP NetWeaver BW databasefor analysis purposes

Online Archive ArchiveReloadOnline

Online Near Line Storage

NLS (SAP NetWeaver BW 7.0)SAP NetWeaver BW analyses havedirect access to NLS dataAvailability of historic datawhile reducing costsReloading of data into the InfoCubeor DataStore Object only necessaryin exceptional cases

© SAP 2009 / Page 18

Archiving Ideas in SAP NetWeaver BW 7.0

ERPBW

ETL/EAI

ADK+ASADK

ILM

NLSArchive

NLS

DAP

Main ideas of the SAP NetWeaver 7.0 BW Archiving interface:

Data Archiving Processes (DAP) allowing for classical archiving as well as NearlineStorage

Near Line Storage (NLS) fills the gap between online storage and offline storageNLS data is read only and directly accessibleNLS data is highly compressedTechnical details depend on Partner Solutions

NLS Data resides on a file system or in a DBMSIndexed: yes or noNLS based on ADK: yes or no

© SAP 2009 / Page 19

Near-line Storage - New Functionality in SAPNW 7.01 and 7.30

Transparent access to Near-Line Storage archives for MultiProvidersPlanned for Release 7.30:

Near-Line Storage (NLS)Integration into Data Mart Layer (Support of MultiProviders)Enable NLS in Query properties from BEx Query Designer(not necessary to do this in RSRT anymore)Selections on navigational attributes will be passed to NLS

Already available since Release 7.01:

NLS Interface supported for write-optimized DataStore objectsADK Archiving supportedBased on request IdLoad date of request or creation date of requestSemantic grouping

NLS Interface extension for lookupsA look up API supports access to NLS partitions of InfoProviders duringtransformations. The API has to be called using ABAP routines.

Archiving of uncompressed InfoCube data© SAP 2009 / Page 20

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 21

confidential

Motivation and FeaturesNew Master Data Deletion

Increase system performanceJoins and look ups from huge master data tables

Improve usabilityRemove unused values from end user value helps

Increase data consistencyReduce number of data quality issues resulting from (semantically) incorrectmaster data (example: planning scenarios, look ups in transformation, …)

Achieve transparencyRun where used check only in order to get clear picture on system usage and datadistribution in the system

…at optimized where-used check and deletion runtimes!Parallel and Package-based Processing supports several Application Serversand Multi-Processor SystemsOptional Background Execution to bypass the maximum Dialog Processing TimeIndex Support on Temporary Where-used-check Table to support SQL JoinOperations

© SAP 2009 / Page 22

confidential

General Availability -New Master Data Deletion

SAP NetWeaver BW 7.0 – SPS 21 / BW Patch 23

SAP Note 1370848 New Master Data Deletion – Information

SAP NetWeaver BW 7.0x – not the default setting,can be switched on in Table RSADMINA or with Transaction RSADMIN

SAP NetWeaver BW 7.30 –default setting

Information in the Help Portalhttp://help.sap.com/saphelp_nw70ehp1/helpdata/en/80/1a6365e07211d2acb80000e829fbfe/frameset.htm

© SAP 2009 / Page 23

confidential

Overview New Master Data Deletion

1. Access / Value Selection 2. Where-used check 3. Master Data Deletion

From the InfoObject contextmenu „Delete Master Data“

From the Master DataMaintenance

Only one usage is sufficient(E)

Only one usage in system(O)

Display where used list fromapplication log

All usages(A)

Only one usage in using object(P)

Delete Master Data (Attributes)and / or Text and SIDs

ReportRSDMDD_DELETE_BATCH

Process Type(planned)

Simulation mode for whereused check only

© SAP 2009 / Page 24

confidential

Parallelization

Customizing the Degree of Parallelization in RSBATCHWhere-Used Check

Master Data Deletion

© SAP 2009 / Page 25

confidential

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 26

confidential

Business dataDeletion of unused dimension entries and partitions

InfoCube DimensionsDeletion of unused dimension entries of an InfoCubecan help keeping to control the growth ofdimension tablesDetection through a standard check in transaction RSRVAlternatively, you can schedule the reportRSDDCVER_DIM_UNUSED for the deletionof unused dimension entries on a regular basis

InfoCube Fact TableIn addition, the report SAP_DROP_EMPTY_FPARTITIONS can be used to dropempty partitions in the InfoCube fact table for incoming requests (F fact table)Please find further information in the following note590370 Too many uncompressed request (f table partitions)

© SAP 2009 / Page 27

confidential

Business dataUsage and filling of aggregates

AggregatesThe table RSDDAGGRDIR provides the information on the number of times an aggregateis called by the query, The field CALLS provides the necessary informationCheck periodically the table and deactivate those aggregates for which CALLS = 0 andobject version OBJVERS = “A”It can also be achieved programmatically by reading the information from the table anddeactivating the aggregates using the function moduleRSDDK_AGGREGATES_DEACTIVATEThe same information is available in the Data Warehousing Workbench (RSA1)

© SAP 2009 / Page 28

confidential

Business dataUsage and filling of OLAP Cache

OLAP CachePlease regularly check on the usage and the filling of your OLAP CacheAdditional Reorganization Options, e.g. unused cache deletion available with EhP1 SP5http://help.sap.com/saphelp_nw70ehp1/helpdata/en/d5/5a393de3bb752de10000000a114084/frameset.htm

Additional Technical Content for OLAP Cache Monitoring with SAP NetWeaver BW 7.01and 7.30

For further information please refer to:OLAP Cache Monitor (Documentation)OLAP Cache Overview (SAP Developer Network)

© SAP 2009 / Page 29

confidential

Business dataDeletion of temporary database objects

Temporary Database Objects

In SAP NetWeaver BW, there are database (DB) objects such as tables or views that havethe '/BW0/0' name prefix.

These objects may be created dynamically during query execution, or other processes thatread data from BW InfoProviders.

With SAP NetWeaver BW 7.X the temporary table management has been improved toprovide a more "self-healing" and "self-managing" behavior.

For further information please refer to SAP Note 1139396 and make sure, that the latestversion for SAP_DROP_TMPTABLES is available (Note 1106393)

Similar information on temporary indices used during BWA processing can be found in SAPNote 1168412.

© SAP 2009 / Page 30

confidential

Statistics and administrative dataDeletion of BW Statistics data

1. Manual deletion of statistics data

In transaction RSDDSTAT. Dates to be deletedare selected.

Using program RSDDSTAT_DATA_DELETE(for query runtime statistics only in 7.0x, inaddition for data warehouse managementstatistics with BW 7.30)

2. Automatic deletion during data load

Per default, with a delta load for query runtime statistics, data of the last 30 days isdeleted out of those BW Statistics Tables the corresponding DataSource extracts from.

This time frame can be customized using the TCT_KEEP_OLAP_DM_DATA_N_DAYSparameter in the RSADMIN table.

Please see following SAP Notes for more information

891740 Query runtime statistics: Corrections for extractors

1095411 BWAC: Reduction in Data volume in QT Statistics Tables

BW Statistics

© SAP 2009 / Page 31

confidential

Statistics and administrative dataDeletion of DTP temporary storage and error logs

DTP Temporary StorageIn the “Temporary Storage” menu in the maintenance of Data Transfer Processes (DTP)you can set options for deleting the DTP Temporary Storage

Please find an additional option for deleting the Temporary Storage in the following SAPNote

1168098 70SP19: Performance during DataStore object extraction

DTP Error LogYou can use the report RSB_ANALYZE_ERRORLOG to analyze which DTPs have createdhow many single record error messages, and to how many requests these messages aredistributed. You can use the report RSBM_ERRORLOG_DELETE for single DTPs todelete the messages for requests

Please find additional information in the following SAP Note

1095924 Correction: Deletion/analysis report for error handling logs

© SAP 2009 / Page 32

confidential

Statistics and administrative dataArchiving Requests Status

Request InformationMany DVM (= Data Volume Management) services reveal that many customers have hugerequest information tables and that is information is no longer needed.Large monitoring tables also impact the performance of the request monitorRequest-archiving is used to archive information mainly from tables RSREQDONE,RSMONMESS, RSSELDONE, RSRULEDONEA new archiving object was created, BWREQARCH. The properties of the object can alsobe seen and changed in transaction AOBJ.Transaction SARA can be used to WRITE, DELETE, or MANAGE the request archivingprocess.Alternatively to using transaction SARA and RSREQARCH the following reports can beused resulting in the same way: RSREQARCH_WRITE, RSREQARCH_RELOAD, andRSREQARCH_RELOADPlease find more information in the Standard Documentation

http://help.sap.com/saphelp_nw70ehp1/helpdata/en/42/ead8d7b55e1bd2e10000000a11466f/frameset.htm

© SAP 2009 / Page 33

confidential

Statistics and administrative dataBW Background Management (RSBATCH)

BW Background ManagementDelete the messages from BW background management as well as the internalparameters of the background processes that are executed by background managementon a regular basis. This prevents table RSBATCHDATA from overflowing.

Please find more information in the standard documentationhttp://help.sap.com/saphelp_nw70ehp1/helpdata/en/48/d2a551003c04e9e10000000a42189c/frameset.htm

© SAP 2009 / Page 34

confidential

Meta DataDeletion of (unused) queries / workbooks / bookmarks

Query, Workbooks and BookmarksTransaction RSZDELETE allows for an advanced where used check on queries, queryviews as well as workbooks including a deletion of objects which are not referenced

Note - 1164904 New revision of transaction RSZDELETE

In addition, the report ANALYZE_RSZ_TABLES allows you to detect inconsistencies inquery definitionsPlease find additional information in the following SAP Notes

1020826 - Hotfixes for check report ANALYZE_RSZ_TABLES792779 - Report ANALYZE_RSZ_TABLES385219 - Reorganization of BW workbooks1089469 - Improvements for report RSWB_ROLES_REORG1388287 - RSRD_BOOKMARK_REORGANISATION: Specified time not used1246860 - Displaying entries in user menu

© SAP 2009 / Page 35

confidential

1. Overview2. Deletion of Persistent Staging Area (PSA) & Change Log3. Database Volume Statistics4. Archiving – Near line storage5. New Master Data Deletion6. Details on further activities7. Further information

Agenda

© SAP 2009 / Page 36

Further information in the SDN

SAP NetWeaver BW - BW Software Lifecycle Management Knowledge Centerhttp://www.sdn.sap.com/irj/sdn/edw-ops?rid=/webcontent/uuid/b039ebc5-304f-2b10-499e-97dabda45e48

SAP NetWeaver BW - the “BW Upgrade Bible” to 7.0xhttp://www.sdn.sap.com/irj/sdn/go/portal/prtroot/docs/library/uuid/60ecc07d-3a4b-2b10-ef83-c988e6ecabf6

SAP NetWeaver BW - Business Warehouse Features delivered with EhP1 (7.01)http://www.sdn.sap.com/irj/sdn/edw?rid=/webcontent/uuid/506d71be-6ff4-2b10-d789-edee2e3e3ca8

Periodic Jobs and Tasks in SAP BWhttp://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/08f1b622-0c01-0010-618c-cb41e12c72

How to Archive PSA Data in SAP NetWeaver Business Intelligence (NW7.0)http://www.sdn.sap.com/irj/scn/index?rid=/library/uuid/a0974515-7225-2b10-14a1-f5abba35b19f

SAP NetWeaver BW 7.0 Documentation - Information Lifecycle Managementhttp://help.sap.com/saphelp_nw70/helpdata/en/d0/84e5414f070640e10000000a1550b0/frameset.htm

Copyright 2009 SAP AGAll rights reserved

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changedwithout prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge and other SAP products and services mentioned herein as well as their respective logos aretrademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned and associated logos displayedare the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

The information in this document is proprietary to SAP. This document is a preliminary version and not subject to your license agreement or any other agreement with SAP. This documentcontains only intended strategies, developments, and functionalities of the SAP® product and is not intended to be binding upon SAP to any particular course of business, product strategy,and/or development. SAP assumes no responsibility for errors or omissions in this document. SAP does not warrant the accuracy or completeness of the information, text, graphics, links, orother items contained within this material. This document is provided without a warranty of any kind, either express or implied, including but not limited to the implied warranties ofmerchantability, fitness for a particular purpose, or non-infringement.

SAP shall have no liability for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. This limitationshall not apply in cases of intent or gross negligence.

The statutory liability for personal injury and defective products is not affected. SAP has no control over the information that you may access through the use of hot links contained in thesematerials and does not endorse your use of third-party Web pages nor provide any warranty whatsoever relating to third-party Web pages

Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die ausdrückliche schriftliche Genehmigung durchSAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankündigung geändert werden.

Einige von der SAP AG und deren Vertriebspartnern vertriebene Softwareprodukte können Softwarekomponenten umfassen, die Eigentum anderer Softwarehersteller sind.

SAP, R/3, xApps, xApp, SAP NetWeaver, Duet, SAP Business ByDesign, ByDesign, PartnerEdge und andere in diesem Dokument erwähnte SAP-Produkte und Services sowie diedazugehörigen Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und in mehreren anderen Ländern weltweit. Alle anderen in diesem Dokument erwähnten Namenvon Produkten und Services sowie die damit verbundenen Firmenlogos sind Marken der jeweiligen Unternehmen. Die Angaben im Text sind unverbindlich und dienen lediglich zuInformationszwecken. Produkte können länderspezifische Unterschiede aufweisen.

Die in diesem Dokument enthaltenen Informationen sind Eigentum von SAP. Dieses Dokument ist eine Vorabversion und unterliegt nicht Ihrer Lizenzvereinbarung oder einer anderenVereinbarung mit SAP. Dieses Dokument enthält nur vorgesehene Strategien, Entwicklungen und Funktionen des SAP®-Produkts und ist für SAP nicht bindend, einen bestimmtenGeschäftsweg, eine Produktstrategie bzw. -entwicklung einzuschlagen. SAP übernimmt keine Verantwortung für Fehler oder Auslassungen in diesen Materialien. SAP garantiert nicht dieRichtigkeit oder Vollständigkeit der Informationen, Texte, Grafiken, Links oder anderer in diesen Materialien enthaltenen Elemente. Diese Publikation wird ohne jegliche Gewähr, wederausdrücklich noch stillschweigend, bereitgestellt. Dies gilt u. a., aber nicht ausschließlich, hinsichtlich der Gewährleistung der Marktgängigkeit und der Eignung für einen bestimmten Zwecksowie für die Gewährleistung der Nichtverletzung geltenden Rechts.

SAP übernimmt keine Haftung für Schäden jeglicher Art, einschließlich und ohne Einschränkung für direkte, spezielle, indirekte oder Folgeschäden im Zusammenhang mit der Verwendungdieser Unterlagen. Diese Einschränkung gilt nicht bei Vorsatz oder grober Fahrlässigkeit.

Die gesetzliche Haftung bei Personenschäden oder die Produkthaftung bleibt unberührt. Die Informationen, auf die Sie möglicherweise über die in diesem Material enthaltenen Hotlinkszugreifen, unterliegen nicht dem Einfluss von SAP, und SAP unterstützt nicht die Nutzung von Internetseiten Dritter durch Sie und gibt keinerlei Gewährleistungen oder Zusagen überInternetseiten Dritter ab.

Alle Rechte vorbehalten.