Enterprise Data Warehousing With Sap Bw - Overview

Enterprise Data Warehousing with SAP BW An Overview

White Paper Version 2.0

August 18th, 2003

[email protected]

SAP (SAP AG and SAP America, Inc.) assumes no responsibility for errors or omissions in these materials.

These materials are provided as is without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials.

SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.

ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW V2.0

Table of Contents ENTERPRISE DATA WAREHOUSING WITH SAP BW AN OVERVIEW....................................... 1

1 INTRODUCTION........................................................................................................................... 1 1.1 SUPPORTED SOFTWARE VERSIONS ........................................................................................... 1 1.2 THIS WHITE PAPER .................................................................................................................. 1

2 AIMS OF THE WHITE PAPER ..................................................................................................... 1

3 MOTIVATION................................................................................................................................ 2 3.1 THE MARKET............................................................................................................................ 2 3.2 WHY DO YOU NEED A CORPORATE BW STRATEGY?................................................................. 3

4 ENTERPRISE DATA WAREHOUSING WITH SAP BW.............................................................. 5

5 BW ENTERPRISE ARCHITECTURE........................................................................................... 6 5.1 ASPECTS OF A DATA WAREHOUSE ARCHITECTURE .................................................................... 6 5.2 DATA STORE ARCHITECTURE - BW DATA LAYER ..................................................................... 10

5.2.1 BW Architected Data Mart Layer .................................................................................. 11 5.2.2 BW Data Warehouse Layer .......................................................................................... 14 5.2.3 BW Extraction and Staging........................................................................................... 19 5.2.4 BW Support for Operational / Real Time Reporting ..................................................... 20

5.3 DATA ARCHITECTURE - BW DATA MODEL ............................................................................... 24 5.3.1 Operative Data Models and Data Warehouse Data Model .......................................... 24 5.3.2 The BW Data Model ..................................................................................................... 25

5.4 TOPOLOGIES BW LANDSCAPES ........................................................................................... 29 5.4.1 Consistency in a distributed BW Landscape ................................................................ 30 5.4.2 BW Landscapes und Technical Parameters ................................................................ 31 5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse...... 32 5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses................ 34

6 CENTRAL BW APPLICATION DEVELOPMENT...................................................................... 38

7 DATA AND DATA MODEL INTEGRATION .............................................................................. 40 7.1 CHALLENGES POSED BY DATA INTEGRATION............................................................................ 40 7.2 DATA MODEL INTEGRATION..................................................................................................... 42 7.3 THE ROLE OF SAP MASTER DATA MANAGEMENTS (MDM) ...................................................... 43

8 SUMMARY.................................................................................................................................. 45

TABLE OF ILLUSTRATIONS 43

2003 SAP AG AND SAP AMERICA, INC. CONTENT

BW Enterprise Data Warehousing Overview V2.0

1 Introduction

1.1 Supported Software Versions This document is relevant for BW Versions 3.x.

1.2 This White Paper The subject of this document is very complex. Therefore, we do not claim to cover all the various aspects of this topic, or to offer an exhaustive description of the areas discussed.

To see updated versions of this document, visit the SAP Service Marketplace regularly.

2 Aims of the White Paper This White Paper offers an overview of the implementation of SAP Business Information Warehouse (SAP BW) from a corporate perspective.

It considers the following issues:

Architectural aspects: data management, data models, topologies etc. on a conceptual level

Integration aspects: supporting corporate strategy as far as the harmonization of master data is concerned

Solution development: the efficient, consistent development of BW-based applications.

Enterprises differ in terms of their organization and their business. This implies that there can be no standard solution for a corporate BW implementation strategy. However there are basic truths that always have to be considered, and patterns that arise within specific business types. Above all, this White Paper will describe fundamental aspects of a corporate-wide BW implementation.

This White Paper is no substitute for a business-specific consultation on BW architecture.

This White Paper focuses on the general principles involved in a strategic corporate BW implementation and not on exceptions to this. It follows the 80-20 rule.

The White Paper avoids details wherever possible, so as not to lose sight of the overall context.

Knowledge about BW is useful to read this White Paper but no prerequisite for it.

2003 SAP AG AND SAP AMERICA, INC. 1


3 Motivation 3.1 The Market At the time of writing, there are more than 6000 BW installations within the market.

The fundamental advantages of BW have led more and more customers to center their corporate data warehousing strategy around BW. Customers frequently stress the end-to-end conception of the BW in this context. In comparison to fragmented technologies, the integrated metadata concept, from data integration right through to analysis, leads to lower total costs for the project overall.

The BW has moved away from isolated instances to incorporate an architecturally-based view. In this way, the BW has come to set new standards of quality in terms of its positioning within an enterprise.

The following graphic shows various rudiments of this strategy:

SAP AG 2003, Enterprise Data Warehousing based on SAP BW, J. Haupt 3 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 3

Evolution of SAP BW

Strategic Corporate BW Strategic Corporate BW ImplementationsImplementations

Isolated BW Isolated BW ImplementationsImplementations

BW1

BW2

BW3

BWn

BW1

BW2

BWn

GlobalBW

Others

Headquarter Headquarter ReportingReportingScenarioScenario

LocalBW

LocalBW

LocalBW

GlobalBW

Architected Architected BW LandscapeBW Landscape

ScenarioScenario

EnterpriseBW

BW EnterpriseBW EnterpriseDataData

WarehouseWarehouseScenarioScenario

Issues: Synergy Integration Consistency

SpokeBW

SpokeBW

Illustration 1: Evolution of the SAP BW

This leads to one main motivation for this paper. It aims to support the evolution-process of BW offering an overview of the essential criteria for corporate architected BW implementations for customers and consultants.

On the other hand, there is still a large number of customers who do not have a corporate data warehouse strategy and who have implemented BW in a more isolated and restricted way.

Thus, another motive is the hope of making people aware of the prospects and benefits of a business-wide, architecturally-based BW implementation.



3.2 Why Do You Need A Corporate BW Strategy? There are many training guides, enhanced documentations, ASAP Accelerators and How to Guides available that offer support to BW users. These documents all focus on offering support with concrete challenges that arise during a project.

This is obviously important and necessary. However there are also short-comings as projects are always part of an incremental BW implementation:

From project to project, project goals are modeled and realized in BW. These goals are typically reporting- and analysis-driven, and the overall success of the project is then often measured by decision-makers in terms of how far these reporting and analysis demands are met. As such, the focus is at solution level or using a data warehouse term the focus is at data mart level.

Less attention is accorded to decisive factors for the mid- and long-term success of an investment in BW:

Redundancy has to be controlled in all areas!

Business-wide guidelines rarely exist regarding:

The persistent BW data stores and their design The BW Data Warehouse data model and its management The BW landscape (the role of BW instances and which conditions are valid for new

instances).

Aspects of data integration are often neglected too, and applications in various organization units that feature a BW are developed asynchronously and redundantly.

Altogether, from an overall business perspective, you are left with a costly and less efficient procedure that delivers neither consistent, reliable, nor integrated information.

The following graphic clarifies the solution-focus issues:


Redundancy and Data Mart (Solution) Focus

Real World

InformationRequirements

(grouped to Scopes)

U V W ....Schema-Modeling

InfoCubes/ODS-Objects

Extraction

Sources

Business Rules

Transformation

BW Application Project Team

Successful Data Warehousing means Controlled Redundancy !Successful Data Warehousing means Controlled Redundancy !Successful Data Warehousing means Controlled Redundancy !

Impacts of non-existing corporate BW guidelines:

Redundant Data Redundant Extraction Redundant Transformation Redundant Business Rules Redundant Masterdata......

Illustration 2: Redundancy and Solution focus



The next graphic clarifies the multiple BW landscape issues:


Redundancy and Multiple BW Landscape

Sub-Division AR/3

Global

Sub-Division B R/3

only Germany

Sales EuropeR/3

BW Local

Global

HQBW

SubDiv BW

MDMS

BWS-America

BWAsia

BWN-America

BWSales Europe

Source Systems

....................DivR/3 DivR/3 DivR/312 systemsworldwide

Successful Data Warehousing means Controlled Redundancy !Successful Data Warehousing means Controlled Redundancy !Successful Data Warehousing means Controlled Redundancy !

A Real

Life

BW La

ndsca

pe Ex

ample

Impacts of non-existing corporate BW guidelines:

Redundant Data FlowsRedundant ExtractionRedundant DevelopmentRedundant Models......

Illustration 3: Redundancy and multiple BW landscape

The more complex an enterprises structures, the more important it is to have corporate-wide guidelines and recommendations for the implementation of BW. They guarantee long-term consistency and reduced costs.



4 Enterprise Data Warehousing with SAP BW When we talk about enterprise data warehousing it is first necessary to ask what we understand by this term: Enterprise data warehousing is the sum of all the decisions that have to be taken at corporate level in order to obtain a durable solution that adheres to business-wide, specified guidelines, and fulfills all requirements for integrated, consistent and structured information.

These decisions are mirrored in the enterprise data warehouse architecture.

In this chapter we will consider the aspects of such an architecture so that we can then look more closely at how BW supports an architecturally-based, consistent, corporate data warehouse solution.

As well as a BW architecture that ensures consistency, we want to consider in what ways the BW offers support in generating consistent application solutions.

Furthermore, in discussing a corporate BW strategy, we have to include BW in the data integration strategy of the company. This is done in the last chapter.

As with all important decisions, we concentrate on the fundamental aspects, following the 80:20 rule.



5 BW Enterprise Architecture An architecture can be seen as a system design decision that has a specified duration. In other words, once it has been implemented it is not easy to change. In defining a corporate BW architecture it is clear that the term architecture is used in various senses, and that aspects such as where, what and how often become confused. This leads to confusion and is dangerous in that essential factors may be considered from a lop-sided perspective, or not at all.

5.1 Aspects of a Data Warehouse Architecture In discussing an optimal BW architecture, an enterprise often focuses too quickly on physical BW instances. Is more than one BW instance needed and if yes, where are they to be institutionalized? How will they work together? These questions concern a BW landscape, or as we prefer to call it, a consistent enterprise BW topology. This means that the second or even third architecture elements are being considered before the first. How can individual BW instances and their tasks be discussed without having first defined how data is consistently handled and saved in such a BW instance? We must start with the BW data store or data layer architecture: The layer architecture defines persistent data stores and their storage schemas from extraction through to the end-user, and refinement processes between the layers. Qualified data status and the respective layers:

Staging Area - row Data Warehouse Layer - integrated, granular, not application-specific, historical Data Mart Layer - integrated, aggregated, application-specific Operational Data Store (ODS) integrated, granular, application-specific, close to event




Conceptual Layers of Data Warehousing

Infor-mationAccess

Non-SAPSource

SAPSource

PersistentStaging

Area

Operational Data Store

Archiving & Near-Line Storage

Data Warehouse

Archi-tected DataMarts

Illustration 4: The conceptual layer architecture of data warehousing Repeatedly saving raw data in various refinement situations means on the one hand side redundancy. On the other hand side means a consistent layer architecture the only way to control the redundancy of data warehousing, which derives from its very nature. This has to be taken into consideration with the layer data store schemas. However, a suitable layer support is not the only necessary condition for a corporate-wide data warehouse architecture that can secure consistency in the mid- and long-term. This can only be achieved if the operative data models are mapped consistently to the model of the data warehouse layer and the model of the data mart layer. Such a consistent data (model) architecture therefore describes the model of each layer and how the models relate to each other. As such we need a data architecture that guarantees, for example, that operative material terms, as they are found in the respective sales or distribution area, or in materials management, are reproduced consistently in the material subject area. Ideally you would have a single operative enterprise data model and use that to derive the model for each layer. However, a corporate data model of this sort does not normally exist, particularly if legacy systems are being used. This means that often no or only limited guidelines and rules exist how to map the operative models to a single data warehouse layer model and a single data mart layer model because of the difficulties to achieve this. The result is unsynchronized isolated solutions (stovepipe solutions). This is illustrated in the following graphic:




Operational Data Models and the Data Warehouse Data Model

Not aligned operational data models Consistent enterprise data model

Inconsistent data warehouse data models stovepipe solutions

Consistent data warehouse data model long term success

A data warehouse A data warehouse consistency challengeconsistency challenge

Illustration 4: Operational Data Models and DWH Data Model A missing data model architecture is one reason why corporate data warehouse strategies fail. Only when there is clarity regarding the data layer and data model does it make sense to discuss the corporate-wide topological aspects of a BW architecture. Because topological aspects are influenced by the layer architecture. Two basic topologies are differentiated: The BW enterprise data warehouse landscape with BW as the central information hub Landscape based on local BWs with a higher level global BW

LocalBW

LocalBW

LocalBW

GlobalBW

OutsideOutside--ININBWBW

LandscapeLandscapeOthers

LocalBW

LocalBW

LocalBW

GlobalBW



EnterpriseBW

InsideInside--OutOutBWBW EnterpriseEnterprise

DataDataWarehouseWarehouse SpokeBW

SpokeBW

Others

SpokeBW

EnterpriseBW



SpokeBW

Others

SpokeBW

Illustration 5: BW basic topologies When determining the BW topology, political, organizational and technical factors play an important role. As a result you cannot really talk about a generally valid optimal BW topology. We summarize that it is necessary to make definitions that are valid enterprise-wide in the following areas:



BW data layer architecture Which data statuses does it make sense to save persistently? Which designs are approved for the various data stores?

BW data model architecture Each data warehouse layer requires a data model that describes objects and their

relationships The data models of the various layers have to be derived from each other consistently

BW topology architecture Different corporate cultures, geographical, and business diversifications lead to different

data warehouse landscapes.



5.2 Data Store Architecture - BW Data Layer It is generally accepted that the data store that supports analysis and reporting is to be kept separate from the data store that deals with any sort of data acquisition. Analysis and reporting are handled by data marts. Close to event-level operative reporting is handled by the operational data store. BW supports multidimensional data marts in an extended star schema. This is made up of BW InfoCubes and over-arching BW InfoObject master data dimensions. Operational data stores are supported by flat BW ODS objects. Here too BW master data acts as structures that span ODS objects. Data acquisition (processes that secure quality and integration) are referred to as staging processes. The persistent saving of staging process data occurs in staging areas. Data staging stores are realized by the Persistent Staging Area (PSA) Objects and by ODS objects. The granular, integrated result of the staging process reproduces data in optimal condition. This is referred to as the data warehouse layer. BW ODS objects are the central storage objects that build a BW data warehouse layer. This gives us the following general picture of a BW layer architecture:

Finance

Logistic Open

Distr

ibution

SAP Business Information WarehouseSAP Business Information Warehouse

StagingStagingAreaArea

CleansingCleansingTransformTransform

MasterReference

Data

Data Data WarehouseWarehouse

ODSODS

PrimaryPrimaryData Data

ManagementManagement

Extra

ction /

Ope

n Stag

ing

DataDataAcquisitionAcquisition

Data Data DeliveryDelivery

any ta

rget

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas

Flow Control Flow Control

MetaMeta DataData

any s

ource Transaction

Data

ArchitectedArchitectedData MartsData Marts

FinanceFinance

LogisticLogistic Open

Distr

ibution




MasterReference

Data


ODSODS



Extra

ction /

Ope

n Stag

ingEx

tractio

n / O

pen S

taging



any ta

rget

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas


MetaMeta DataData

any s

ource Transaction

Data


Illustration 5: BW layer architecture - overview



5.2.1 BW Architected Data Mart Layer InfoCube data marts allow the flexible generation of queries and navigation to integrated, granular or aggregated data. In order to make comparisons possible, an amount of historical data (defined by you) can be retained. InfoCubes are multidimensional data structures (also called star schemas) that organize data in such a way that key figures (or facts or measures) are affixed to characteristics and their attributes, which form the so-called dimensions (BW InfoCube dimensions and master data). To increase the capacity of an InfoCube schema, dimensions are separated into local and global operative parts. Global, because these dimension-parts are only saved once and are conjointly addressed by various InfoCubes.

FACT Table

Material_Dimension_IDSalesOrg_Dimension_IDTime_Dimension_IDCustomer_Dimension_ID

Sales AmountQuantity

InfoCubeInfoCube

Gebiet 1 Gebiet 2 Gebiet 3

Bezirk 1

Gebiet 3a

Bezirk 2

Region 1

Gebiet 4 Gebiet 5

Bezirk 3

Region 2

Gebiet 6

Bezirk 4

Gebiet 7 Gebiet 8

Bezirk 5

Region 3

Vertriebsorganisation

Material Group

Material Hierarchy Table

Material NumberLanguage Code

Material NumberLanguage CodeMaterial Name

Material Text TableMaterial_Dimension_ID

Material NumberMaterial Dimension

Table

Material Master Table

Material NumberMaterial Number

Material Type

MaterialMaterialDimensionDimension

Shared, globalShared, globalPart of

Material Dimension

LocalLocal Part of Material Dimension

BW Extended Star Schema

FACT Table





InfoCubeInfoCube

Gebiet 1 Gebiet 2 Gebiet 3

Bezirk 1

Gebiet 3a

Bezirk 2

Region 1

Gebiet 4 Gebiet 5

Bezirk 3

Region 2

Gebiet 6

Bezirk 4

Gebiet 7 Gebiet 8

Bezirk 5

Region 3

Vertriebsorganisation

Material GroupMaterial Group

Material Hierarchy Table

Material NumberLanguage Code

Material NumberLanguage CodeMaterial NameMaterial Name

Material Text TableMaterial_Dimension_ID

Material NumberMaterial Dimension

Table

Material Master Table

Material NumberMaterial Number

Material TypeMaterial Type

MaterialMaterialDimensionDimension

Shared, globalShared, globalPart of

Material Dimension

LocalLocal Part of Material Dimension

BW Extended Star Schema

Illustration 5: The BW extended star schema In this way BW InfoCubes support the concept of conformed dimensions, which form a sort of glue between InfoCubes.

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster

Horizontal Consistency:Horizontal Consistency:Conformed Structures Conformed Structures

BW InfoObjectsBW InfoObjectsmaster datamaster data

CUSTOMER

MATERIAL

Horizontal Consistency: BW Architected Data Mart Layer

InfoCubeInfoCube InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster



CUSTOMER

MATERIAL

Horizontal Consistency: BW Architected Data Mart Layer

InfoCubeInfoCube

Illustration 6: Horizontal consistency in the BW architected data mart layer



Conformed dimensions essentially support horizontal consistency within the data mart layer. They are part of the BW concept and help prevent the occurrence of isolated solutions (stovepipes). The BW term for these conformed dimensions is master data. This is one reason to qualify the BW data marts as Architected data marts. The effectiveness of the BW InfoCube schema does not just support consistency but is obviously also essential for mapping end-users demands. Above all we have to mention here the ability to map entities from different historical views to the same key figure. By using surrogate keys in the InfoCube and because of the master data/ conformed dimension concept, the BW makes it possible to simultaneously map different views to key figures in one InfoCube.

SAP AG 2003, Corporate Data Warehousing based on SAP BW, J. Haupt 6 SAP AG 2001 BW - The Open Business Intelligence Platform/ J. Haupt / 6

BW Support of Different Historical Views

Constellation 09/1998Customer Mother-Company

AAA XBBB XCCC Y

Constellation 10/1998

Customer Mother-Company AAA XBBB Y (changed)CCC Y

Customer Date Revenue

AAA 09/1998 100BBB 09/1998 100CCC 09/1998 100BBB 10/1998 100CCC 10/1998 100

Fact Table

Mother-Comp Rev 09/98 Rev 10/98

X 100 0Y 200 200

Report using todays (10/98) constellation


X 200 100Y 100 100

Report using 09/98 constellation


X 200 0Y 100 200

Report showing historical truth


X 100 0Y 100 100

Report showing comparable results

BW supported Views in

a single InfoCube

Extended Star Schema

Illustration 7: BW and slowly changing dimensions 'Today is Yesterday' : Show key figure values, including historical key figure values, from the

currently valid perspectives (current view of dimension attributes) 'Yesterday is Today' : Show key figure values from user-defined perspectives (key date oriented

view of dimension attributes) 'Historical Truth' : Show key figures (transactions) in the same state (historicization of dimension

attributes), as at the time of posting historically correct

Scenarios can also be presented in which only key figure values with unchanged dimension attribute constellations are reported. In order to reduce redundancy and counteract the one report is equal to one InfoCube error, virtual multidimensional structures are supported for persistent InfoCubes. These are called BW MultiProviders and BW Queries.




Multi-dimensional Schemas in BW

EndEnd--User User Reporting & Analysis needReporting & Analysis need

MultiMulti--dimensional modeldimensional model

BW InfoCube(s)BW InfoCube(s)z physical implementation

z basic factsznot report specific

BW MultiProviderBW MultiProviderzvirtual Structureszbasic facts

znot report specificzoptional

BW BEX QueryBW BEX Queryzvirtual Structureszcomplex KPIs

znavigation behaviorzreport (type) specific

BW BEX ReportBW BEX Reportzend-user requirement

specific

Illustration 8: Persistent and virtual multidimensional structures in BW Despite the capabilities of BW data mart layers, the limitations and dangers of a data mart-focused implementation are also evident: Limited reusability of data and loss of historical conditions

Data is dealt with in the data mart layer in a scope-specific manner: Aggregation/ manipulation/ overwriting of data is normal. It follows that the usage of these data for other purposes is limited.

The data mart layer is the direct access layer for the end-user. Performance, therefore, plays a decisive role. For this reason only data that is actually needed is retained in the data mart layer. Saving data just in case will quickly lead to a large data volume and impact negatively on performance. Overwriting old images, as normally happens in the conformed dimensions (master data), is therefore valid and indeed desired. However, this means that history is lost.

Redundant data storage (the same data is held in various InfoCubes and often in different degrees of granularity)

Redundant data acquisition (staging) Redundant data extraction Limited ability to trace data. Data staging is required by the BW layer architecture here. Factors arising from contact with the data mart data model that threaten consistency are discussed in the chapter on data model architecture.



5.2.2 BW Data Warehouse Layer In order to overcome the limitations and dangers of a data mart-focused BW implementation, the corporate BW architecture has to answer demands such as:

extract once deploy many single point of truth

Furthermore, the contradiction between quite sensibly reducing information in the data mart layer if it is not currently required by the end-user, and the claims the BW makes to flexibility, has to be resolved. The BW also has to be able to respond quickly to new requests. The answer is to build a separate layer that saves the integrated results of the data transformation and data quality-ensuring processes in a consistent, granular and historical form: The BW data warehouse layer. Ideally this is built centrally and extends business-wide. We can then talk about a BW enterprise data warehouse (layer).


A Matter of Consistency and Reliability The BW Data Warehouse (Layer)

Finance

Logistic Open

Distr

ibution




MasterReference

Data


ODSODS



Extra

ction /

Ope

n Stag

ing



any ta

rget

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas


MetaMeta DataData

any s

ource Transaction

Data


Illustration 9: SAP BW data warehouse layer The following description of a BW data warehouse layer is based on William H. Inmons definition: A BW data warehouse layer offers:

o Subject-oriented, integrated data o Original data that will not be changed in relation to the goals of a particular project

(unflavored data) o Granular data

Non-aggregated data o Historical data (that cannot be overwritten or deleted)



o A single point of truth The BW data warehouse layer is the only source for BW architected data marts. (extract once deploy many).

In this way a BW data warehouse offers the following benefits: I. Flexibility

o New InfoCube data mart solutions that include history and new extraction can be built in real-time

o The BW can respond quickly to unpredictable, one-off user requests, without having to build persistent structures (Although the BW data warehouse layer is not to be misused for standard analysis as it does not possess the optimal structures for this).

II. Reliability and Reproducibility o The BW data warehouse layer is centrally administrated and not subjected to arbitrary

projects. The data warehouse layer is the common source for all the data provided in the data marts.

III. Complete history IV. Consistency

o Extraction, staging and integration are managed centrally o Data is extracted only once o Redundant staging processes are avoided.

A BW data warehouse (layer) is an enterprises memory, a repository for information for the whole enterprise. In addition to offering horizontal consistency within the BW data mart layer, the BW data warehouse layer offers a sort of vertical consistency that the data mart layer alone is not able to guarantee.



Vertical Consistency: BW Data Warehouse Layer

Source 1 Source 2 Source 3 Source 4 Source 5

Vertical Consistency:Vertical Consistency:Single Point of Truth Single Point of Truth

BW DWH Layer ODSBW DWH Layer ODS--ObjectsObjects

Transaction / Master Data

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster



CUSTOMER

MATERIAL

InfoCubeInfoCube

Vertical Consistency: BW Data Warehouse Layer

Source 1 Source 2 Source 3 Source 4 Source 5

Vertical Consistency:Vertical Consistency:Single Point of Truth Single Point of Truth

BW DWH Layer ODSBW DWH Layer ODS--ObjectsObjects

Transaction / Master Data

InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster



CUSTOMER

MATERIAL

InfoCubeInfoCube InfoCube

CUSTOMER

MATERIAL

CUSTOMER

MATERIALmaster

CUSTOMERmaster



CUSTOMER

MATERIAL

InfoCubeInfoCube

Illustration 10: BW data warehouse layer and vertical consistency The question of how to organize data warehouse layers often causes arguments. Extreme positions on this range from 3NF to de-normalized star schemas. We do not want to give a definite answer here, but suggest taking a pragmatic approach. As the data warehouse layer forms the basis for supplying the data marts, data should be saved so that it can be easily transported into conformed dimensions and InfoCubes. This means complex join operations for normalized data warehouse objects on the way to the data marts should be avoided. It is therefore, for example, legitimate to store position data and header information for a sales order together in the data warehouse layer.

To the question, "what kind of database design is appropriate for a data warehouse?", W.H. Inmon offers the following answer: For a data warehouse, it is always better to have a lightly normalized design. Star schemas fit elsewhere (s. Data Marts t. author). The data warehouse design is not perfectly normalized because a few alterations are made for common usage, such as creating arrays of data when data is always used together, creating a small and selective amount of redundancy where appropriate, merging tables that are commonly used together, and so forth. At the end of the day, a data warehouse design is distinctively normalized. This answer presumes that you know the difference between a data warehouse and a data mart.

Restatement or regeneration of some of these higher levels of information requires that we keep available a detailed historical data store. This should be clean (free of errors and conformed to the Data Warehouse master data model) to avoid unnecessary reprocessing of source data. (W. H. Inmon, Architecture 2002)



The BW does not offer a data store specific for the data warehouse layer. Transaction data is saved in flat ODS objects, master data, reference data, in InfoObjects or ODS objects. Handling history is supported in BW by staging processes (transfer/ update rules) and staging ODS objects.

Customer CustomerGroupA YB Z

Customer Source Activ From Activ To CustomerGroup

A S1 20020101 20020331 XB S1 20020101 99991231 ZA S1 20020401 99991231 Y

Customer CustomerGroupA XB ZA Y

Data MartMaster Data Table

DWH Object

Sourcesystem S11st Load: 20020101

after 2nd Load: 20020401

after 2nd Load: 20020401

2nd Load: 20020401

Illustration 11: Example - BW data warehouse: Historization of master data With master data, the source system normally does not deliver versioned or historical data images. Furthermore full-loads are often provided for master data. If the source system does not provide any historization, the BW staging processes must provide this information. The management of activity time slots is supported by BW for data warehouse InfoObjects automatically. For data warehouse ODS objects activity time slots must be maintained during staging. It is also the task of the BW with full loads to determine the changed records. This is done using special staging ODS objects. These tasks are omitted when SAP Master Data Management (SAP MDM) historicized Delta images are provided. Handling transaction data is easier, as long as the transaction is not to be posted again. With changeable transactions a simple time stamp is often not enough to prevent it from being overwritten.



Customer Dimension Material DimensionA 4712

4713

Customer Material TimeAmount

Company Currency

A 4712 200211 100A 4713 200211 150

Time Dimension200211

Customer Time DocNo Pos MaterialAmount Local

Currency

Amount Company Currency

A 20021107-10am 1 10 4711 100 50A 20021107-3pm 1 10 4712 200 100A 20021107-4pm 2 10 4713 300 150

Customer Time DocNo Pos MaterialAmount Local

CurrencyA 20021107-10am 1 10 4711 100 New bookingA 20021107-3pm 1 10 4712 200 Correction bookingA 20021107-4pm 2 10 4713 300 New booking

Sourcesystem

BW Data Warehouse Layer

daily

weekly dailymonthly

other InfoCubeother InfoCube

BW Architected Data Mart Layer

InfoCube

Illustration 12: Example - BW data warehouse: Transaction data The quantity of data in a BW data warehouse layer mounts quickly. The BW/ SAP Archiving Development Kid (ADK) provides the prerequisites for the data warehouse layer to cope with the demand, without the memory space used escalating exorbitantly. If a large data volume is to be made available online, the BW supports the usage of nearline storage.



5.2.3 BW Extraction and Staging In order to guarantee the quality of data in the BW data warehouse layer, corporate-wide rules with respect to extraction and staging are important. First and foremost, these should avoid redundant extraction and transformation. Using BW as the basis for enterprise data warehousing requires that all data sources can be addressed. For this reason the BW offers a wide spectrum of support for extraction:

Predefined, enhanceable extractors for practically all SAP application data The possibility to generate extractors for customer-specific SAP applications like COPA Generic extractors for customer-specific enhancements of mySAP applications Direct extraction from databases supported by SAP (BW DB-Connect Feature), based on table- or view-definitions Flat file extraction Integration with Ascentials Datastage BW 3.5 it is scheduled to support extraction from sources that can be addressed using JDBC or ODBO .

Most extractors for SAP application transaction data are delta-enabled. This means that transactions can be written to a delta queue at the time of posting. They are then extracted from this delta queue into the BW. It is evident that as well as delta-enabling, important design possibilities for BW scenarios are also derived from this, as master data information like price, for example, can also be saved at the time of posting. Extracted data is the first status at which it is relevant to save data persistently. This raw data is saved in the persistent staging area (PSA) on a 1:1 basis. The PSA forms a backup for newly designed staging processes.

StagingStagingArea:Area:PSAPSA


BW Data BW Data WarehouseWarehouse

Extr

actio

n /

Ope

n St

agin

g

BW ODSBW ODSany

sou

rce

StagingStagingArea:Area:PSAPSA


BW Data BW Data WarehouseWarehouse

Extr

actio

n /

Ope

n St

agin

gEx

trac

tion

/ O

pen

Stag

ing

BW ODSBW ODSany

sou

rce

Illustration 13: BW staging Whether and which data statuses between raw and integrated data is saved persistently has to be decided on a case by case basis. This decision depends on the quality of data and the outlay for potential data harmonization. BW ODS objects are used as data stores.



5.2.4 BW Support for Operational / Real Time Reporting Staging, data warehouse and data mart layers constitute classic data warehousing. Classic in the sense that data is subjected to an exact and predefined procedure in order to guarantee the quality and integrity of reporting and analysis. Classic data warehousing also means the ability to take the burden away from the operative systems.

The classic data warehousing approach: Dedicated load and staging processes High quality Optimized retrieval structures Reduce workload on OLTP systems

DataMart

Finance


DataMart

Logistic Tactic

al / St

rateg

ic

MasterReference

Data

TransactionData


Extra

ction /

Ope

n Stag

ing

DataMart

Finance

DataMart

Finance


DataMart

Logistic

DataMart

Logistic Tactic

al / St

rateg

ic

MasterReference

Data

TransactionData


MasterReference

Data

TransactionData


Extra

ction /

Ope

n Stag

ingEx

tractio

n / O

pen S

taging

Illustration 14: Classic data warehousing Alongside the classic layers (staging, data warehouse and data mart) an additional data area is often postulated: The Operational Data Store (ODS). The operational data store supports operative reporting, whereas data marts are data structures for tactical and strategic reporting and analysis. In practice, if operative reporting does not demand real-time data, the physical division between the data mart and the ODS becomes artificial. Mostly loading data two or three times daily is sufficient for operative reporting. Updating data on a daily basis may even be enough. The greater the latency (the time difference between the operative event and the acquisition of this event for reporting purposes), the more sensible it is to store granular data in data mart InfoCubes too, and to use this both for tactical analysis and operative reporting. Flat BW ODS objects are better suited to purely listing data. In this way, most BW implementations already support operative reporting and so relieve the pressure on operative systems.



In most of the cases operational reporting needs can be supportedin BW using the classic data warehouse process

DataMart

Finance

ArchitectedArchitectedData Marts/ Data Marts/ ODSODS--ObjectsObjects

DataMart

Logistic

Tactic

al / St

rateg

ic / Op

eration

al

MasterReference

DataTransaction

Data


Extra

ction /

Ope

n Stag

ing

ODS-ObjectSales Orders

..most of the cases..: classical data warehousing has its limits if we want to provide event-level or close to event-level operational reporting

(near real time reporting) and/ or if we are confronted with huge data volumes

In most of the cases operational reporting needs can be supportedin BW using the classic data warehouse process

DataMart

Finance


DataMart

Logistic

Tactic

al / St

rateg

ic / Op

eration

al

MasterReference

DataTransaction

Data


Extra

ction /

Ope

n Stag

ing


DataMart

Finance

DataMart

Finance


DataMart

Logistic

DataMart

Logistic

Tactic

al / St

rateg

ic / Op

eration

al

MasterReference

DataTransaction

Data


MasterReference

DataTransaction

Data


Extra

ction /

Ope

n Stag

ingEx

tractio

n / O

pen S

taging


..most of the cases..: classical data warehousing has its limits if we want to provide event-level or close to event-level operational reporting

(near real time reporting) and/ or if we are confronted with huge data volumes

Illustration 15: Classic data warehousing supports operatives reporting BW supports this tactical analysis and operational reporting on the same InfoCube through pre-calculated aggregates, aggregation awareness in the OLAP Engine, and query catching. However it is to be pointed out that the utilization profile of data for operative usage is different in regard to both its history and its granularity to data for analytical usage. With large volumes of transaction data this has to be accounted for in the schema modeling. With delayed operative reporting, which follows the path of classic data warehousing, there is no need to define a dedicated BW operational data store. You do then have to be aware of the possible consequences of basing analysis and operative reporting on the same structures. One the other hand, if data is requested close to the event time in the operative system, the classic pull principle of the BW data warehousing boundaries are set. In this case we talk about real-time or near real time operational reporting. The data extracted (in real time) is written straight to ODS objects, without going through the data warehouse layer. These objects form the BW operational data store.




Near Real Time Data Warehousing (Apollo) BW Operational Data Store

Finance

Logistic Open

Distr

ibution




MasterReference

Data


ODSODS



Extra

ction /

Ope

n Stag

ing



any ta

rget

Ext

ende

d St

ar S

chem

asE

xten

ded

Star

Sch

emas


MetaMeta DataData

any s

ource Transaction

Data


Illustration 16: BW operational data store: Near real time data warehousing Data from an operational data store is not to be processed directly in the data mart layer for reasons of data consistency. If data from an operational data store is needed in a data mart scenario, you should always go through the (enterprise) data warehouse. To summarize, the functionalities planned and offered by BW to support real time reporting scenarios are: BW Remote InfoCubes external InfoCube fact tables

BW allows you to define InfoCubes whose fact tables are themselves not persistent in the BW. The data is then read by an extractor during the query runtime and subjected to normal InfoSource treatment as far as transfer and update rules are concerned. It can then be made available to the OLAP processor. All data sources that can be addressed by BW can make external fact tables available in this way.

BW Virtual InfoCubes a program that behaves like a fact table BW near real time data warehousing (next main BW release)

With the next main BW release it will be possible to push data into BW ODS objects directly after either its emergence or the change event.

Direct reporting to operative structures (next main BW release) Reporting directly to operative structures will also be an option if integration with other data is not required. However the burden produced (lack of data integration and data quality controls) means tight limitations are set for this option.




BW and Operational/ Real Time Reporting

Key-Questions:

Data Latency ? Data Integration ?Workload on OLTP Systems ?

Ul

BWBW

OLTP

BW Data IntegrationBW Data Acquisition

BW Near real timedata warehousing

BW classicdata warehousing

BW remoteInfoCube

BW virtualInfoCube

ApolloApollo

Adaptors

JDBC ODBO XMLA SAP Query

Illustration 17: BW and operational / real time reporting .



5.3 Data Architecture - BW Data Model The BW layer architecture describes the life-cycle of data from the event through to the retrieval-relevant data mart or flat ODS object structure. One thing is intuitively clear: persistent stores and their storage schemata alone are not sufficient to ensure controlled redundancy. This requires a data warehouse data model.

5.3.1 Operative Data Models and Data Warehouse Data Model The operative applications entity relationship model serves as the basis for a data warehouse data model. Ideal examples of an business-wide, operational data model are scarce. The challenge then is to derive a consistent data warehouse data model from a multitude of operative models. The following graphic illustrates this:









Illustration 18: Operational data model and data warehouse data model Generating a business-wide data warehouse data model based on operative data models, which are not harmonized, is a complex and political procedure. This is why the data warehouse data model architecture is often neglected. The result is isolated applications or stovepipe data marts that are not integrated. In this regard BW customers finds themselves in a good position. BW offers an extensible data warehouse data model that represents the operative models of the SAP applications and a large number of industry-specific solutions consistently as part of the BW Business Content. Data models also exist for specific applications from the competitive environment (Oracle Financials, for example). BW customers with a high operative degree of coverage from SAP applications can enjoy being in the situation illustrated in the following graphic:



N o t a lig n e d o p e ra tio n a l

d a ta m o d e ls S A P e n te rp ris e d a ta m o d e l

B W C o n s is te n t d a ta w a re h o u se d a ta m o d e l fo rS A P A p p lic a tio n s a n d o th e rs

lo n g te rm su cce ss

A d a ta w a re h o u s e A d a ta w a re h o u s e c o n s is te n c y c h a lle n g ec o n s is te n c y ch a lle n ge

N o t a lig n e d o p e ra tio n a l d a ta m o d e ls S A P e n te rp ris e d a ta m o d e l

B W C o n s is te n t d a ta w a re h o u se d a ta m o d e l fo rS A P A p p lic a tio n s a n d o th e rs

lo n g te rm su cce ss

A d a ta w a re h o u s e A d a ta w a re h o u s e c o n s is te n c y c h a lle n g ec o n s is te n c y ch a lle n ge

Illustration 19: Data warehouse data models and the situation for BW customers The BW Business Content data model watches over the incremental generation of InfoCube solutions and ensures that they match, often without the customer even having to be aware of this. This is the true value of BW Business Content from an architecture point of view, and one reason for the success of BW.

5.3.2 The BW Data Model Operative entities and attributes correspond to BW InfoObjects.

Enterprise Data Model:e.g. mySAP Component

Object Models

0MATERIAL

Customer Material Sales Person

Sales Transaction

Materialgroup Sales ORGMaterialType

0MATTYPE

0MATGR

BW Data Model0CUSTOMER

0DOCNO0SALESPERS

0AMOUNT

0SALESORG

Entities, Attributes ->BW InfoObjects

operative entities and attributes BW InfoObjects


Object Models

0MATERIAL


Sales Transaction


CustomerCustomer Material Sales Person

Sales Transaction


0MATTYPE

0MATGR

BW Data Model0CUSTOMER

0DOCNO0SALESPERS

0AMOUNT

0SALESORG

Entities, Attributes ->BW InfoObjects

operative entities and attributes BW InfoObjects

Illustration 20: Enterprise data model and enhanceable BW data model: InfoObjects



Alongside technical properties like format and length, InfoObjects carry data warehousing-relevant properties:

Aggregation behavior (key figures) Standard displays (key figures) Textual description Language-dependant descriptions External hierarchies

Clustering operative entities and attributes to subject areas takes place through InfoSources. Master data and transaction data InfoSources are differentiated:


Object ModelsEnterprise Data Model:e.g. mySAP Component

Object Models

BWData Model defined by

Business ContentBW

Data Model defined by Business Content


Sales Transaction



Sales Transaction


BCT InfoSources Subject Areas

Master Data Transaction Data0MATERIAL 0MATGR 0MATTYPE0MATERIAL 0MATGR 0MATTYPE 0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT0DOCNO 0SALESDAT 0CUSTOMER 0SALESPERS 0MATERIAL 0AMOUNT

Clustering operative entities and attributes Data Warehouse subject-areasBW InfoSources built of InfoObjects

Illustration 21: BW data model: InfoSources and subject areas InfoSources always represent a set of InfoObjects and describe relations between these. InfoSources are the central bridge between the operative and the BW data model. In the operative system they correspond in form to a DataSource that, in turn, represents an

extractors result structure. In the BW data mart layer an InfoSource either defines a conformed dimension (master data

InfoSources), or a relation between conformed dimensions (transaction data InfoSources), in other words, the basis for InfoCube schemata.

The following graphic illustrates the role of BW InfoSources with regard to the data mart layer data model:





Object Models

0MATERIAL0MATGR0MATTYPE......

BW Data Mart Layer Data Model defined by

Business ContentBW Data Mart Layer



Sales Transaction



Sales Transaction



0SALESPERS0SALESORG.....

0CUSTOMER00CUSTGR......

0CUSTOMER 0MATERIAL 0AMOUNT

Shared/ Conformed DimensionsShared/ Conformed Dimensions InfoCubes with Local DimensionsInfoCubes with Local Dimensions

BCT Extractors/ DataSource


BW Extended Star SchemasBW Extended Star Schemas

Illustration 22: BW data model: data mart layer data model In the BW (enterprise) data warehouse a master data InfoSource and additional time-slice or

time-stamp attributes defines a data warehouse ODS object, which is able to store different versions of a subject area, which occur over time. Time-slices or time-stamps for master data are normally not offered by the source systems. This information has to be provided in the transfer or update rules during staging. As a rule of thumb a transactional-InfoSources define the structure of a data warehouse ODS object for transactional data (see following graphic):





Object Models

BW Data Warehouse Layer Data Model defined by

Business ContentBW Data Warehouse Layer



Sales Transaction

Material group Sales ORGMaterial Type


Sales Transaction

Material group Sales ORGMaterial Type


BCT Extractors/ DataSource


+ Insert Timestamp/ Activity Time Slice

+ Source

EDW InfoObject/ ODS-Object ODS-Object

If Overwrite:Unique Identifier

+Source

Illustration 23: BW data model: Data warehouse layer data model BW Business Content offers a pre-defined extensible data model for SAP components based on InfoObjects and InfoSources. This data model describes the BW data mart layer and also extends the BW data warehouse layer to include time slices. If the Business Content data model for mySAP components is used throughout, data mart InfoCube scenarios for a particular themed area can be built without producing isolated applications.



5.4 Topologies BW Landscapes The topology aspects of a BW architecture often interest people most. Some reasons for this are:

The short-term need to optimize already existing BW landscapes The parallel implementation of BW and R/3 Or simply a short-term focus on factors pertaining to costs.

However, the discussion in the previous chapters has made it clear that focusing on landscape aspects alone is superficial and in no way productive if strategies regarding the persistent layer and its data models are not also developed. If this is now clear, we can pose the question: What does the ideal BW landscape for my enterprise look like?

LocalBW

LocalBW

LocalBW

GlobalBW



LocalBW

LocalBW

LocalBW

GlobalBW



EnterpriseBW



SpokeBW

Others

SpokeBW

EnterpriseBW



SpokeBW

Others

SpokeBW

The Ideal Corporate BW Landscape ?

Strategic Corporate Implementation Options:Strategic Corporate Implementation Options:

Illustration 24: The ideal corporate BW landscape? Unfortunately we have to disappoint you: There is no one size fits all BW landscape Naturally there are solutions that are to be favored in terms of consistence, but enterprise-specific circumstances often prevent such solutions. Decisions on BW landscape architecture and ultimately on all aspects of the architecture have to be made amidst the following conflicts:




Enterprise-wide Strategies

Inside-Out versus Outside-In Central Data Hub, Local Spokes Local Hubs, central aggregation

Centralized versus Decentralized Low dispersal, high centralization High dispersal, low centralization High dispersal, high centralization

Global Integrations versus Local Responsiveness Global scale Global standardization Global demand Local customer needs Local market conditions Local governmental regulations

Select the [IT] approach thatfits most closely with corporate strategy

Prof. Sethi, University of TexasInformation&Management, 2/01

Illustration 25: Enterprise-wide strategies and BW architecture Further parameters relevant to this decision are:

Master data integration status and strategy To what extent is an awareness of the essential roles of consistent data available? Budget Sponsorship IT Strategy - Operative System Landscape Competitive environment.

As a basic principle the following advice should be followed: Select the [IT] approach that fits most closely with corporate strategy (Prof. Sethi, University of Texas Information&Management, 2/01) In other words, a business-wide BW enterprise data warehousing strategy will hardly ever be successful when it contradicts the corporate culture.

5.4.1 Consistency in a distributed BW Landscape The corporate BW landscape will often consist of several BW instances and possibly also external data warehouse tools. Alongside general decisions that ensure consistency within and between BW layers, rules must also be defined to ensure the consistency of data and metadata in a distributed data warehouse landscape. BW supports distributed (BW) landscapes with a multitude of functionalities: Data consistency

The distribution of data in a landscape like this has to be controlled (delta handling). This is guaranteed by BW functionalities:



o Data Mart Interface (BW-BW data distribution) o Open Hub Service (BW-Third Party data distribution)

Metadata consistency This is realized and controlled using the BW development environment and the BW metadata interface (XML for 3rd-party)

Daily Operations Partnerships with producers of automization tools for data centers guarantee that the data distribution process (BW process chains) can be controlled and scheduled from one central point.

These functionalities are the same whatever BW landscape architecture you choose.

5.4.2 BW Landscapes und Technical Parameters Diverse technical parameters also influence what BW landscape is suitable for your enterprise. Geographical distribution of users

In the most extreme cases users can be spread around the whole world. This graphic illustrates challenges that then arise:


Availability and Maintenance

Availability and Maintenance 24 x 7 Access

Performance impacts

Data actuality

Data backup, repair and recovery

System upgrades and maintenance

Zero Downtime B r u s s e l s L . A . T o k y o R e p o r t i n g T i m e

1 2 a m 3 p m 8 a m

2 a m 5 p m 1 0 a m

4 a m 7 p m 1 2 p m

6 a m 9 p m 2 p m

8 a m 1 1 p m 4 p m

1 0 a m 1 a m 6 p m

1 2 p m 3 a m 8 p m

2 p m 5 a m 1 0 p m

4 p m 7 a m 1 2 a m

6 p m 9 a m 2 a m

8 p m 1 1 a m 4 a m

1 0 p m 1 p m 6 a m

B r u s s e l s L . A . T o k y o R e p o r t i n g T i m e

1 2 a m 3 p m 8 a m

2 a m 5 p m 1 0 a m

4 a m 7 p m 1 2 p m

6 a m 9 p m 2 p m

8 a m 1 1 p m 4 p m

1 0 a m 1 a m 6 p m

1 2 p m 3 a m 8 p m

2 p m 5 a m 1 0 p m

4 p m 7 a m 1 2 a m

6 p m 9 a m 2 a m

8 p m 1 1 a m 4 a m

1 0 p m 1 p m 6 a m

B r u s s e l s R e p o r t i n gF r 4 p m

8 p m

S a 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

S u 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

M o 1 2 a m

4 a m

8 a m

1 2 p m

B r u s s e l s R e p o r t i n gF r 4 p m

8 p m

S a 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

S u 1 2 a m

4 a m

8 a m

1 2 p m

4 p m

8 p m

M o 1 2 a m

4 a m

8 a m

1 2 p m

Illustration 26: BW landscape and availability / maintenance Code pages in different languages have to be supported

BW is Unicode-compatible. Concrete solution scenarios have to be agreed with BW product management.



5.4.3 Inside-Out Landscape Architecture - BW as Central Enterprise Data Warehouse It is indisputable that business-wide, consistent information can soonest be guaranteed where there is a central BW instance offering data warehouse layer services. This means the business-wide consolidation of all relevant data in a granular, integrated, non-volatile, unflavored and subject-oriented form. William H. Inmon forwards this solution in the corporate information factory.


BW as Central Enterprise Data Warehouse (EDW)

BW Enterprise Data Warehouse:BW Data Warehouse layer as single point of truth for the entire corporation

EnterpriseBW



SpokeBW

SpokeBW

Illustration 27: BW as Corporate Information Factory All data that acts as the basis for tactical and strategic data marts has to go through the enterprise data warehouse. Extraction straight into data marts, ignoring the enterprise data warehouse, is not permitted. Exceptions have to be controlled. There must be a strategy in place for using these data marts as the basis for operative reporting, as well as for using the enterprise data warehouse itself for operative reporting. A landscape of this sort with the inherent demands

single point of truth and extract once, deploy many

offers the ideal prerequisites for controlling redundancy at corporate level. It is often assumed that this solution, although ideal, is practically unrealizable. Without going into the arguments in detail, it is sufficient to say that this is not necessarily the case for BW customers.



Many large companies that operate globally make a large investment in order to unify their operative landscapes by implementing SAP R/3. As a result, the ideal prerequisites for a central enterprise data warehouse approach emerge: integration of data (common coding), integration of data models into the SAP R/3 data model, the building of an operative R/3 landscape, which takes the company organizational and technical conditions into account. In other words, a centralized approach in the operative environment leads directly to a solution option for the enterprise data warehouse topology. This type of ideal environment is often found in single division enterprises that find themselves in strongly competitive environments and as a result, have a high sense of awareness for the usage of more reliable and efficient IT solutions. This awareness mostly involves a high degree of support from the highest management level for a BW enterprise data warehouse solution.


Inside-Out: Central BW Instance Covers All

BW EDWBW EDW

BW Architected Data MartsBW Architected Data Marts

tactical/operational like

reporting

strategicreportingintegrated

non-volatileno flavor

Inside-Out:Central EDW BW Instance Covers All

rr

PSAPSA

Stagin

g Area

EnterpriseBW



SpokeBW

SpokeBW

BW ODSBW ODSoperational/ near real timereportingr

ExternalExternal Data MartsData Marts

aggregatedless granulagranula

granula

Illustration 28: Scalability of a BW Corporate Information Factory I A BW enterprise data warehouse based landscape will be implemented incrementally for cost reasons. Often, first a BW instance will be built that offers data warehouse layer as well as data mart layer services, including operative-level reporting. Obviously the aim of the enterprise data warehousing strategy also has to be to source all external data marts with data from the BW enterprise data warehouse only! It is then possible to separate reporting and analysis services as the load increases.




Inside-Out: Central EDW BW Instance and BW Spoke Instances

EnterpriseBW



SpokeBW

SpokeBW

BW EDWBW EDW

integratedconsistent

Inside-Out:Central EDW BW Instance as Information HUBArchitected Data Marts Spoke Instances

r

strategicreporting

Corporate BWCorporate BWData Mart Data Mart

r

PSAPSA

Stagin

g Area tactical/operational like

reporting

BW ODSBW ODSoperational reportingr

BW OBW Ooperational reportinggranular

PSAPSA

BW Architected Data MartsBW Architected Data Martstactical/

operational likereporting

r

ri

BW ODSBW ODSoperational/ near real timereportingr

ExternalExternal Data MartsData Marts

granulaagg egated

granula

DSDS

less granula

aggegatongranula

Illustration 29: Scalability of a BW Corporate Information Factory II To summarize, the conditions for a central BW enterprise data warehouse solution are always most favorable within a single division company with strong headquarter.

5.4.4 Outside-In Landscape Architecture - Decentralized BW Data Warehouses

Even where it is preferable to have a single BW enterprise data warehouse for the whole enterprise, the complexity of the different business areas in a company that operates globally can lead to shared BW landscapes. On this issue William H. Inmon says:

Simply building a central single data warehouse does not address the size and complexity of the problem posed by the need for integrated, historical easily accessible data across a complex global enterprise. W.H. Inmon Managing Multiple Data Warehouse Development

Often political circumstances or a companys lack of an over-arching concept lead to shared BW landscapes too, even when the business conditions seem to be favorable.

Enterprises that choose diversification should generally check that the outlay involved in integrating data on a granular level, as required with the enterprise data warehouse approach, is not greater than the benefit.

Even when the ideal prerequisites exist such as for a single division enterprise, certain conditions (a regional enterprise structure, for example, without process integration between the regions), coupled with technical and political factors, can lead to an enterprise moving away from the central BW enterprise data warehouse concept. This leads to a landscape that comprises several local BWs. Local here means that one of these BWs covers only one part of the organization. The description of what local can mean in individual cases depends on the size and the complexity of the company.



Each local BW should operate like an enterprise data warehouse for its own area. In this way, the sum of the local enterprise data warehouses forms a quasi virtual enterprise data warehouse. Unlike the central enterprise data warehouse that requires all corporate data to be integrated at granular data level, this is not necessary with local enterprise data warehouses, although it is still preferable. Each local enterprise data warehouse necessarily ensures a local granular data integration. Having a landscape architecture based on local BWs means that for over-arching reporting and analysis, at least one additional integration layer is required to consolidate the data from the local BWs. A BW that consolidates the data from other BWs and offers this in an integrated form is called a global BW. Again, the term global, and the scope of a particular global BW can only be defined on an individual, company-specific basis.


Architected Multiple BW Landscape

DivisionalGlobal

BWBW Europe

LocalRegional Data Warehouse

ODSData Marts

Global BW Global BW Division AA

Global BWGlobal BWHeadquarter

R/3-ERPAmericas I

R/3-ERPAsia

R/3-ERPEurope I

mySAPCRM

Staging

BWBW Americas

Data WarehouseODSData Marts

Staging

R/3-ERPAmericas II

BWBW Asia


Staging

virtualBW Enterprise

Data WarehouseStaging

Illustration 30: Local - Global BW Landscape In terms of an enterprises data integration strategy, it is important that master data harmonization takes place on an aggregated level (global BW)1. In these terms a global BW of this sort is always an integrator.

1 Integration is then only demanded at the higher hierarchy levels of a subject area or dimension. This is generally easier to achieve than integration at granular level. As a result, a topology of this kind often mirrors the integration strategy of an enterprise that does not choose granular common coding.



The situation of local BWs or legacy systems with respect to harmonization determines the integration efforts at global BW site: 1. Local BWs are to be integrated

I. It is the task of the global BW to gather data from local BWs (Data values are already common coded, local-global data models are integrated, a data model architecture exists)

II. It is the task of the global BW to consolidate and harmonize data (mapping) (Data values are not common coded, local-global data models are integrated, data model architecture exists)

III. It is the task of the global BW to consolidate and harmonize data (mapping) and to integrate the local-global data models. (A data model architecture either does not exist, or is not controlled. Statements about model consistency cannot be made without doing individual checks.)

2. Local BWs and legacy systems are to be integrated I. For legacy systems data and data model integration is necessary.

These scenarios copy the different corporate data warehousing strategies. The essential difference lies in whether local and global BWs are developed according to common architecture guidelines (scenario 1.I and 1.II). If this is the case we can talk about an architected outside-in BW landscape. This harmonized, architecture-based interaction of local and global BW instances is a serious option for building a BW based corporate data warehousing strategy from the very beginning. It is supported by the central development of BW templates for local reporting and analysis.


Architected Multiple BW Landscape: BW Template Roll-Out

DivisionalGlobal

BWBW Europe

LocalRegional Data Warehouse

ODSData Marts

Global BWGlobal BWHeadquarter

R/3-ERPAmericas I

R/3-ERPAsia

R/3-ERPEurope I

mySAPCRM

Staging

BWBW Americas


Staging

R/3-ERPAmericas II

BWBW Asia


Staging

virtualBW Enterprise

Data WarehouseStaging

Architected Multiple BW Landscape: BW Template Roll-Out

R/3 templateroll-out

BW templateroll-out

Global BW Global BW Division AA

Illustration 31: Architected Multiple BW Landscape and BW Template roll-out



However, if dealing with a mature BW landscape, you will probably echo scenario 1.III and scenario 2. There is obviously only a restricted corporate strategy for local BWs and it is the task of the global BW to take the first step towards a corporate strategy (catch the BWs scenario).

BWlocalBWlocal

ERP/ Legacy

othersothers

BW GlobalBW Global

ERP/ Legacy Legacy/ Files

BWlocalBWlocal

Focus on Integration andHeadquarter Reporting Global BW as

the Corporate Integrator

BWlocalBWlocal

ERP/ LegacyERP/ Legacy

othersothers

BW GlobalBW Global

ERP/ LegacyERP/ Legacy Legacy/ FilesLegacy/ Files

BWlocalBWlocal

Focus on Integration andHeadquarter Reporting Global BW as

the Corporate Integrator

Illustration 32: Global BW as corporate Integrator



6 Central BW Application Development As well as the need for consistent information on all organizational levels, avoiding redundant application development (data marts) is also a motivation for a business-wide BW strategy.

Redundant application development poses a constant threat to data consistency. Redundant application development is often accompanied by a certain arbitrariness in the management of persistent data stores and BW data models.

The general solution approach is to develop applications or templates for all the organization units of an enterprise centrally.

To what extent applications are generated on a local level (local-local applications) and/ or corporate templates are adaptable locally, is always company-specific. It is symptomatic that this decision is influenced more by political than functional factors.

Two extreme positions stand out regarding the local management of templates:

Stable templates

No local changes to the central template are permitted

Example templates

The centrally produced template acts as an example only.

The first is self-evident: The fewer changes are allowed to the central template, the easier the central further development and maintenance of these templates, and the better it is for overall consistency in the BW.

The description of this scenario sounds very rigid. This is not entirely true as it can work very well if local power users are allowed to g

Documents

Enterprise Data Warehousing With Sap Bw - Overview