DATA WAREHOUSING

PRESENTED BY:

Mrs. SUNNY TALWAR

CONTENTS

WHAT IS DATA WAREHOUSING.

PURPOSE OF DATA WAREHOUSING.

DATA WAREHOUSE COMPONENTS.

DATA WAREHOUSE FUNCTIONALITY.

EVOLUTION ARCHITECTURE OF DATA WAREHOUSE.

COMPLEXITIES OF CREATING A DATA WAREHOUSE.

SUCCESS & FUTURE OF DATA WAREHOUSE.

DATA WAREHOUSE PITFALLS.

Data Warehouse A collection of corporate information, derived directly from operational systems and some external data sources. Its specific purpose is to support business decisions, not business operations.

DEFINITION

In order to manage millions of accounts , the bank has to maintain loads of data. For implementing any new business idea the bank has to analyze the data at different locations and base its decision on its data and hence the data should be in a format that allows the business analysts to have clarity of the account-holders and their transactions.

EXAMPLE OF DATA WAREHOUSING

Consider a Multi Location Banking System.

THE PURPOSE OF DATA WAREHOUSING

Realize the value of data Data / information is an asset Methods to realize the value, (Reporting,

Analysis, etc.)

Make better decisions Turn data into information Create competitive advantage Methods to support the decision making

process.

Architecture

CREATING A DATA WAREHOUSE

Capacity

integrateIntegration

PhysicalOrganizaztion

Schemaand View

Sources

End – UserApplication

Scripts

Meta Data

Data Warehouse Components

• Staging Area• A preparatory repository where

transaction data can be transformed for use in the data warehouse

• Data Mart • Traditional dimensionally modeled set of

dimension and fact tables• Per Kimball, a data warehouse is the union

of a set of data marts • Operational Data Store (ODS)

• Modeled to support near real-time reporting needs.

DATA WAREHOUSE FUNCTIONALITY

Data Warehouse Engine

Optimized LoaderExtractionCleansing

AnalyzeQuery

Metadata Repository

RelationalDatabases

LegacyData

Purchased Data

ERPSystems

VERY LARGE DATA BASES

Terabytes -- 10^12 bytes:

Petabytes -- 10^15 bytes:

Exabytes -- 10^18 bytes:

Zettabytes -- 10^21 bytes:

Zottabytes -- 10^24 bytes:

Wal-Mart -- 24 Terabytes

Geographic Information Systems

National Medical Records

Weather images

Intelligence Agency Videos

WAREHOUSES ARE VERY LARGE DATABASES

EVOLUTION ARCHITECTURE OF DATA WAREHOUSE

Top-Down Architecture

Bottom-Up Architecture

Enterprise Data Mart Architecture

Data Stage/Data Mart Architecture

GO TO DIAGRAM

GO TO DIAGRAM

GO TO DIAGRAM

GO TO DIAGRAM

BACK TO ARCHITECTURE

Top-Down Architecture

BACK TO ARCHITECTURE

Bottom-Up Architecture

Enterprise Data Mart Architecture

BACK TO ARCHITECTURE

Data Stage/Data Mart Architecture

BACK TO ARCHITECTURE

COMPLEXITIES OF CREATING A DATA WAREHOUSE

Incomplete errors Missing FieldsRecords or Fields That, by Design, are

not Being Recorded

Incorrect errorsWrong Calculations, AggregationsDuplicate RecordsWrong Information Entered into Source

System

SUCCESS & FUTURE OF DATA WAREHOUSE

The Data Warehouse has successfully supported

the increased needs of the State over the past

eight years.

The need for growth continues however, as the

desire for more integrated data increases.

The Data Warehouse has software and tools in

place to provide the functionality needed to

support new enterprise Data Warehouse projects.

The future capabilities of the Data Warehouse can

be expanded to include other programs and

agencies.

DATA WAREHOUSE PITFALLS

You are going to spend much time extracting, cleaning, and loading data

You are going to find problems with systems feeding the data warehouse

You will find the need to store/validate data not being captured/validated by any existing system

Large scale data warehousing can become an exercise in data homogenizing

DATA WAREHOUSE PITFALL

The time it takes to load the warehouse will expand to the amount of the time in the available window... and then some

You are building a HIGH maintenance system You will fail if you concentrate on resource

optimization to the neglect of project, data, and customer management issues and an understanding of what adds value to the customer

BEST PRACTICES

Complete requirements and design

Prototyping is key to business understanding

Utilizing proper aggregations and detailed

data

Training is an on-going process

Build data integrity checks into your system.

Thank You