DATA WAREHOUSE

Data Warehousing • DW: Pool of data produced to support decision making.• Structured to be available in ready to use form• Subject Oriented • Integrated • Time-variant• Nonvolatile• Additional characteristics like

1.Web based2.Relational/multidimensional3.Client/Server4.Real time(recent trends)5.Include metadata

Types of Data warehouseDATA Mart

• Dependent

– Created from warehouse

– Replicated • Functional subset of warehouse

• Independent

– Scaled down, less expensive version of data warehouse

– Designed for a department or SBU

– Organization may have multiple data marts• Difficult to integrate

• Operational DATA Stores: Provides a fairly recent form of customer information file(CIF)

• Enterprise DATA Warehouses: Used across the enterprise for decision support

• METADATA: Describes the structure of and meaning about data, contributing to their effective use.

Data warehousing process overview

Major components

• Data sources

• Data extraction

• Data loading

• Comprehensive database

• Metadata

• Middleware tools

Data Warehousing Architectures • May have one or more tiers

– Determined by warehouse, data acquisition (back end), and client (front end)

• One tier, where all run on same platform, is rare

• Two tier usually combines DSS engine (client) with warehouse– More economical

• Three tier separates these functional parts

Data Integration, Extraction And Load process

1.DATA INTEGRATION

Comprises three major processes

• Data Access: ability to access & extract data from any data source

• Data federation: Integration of business views across multiple data store

• Change capture: Based on the identification, capture, and delivery of the changes made to enterprise data source.

2.Extraction, Transformation And Load(ETL)

• Is an integral component in any data-centric project.

• ETL consists:

Extraction-From all relevant sources

Transformation-Converting extracted data in the form so it can place in data warehouse or another database

Load-Putting the data in the data warehouse.

ETL Process

Transient Data

source DataWarehouse

DataMart

Packagedapplication

Legacysystem

Extract

Other Internal

applications

Transform Cleanse Load

Benefits of Data Warehouse

• Allows extensive analysis in numerous ways.

• A consolidated view of corporate data.

• Better and more timely information.

• Enhance system performance.

• Simplification of data access.

• Enhance business knowledge, enhance customer service and satisfaction, facilitate decision making.

Data Warehouse development Approaches

The Inmon Model: The EDW Approach

• Emphasizes top-down development

• Employing established database development methodologies and tools

The Kimball Model: The Data Mart Approach

• Plan big, build small

• Subject oriented or department oriented

• Focus on the requests of a specific department.

Successful Implementation of Data warehouse

• Establishment of service-level agreements and data-refresh requirements.

• Identification of data sources and their governance policies.

• Data quality planning & model designing.

• ETL tool selection.

• Relational database software and platform selection.

• Data transport and data conversion.

• Reconciliation process

• End-user support

Real-Time Data warehousing

• Also knows as active data warehousing.

• Process of loading & providing data via the data warehouse.

• Evolved from EDW (Enterprise Data Warehousing)

concept.

• Allows information-based decision making at finger tips.

• Positively affect almost all aspects of customer service, SCM, logistics.