Enterprise Architectures Data Warehousing Business ... · Management Timo Itälä, Paavo Kotinurmi, Matti Hämäläinen 2 SoberIT Software Business and Engineering Institute HELSINKI

1

SoberITSoftware Business and Engineering Institute

HELSINKI UNIVERSITY OF TECHNOLOGY

Enterprise ArchitecturesData Warehousing

Business IntelligenceCustomer Relationship

Management

Timo Itälä, Paavo Kotinurmi, Matti Hämäläinen

2



Timo Itälä

Course Map

13.9: Enterprise Architectures Overview

20.9: ERP and PDM systems

27.9: BI and Data Warehousing

4.10: BPM and SOA

11.10: Content Management Systems

18.10: Enterprise Architecture Summary

3



Timo Itälä

Lesson 3: Data Warehousing

Enterprise Information Cycle

Data, Information, Decisions, Intelligence

Reporting with operational systems

Data Warehouse concept

Data Warehouse architectures

Customer focused retailing

Performance dashboards and scorecards

Customer Relationship Management

4



Timo Itälä

Enterprise Information Cycle

Set Mission, Goals

Prepare Plans, Set Objcetives, Set Measures

Execute the plan

Follow-up

Analyse, Try to understand

Learn

Repeat the above

5



Timo Itälä

Operational systems and reportingRoutine reports from operational systems

Predefined contentPredefined timesPredefined formatShort history

What makes good information?The right informationThe right timeThe right format

Example of an reporting system:53 different reports from an inventory control application

Processes

Data

Decision Support

Information

Dilemma with IT department:Process needs are different from decision support needsOne good answer leads to three new questions

6



Timo Itälä

Multiple operational systems

Multiple departments and their operational and reporting systems

Different conceptual modelDifferent data modelDifferent coding schemesDifferent reporting intervalsDifferent versions of truth

Very difficult to get a holistic view of the enterprise

7



Timo Itälä

Data Warehouse conceptUnbundling environments for operational systems and decision support systems

A Data Warehouse isIntegratedSubject-OrientedTime-VariantNonvolatileDatabase that provides support for decision making

Bill Inmon, ”the father of Data Warehouse”

Data Warehouse

8



Timo Itälä

Early innovators

Industries with mass marketsRetail

Banking

Telecommunications

Airlines

Characteristics of enterprisesHeavy competition

Sufficient size

Marketing and sales orientation

Data Warehouse

9



Timo Itälä

Early examples

RetailK-Mart Marketing: Balance between surplus and empty shelf

Wall-Mart Logistics: From push-to-store to pull-by-customers

TelecommunicationsAmeritech: Usage of the network, customer retention, competition follow-up

Banking and insuranceCustomer management

Risk management

Fraud detection

Data Warehouse

10



Timo Itälä

“Atomic cube”Atomic data

sales

by date

by product

by store

Dimensions

Measures

Days

Pro

du

cts

Store

s

11



Timo Itälä

Adding facts by dimensionsSummary

total sales

at store #4

today

Days

Pro

du

cts

Store

s

12



Timo Itälä

Using summaries in caluculationsAverage

sum sales

of product #5

at store #1

last month

Days

Pro

du

cts

Store

s

13



Timo Itälä

Star schema database

Dimensional model

Fact tableDimension keys

Measures

Dimension tablesDimension keys

Attributes

14



Timo Itälä

Using dimensional model

Dimension attributes are used as column headings

Facts are summarized

Sort order can change when needed

by city, by product

Subtotals when dimension value changes

Grand total in the end

15



Timo Itälä

OLAP On-Line Analytical ProcessingThe ”Cube paradigm”

Drill DownSlice and Dice

Why drop in Oulu?

What caused dress department drop in Oulu?

Why drop on week 35?

16



Timo Itälä

Atomic data; Granularity

Point-of-Sales:Customer purchase receipt row

POS data

17



Timo Itälä

Example: POS dataIndependent Data Marts

SalesPurchasesEmployees

…

Staging Area

Source systems

Data Warehouse

End user access and applications

Extract

Transform

Load

(ETL)

Dimensions:

Stores, Products, Customers, Time

Transaction Data

Summary Data

OLAP tools

18



Timo Itälä

Example: Market situation, ice cream manufacturer

Question of product marketing managerBest products?Best customers?Best regions?Best seasons?How are we doing against competition?How effective are our marketing campaigns?What is the effect of sunny weather?

Sales of ice creamby product, customer, region, time

from manufacturerInternal

Sales of ice creamby product, customer, region, time

from retail stores

Marketing of ice creamby product, media, region, time

from marketing department

External

Internal

Weather reports

19



Timo Itälä

Data Warehousing architectures

Independent data marts

Data mart bus architecture with linked dimensional data marts

Hub and spoke architecture

Centralized data warehouse

Federated architecture

20



Timo Itälä

Independent data martsIndependent Data Marts

SalesPurchasesEmployees…

Dimensional data model

Staging Area

Source systems

Data Mart


21



Timo Itälä

Data mart bus architecture with linked dimensional data marts (Kimball)

Process transactions

Raw material

Work in progress

Finished products

…l

Staging Area

Source systems

Data Marts with

confirmad dimensions


Facts and dimensions

22



Timo Itälä

Hub and spoke (Bill Inmon) Enterprise wide DW

Internal dataExternal data

…

Dependent data marts for specific user groups

Staging Area

Source systems

Data Warehouse

(normalized)


Dependent Data Marts

23



Timo Itälä

Centralized Data Warehouse

Enterprise wide DW

Internal data

External data

…

Direct Access to DW

Staging Area

Source systems

Data Warehouse

(normalized)


24



Timo Itälä

Federated architecture

Staging Area

Source systems

Data Warehouse

(normalized)


Dependent Data Marts

Logical/Physical

Integration

25



Timo Itälä

Architecture and methodology

ArchitectureComponent parts, their characteristics and relationships among the other parts

Hub and spoke vs. confirmed Data marts

MethdologyActivities

Sequencing of activities

Top-down vs. Bottom-up

26



Timo Itälä

Selection Factors

Information interdepence between organizationsUpper Management´s information needsUrgency of need for data warehouseNature of end user tasksConstraints on resourcesView of the data warehouse prior to implementationExpert influenceCompatibility with existing systemsThe perceived ability of the in-house IT staffSource of sponsorshipTechnical issues

27



Timo Itälä

Popularity

Source: Data Warehuse Architectures: Factors in the Selection Decision and the Success of the Architectures.

Hugh J. Watson, Terry College of Business, University of Georgia, Athens, Georgia 30602

Thilini Ariyachandra, College of Business, University of Cincinnati, Cincinnati, Ohio 45221

39% Hub and spoke

27% Data mart bus

17% Centralized data warehouse

13% Independent data marts

4% Federated architecture

Number of respondents: 454

28



Timo Itälä

Operational Data Store

DW

Operational Data Store(up-to-date current data)

Decisions based on current situation

29



Timo Itälä

Tools and products

Portal tools

ETL Tools

Data Base Management

System

Business Intelligence Tools:

Reporting

OLAP

Dashboards

Scorecards

ETL Tools

30



Timo Itälä

Extract-Transform-Load process

ETL Tools

Extract dataExtract from source databaseReceive messagesNew data after previous extract

TransformFrom source to target presentation

Load

ConsiderationsData Quality ProfilingData Quality CleansingScheduling

31



Timo Itälä

Metadata

Business metadata (concepts, measures, calculations…)Technical metadata (structures, transformations…)Operational metadata (tasks, schedules, logs,…)

32



Timo Itälä

Example: Retailers vital data

Sales and Product Data

Merchandising & BuyingSupplyChain

TradingLocation

Our Store

OurStore

Our Store

MarketingCustomer

Data

33



Timo Itälä

Customer Focused Marketing

Identify your customersWho they are Where they are What they are like

Capture shopping dataAnalyse basket dataIdentify common attributes

Plan targeted marketing and merchandising activities … … to change their behaviourMeasure the impact on the customers

34



Timo Itälä

The Vital Questions

Which customers come to the stores?

What are their shopping habits?

Are they right for the future?

How valuable are they today?

What is their lifetime value?

How do they react to marketing campaigns?

How can we target, attract and retain them?

35



Timo Itälä

The Future

Customer-centric data warehouses will become an essential tool for many retail organisations

Gradual move from analysis to modelling and predicting future customer purchasing behaviour

The winners will understand their customers

36



Timo Itälä

Evolution of a Solution

The search for the perfect “business insight system”:

1980s Executive information systems (EIS)

Decision support systems (DSS)

1990sData warehousing (DW)

Business intelligence (BI)

2000sCorporate performance management (CPM)

Performance dashboards

37



Performance Dashboards

Source: The Data Warehousing Institute (TDWI)

38



Timo Itälä

Two Metaphors

Dashboard Performance Chart

Performance Dashboard

39



Timo Itälä

Two Disciplines

+Business Intelligence

Performance Management

=

Performance Dashboards

Information

Knowledge

Plans

Act

Data Warehouses

Analytical Tools

Rules and Models

Review, Measure, Refine

Data

Events

Data

Wisdom

DATA REFINERY

STRATEGY

EXECUTION

1. Strategize 2. Plan

3. Monitor/Analyze

4. Act/Adjust

IntegratedData

Miss

ion,

Value

s, G

oals

Objec

tives

, Inc

entiv

es

Stra

tegy

Map

s

Budgets, Plans,

Forecasts,

Models

Initiatives,

Targets

BI/D

W

Perfo

rman

ce

Dashb

oard

sActions,

Decisions,

Revisions

40



Timo Itälä

Why Performance Dashboards?Resonate with users

Monitor status of several areas on one screenGraphical view of key metricsAlerts users to exception conditionsClick to analyze and drill to detailCustomized views based on rolePersonalized views based on interestNo training required

Rich data Blends data from multiple sourcesBoth detailed and aggregated Both historical and real-time

41



Timo Itälä

Why Performance Dashboards?

Aligns the businessEveryone uses the same data

Everyone uses the same metrics

Everyone works off the same objectives

Optimizes performance and complianceCloses gap between strategy and execution

Greater visibility into business

Makes processes more efficient

Makes workers more effective

42



Timo Itälä

Performance Dashboard

Three Layers of Information

Detailed, Operational DataDW queries, Operational reports

Summarized Dimensional DataDimensions, hierarchies, “slice/dice”

Graphical Abstracted DataGraphs, Symbols, Charts

Pla

nnin

gP

lans

, mod

els,

fore

cast

s, u

pdat

es

Monitoring

Analysis

Reporting

Collaboration

43



Timo Itälä

Dashboards vs Scorecards

SymbolsCharts Display

SummariesEventsData

Periodic snapshots“Right time” feedsUpdates

Executives, managers, staff Executives, managers, staffUsers

Charts progressMeasures current activityPurpose

ScorecardDashboard

Dashboards and scorecards are visual interfaces for monitoring business performance

44



Timo Itälä

Three Types

CollaborationAnalysisMonitoringEmphasis

“Dashboard”

Intra-day

Detailed

Operational

Supervisors+

Monitor operations

Operational

“Scorecard”

Monthly/Quarterly

Summary

Enterprise

Executives+

Execute strategy

Strategic

“BI Portal”“Looks like a…”

Daily/WeeklyUpdates

Detailed/SummaryInformation

DepartmentalScope

Managers+Users

Optimize processApplication Emphasis

Tactical

45



Timo Itälä

Tactical Dashboard CaseInternational Truck and Engine

$9.7 billion manufacturer of trucks, buses, diesel engines, and parts based in Illinois

Key business issues: Market reality: Global competition, new regulations, emerging markets

Goals: 1) $15b in revenues 2) reduced costs, 3) improved quality, 4) reduced risk

46



Timo Itälä

KBI Portal

Purpose Deliver actionable information to financial analysts

ScopeSpans 32 source systems across five divisions

130 key business indicators, updated daily

Supports 500 financial executives, managers, and analysts

UpshotBridges gulf between finance and operations

Replaces hodge-podge of paper reports

Saves analysts time creating custom reports

Shuts down dozens of reporting systems

47



Timo Itälä

Architecture

Source Systems

Star Schema Database

OLAP Cubes

Reporting Portal

Purpose

Staging Area Database

TransactionalRun the business

Gather all data in one place. Keep a copy for future reuse.

Integrate data for easy loading into OLAP cubes.

Store data dimensionally to support fast queries and easy navigation.

Data

Display key metrics so they can be viewed at a glance.

Transactional

Transactional & lightly summarized

Moderately summarized

Highly summarized

ETLTools

ETLTools

Web Server

Data Warehouse

48



Timo Itälä

Monitoring Layer

49



Timo Itälä

Detail Transaction Layer

‘Click’ any VIN to expandfull page order/build report

‘Click’ any VIN to expandfull page order/build report

50



Timo Itälä

Analysis Layer

51



Timo Itälä

BI Maturity Model

1. Prenatal 2. Infant 3. Child 4. Teenager 5. Adult 6. Sage

“Production Reporting”

“Spreadmarts”

“Data Marts”

“Data Warehouses

”

“Enterprise DW”

“Analytic Services”

GULF CHASM

Business Value Semantic Integration Data Consolidation

52



Timo Itälä

Customer relationship management

Customer relationship management (CRM) encompasses the capabilities, methodologies, and technologies that support an enterprise in managing customer relationships.(Source:Wikipedia)

CRM is a holistic change in an organisation's philosophy which places emphasis on the customer.

53



Timo Itälä

Application architecture of CRM

Operational - automation to the basic business processes (marketing, sales, service)Analytical - support to analyze customer behavior, implements business intelligence alike technologyCollaborative - ensures the contact with customers (phone, email, fax, web, sms, post, in person)Most successful analytical CRM projects take advantage of a data warehouse to provide suitable data.

54



Timo Itälä

Enterprise Architecture

And the journey continues..Business

Data

Applications

Technology

Questions?

Documents

Enterprise Architectures Data Warehousing Business ... · Management Timo Itälä, Paavo Kotinurmi, Matti Hämäläinen 2 SoberIT Software Business and Engineering Institute HELSINKI