Gathering Requirements for Effective Dimensional ModelingGathering Requirements for Effective Dimensional Modeling Gathering Requirements for Effective Dimensional Modeling Margy Ross

© 2003 Kimball Group All rights reserved.

GatheringRequirements for

EffectiveDimensional Modeling

GatheringRequirements for

EffectiveDimensional Modeling

Margy RossDec 17, 2003


Kimball Group

ConsultingProject assessments and

strategyRequirements analysis

Dimensional modeling anddesign reviews

Technical architecture

[email protected]

EducationPublic and on-site classes

Dimensional ModelingData Warehousing

Lifecycle

www.kimballuniversity.com

“Definitive Source for the Dimensional Approach”


Reference Materials

r Course materials adapted from...§ The Data Warehouse Lifecycle Toolkitè R. Kimball, L. Reeves, M. Ross,

W. Thornthwaite (Wiley 1998)

§ The Data Warehouse Toolkit, 2nd Editionè R. Kimball, M. Ross (Wiley 2002)

§ Kimball Universityè Design Tipsè Intelligent Enterprise articlesè www.KimballUniversity.com


Session Agenda and Goals

r Agenda:§ Introductions§ Lifecycle Approach§ Dimensional Model Overview§ Gathering Requirements for Dimensional

Modeling

r Today’s Goals:§ Learn proven approach for gathering data

warehouse business requirements§ Learn specific concepts and practical

techniques


Top 10 ReasonsData Warehouses Miss the Mark

q Focus on technology and data rather thanbusiness requirements and goals.

q Fail to secure business sponsorship.

q Pursue all-or-nothing approach rather thancompelling iterations.

q Spend time and energy building normalizeddata foundation, with nothing left forpresentation.

q Concentrate on operational performance anddevelopment rather than query performanceand ease of use.


Top 10 ReasonsData Warehouses Miss the Mark

q Make the user view of the data too complex.

q Build isolated data silos (marts orwarehouses).

q Only load summarized data into the userpresentation area.

q Presume business requirements are static.

q Fail to recognize that the data warehouse’ssuccess is tied directly to user acceptance.


BusinessDimensional

Lifecycle Approach

BusinessDimensional

Lifecycle Approach


Business Dimensional Lifecycle

TechnicalArchitecture

Design


Design

ProductSelection &Installation

ProductSelection &Installation

AnalyticApplication

Specification

AnalyticApplication

Specification

AnalyticApplication

Development

AnalyticApplication

Development

ProjectPlanningProject

Planning

Business

Requirement

Definition

Business

Requirement

Definition

DeploymentDeploymentMaintenance

andGrowth

Maintenanceand

Growth

Project ManagementProject Management

DimensionalModeling

DimensionalModeling

PhysicalDesign

PhysicalDesign

Data StagingDesign &

Development

Data StagingDesign &

Development


DimensionalModeling Concepts

DimensionalModeling Concepts


SimplifiedElements of Data Warehouse

Data StagingArea

Data Warehouse Bus:Conformed facts anddimensions

Data Warehouse Bus:Conformed facts anddimensions

Services:Transform from

source-to-targetMaintain conform

dimensionsNo user query

support

Data Store:Flat files or

relational tables

Design Goals:Staging

throughputQuality /

consistency

Services:Transform from

source-to-targetMaintain conform

dimensionsNo user query

support

Data Store:Flat files or

relational tables

Design Goals:Staging

throughputQuality /

consistency

PresentationArea

Ad Hoc Query &Analysis Tools

Standard Reports

AnalyticApplications:

ModelingForecastingScoringData mining

Ad Hoc Query &Analysis Tools

Standard Reports

AnalyticApplications:

ModelingForecastingScoringData mining

Data AccessTools

SourceSystems

Extract

Load

Extract

Extract

DimensionalAtomic AND

summary dataBusiness-process

centric

Design Goals:Ease-of-useQuery

performance

DimensionalAtomic AND

summary dataBusiness-process

centric

Design Goals:Ease-of-useQuery

performance

Access


PRODUCT KEY

Product Desc.SKU #SizeBrand Desc.Class Desc.

Terminology: Dimensions

r Set of attributes (columns) related to asubject/object§ Who, what, when, where, why, how§ Product, Customer, Date, Patient, Vendor,

Facility, …r Each dimension row is a unique

occurrence§ One row per product, customer, day, …

r Dimension attributes:§ Report labels and query constraints§ “By” words and “where” clauses§ Verbose descriptive attributes, in addition to

codes§ Hierarchical relationships


DATE KEYPRODUCT KEYSTORE KEYPROMOTION KEY

$ SalesUnit Sales

Terminology: Facts

r Result from a businessprocess or businessevent§ Facts are usually numeric

and additive

r Granularity/grain§ Identifies the fact level of

detail§ One row per sale,

one row per service call,one row per claim, …

§ Atomic grain is mostflexible


Terminology: Dimensional Modelor Star Schema

r Fact table perbusiness process /event, plus relevantdimensions

r Benefits:

§ Easier to understand

§ Better performancefrom fewer joins

§ Extensible to handlechange

StoreAttributes

PromoAttributes

ProductAttributes

Product KEY Store KEY

Promo KEYFacts

Product KEYDate KEYStore KEYPromo KEY

DateAttributes

Date KEY

DimensionTables

DimensionTables

FactTable


“Dimensions”“Dimensions”Report, row and column headings

“Facts”“Facts”Numeric report values

Sales Rep Performance ReportCentral Region

Jan 2003 Feb 2003Dollars Dollars

Chicago District Adams 990 999

Brown 900 999

Frederickson 990 999

Minneapolis District Andersen 950 999

Smith 950 999

Central Region Total 4,780 4,995

Sample Report Translationof Dimensional Model


Terminology:Conformed Dimensionsr One row for each instance of a business

subject or object (product, customer, etc.)r All fact tables use same standard dimensions

§ Established via Bus Matrix, enforced in ETL

r Consistent – apples to apples acrossprocesses

ProductAttributes

Product KEY

Shipment Facts

Product KEY…

Shipment Qty

Order Facts

Product KEY…

Inventory Qty

Inventory FactsProduct KEY…

Order Qty


Terminology: Enterprise DataWarehouse Bus Architecture

Store Inventory

Store Sales

Date Store Promo Distr.Center

Shipper VendorProduct

Purchase Orders


Terminology:Data Warehouse Bus Matrix

r Rows = Business processesr Columns = Conformed dimensions

Date Product Store Promo Dist Ctr Shipper VendorStore SalesStore InventoryStore DeliveriesDist Ctr InventoryDist Ctr DeliveryPurchase Orders


Creating Conformed Dimensions

r Assign surrogate key to every dimension row§ Buffer the DW from operational system changes§ Integrate data from multiple sources§ Query performance advantages§ Prerequisite for slowly changing dimension

r Combine all attributes intoMaster dimension table

r Use the Master tomap surrogatekeys to fact rows

Product CodeDescriptionBrandCategoryHeightWidthWeightStandard Cost

Product KEY

Marketing

Logistics

Cost Acctg.


Dimensional Modeling Processr Develop the Data Warehouse Bus matrixr Follow the 4-step method

§ Step 1: Identify the business process (matrixrow)

r Research data sources, including direct dataexploration

r Complete the 4-step method§ Step 2: Declare the grain§ Step 3: Identify the dimensions§ Step 4: Identify the facts

r Diagram the dimensional model


Key Input to Dimensional Model

DimensionalModel:

Business ProcessGrain

DimensionsFacts

BusinessRequirements

DataRealities


Gathering BusinessRequirements

Gathering BusinessRequirements


ProjectScope

BusinessRequirements

Maintenance andGrowth

ApplicationSpecification


Deployment Plan

PhysicalDesign

DimensionalModel

Data StagingDesign

Importance ofBusiness Requirements


Recommended Approach forUncovering Requirements

r Start with business users to understand...§ Business objectives§ Information/analysis themes§ Decision-making process

r Interweave data “reality” meetings withsource system experts or DBAs§ Ability to support user themes§ Data gotchas


Techniques forUncovering Requirements

r Interviews: All voices heard: Less interviewee time required

r Facilitated Group Sessions: Less elapsed time, but more participant time0 Limited # participants: Brainstorming, consensus and issue resolution

r Interviews AND Facilitated Groups


Preparation:Research and Team Roles/Tools

r Don’t presume you already know it all§ Research available resourcesè Annual report, marketing literature, web content

§ Organization chart§ Previous warehousing initiatives

r Identify interview team roles§ Lead interviewer, scribe and observers

r Prepare interview questionnaire§ Specific to interviewee level and function§ 1-page fallback device, not script


Preparation:Select Interviewees

r Horizontal user representation (breadth)§ Understand common vocabulary and data across

core functions§ Critical to Data Warehouse Bus framework

r Vertical user representation (depth)§ Vision à tactics; full perspective in target area§ Executives, managers and analysts

r Preliminary IT representatives§ Source experts, DBAs, IT liaisons and IT mgmt


Preparation:Schedule/Prepare Intervieweesr Scheduling logistics:

§ Solicit administrative help§ Individual sessions for execs§ Individual or small groups for othersè Max 2 - 3 people; homogeneous; < 2 levels

§ Maximum 3 - 4 interviews per dayè One hour for individuals; 1 1/2 hour for groupsè Minimum 1/2 hour between interviews

r Prepare users:§ Conduct User Kick-off Meeting - “ownership”§ Follow-up with pre-interview letterè Bring copies of key analyses


Interview Flow:Introduce the Interview

r Set the stage…§ Review project and interview objectives§ Introduce team players and roles§ Confirm time

r Establish overall tone§ Practice in advance§ No technical jargon; No phones or pagers

r Get users to talk about what they do§ Job responsibilities and organizational fit?


Business ExecutiveInterview Contentr Big-picture understanding and vision

§ Objectives of your organization? What are youtrying to accomplish?

§ How do you measure success? How do you knowdoing well? How often measured?

è Identify key business processes and facts

§ Key business issues? Vision for organization?

§ Opportunities to better leverage information withinorganization? Impact on business?

è Identify expectations and business benefits


Business Manager/AnalystInterview Contentr Similar to Business Exec, but more detailed

§ Objectives of your department? How measuresuccess? Key metrics? How often?è Identify key business processes and facts

§ How distinguish between products? Natural waycategorize products? How narrow list of products?è Identify dimension attributes and hierarchies

§ Types of routine/ad hoc analysis performed? Howoften? Improvements to current methods?è Identify data access tool requirements &

application templates

§ Opportunity to impact business with improvedaccess to information? Financial impact?


Business AnalystReport / Spreadsheet Review

r Understand current analyses andopportunities for improvement§ What data on the report is important?§ How do they use the report today?§ If report were dynamic, what’s different?

r Remember: Designing analyticenvironment, not reporting system§ Resist temptation to focus on top five reports


IT Data AuditInterview Contentr Feasibility of supporting requirements

§ Overview key operational source system

§ Update frequency? Availability of historical data?§ Known data caveats or quality issues?è Identify data availability and gotchas

r Dig until confident that core data exists§ More detailed data analysis will follow for

dimensional modeling

§ Beware of classic “profitability” data trap


Interview Flow:Wrap-Up

r Ask about project success criteria§ What is the #1 thing the project must accomplish

to be deemed successful?è Identify measurable success criteria

r Review next steps§ General disclaimer to manage expectations§ Deliverables and next feedback opportunity

r Thank for participation


r Remember interview role§ Do LISTEN; don’t defend, sell, try to impress, etc.

r Establish peer basis§ Use their vocabulary§ Don’t dive too quickly

r Strive for conversational flow

r Verify communication§ Capture terminology precisely

r Maintain interview schedule flexibility§ Can add new folks, but avoid interview burn-out

Interview Flow:Ground Rules


Post-Interview:Review Interview Results

r Informally debrief with team§ Common themes§ Do-ability§ Areas requiring clarification§ User analytical / technical sophistication

r Fill-in rough interview notes§ Highlight key points and vocabulary


Post-Interview:Publish Deliverables

r Don’t overlook formal documentation§ Validation§ Reference material

r Individual interview write-ups§ Summary, not transcriptè Business Objectivesè Analytic and Info Requirementsè Project Success Criteria

r Consolidated findings document


Post-Interview:Publish Deliverables cont’d

Consolidated Requirements Findings

l Executive Overview

l Project Overview

l Business Requirementsl Business Process Opportunities (matrix rows)

q Description

q Typical questions

q Data feasibility

l Preliminary Data Warehouse Bus Matrix

l Success Criteria


Low

High

High

Low Feasibility

PotentialBusiness

Impact

C A

DE

Post-Interview:Facilitation for Next Stepsr Session with Business

and IT management§ Confirm findings§ Get commitment§ Prioritize

r For each requirement§ Business impact / value§ Feasibility

r Outcomes:§ “Right” opportunities§ Consensus§ Ownership§ Roadmap for growth

B


Linking Data Warehouse BusMatrix and Prioritization Grid

PotentialBusiness

Impact

Low

High

High

Low Feasibility

C A

DE

B

Dim1 Dim2 Dim3 Dim4 Dim5Biz Proc ABiz Proc BBiz Proc CBiz Proc DBiz Proc E

Data Warehouse Bus Matrix


Tips for Handling Obstacles

“Abused User” è Review documentation“Validate” earlier inputUse alternative forum

“Overbooked User” è Get sponsor’s helpDefer, if prevalent

“Comatose User” è Don’t prolong agonyFind replacement

“Overzealous User” è Assess homogeneityReschedule sessions

“Non-Existent User” è Typically fatal - STOP!


Business RequirementsWarning Signs§ “We already understand the requirements -- better than

the users do.”

§ “We’ll send out a survey.”

§ “We’ll just look at their current reports and data files tounderstand users’ requirements.”

§ “Since each interview lasts one hour, we’ve scheduled2 days of interviews with 8 interviews per day.”

§ “We don’t have time to document the interviews.”


Business RequirementsSummary

r Understanding business requirements isCRITICAL to successful data warehousing

r Don’t overlook the up-front preparation

r Focus on listening

r Document what you’ve heard

r Close the loop following requirements

Documents

Gathering Requirements for Effective Dimensional ModelingGathering Requirements for Effective Dimensional Modeling Gathering Requirements for Effective Dimensional Modeling Margy Ross