Upload
fahri-firdausillah
View
3.477
Download
7
Tags:
Embed Size (px)
DESCRIPTION
Presentation of Chapter 5 & 6 from Ponniah Paulraj book in Data Warehousing
Citation preview
Planning & Project Planning & Project ManagementManagement
Fahri Firdausillah [M031010012]
Joke First, Serious LaterJoke First, Serious Later Consultant:Consultant: So, your company is into data So, your company is into data
warehousing? How many data marts do you have?warehousing? How many data marts do you have?
Project Manager:Project Manager: Eleven.Eleven.
Consultant:Consultant: That’s great. But why so many?That’s great. But why so many?
Project Manager:Project Manager: Ten mistakes.Ten mistakes.
Defining the Business Defining the Business RequirementsRequirements
Chapter 5Chapter 5
PreamblePreamble OLTP and DW planning is different in term of OLTP and DW planning is different in term of
requirements clarityrequirements clarity
Planning DW is about solving users’ problems and Planning DW is about solving users’ problems and providing strategic information to the user.providing strategic information to the user.
OLTP systems are primarily data capture systems. On OLTP systems are primarily data capture systems. On the other hand, data warehouse systems are the other hand, data warehouse systems are information delivery systems.information delivery systems.
Unlike an OLTP system, which is needed to run the day-Unlike an OLTP system, which is needed to run the day-to-day business, no immediate payout is seen in a to-day business, no immediate payout is seen in a decision support system.decision support system.
Dimensional AnalysisDimensional Analysis
the users are generally unable to define the users are generally unable to define their requirements clearly.their requirements clearly.
For most of the users, this could be the For most of the users, this could be the very first data warehouse.very first data warehouse.
How can you build something the users are How can you build something the users are unable to define clearly and precisely?unable to define clearly and precisely?
Need different approach of requirements Need different approach of requirements gathering.gathering.
Dimensional Analysis (cont'd)Dimensional Analysis (cont'd) They can tell you what measurement units are They can tell you what measurement units are
important for them, how they combine the important for them, how they combine the various pieces of information for strategic various pieces of information for strategic decision making.decision making.
Although the actual proposed usage of a data Although the actual proposed usage of a data warehouse could be unclear, the business warehouse could be unclear, the business dimensions used by the managers for decision dimensions used by the managers for decision making are not nebulous at allmaking are not nebulous at all
Dimensional Analysis “in Action”Dimensional Analysis “in Action”
More Complex Dimensional ModelMore Complex Dimensional Model
Information PackagesInformation Packages The business dimensions and their hierarchical levels form The business dimensions and their hierarchical levels form
the basis for all further development phases.the basis for all further development phases.
The dimension hierarchies are the paths for drilling down or The dimension hierarchies are the paths for drilling down or rolling up in our analysisrolling up in our analysis
Requirements Gathering MethodsRequirements Gathering Methods
Questionn
aires Group Session
InterviewTypes of Questions Open Ended QuestionThese open up options for interviewees to respond
Closed QuestionThese allow limited responses to interviewees
ProbesThese are really follow-up questions. Probes may be used after open-ended or closed questions
Sample Expectation from Sample Expectation from InterviewsInterviews
Senior Executives Dept. Managers IT Dept. Professionals
Organization objectives
Criteria for measuring success
Key business issues, current
& future
Problem identification
Vision and direction for the
organization
Anticipated usage of the DW
Departmental objectives
Success metrics
Factors limiting success
Key business issues
Products & Services
Useful business dimensions
for analysis
Anticipated usage of the DW
Key operational source
systems
Current information delivery
processes
Types of routine analysis
Known quality issues
Current IT support for
information requests
Concerns about proposed DW
Adapting JADAdapting JAD
1. P
roje
ct
Defi
nitio
n 2. Research3. Pre
para
tion
4. JAD Sessions
5. F
inal
Docu
ment
1.1. Identify project objectives and Identify project objectives and limitationslimitations
2.2. Identify critical success factorsIdentify critical success factors
3.3. Define project deliverables Define project deliverables
4.4. Define the schedule of workshop Define the schedule of workshop activitiesactivities
5.5. Select the participantsSelect the participants
6.6. Prepare the workshop materialPrepare the workshop material
7.7. Organize workshop activities and Organize workshop activities and exercisesexercises
8.8. Prepare, inform, educate the Prepare, inform, educate the workshop participantsworkshop participants
9.9. Coordinate workshop logisticsCoordinate workshop logistics
Requirement Definition: Scope & Requirement Definition: Scope & ContentContent
Requirements definition document is the basis for the Requirements definition document is the basis for the next phases. Formal documentation will also validate next phases. Formal documentation will also validate your findings when reviewed with the usersyour findings when reviewed with the users Data SourcesData Sources
Data TransformationData Transformation
Data StorageData Storage
Information DeliveryInformation Delivery
Information Package DiagramsInformation Package Diagrams
Requirements Definition Document Requirements Definition Document OutlineOutline
1.1.IntroductionIntroduction
2.2.General Requirements DescriptionsGeneral Requirements Descriptions
3.3.Specific RequirementsSpecific Requirements
4.4.Information PackagesInformation Packages
5.5.Other RequirementsOther Requirements
6.6.User ExpectationsUser Expectations
7.7.User Participation and Sign-OffUser Participation and Sign-Off
8.8.General Implementation PlanGeneral Implementation Plan
Requirements as the Requirements as the Driving Force for Data Driving Force for Data
WarehousingWarehousing
Chapter 6Chapter 6
PreamblePreamble If accurate requirements definition is important for any If accurate requirements definition is important for any
operational system, it is many times more important for operational system, it is many times more important for a data warehousea data warehouse
extremely important that your datawarehouse contains extremely important that your datawarehouse contains the right elements of information in the most optimal the right elements of information in the most optimal formatsformats
Every task that is performed in every phase in the Every task that is performed in every phase in the development of the data warehouse is determined by development of the data warehouse is determined by the requirementsthe requirements
Every decision made during the design phase is totally Every decision made during the design phase is totally influenced by the requirements.influenced by the requirements.
Data DesignData Design
Data Design (cont'd)Data Design (cont'd) Structure for Business DimensionsStructure for Business Dimensions
Importance of having the appropriate dimensions and the right Importance of having the appropriate dimensions and the right contents in the contents in the information package diagramsinformation package diagrams..
Structure for Key MeasurementsStructure for Key Measurements
Users measure performance by using and comparing key Users measure performance by using and comparing key measurementsmeasurements
In order to review using proper key measurements, DW has to In order to review using proper key measurements, DW has to guarantee the information package diagrams contain all the relevant guarantee the information package diagrams contain all the relevant keys.keys.
Levels of DetailLevels of Detail
DW needs to provide drill-down and roll-up facilities for analysisDW needs to provide drill-down and roll-up facilities for analysis How deep detail of data is needed in DWHow deep detail of data is needed in DW
Data Design “in Action”Data Design “in Action”
Structure for Business Dimensions
Structure for Key Measurements
Levels of Detail
The Architectural PlanThe Architectural Plan
Source DataSource Data Production Data: Data get from operational system. Production Data: Data get from operational system.
Normally include financial system, customer Normally include financial system, customer relationship system, manufacturing system, etc.relationship system, manufacturing system, etc.
Internal Data: Private data keep by internal Internal Data: Private data keep by internal organization. Could be spreadsheets, documents, even organization. Could be spreadsheets, documents, even departmental databasedepartmental database
Archived Data: Old data that is already not to be used Archived Data: Old data that is already not to be used in operational system.in operational system.
External Data: Data from outside systems, it can also External Data: Data from outside systems, it can also from outside company. This type of data usually do not from outside company. This type of data usually do not conform internal formatconform internal format
Data StagingData StagingBad data lead to bad decision, Bad data lead to bad decision,
data quality in data warehouse is sacrosanctdata quality in data warehouse is sacrosanct ETL process ensure data to be ready stored and processed ETL process ensure data to be ready stored and processed
in DW.in DW.
In many cases data need to be extracted from sources in In many cases data need to be extracted from sources in different scheme, different vendor, even in different format different scheme, different vendor, even in different format of flat files.of flat files.
If data extraction for a DW poses great challenges, data If data extraction for a DW poses great challenges, data transformation presents even greater challenges.transformation presents even greater challenges.
Data need to be cleaned from misspelling, resolution Data need to be cleaned from misspelling, resolution conflict, duplication, setting default missing values, etc.conflict, duplication, setting default missing values, etc.
Initial load moves very large volumes of data. After that data Initial load moves very large volumes of data. After that data staging will continuously extract the changes from sources.staging will continuously extract the changes from sources.
Extract Transform Load
Sample ArchitectureSample Architecture
Data Storage SpecificationsData Storage Specifications DBMS SelectionDBMS Selection
User requirements affect the selection of the proper DBMS.User requirements affect the selection of the proper DBMS.
Choice of the DBMS may be conditioned by its tool kit Choice of the DBMS may be conditioned by its tool kit component.component.
Features to be considered: Level of User Experience, Types of Features to be considered: Level of User Experience, Types of Queries, Need for Openness, Data Loads, Metadata Management, Queries, Need for Openness, Data Loads, Metadata Management, Data Repository Locations, Data Warehouse Growth.Data Repository Locations, Data Warehouse Growth.
Storage SizingStorage Sizing Determined by how many data source and how much the Determined by how many data source and how much the
data will grows continuously.data will grows continuously. If DW is expected to support Online Analytical Processing If DW is expected to support Online Analytical Processing
OLAP, then how much OLAP is necessary.OLAP, then how much OLAP is necessary.
Information Delivery StrategyInformation Delivery Strategy
MetadataMetadata Operational Metadata:Operational Metadata:
When deliver information to the end-users, you must be able When deliver information to the end-users, you must be able to tie that back to the original source data sets. Operational to tie that back to the original source data sets. Operational metadata contain all of this information about theoperational metadata contain all of this information about theoperational data sources.data sources.
Extraction and Transformation Metadata:Extraction and Transformation Metadata:Storing information of extraction frequencies, extraction Storing information of extraction frequencies, extraction methods, and business rules for the data extraction.methods, and business rules for the data extraction.
End-User Metadata:End-User Metadata:Navigational map of the data warehouse, allows the end-Navigational map of the data warehouse, allows the end-users to use their own business terminology and look for users to use their own business terminology and look for information in those ways.information in those ways.
Management & ControlManagement & Control
Sits on top of all the other components.Sits on top of all the other components. Controls the data transformation and the Controls the data transformation and the
data transfer into the data warehouse data transfer into the data warehouse storage.storage.
Interacts with the metadata component to Interacts with the metadata component to perform the management and control perform the management and control functions.functions.
Metadata is the source of information for Metadata is the source of information for the management module.the management module.
End of PresentationEnd of Presentation&&
Thank You Very MuchThank You Very Much