8/12/2019 Integrated EDW Kimball
1/18
EssentialStepsforEssentialStepsforEssentialStepsforEssentialStepsfor
theIntegratedEDWtheIntegratedEDWtheIntegratedEDWtheIntegratedEDW
AKimballGroupWhitePaper
ByRalphKimball
8/12/2019 Integrated EDW Kimball
2/18
TableofContentsTableofContentsTableofContentsTableofContents
ExecutiveSummary.................................................................................3
AbouttheAuthor......................................................................................3
WhatDoesanIntegratedEnterpriseDataWarehouse(EDW)Deliver?....4DrillingAcrossistheUltimateLitmusTestforIntegration.........................4
TheOrganizationalChallengesofProvidinganIntegratedEDW..............5
ConformedDimensionsandFacts...........................................................6
UsingtheBusMatrixasaWaytoCommunicatewithExecutives.............6
ManagingtheBackboneoftheIntegratedEDW.......................................7
TheDimensionManager..........................................................................8
TheFactProvider..................................................................................11
ConfiguringBIToolstoUsetheIntegratedEDW....................................12
AdvancedTopics...................................................................................13
Conclusion.............................................................................................18
8/12/2019 Integrated EDW Kimball
3/18
EssentialStepsfortheIntegratedEDWCopyright2008byKimballGroup.Allrightsreserved.
ExecutiveSummaryExecutiveSummaryExecutiveSummaryExecutiveSummary
Inthiswhitepaper,weproposeaspecificarchitectureforbuildinganintegratedenterprisedatawarehouse(EDW).Thisarchitecturedirectlysupportsmasterdatamanagementeffortsandprovidestheplatformforconsistentbusinessanalysisacrosstheenterprise.Wedescribethescopeandchallengesofbuildingan
integratedenterprisedatawarehouse,andweprovidedetailedguidancefordesigningandadministeringthenecessaryprocessesthatsupportintegration.ThiswhitepaperhasbeenwritteninresponsetoalackofspecificguidanceintheindustryastowhatanintegratedEDWactuallyis,andwhatnecessarydesignelementsareneededtoachieveintegration.
AbouttheAuthorAbouttheAuthorAbouttheAuthorAbouttheAuthor
RalphKimballfoundedtheKimballGroup.Sincethemid1980s,hehasbeenthedatawarehouse/businessintelligence(DW/BI)industrysthoughtleaderonthedimensionalapproachandtrainedmorethan10,000ITprofessionals.PriortoworkingatMetaphorandfoundingRedBrickSystems,Ralphco-inventedtheStarworkstationatXeroxsPaloAltoResearchCenter(PARC).RalphhashisPh.D.inElectricalEngineeringfromStanfordUniversity.
TheKimballGroupisthesourcefordimensionalDW/BIconsultingandeducation,consistentwithourbest-sellingToolkitbookseries,DesignTips,andaward-winningarticles.Visitwww.kimballgroup.comformoreinformation.
8/12/2019 Integrated EDW Kimball
4/18
EssentialStepsfortheIntegratedEDW
WhatDoesanIntegratedEnterpriseDataWarehouse(EDW)WhatDoesanIntegratedEnterpriseDataWarehouse(EDW)WhatDoesanIntegratedEnterpriseDataWarehouse(EDW)WhatDoesanIntegratedEnterpriseDataWarehouse(EDW)Deliver?Deliver?Deliver?Deliver?
ThemissionstatementfortheintegratedEDWistoprovidetheplatformforbusinessanalysistobeappliedconsistentlyacrosstheenterprise.Aboveall,thismissionstatementdemandsconsistencyacrossbusinessprocesssubjectareas
andtheirassociateddatabases.Consistencyrequiresdetailedtextualdescriptionsofentitiessuchascustomers,products,locations,andcalendarstobeapplieduniformlyacrosssubjectareas,usingstandardizeddatavalues.Ofcourse,thisisafundamentaltenetofmasterdatamanagement(MDM).
Consistencyrequiresaggregatedgroupingssuchastypes,categories,flavors,colors,andzonesdefinedwithinentitiestohavethesameinterpretationsacrosssubjectareas.Thiscanbeviewedasahigherlevelrequirementonthetextualdescriptionsdescribedinthepreviousparagraph.
ConsistencyrequiresthatconstraintsposedbyBIapplicationswhichattemptto
harvestthevalueofconsistenttextdescriptionsandgroupingsbeappliedwithidenticalapplicationlogicacrosssubjectareas.Forinstance,constrainingonaproductcategoryshouldalwaysbedrivenfromafieldnamedCategoryfoundintheProductdimension.
Consistencyrequiresthatnumericfactsarerepresentedconsistentlyacrosssubjectareassothatitmakessensetocombinethemincomputationsandcomparethemtoeachother,perhapswithratiosordifferences.Forinstance,ifRevenueisanumericfactreportedfrommultiplesubjectareas,thenthedefinitionsofeachoftheserevenueinstancesmustbethesame.
Consistencyrequiresthatinternationaldifferencesinlanguages,locationdescriptions,timezones,currencies,andbusinessrulesberesolvedtoallowalloftheaboveconsistencyrequirementstobeachieved!
Consistencyrequiresthatauditing,compliance,authentication,andauthorizationfunctionsbeappliedinthesamewayacrosssubjectareas.
Finally,consistencyimpliescoordinationwithindustrystandardsfordatacontent,dataexchange,andreporting,wherethosestandardsimpacttheenterprise.TypicalstandardsincludeACORD(insurance),MISMO(mortgages),SWIFTandNACHA(financialservices),HIPAAandHL7(healthcare),RosettaNet(manufacturing),andEDI(procurement).
DrillingAcrossistheUltimateLitmusTestforIntegrationDrillingAcrossistheUltimateLitmusTestforIntegrationDrillingAcrossistheUltimateLitmusTestforIntegrationDrillingAcrossistheUltimateLitmusTestforIntegration
EvenanEDWthatmeetsalloftheconsistencyrequirementsdescribedabovemustadditionallyprovideamechanismfordeliveringintegratedreportsandanalysesfromBItools,attachedtomanydatabaseinstances,possiblyhostedonremote,incompatiblesystems.Wecallthisdrillingacross.DrillingacrossistheessentialactoftheintegratedEDW.Whenwedrillacross,wegatherresultsfromseparatebusinessprocesssubjectareasandthenalignorcombinetheseresultsintoasingleanalysis.
Forexample,supposeourintegratedEDWspansmanufacturing,distributionand
8/12/2019 Integrated EDW Kimball
5/18
EssentialStepsfortheIntegratedEDW
retailsalesinabusinessthatsellsaudio/visualsystems.Wellassumethateachofthesesubjectareasissupportedbyaseparatetransactionprocessingsystem.AproperlyconstructeddrillacrossreportcouldlooklikeFigure1.
Figure1.AThreeFactTableDrillAcrossReportFigure1.AThreeFactTableDrillAcrossReportFigure1.AThreeFactTableDrillAcrossReportFigure1.AThreeFactTableDrillAcrossReport
ThefirsttwocolumnsarerowheadersfromtheProductandCalendarconformeddimensions,respectively.Theremainingthreefactcolumnseachcomefromseparatedatabases,namelymanufacturing,distribution,andretailsales.ThisdeceptivelysimplereportcanonlybeproducedinaproperlyintegratedEDW.Inparticular,theProductandCalendardimensionsmustbeavailableinallthreeseparatedatabases,andtheCategoryandPeriodattributeswithinthosedimensionsmusthaveidenticalcontentsandinterpretations.Althoughthemetricsinthethreefactcolumnsaredifferent,themeaningofthemetricsmustbeconsistentacrossproductcategoriesandtimes.
YoumustunderstandandappreciatethetightconstraintsontheintegratedEDWenvironmentdemandedbytheabovereport.Ifyoudont,youwontunderstandthiswhitepaper,andyouwonthavethepatiencetostudythedetailedstepsdescribedbelow.Or,toputthedesignchallengeinotherterms,ifyoueventuallybuildasuccessfulintegratedEDW,youwillhavevisitedeveryissueinthispaper.So,withthosewarnings,readon!
TheOrganizationalChallengesofProvidinganIntegratedEDWTheOrganizationalChallengesofProvidinganIntegratedEDWTheOrganizationalChallengesofProvidinganIntegratedEDWTheOrganizationalChallengesofProvidinganIntegratedEDW
TheintegratedEDWdeliverablesdescribedaboveareadauntinglistindeed.But
forthesedeliverablestoevenbepossible,theenterprisemustmakeaprofoundcommitment,startingfromtheexecutivesuite.Theseparatedivisionsoftheenterprisemusthaveasharedvisionofthevalueofdataintegration,andtheymustanticipatethestepsofcompromiseanddecisionmakingthatwillberequired.Thisvisioncanonlycomefromtheseniorexecutivesoftheenterprise,whomustspeakveryclearlyonthevalueofdataintegration.
ExistingmasterdatamanagementprojectsprovideanenormousboostfortheintegratedEDW,sincepresumablytheexecutiveteamalreadyunderstandsandapprovesthecommitmenttobuildingandmaintainingmasterdata.AgoodMDM
8/12/2019 Integrated EDW Kimball
6/18
EssentialStepsfortheIntegratedEDW
resourcegreatlysimplifies,butdoesnoteliminate,theneedfortheEDWteamtobuildthestructuresnecessaryfordatawarehouseintegration.
Inmanyorganizations,achicken-and-eggdilemmaexists,astowhetherMDMisrequiredbeforeanintegratedEDWispossible,orwhethertheEDWteamcreatestheMDMresources.Often,alowprofileEDWefforttobuildconformeddimensionssolelyfordatawarehousepurposesmorphsintoafull-fledgedMDM
effortthatisonthecriticalpathtosupportingmainlineoperationalsystems.Inourclassessince1993,wehaveshownabackwardpointingarrowleadingfromcleaneddatawarehousedatatooperationalsystems.Intheearlydays,wesighedwistfullyandwishedthatthesourcesystemscaredaboutclean,consistentdata.Now,morethanfifteenyearslater,weseemtobegettingourwish!
ConformedDimensionsandFactsConformedDimensionsandFactsConformedDimensionsandFactsConformedDimensionsandFacts
Sincetheearliestdaysofdatawarehousing, conformeddimensionshavebeenusedtoconsistentlylabelandconstrainseparatedatasources.WelearnedaboutconformeddimensionsfromA.C.Nielsenin1983when,atMetaphorComputerSystems,webroughtNielsenssyndicatedscannerdatatogetherwithproduct
shipmentsdataatconsumerpackagegoodscompanies.Theideabehindconformeddimensionsisverysimple:twodimensionsareconformediftheycontainoneormorecommonfields,whosecontentsaredrawnfromthesamedomains.Thatresultsinconstraintsandlabelshavingthesamecontentandmeaningwhenappliedagainstseparatedatasources.
Conformedfactsaresimplynumericmeasuresthathavethesamebusinessandmathematicalinterpretationssothattheymaybecomparedandcomputedagainsteach
