Embed Size (px)
Citation preview
Worst Practices in Master Data Management
Mike FergusonManaging DirectorIntelligent Business StrategiesInformation Builders WebinarApril 2016
2
About Mike Ferguson
Mike Ferguson is Managing Director of Intelligent Business Strategies Limited. As an independent analyst and consultant he specializes in business intelligence, analytics, data management and big data. With over 34 years of IT experience, Mike has consulted for dozens of companies, spoken at events all over the world and written numerous articles. Formerly he was a principal and co-founder of Codd and Date Europe Limited – the inventors of the Relational Model, a Chief Architect at Teradata on the Teradata DBMS and European Managing Director of DataBase Associates.
Twitter: @mikeferguson1Tel/Fax (+44)1625 520700
3
Topics
Ten worst practices in MDM
Succeeding with MDM – a few guidelines for success
4
Worst Practice #1 - Implementing MDM With No Business Case
MDM System
CR
U
D
Prod
Asset
Cust
ImpactProject fails because of lack of sponsorshipCost of operating remains higher than it should be
Duplicate business process consolidation and rationalisation not possible
Return on investment
5
Building A Business Case For MDM Example – Customer Master Data Anomalies In ERP System
ERP
What happens if you have to invoice a customer?
What happens when you receive a payment from a customer?
Do you have duplicate customers in your ERP system(s)?
Duplicate customers?
Change customer details
If you change the details of a customer address do you change all duplicates?
Does your ERP system send customer data to other systems? If so does it send all duplicates? What happens if duplicates are not in sync?
6
Master Data Maintenance - The Problem of Multiple Data Entry Systems and Master Data Synchronisation
Mortgage System
Customerdata subset
BranchBankingSystem
Customerdata subset
LoansSystem
Customerdata subset
ERPSystem
Customerdata subset
Credit Card
SystemCustomer
data subset
Call CentreSystem
Customerdata subset
The “synchronisation
nightmare”
This has to be done for changes to EVERY master data entity
The problem gets worse as you add more applications
7
Master Data Synchronisation – The Spaghetti ArchitectureComplexity & Lack of Integration Works Against Business
Where is the complete set of master information?
How do I get the master data I need when I need it?
With so many definitions for master data what does it mean?
Can I trust it? Is it complete and correct? How do I get it in the form I
need? How do I know where it
goes and if it is correct? How do I control it?
Spaghetti Interfaces between systems
How much does it cost to operate this way??!
8
Inconsistent Master Data Can Disrupt Operations and Drive Up Costs Due To Manual Intervention Being Needed
order credit check
fulfill ship invoice paymentpackage
Order to cash process
prod custasset
Master data
X
How many people do you employ to fix and reconcile data because it is not synchronised?
What master data entities are used in your core processes
In what systems in your core processes does it reside?
Where in your core processes is master data created?
Where in your core processes is it consumed?
9
XYZ Corp.
Many Companies Have Business Units, Processes & Systems Organised Around Products and Services
Customers/Prospects
Product/service line 1
order credit check
fulfill ship invoice paymentpackage
Product/service line 2
Product/ service line 3Cha
nnel
s/O
utle
ts
order credit check
fulfill ship invoice paymentpackage
order credit check
fulfill ship invoice paymentpackage
Order(product line 1)
Order(product line 2)
Order(product line 3)
Enterprise
What is the im
pact on the customer is customer and product
master data is not managed across these processes?
10
Business and Data Complexity Can Spiral Out Of Control if Processes & Systems Are Duplicated Across Geographies
Product line 1
Product line 2
Product line 3
Product line 1
Product line 2
Product line 3Product line 1
Product line 2
Product line 3
Product line 1
Product line 2
Product line 3
Product line 1
Product line 2
Product line 3
Suppliers
Products/Services
AccountsAssets
Employees
Customers Partners
Materials
11
MDM Business Case Recommendations
Quantify the business impact of anomalies in your business caused by lack of synchronised master data on• Core operational business processes• Decision making • Compliance
Quantify the cost of fixing those anomalies by implementing MDM
Put together a set of candidate business cases ranked in order of return on investment
Use business SMEs to help you
MDM System
CR
U
D
Prod
Asset
Cust
12
Worst Practice #2 - Using A Data Warehouse As Your MDM System
EDWmart
DW & marts
order fulfill ship invoice paymentpackage
New customer?
Create customer process
Impact• Turns the DW into an OLTP system• What if several DWs exist?• DW becomes embroiled in synchronisation of all
OLTP systems• OLTP and analytics become entangled• If transactions occur 24 x 365 then the DW must
become 24 x 365• DW tables are typically de-normalised• Changes to DW impact OLTP systems• ……
13
MDM System
C
R
U
D
Prod
Cust
Asset
MDM Versus A Data Warehouse – They Should Be Separate Systems
Versus
Integrated Master DataNormalised Master DataHistorical Master DataSingle Customer ViewSingle Product ViewMaster Data de-coupled from all systemsMaster data optimised for CRUDMDM System is a data source feeding BI System AND Operational Systems
Integrated Master DataDe-Normalised Master DataHistorical Master DataSingle Customer ViewSingle Product ViewMaster Data in the BI System OnlyMaster data optimised for analysisEnterprise DW feeds data marts
DW
mart
mart
mart
Dat
a in
tegr
atio
n
Operational systems BI System
Mas
ter d
ata
inte
grat
ion
Operational systems
14
(SBV definitions)
C
R
U
D
prod cust
asset
Impact of Master Data Management on DW/BI Systems
Mas
ter d
ata
inte
grat
ion
Operational systems
MDM System
Enterprise Data Warehouse has shared common dimension data
transaction data
DW
Historic data
DF
D D
Dtime
product
Customer
FD
location
Dat
a in
tegr
atio
n
BI Tools(Reporting and Analysis)
Data Virtualization
MDM is a data source to data warehousing systems to improve consistency of dimensional data
15
Worst Practice #3 - Building An MDM System Without Understanding How Your Master Data Is Maintained
Where is master data maintained?• What processes?• What applications?
Call CentreSystem
Customerdata subset
ProductData subset
BranchBankingSystem
Customerdata subset
ERPSystem
Customerdata subset
Credit Card
SystemCustomer
data subsetProduct
data subsetProduct
data subset
changes changeschanges changesDataEntry
Systems
Mortgage System
Customerdata subset
LoansSystem
Customerdata subset
Productdata subset
Productdata subset
changes changes
Are your processes duplicated?
You WILL compromise the integrity of master data if you don’t know where it is
maintained
16
Call CentreSystem
Customerdata
subset
Productdata
subset
If The MDM System Becomes A System Of Record (SOR) Then DES Changes To Master Data Flow To MDM System
BranchBankingSystem
Customerdata
subset
ERPSystem
Customerdata
subset
Credit Card
SystemCustomer
data subset
CustomerSOR
Productdata
subset
Productdata
subset
ProductSOR
changes changeschanges changes
changes changes
Multiple Data Entry Systems still
maintain the master data
Mortgage System
Customerdata
LoansSystem
Customerdata
Productdata
subset
Productdata
subset
SOR = System Of RecordDES= Data Entry System
DataEntry
Systems
DataEntry
Systems
17
Call CentreSystem
Customerdata
subset
Productdata
subset
If The MDM System Becomes A Data Entry System Then Do You Know The Impact of Change? It Is Very Significant
Mortgage System
Customerdata
subset
BranchBankingSystem
Customerdata
subset
LoansSystem
Customerdata
subset
ERP System
Customerdata
subset
Credit Card
SystemCustomer
data subset
CustomerDES& SOR
Productdata
subset
Productdata
subset
Productdata
subset
Productdata
subset
ProductDES & SOR
changes changeschanges changes
changes changes
Customer master data changes
Product master data changes
SOR = System Of RecordDES= Data Entry System
Do you understand the impact of introducing centralised data
entry on an MDM system?
18
Recommendation– Identify Master Data “Producer” and “Consumer” Applications In ALL Your Processes
C
R
U
D
customer
New Customer Process
CreateNew Customer
What processes andapplications create master data?
InboundWhat processes use master data created elsewhere in the business?
Outbound
Customer serviceFinanceDistributionoperational & BI systems
Sales & Marketing
E.g. Manufacturing
19
Worst Practice #4 - Multiple MDM Systems For The Same Master Data Entity
MDM System
C
R
U
D
Customer
MDM System
C
R
U
D
Customer
MDM System
C
R
U
D
Customer
ImpactCost! - OverspendMaster data integrity potentially compromised?
Duplicate master data with different IDs?
Maintenance?
Major opportunity for error if these are silos
Which one is the master?
20
Distributed Master Data With Multiple Overlapping Subsets Needs Very Careful Management
C
R
U
D
prod custasset
C
R
UD
C
R
UD
C
R
UD
Global ID Col 1
Col 2 Col 3 Col 4
Shared attributes
Global ID Col 1
Col 2 Col 3 Col 4 Col 11 Col 12
Global ID Col 8 Col 9 Col 10Global ID Col 1
Col 2 Col 5 Col 6 Col 7
Local hub Local hub
Local hubmust remain
read only
must remain read only
Identical to central master
Identical to central master
Identical to central master
exclusive to local environment
exclusive to local environment
exclusive to local environment
can be created/updated
sync
sync
sync
line of business 1
line of business 2
Enterprise wide
line of business 3
sync
MUST remain read only as maintenance of shared attributes is done centrally
21
A More Efficient Way Might Be To Combine MDM And Data Virtualisation To Get Multiple Virtual Views
MDM System
C
R
U
D
Customer
ViewView View
Data Virtualisation
ViewViewView
22
Worst Practice #5 - Building an MDM System And Thinking You Are Done
MDM System
C
RU
D
Prod
Asset
Cust
budget
Change management
No budget !
23
Where Do You Start The Change Process?
You need to assemble people on your team that know• Your existing processes associated with master data• Your existing Data Entry Systems where master data is
maintained• How master data flows between people and existing systems
A master data change management program has to simplify your ‘spaghetti architecture’ to create benefits
Simplify
24
Understanding The Impact of Introducing An Enterprise Master Data Management System
Applications
Data
Processes & workflows
User Interfaces
People Documents
ChangeManagement
Changes need to be made to all of
these
25
Worst Practice #6 – Not Including Master Data Changes In Inbound MDM System Data Integration
BranchBankingSystem
Customerdata
subset
ERPSystem
CustomerData
Credit Card
System
changes changesDataEntry
Systems
Customerdata
subset
MDM System
C
RU
D
Prod
Asset
Cust
Call CentreSystem
Customerdata
subset
BI/DWSystem
changes
✓You WILL
compromise the integrity of master data if you do this
✓
26
Worst Practice #7 – Not Using MDM For Outbound Synchronisation
BranchBankingSystem
Customerdata
subset
ERPSystem
CustomerData
Credit Card
System
changes changesDataEntry
Systems
Customerdata
subset
MDM System
C
RU
D
Prod
Asset
Cust
Call CentreSystem
Customerdata
subset
BI/DWSystem
changes
✓
✗✗
Should come from MDM
You WILL compromise the
integrity of master data if you do this
27
Worst Practice #8 NOT Recognising The Value Of MDM In A DQ Firewall To Validate Transaction Data
order credit check
fulfill ship invoice paymentpackage
Order-to-Cash Process
An ideal situation would be smooth operation, increased automation, no delays, no defects and no unplanned operational cost
Orders
Business operational transaction processing – the ideal situation
28
Data Issues In Transaction Processing Can Impact Profitability By Causing Unplanned Operational Costs
order credit check
fulfill ship invoice paymentpackage
Data errors
Orders
Order-to-Cash Process
errorserrors
££
data qualityproblems e.g.
missing or wrong data on order entry
£
Unplanned operational cost = (£ + £££ + ££) * Number of Orders
£££
manual interventionand process
delays
All these defects add up to unplanned operational cost of processing an Order
Whatever you do has to reduce unplanned operational cost
Domino impact
What about other types of transactions
have data related problems?
29
The Impact of Data Anomalies In Transaction Processing As The Business Scales Can Be Considerable
order credit check
fulfill ship invoice paymentpackage
Data errors
Orders
Order-to-Cash Process
errorserrors
££££
data qualityproblems e.g.
missing or wrong data on order entry
£££
Unplanned operational cost increases as the business scales if anomalies are not fixed and data is not governed
• E.g. growth in back office headcount to deal with anomalies
£££££££
manual interventionand process
delays
Domino impact
30
The Value Of MDM In A Data Quality Firewall In Validating In-Bound Transaction Data
Validate& enrich
C
R
Uprod client
asset
D
e.g. Order
Validate, enrich and resolve identity
Master data
DQ services
DQ Firewall
31
Call CentreSystem
Customerdata
subset
Worst Practice #9 – Allowing Conflicting Constraints In ‘Consumer’ Systems On Outbound Master Data Synchronisation
Risk Management
BI SystemMortgage System
BranchBankingSystem
LoansSystem
ERPSystem
Credit Card
SystemCustomer
data subset
Customerdata
subset
Customerdata
subset
Customerdata
subset
Customerdata
CustomerDES & SOR
Transactions
Customer master data
changes
Transactions
Transactions TransactionsTransactions Transactions
DELETE CascadeUPDATE Cascade
Customerdata
subset
DELETE RestrictUPDATE Restrict
DELETE Set NullUPDATE Set Null
Nulls
NOT Nulls
CHECK column IN (‘A’, ‘B’, ‘C’)
CHECK column IN (‘A’, ‘D’, ‘E’, ‘F’)
SOR = System Of RecordDES= Data Entry System
You WILL compromise the
integrity of master data if you allow this
32
Outbound Master Data Synchronisation Recommendations
Master data synchronisation processes are needed• The master data hub is the source• The disparate consumer system is the target
Synchronisation CANNOT be based on a “fire and forget” approach
Schema and constraint correction may be needed in disparate systems to uphold master data integrity across the enterprise
33
Worst Practice #10 – Not Using Common Processes And Services To Maintain Master Data
MDM System
Prod
Asset
Cust
MDM System
C
RU
D
Prod
Asset
Custversus
Common services and processes • Facilitate re-use across multiple
applications• Reduce application development and
maintenance costs • Drive consistency across applications
Master Data Processes e.g. New Customer, New Product
Succeeding With Master Data Management- A Few Guidelines For Success
35
Know Your Processes !
You won’t find good business cases, and you can’t implement MDM or master data change
management unless you know how existing processes and applications work with master data
36
Identify Processes And Their Activities That Use and Maintain Master Data
Process can span multiple organisational
departments
Order Entry, Fulfilment and Tracking Process
Maintains Customer data +Access product data
Access product data
Access Customer data +Access product pricing data
Access customer data
37
Establish A Data Governance Operating Model – Master Data Entity and Transaction Based Approach
Data Gov control board
Business data
steward
Business data
steward
Business data
steward
Data Gov control board
Business data
steward
Business data
steward
Business data
steward
Data Gov control board
Business data
steward
Business data
steward
Business data
steward
Enterprise Data Gov control board
All control boards have a dispute resolution process
Control board approval processes for data naming, integrity rules….
ProductClient OrdersDispute
Resolution process
Dispute Resolution
process
Dispute Resolution
process
Dispute Resolution
process
Customer OrdersProduct
Operating model is independent of
location and line of business
Virtual community Virtual communityVirtual community
sponsor
38
Data Stewards Should Be Accountable For The Consistency And Quality Of Master Data Across Processes
ordercredit check fulfil ship invoice paymentpackage
Process Example - Manufacturing Order to cash
schedule
Order entry
system
Finance credit control system
Production planning & scheduling
system
CAMsystem
Inventory system
Distribution system
Billing Gen Ledger
Orders data Customer data Product data
Data steward(Customer data)
Data steward(Customer data)
Data steward(Customer data)
39
To Implement MDM A Methodology Needs To Be Applied To EACH Master Data Entity To Bring It Under Control
SBV
Model
Discover
Map
ProfileClean
Integrate
Provision
Monitor
IDE
NTI
FY
ASSESS
SER
VE
INTEGRATE
DEFINE
CONTROL
CUSTOMER Data
Governance
SBV
Model
Discover
Map
ProfileClean
Integrate
Provision
Monitor
IDE
NTI
FY
ASSESSSE
RVE
INTEGRATE
DEFINE
CONTROL
PRODUCT Data
GovernanceSBV
Model
Discover
Map
ProfileClean
Integrate
Provision
Monitor
IDE
NTI
FY
ASSESS
SER
VE
INTEGRATE
DEFINE
CONTROL
SUPPLIERData
Governance
Customer Data Governance Product Data Governance
Supplier Data Governance
Also ASSETEMPLOYEEACCOUNTMATERIAL…..
40
A Shared Business Vocabulary Is The Anchor Point For Any MDM Project
IDE
NTI
FY
ASSESS
SER
VE
INTEGRATE
DEFINE
CONTROLSBV
Model
Discover
Map
ProfileClean
Integrate
Provision
Monitor
Define all Common
Master Data,attributes
41
Discover And Map Disparate Master Data in Different Systems To Standard, SBV Defined Master Data Entities
Standard SBV Model for Customer Master Data
Sales Force Automation Schema
Disparate Customer data
SFA System Customer Mapping
Branch System Customer Mapping
Billing System Customer Mapping
Branch System SchemaDisparate Customer
data
Billing System SchemaDisparate Customer
data
Data Discovery
42
To Get A Single View Of Master Data You Need To Have Global IDs, A Master SBV, AND Know All Data Mappings
ID = Party_ID Party_FirstNamePary_SurnameParty_StreetNo
Party_StreetNameParty_City
…….
C_NameC_Address
C_City
Client_NameClient_Addr
CardHolder_ForenameCardholder_SurnameCardholder_Address
Acc_NameAcc_Addr1Acc_Addr2
Acc_BirthdateCustomerMaster Data
ID = Loan Number
ID = Mortgage Number
ID = Account Number
ID = CCard Number
Loan system
Mortgage system
Savings system
Card system
43
Use People, Processes, Policies And Technology To Implement MDM
SBV
Model
Discover
Map
ProfileClean
Integrate
Provision
Monitor
IDE
NTI
FYASSESS
SER
VE
INTEGRATE
DEFINE
CONTROL
CUSTOMER Data
Governance
Data & Metadata
Relationship Discovery
Tools
Data Quality
Profiling & Monitoring
Tools
Data Modelling &
Data Integrity Tool
DataCleansing &
MatchingTools
Data Integration
Tools
metadata
Enterprise Data Management Platform – All tools share a common repository
BusinessGlossary
Tool
Data Governance Console
C
R
Uprod cust
asset
D
44
Implement Shared Master Data Services To Provide A Common Approach To Master Data Maintenance
Master data access and maintenance services• A common set of shared services to access
and maintain master data i.e. for use as a data entry system (DES)
• Allows an MDM system to be integrated with applications, processes and portals to ‘consume’ master data
• May have support for other APIs in addition to web service interface e.g. Java and .Net
Integration with Enterprise Service Bus / Message Broker software to manage synchronisation of changes to master data to all operational and analytical systems that use complete sets or subsets of this data
C
R
U
D
prod custasset
Worst Practices in MDM
Jake FreivaldVice PresidentInformation Builders
46
Founded: 1975 in NYCEmployees: 1,600Direct customers: 9,000+OEMs: 10,000+End users: Tens of millions
Goal: Deliver the industry’s best-engineered software and top customer service to ensure customer success.
Corporate SummaryInformation Builders
50+ locations
Software Data integration (iWay) Data integrity (iWay) BI and analytics (WebFOCUS)
Customer Support Global 24x7, local and online
Consulting services Expertise, mentoring, and rapid
application developmentEducation
Our facilities, your facilities, or onlineCustomer communications
Newsletters, IB Magazine, and onlineUser community
Summit conference, local groups, advisory councils, FocalPoint social network , Facebook, Twitter, etc.
Information BuildersAward-Winning Customer Service
48
Customer
User Community &Social Network
CustomerSupport
Consulting Education
Documentation Premium
Wisdom of CrowdsTM
Mastering Master Data Management
49
Understanding Processes and Their ValueInformation Management
50
MultipurposeReal-timeLifeWatch
MultisystemDQ firewallTRAC
MultidomainSingle, 360° viewLA CAFE
Project Components Integration Quality and standardization Mastering Generation of consumption
artifacts Historical data management Testing Issue remediation
The “Real Work” Behind MDM
Project Characteristics Transformational at the
business level Technically complex Dependent on solid
architecture
Knowledge requirements Staffing requirements Integration expertise Mastering expertise Business process transformation experience Testing and quality assurance
The “Real Work” Behind MDM
MDM App(End State)
• MDM hub• Data warehouse• Partner interface
• Quality process
• Operational systems• BI/analytics app
Agile Approach to MDM and Data IntegrationTypical Data Integration, Quality, & Mastering Approach
• MDM hub• Data warehouse• Partner interface
• Quality process
• Operational systems• BI/analytics app
MDM App(End State)
Omni-Gen
Drive from the business (#1) • Deliver mapping from many systems of record (#3) • Flexible system enables use of only one MDM repository (#4) • Eliminate one-and-done mentality with business-user remediation (#5) • Facilitate rapid change, with master data changes in inbound MDM data integration (#6) and changes to outbound MD synchronization considered (#9) • Include a DQ firewall as part of MDM (#8) • Use common processes and reuse rules to maintain consistency (#10) .
Traditional MDM Development ProcessWithout Omni-Gen
55
Data
Mgm
t Te
amAn
alyti
cs
Team
Busin
ess T
eam
Define Golden Record
Test
Dat
a Re
adin
ess
Go-li
ve
Define Analytics
Define Cleansing Rules
Define Master Rules
Attach History
Create API & Feeds
CreateGovernance Portal
Define Master Data Store
Write Mastering Rules
Write Cleansing Rules
Analytics Initial Implementation
Define Mapping Rules
Rapid, Automated, and Parallel Development ProcessWith Omni-Gen
56
Data
Mgm
t Te
am Write Cleansing Rules
Define Mapping Rules
Write Mastering Rules
Auto
mati
c
Attach History
Create API & Feeds
Generate Interface Doc Specification (IDS)
Anal
ytics
Te
am Analytics Initial Implementation
Busin
ess T
eam
Define Golden Record
Test
Dat
a Re
adin
ess a
nd G
o-liv
e
Shorter, automated, and
rule-driven lifecycle
means more iterative
development and
compressed cycle times
Define Governance Rules
Define Cleansing Rules
Define Mastering Rules
Define Analytics
Omni-Gen
Profiling Integration Cleansing Mastering
Integration Edition
Data Quality Edition
Master Data Management Edition
Product ViewOmni Information Management Platform
Omni-Gen
Profiling Integration Cleansing Mastering
Integration Edition
Data Quality Edition
Master Data Management Edition
Omni-Patient Omni-Payer Other and Custom Omni Applications
Product ViewOmni Information Management Platform
The DW is a client of the MDM system (#2) • Synchronization managed through outbound facilities of the MDM system (#7)
Onboarding Data for Improved Collaboration and OutcomesOmni-Payer Benefits
PrivatePractice
HEDIS / CMS
Hospitals
Informaticsand Analytics
ProviderRelations
CommunityCare
Thank you!
