Embed Size (px)
DESCRIPTION
In this Webinar slide deck, we look at Enterprise Reference Data Management. We examine how the old way of doing things is straining under not only the new regulatory climate but also how Reference Data may evolve to a competitive advantage. What you will learn: • How the cost of traditional ETL is not only measured in time and development spent on delivery but also on opportunity costs and downstream breaks. • Why an RDBMS is not suitable as a “golden copy” for Enterprise Reference Data • How a multi-variant Enterprise NoSQL database is the best platform for managing Enterprise Reference Data • How Semantics and ontologies play a role • The concept of an “executable data dictionary"
Citation preview
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. © COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Enterprise Reference Data Management
Ken Krupa MarkLogic
Dean Allemang Rupert Brown MarkLogic Working Ontologist
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 2
Trade Lifecycle (Simplified)
Front Office
• Trade capture • Execution
Middle Office
• Validation • Booking • Confirmations
Back Office
• Clearing • Settlement • Accounting
Pre-execution
Record Retention
Surveillance, Risk, Compliance
Enterprise Reference Data
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 3
Trade Lifecycle (Simplified)
Front Office
• Trade capture • Execution
Middle Office
• Validation • Booking • Confirmations
Back Office
• Clearing • Settlement • Accounting
Pre-execution
Record Retention
Surveillance, Risk, Compliance
Enterprise Reference Data
YOU ARE HERE
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 4
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 4
Functions & Linkages Across an Investment Bank
Client Onboarding
Research / Advisory
Portal and Channels
Execution & Position
Management
Treasury (Funding and
Liquidity)
Finance & General Ledger
Risk Management
Operations
Corporate Finance /
M&A
Legal / Compliance / Surveillance
Reference Data
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 5
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 5
Functions & Linkages Across an Investment Bank
Client Onboarding
Research / Advisory
Portal and Channels
Execution & Position
Management
Treasury (Funding and
Liquidity)
Finance & General Ledger
Risk Management
Operations
Corporate Finance /
M&A
Legal / Compliance / Surveillance
Reference Data
Warning
When reference data breaks, everything else breaks!
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 6
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 6
Functions & Linkages Across an Investment Bank
Client Onboarding
Research / Advisory
Portal and Channels
Execution & Position
Management
Treasury (Funding and
Liquidity)
Finance & General Ledger
Risk Management
Operations
Corporate Finance /
M&A
Legal / Compliance / Surveillance
Reference Data
The latency of functions varies – from real-time to daily or slower.
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 7
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 7
Vendor Control
Data Operations
Exceptions Dashboard
Corrections Interface
Distributions Dashboard
Vendor Feeds Ingest / Conversion Golden Copy Distribution
Thomson Reuters
Bloomberg
Markit
Telekurs
CME Direct
Etc.
Exchange Feeds
Staging Prep
Staging Tables
ETL Routines
Exception Handling Routines
Bi-Temporality
Layer
Golden Copy
Distribution ETL
Flat Files
Managed RDBMS Copies
Message Bus
NoSQL Copies
✓
Sample Reference Data System Architecture
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 8
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 8
Vendor Control
Data Operations
Exceptions Dashboard
Corrections Interface
Distributions Dashboard
Vendor Feeds Ingest / Conversion Golden Copy Distribution
Thomson Reuters
Bloomberg
Markit
Telekurs
CME Direct
Etc.
Exchange Feeds
Staging Prep
Staging Tables
ETL Routines
Exception Handling Routines
Bi-Temporality
Layer
Golden Copy
Distribution ETL
Flat Files
Managed RDBMS Copies
Message Bus
NoSQL Copies
✓
Sample Reference Data System Architecture
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 9
Business Value of Good Reference data Data Errors and Repairs Reconciliation, exception processing, convoluted models, trade repairs, valuation miscalculation, settlement instruction mismatches, bad corporate action processing Data Utilization Manual processes, maintenance of proprietary feeds and interfaces, redundant systems, duplicate master files, duplicate accounts, integration and transformation challenges Data Reporting: Accurate reporting for Risk / Reg oversight Avoidance of fines $$
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 10
To Deliver Value a Reference Data Platform MUST:
Maintain complex hierarchies, nested metadata
Handle any arbitrary data model
Maintain the associations
Standardize bitemporal-dependent queries
Adapt to ingestion spikes
Caution
Reference data vendors can change nomenclature at any time!
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 11
Continuing Challenges of Reference Data Endpoint system mismatches
Fixed schemas – (field lengths) Vendor platform ingestion cycles (Periodic batch FTP) Disinvestment in perceived “legacy” platforms
Domain specific “reference data” (technical B2B connectivity settings etc) does not get centrally mastered as it not considered “business relevant”
Outsourcing creates trouble ticket cost recovery mentality
“ The Data Mandate
Systemic risk is real (mitigation is a legal obligation)
Financial stability oversight pressure is on regulators and
market authorities
Fragmented reporting processes (for both systemic risk and transparency goals) require data harmonization for
comparability and analysis
Data harmonization is not the case today (regulators can’t fulfill their obligation)
We finally understand the intractable relationship between data precision and linked risk analysis (rare opportunity to
implement common data infrastructure to support both financial stability oversight and operational efficiency)
The goals are data harmonization, compounding
consistency and process transparency – across the entire financial system
Governance is needed to ensure that the “data mandate”
remains a priority
Two primary tasks: (1) implement the standards-based infrastructure and (2) adopt data management best practice
The Regulators Are Watching Carefully
A bank should establish integrated data taxonomies and architecture across the banking group, which includes information on the characteristics of the data (metadata), as well as use of single identifiers and/or unified naming conventions for data including legal entities, counterparties, customers and accounts. Roles and responsibilities should be established as they relate to the ownership and quality of risk data and information for both the business and IT functions. Basel Committee on Banking Supervision Principles for effective risk data aggregation and risk reporting, Jan 2013 ”
Page 13
Keeping the Regulators Happy …
Page 13
Business Objective
Data as Factors of Input
C O N T R O L S
G O V E R N A N C E
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 14
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 14
The need for flexible schemas: Interest Rate Swap
Interest Rate Swap
Type: Vanilla Leg-1
Cashflow
Type: Fixed
Notional
Cashflow
Cashflows
Payment Date
Payment Date
Payment Amount
Payment Amount
Counter-party
Cashflow
Type: Floating
Cashflow
Cashflows
Payment Date
Payment Date
Payment Amount
Payment Amount
Counter-party
Leg-2
Hierarchical Requires shredding for
RDBMS Potentially long life-
time
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 15
The Need for “Consistent Bitemporality” Financial Instruments have long lifespans – 30 Yr Tbond – UK Gilts etc Derivative Trades exploit these lifespans – Interest Rate Swaps, Convertible
Bonds etc to provide long term volatility protection
BUT… .
Organizations change all time Internal responses to regulation and tax legislation External M&A or non core disposals
N.B. Bitemporality occurs at the leaf nodes and composite entity levels
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 16
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 16
The Need for Sustainable Extensibility A major bank wants to organize reference data. The bank was formed as a merger of dozens of banks and investment houses How to merge all the data bases? … especially when there are always more databases around the corner?
?
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 17
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 17
Sustainable Extensibility – the Solution They built a “model” of the data Turned all the legacy data into graphs Mapped the graphs to the model and each other. As time goes on, they include other datasets incrementally into the federation.
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 18
Where to Begin Start at the “edges” of the process flow Greatly improve the ETL process
Faster time-to-delivery Discovery on ingestion
Improve the downstream distribution processes Schema-on-read to support heterogeneous down-stream representation Leverage enterprise integration capabilities
Over time, become the Golden Copy as benefits are realized
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. SLIDE: 19
Recap:- What we have covered today
Reference Data continues to be an operational challenge for Financial Organizations It has multiple complexity dimensions:- Suppliers, Aggregation, Distribution, Reconciliation… Regulators are scrutinizing and mandating capabilities in every increasing detail
To address these challenges we need to build solutions with:-
Global Coverage
Sustainable Extensibility
Consistent Bi-Temporality
Learn More About Reference Data & NoSQL
Read
nosqlfordummies.com
Listen Learn Meet
marklogic.com/training [email protected] www.po.st/D11POH
© COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. © COPYRIGHT 2013 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
Thank you! For more information
Rupert Brown <[email protected]> & LinkedIn
Dean Allemang <[email protected]>
Ken Krupa <[email protected]> @kenkrupa, also on LinkedIn
