Powering Self Service Business Intelligence with Hadoop and Data Virtualization

Powering Self Service Business Intelligence with Data VirtualizationSean Roberts, HortonworksMark Pritchard, DenodoJanuary 2017

2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved2 © Hortonworks Inc. 2011 – 2017. All Rights Reserved

Powering the Future of DataSean RobertsPartner Engineering


Data Doubles Every Two Years

44ZB By 2020

3 © Hortonworks Inc. 2011 – 2017. All Rights Reserved


The Old WaySystem Centric

Procedural

Hierarchical

Scheduled

Homogeneous

The New WayUser Centric

Agile

Dynamic

Real-Time

Contextual


Data Doubles Every Two Years

44ZB By 2020


Capture streaming data

Deliverperishable insights

Combinenew & old data

Store data forever

Access a multi-tenant data lake

Modelwith more data

DATA AT RESTDATA IN MOTION

ACTIONABLEINTELLIGENCE

Perishable Insights Historical Insights


Secure

Real-time

Streaming

Integrated

Hortonworks DataFlow for Data in MotionPowered by Apache NiFi, Kafka, and Storm


Hortonworks Data Platform for Data at RestPowered by Open Enterprise Hadoop

Open

Interoperable

Ready

Central

8 © Hortonworks Inc. 2011 – 2017. All Rights Reserved 8 © Hortonworks Inc. 2011 – 2016. All Rights Reserved

Claims Optimization

SocialSentiment

Cohort Selection Bill ShockPhysician

NotesDevice

Monitoring

R & DQuality Benchmarks

PatientExperience

Seasonal Staffing

Net Promoter

Score

Supply Chain

SentimentAnalysis

PatientOutreach

360°PatientView

Patient Throughput

Customer Churn

Analysis

STARS Ratings

Genomics

Remote Monitoring

Drug Diversion

CensusProactive Maintenance

PreventativeMedicine

Inventory

MedicationSafety

OPEXReduction

Lab Notes Archive

MainframeOffloads

Device Data

Ingest

Rapid Reporting

DigitalProtection

Dataas a

Service

FraudPrevention

Real-time Decision Support

I N N OVAT E

R E N OVAT E

E X P L O R E O P T I M I Z E T R A N S F O R M

A C T I V EA R C H I V E

E T LO N B O A R D

DATAE N R I C H M E N T

DATAD I S C OV E RY

S I N G L EV I E W

P R E D I C T I V EA N A LY T I C S

H EA LT H C A R E

Care-path Best Practices

OR Optimization

HCAHPSScores

Staffing Predictions

Proactive Outreach

Legacy System

DataImaging Archive

Historical PatientRecords

Improved Drug Yields


Actionable Intelligence Makes Healthcare Precise and Personal

Patient Records

Lab Data

Pharmacy Data

Patient Locations

Wearables

Intra-Network Data

Sensor Data

Claims Data

Social Media Physician

NotesPatient

Satisfaction Data

Clinical (EMR) Data

SINGLE VIEW OF PATIENT

REAL-TIME VITAL S IGN MONITORING

BILLING & REIMBURSEMENTS

EMR OPTIMIZATION

SUPPLY CHAIN OPTIMIZATION


Mercy’s Journey

Mercy Medical System Sought a Data Lake for a Single View of its Patients – “One Patient, One Record”

Existing platform impeded goal of enriching Epic data for 1 million patients over 35 Hospitals and 500 clinicsMoving Epic EMR data to Clarity EDW took 24 hours and was “never goingto enable real-time analytics”. Now that takes 3-5 minutes with HDP.Improved billing processes resulted in $1M additional annual revenuefrom newly documented secondary diagnoses and care


Better Health Through “One Patient, One Record”

ACTIVE ARCHIVELab Notes

DATADISCOVERY

OPEX Efficiency

SINGLE VIEWBilling

DATA DISCOVERYVital Signs

SINGLE VIEWSingle Patient

Record

ACTIVE ARCHIVEEpic EMR

Replication

ACTIVEARCHIVEPrivacy

Database

DATA ENRICHMENT

Epic Enrichment

PREDICTIVE ANALYTICS

Device Data Ingest

Move to Clarity wouldn’t enable real-time analytics

Existing platform impeded goals

Data enrichment needed for 1 million patients

Move off Epic took over 24 hours

S I T UAT I O N

3-5 Minutes

$1M Additional Annual Revenue

From “Never”to “Seconds”

900x Faster

move data off Epic to Clarity with HDP

from improvedbilling process

acceleratedresearcher insight

ingest of ICU vital signs

P R E D I C T I V EA N A LY T I C S

Preventive Care

ETL OFFLOADMedical Decision

Support


Interoperable with Leading Technologies

Partners


Information spread across different systems

IT responds with point-to-point data integration

Takes too long to get answers to business users

The Self-Service Challenge Data Is Siloed Across Disparate Systems

MarketingSales ExecutiveSupport

Database

AppsWarehouse Cloud

Big Data

Documents AppsNo SQL“Data bottlenecks create business bottlenecks.”

– Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015


Denodo and Modern Data Architecture

Powering Self Service Business Intelligence with Data VirtualizationMark PritchardJanuary 2017

Agenda1.Data Virtualization and Benefits

2.Case in Point – Vizient

3.Modern Data Architecture

4.Self-Service BI on Distributed Datasets

5.Denodo Platform for Data Virtualization

6.Q&A

Data Virtualization and Benefits

18

-Source: “Gartner Market Guide for data virtualization – 2016”

Data virtualization technology can be used to create virtualized and integrated views of data in memory (rather than executing data movement and physically storing integrated views in a target data structure), and provides a layer of abstraction above the physical implementation of data.”

Data Virtualization – Definition

19

Data VirtualizationReal-time Data Integration

“Data virtualization integrates disparate data sources in real time or near-real time to meet demands for analytics and transactional data.”

– Create a Road Map For A Real-time, Agile, Self-Service Data Platform, Forrester Research, Dec 16, 2015

Publishes the data to applications

Combines related data into views

Connects to disparate data sources

2

3

1

20

Denodo and Modern Data Architecture

21

Benefits of Data Virtualization“Get it Real-time and Get it Fast!”

Better Data IntegrationLower integration costs by 80%.

Flexibility to change.

Real-time (on-demand) data services.

Complete InformationFocus on business information needs.

Include web / cloud, big data, unstructured, streaming.

Bigger volumes, richer/easier access to data.

Better Business OutcomeProjects in 4-6 weeks.

ROI in <6 months.

Adds new IT and business capabilities

“Benefits of Data Virtualization: get it real-time and get it fast!” – William McKnight, President, McKnight Consulting Group

Case in Point - Vizient

23

Who is Vizient?Network for non-profit hospitals and alliance of academic medical centers

Network of not-for-profit healthcare organizations to

improve performance and efficiency in

clinical, financial and operational

management

Combination of VHA, University

HealthSystem Consortium, Novation,

MedAssets Spend, Clinical Resources

Management and SG2

Experts with purchasing power,

insights and connections that

accelerate performance for

members

24

Purpose, Mission and Strategic Aspirations

Mission To connect members with the

knowledge, solutions and expertise that accelerate performance

Strategic Aspirations To become an indispensable partner to healthcare organizations

Purpose To ensure

members deliver exceptional, cost effective care

25

Vizient delivers brilliant, data-driven resources and insights — from benchmarking and predictive analytics to cost-savings — to where they’re needed most.

Empowering Brilliant Connections

Modern Data Architecture

27


HTTP

FTP

HL7

Flat File

Share

Subscribe

Attach

Initial

Event

Share

Persist

Event

Share

Ongoing

Event

Usage

Purchase Data

OpenData

RAW CLEAN AGGREGATED ENRICHED

RDBMS

RDBMS

ODS

DW

STAGESOURCE PROCESS PERSIST SERVE

Rules

Batch

Human Process

Hadoop

Hadoop

Lake

Machine Learning

Analytics

Consulting

28


HTTP

FTP

HL7

Flat File

Share

Subscribe

Attach

Initial

Event

Share

Persist

Event

Share

Purchase Data

OpenData

RAW CLEAN AGGREGATED ENRICHED

RDBMS

RDBMS

ODS

DW

STAGESOURCE PROCESS PERSIST

Ongoing

Event

Usage

SERVE

Rules

Batch

Human Process

Hadoop

Hadoop

Lake

Machine Learning

Analytics

ConsultingData

Virt

ualiz

atio

n

Powering Self-Service Discovery with Data Virtualization

30

Financial Data MartPrimary Use Caseo Unify disparate accounting and finance data marts across various legacy

organizations into a shared repository

Secondary Use Cases o Provide a unified source for key BI initiatives like the GPO Dashboardo Support reporting needs as legacy systems are migrated or replaced during

integration of Vizient and L-MDAS (dbVision, etc.)o Provide a final resting place for archived legacy sources like Solomon, Epicor,

etc. VHA

MedAssets

UHC

Vizient

31

Financial Data MartArchitectural Approach

Denodo was selected as the data platform in order to utilize the following features of the software:

o Data Virtualization allows sources in various mediums and locations to be integrated without physically moving the data

o Data Abstraction allows data to be represented consistently within the DataMart while data sources are moved or replaced behind the scenes

o Data Integration allows for a single seamless view to be created across a subject area (e.g. “Supplier Sales”) with varied data transformation rules for each data source within the subject area (PRS, dbVision).

32

GPO DashboardPrimary Use Case

o Provide a consolidated view of supplier sales data across all customers of legacy Vizient & Med Assets organizations.

Architectural Approach

o Financial DataMart (on Denodo) for data source o Denodo TDE Exporter Tool for daily data extracts to Tableau:

Report Data Report User Security

o Tableau for report development and distribution

33

GPO DashboardKey Challenges

o Balance between data timeliness and report performance Tableau reports performed best utilizing the TDE format (cached/extracted dataset) as

opposed to a live connection This meant that the report caches required daily refreshes, and data extraction had to be

appropriately tuned Denodo features such as dataset statistics and indexing greatly contributed to this

performance tuning

o Provisioning user security at cell level The requirement for some internal report users to be restricted to the

members/customers to which they are assigned meant that a new report security approach was needed

Reliance on TDEs for report data necessitated the integration of security in the reporting layer

Tableau’s “data blending” feature allows user security to be specified within a separate dataset

This also supports reuse of the security view in other reporting environments.

34

Contract Sales Actualizer DashboardPrimary Use Caseo Integrate Member Spend and Supplier Sales data from all Vizient organizations to identify

opportunities for increasing contract utilization

Other Use Caseso Maintain consistency (Single Source Of Truth) with GPO dashboard regarding:

Supplier Sales Data Dimension Data User Security

Architectural Approach o Data source utilizes Denodo to reuse overlapping datasets (sales, dimensions,

security) while allowing separate virtualized views to be created for new datasets (member spend) which can be also be reused by future projects

o Reporting components match approach used by GPO Dashboard

35

Contract Sales Actualizer DashboardKey Challenges

o Successful integration of Exadata RDM as a data source for Denodo. Approach utilizes the strength of Exadata RDBMS for aggregating large

quantities of data quickly Denodo to integrate the data with similar legacy SQL Server data sources to

create a comprehensive view of Vizient member spend

o Scalability/Configuration Management Advances were made to support parallel development of this project and

continued efforts on GPO dashboard Compartmentalization features within Denodo allow for code changes in each

project to be version controlled and assessed for dependencies Process guidelines are being authored to allow for multiple development efforts

on the same datasets

Denodo Platform for Data Virtualization

37

Accelerate Your Fast Data Strategy with Denodo Platform 6.0New Release of Denodo Platform Delivers Breakthrough Performance, Accelerates Adoption, and Expedites Business Use of Data

Breakthrough PerformanceDynamic Query Optimizer delivers breakthrough performance for big data, logical data warehouse, and operational scenarios.

Data Virtualization In the Cloud

Denodo Platform for AWS accelerates adoption of data virtualization.

Self-service Data Discovery and SearchSelf-service data discovery and search expedites use of data by business users.

“Very happy with Denodo version 6. Well done!” – Claudia Imhoff, President, Intelligent Solutions

38

Common Data Virtualization Use CasesData Virtualization

BIG DATA, CLOUD INTEGRATION

Advanced Analytics Data Warehouse Offloading Big Data for Enterprise Cloud / SaaS Integration

AGILE BUSINESS INTELLIGENCE

Logical Data Warehouse Virtual Data Marts Self-Service BI Operational BI / Analytics

SINGLE VIEW APPLICATIONS

Single Customer View - Call Centers, Portals Single Product View - Catalogs Single Inventory View - Inventory Reconciliation Vertical Specific - Single View of Wells

DATA SERVICES

Unified Data Services Layer Logical Data Abstraction Agile Application Development Linked Data Services

39

Why Denodo Data Virtualization?Data Virtualization

Exceptional pre-sales service and customer support

Technical expertise and response time rated best by costumers.

Enhancement requests typically made available in days / weeks.

Lower overall TCO and ROI Address more use cases with a single platform. Aggressive “all-in-one” pricing for entire platform;

flexible pricing models.

Broad functionality and winning innovation in a fully integrated platform

Flexible and broad use cases. Integrated purpose-built for data virtualization – very

easy to develop / deploy / change models.

Q&A

Thanks!

www.denodo.com [email protected]© Copyright Denodo Technologies. All rights reservedUnless otherwise specified, no part of this PDF file may be reproduced or utilized in any for or by any means, electronic or mechanical, including photocopying and microfilm, without prior the written authorization from Denodo Technologies.

Data & Analytics

Powering Self Service Business Intelligence with Hadoop and Data Virtualization