Page 1 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
The Modern Data Architecture for the Insurance Industry
Page 2 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Today’s Speakers
Tripp Smith, CTO Clarity Solution Group
Cindy Maike, GM-Insurance Hortonworks
Page 3 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
The Insurance Industry Data Equation …the current situation
Page 4 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Trends require a shift from product to prevention
HDP and Hadoop allow insurers to shift interactions from…
Reactive Post-Transaction
Proactive Pre-Decision
…to prevention services customized for needs From traditional coverage
…to proactive advisors From siloed information
…to 1x1 targeting & engagement From “mass-market”
A shift in Customer Engagement
A shift in Products
A shift in Agent/Broker and Call Center Support
…to ‘valid and pay’, anomaly detection and severity From “a claim is a claim”
A shift in Claims Management
Page 5 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Consensus of Analysts estimate enterprise data growth of 50x year over year through 2020
Data is growing exponentially at unprecedented rates
0 5 10 15 20 25 30 35 40
2020
2018
2015
2013
The “Digital Universe” expressed in Zettabytes* 85% of growth from new types of data with machine-generated data increasing 15x *Multiples of Bytes
Kilobyte Megabyte Gigabyte Terabyte Petabyte Exabyte Zettabyte Yottabyte
Page 6 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Clickstream Capture and analyze website visitors’ data trails and optimize your website
Sensors Discover patterns in data streaming automatically from remote sensors and machines
Server Logs Research logs to diagnose process failures and prevent security breaches
New types of data
Sentiment Understand how your customers feel about your brand and products – right now
Geographic Analyze location-based data to manage operations where they occur
Unstructured Understand patterns in files across millions of web pages, emails, and documents
Data is growing exponentially – causing IT delays
Page 7 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Enabling a Mature Enterprise Strength Hadoop
Opportunities with Using Hadoop
Information Agility Increased Opportunity
• Single point of access to high-priority enterprise assets, transactional assets and Dark Data
• Centralization of data combined with decentralization of analytic capabilities
Processing Horsepower Increased Capacity
• Near-linear hardware capacity scalability • Portfolio of components that scale to data or
computational complexity
New or Expanded Analytics Expanded Capability
• Increased depth of conventional analysis • Application of analytics to real-time needs • Deep machine learning and discovery analytics
Cost Containment Reduced Expense
• Cheaper than enterprise SAN or proprietary RDBMS • Scalable with inexpensive hardware vs. expensive
optimization or recoding
"Out of the Box" Challenges
Manageability – Risk – TCO Spiral
• Platform security and repeatable processes for securing data
• Common vocabulary and business data definitions
• Consistently applied data integration and transformation processes
• Transparent data quality and data lineage
• Ability to manage complex mixed workloads and a variety of access patterns to support disparate user groups and use cases
• Time which instead is spent on data forensics rather than analysis
Operational and Interactive Platforms
Page 8 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
MESH achieves agility with integrity
The Mature Enterprise Strength Hadoop (MESH) framework addresses scaling a Hadoop ecosystem that meets enterprise needs
Matrix of architecture, governance and enablement capabilities
Integrated ecosystem addressing the full breadth of enterprise analytics, users and use cases
Enterprise-strength security and achievable data governance
Automation and acceleration across implementation, governance and enablement vectors
Operational roadmap and tool kit to activate business value through analytic agility
Managed Raw Materials
Structured Integration and
Discovery
Governed End-User
Consumption
Organic, Process-Driven
Information Refinement
Page 9 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
3 Dimensions of Success: Architecture, Governance, and Enablement
Security and Role Based Access Controls
Acquisition and Ingestion
Archival Data Management
Event Processing
Data Transformation
Master Data Integration
Information Delivery
Discovery Analytics
Machine Learning
Common Vocabulary and Data Definitions
Tool Rationalization
Process Automation
Testing and Quality Assurance
Resource and Workload Management Processes
Data Quality and Stewardship
Page 10 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
The components beyond the warehouse
Iterative workspace for deep analytics and data discovery
Enablement and alignment
Streamlined service-driven data integration
Incremental enrichment of analytic data service
Governance and operational clarity
Page 11 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Big Data & Hadoop market drivers and opportunities
Business Drivers
• From reactive analytics to proactive customer interaction
• Insights that drive competitive advantage and optimal returns
Financial Drivers
• Cost of data systems, as % of IT spend, continues to grow
• Cost advantages of commodity hardware and open source software
$
Technical Drivers
• Data is growing exponentially and existing systems are overwhelmed
• Predominantly driven by NEW types of data that can inform analytics
Page 12 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
The Modern Data Architecture …use case examples of Insurers using Hortonworks Data Platform
Page 13 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Customers using HDP to meet Insurance challenges
Top 3 Challenges Relevant Insurance Use Cases
Change in Customer Engagement Model
Rising Claims Costs (frequency and severity)
Data Explosion (Complexity of Risk/
Underwriting Information)
Personalization / Next Best Action
720º Degree Customer Visibility
Risk/Underwriting Profile Analysis
Sensor-based Telematics (Prevention Services, UBI)
ETL/EDW Optimization
Claim anomaly and fraud detection
Page 14 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Use Case Analysis
Page 16 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Enhanced Insurance cross-sell and catching fraudulent claims
Problem: ETL challenges with multiple data streams hampers analysis on ways to improve customer service
• Traditional and newer types of data were difficult to combine in the EDW, because of
“schema on write” architecture some data was discarded
• Company missed data-driven ways to serve customers better
• Poor data visibility hampered analysis separating legitimate from fraudulent claims
Solution: Data lake to improve up-sell and identify fraud
• “Schema on read” architecture ingests more data sources for predictive analytics
• Agents use new insights to provide higher service levels to valued customers
• Claims analysts and underwriters process streaming data to quickly flag fraud risks and
fast-track legitimate claims
Insurance – Health
Large US medical insurer
IH2
Why Hadoop?
Data Systems Optimization
Claim Anomalies & IT Optimization
Page 17 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Improved risk analysis and margins for usage-based car insurance
Problem: Slow ETL processing hampered speedy underwriting of usage-based car insurance
• Usage-based car insurance requires rapid ingest and analysis of sensor data
• Volume, velocity and variety of incoming data taxed existing systems and the high cost
of storage eroded margins
• ETL process only captured 25% of the dataset and took 5-7 days to complete
Solution: Faster time-to-insight, improved ETL & predictive analytics
• Built Azure POC cluster to justify the big data project before launching HDP on site
• Improved performance and predictive analytics with Apache Hive
• Faster ETL in Hadoop now processes 100% of the data in three days or less
Insurance – Property & Casualty
Personal auto & other property-casualty insurance
IP1
Why Hadoop?
Predictive Analytics
Telematics/UBI New Analytic Applications
Sensor Data and ETL Offload
Page 18 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Improved visibility and accuracy for P&C Insurance claim analysis
Problem: Analysis of unstructured data did not keep pace with mature systems for analyzing structured data
• Large P&C insurance provider had systems for analyzing structured data at scale
• Unstructured data from claims notes and social media data could add valuable
information to claims analysis, but is was unable to analyze this data at scale
• Impartial data visibility hampered underwriting and claims, driving up costs, eroding
margins and blocking efforts to reduce fraudulent claims
Solution: Join structured and unstructured data for accuracy in claims processing, reducing risk, processing costs and fraud
• “Schema on read” architecture captures more data sources (text and social data)
• Larger data sets fed to front-end business tools provided by Hortonworks partners: SAS,
Tableau and QlikView
Insurance – Property & Casualty
Major provider of property casualty, life and mortgage insurance
IP2
Why Hadoop?
Data Discovery
Claims: New Analytic Applications Structured, Social & Unstructured Data
Page 19 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Case Study: 12-month Hadoop evolution at TrueCar
Dat
a Pl
atfo
rm C
apab
ilitie
s
12 months execution plan
June 2013 Begin Hadoop Execution
July 2013 Hortonworks Partnership
May ‘14 IPO
Aug 2013 Training & Dev Begins
Nov 2013 Production Cluster 60 Nodes 2 PB
Jan 2014 40% Dev Staff Perficient
Dec 2013 Three Production Apps (3 total)
Feb 2014 Three More Production Apps (6 total)
12 Month Results at TrueCAR • Six Production Hadoop Applications • Sixty nodes/2PB data • Storage Costs/Compute Costs
from $19/GB to $0.23/GB
“We addressed our data platform capabilities strategically as a pre-cursor to IPO.”
Leverage commodity hardware for efficient data storage
Page 20 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Optimized marketing impact with advanced customer analytics
Problem: Difficulty understand marketing effectiveness and customer behavior
• 25M customers and prospects conducting over 10M weekly corporate interactions
• Lack of visibility into effectiveness of marketing spend and impacts on consumer
behavior
• Difficulty modelling behavior across disparate data sources from internal enterprise
master data and external vendors
Solution: Advanced analytics and experiment design
• Closed-loop marketing analytics across key enterprise business units
• Informed tactical decisions that ensure efficient marketing spend
• Quantification of marketing effectiveness and audience behavior
• Strategic insight for enterprise marketing efforts to evolve cross-sell to customers and
drive revenue generation
Insurance
Large US-based financial services and insurance company
IH1
Why Clarity?
Advanced Customer Engagement
Personalization / Next Best Action
Page 21 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Consumer 720 – Enterprise data with new data for personalized customer engagement
720º Degree Customer Visibility
Enterprise Master Data Evolution
• Companies continue to evolve the single version of key Customer data for use throughout the enterprise
Social and Interactive Data Challenges
• Additional data from Social Media sources provide additional consumer insight, but data integration challenges prevent this insight from turning into action
Limited Options for Enablement
• There is no solution in the marketplace today that allows companies to seamlessly integrate core customer data needed to manage a business along with the vast amount of social data that these customers use to express affinity
Inside the Enterprise
Outside the Enterprise In-store
Activity
Service & Support
Data Enrichment
Online Purchases Enterprise Social
360 360
Consumer Engagement Roadblocks
• Inability to accurately identify customers across the enterprise
• Key marketing systems (campaign management, analytics, CRM, etc.) unable to leverage holistic customer data
• Time and effort wasted with validating, integrating and managing consumer data across the enterprise
Co
nsum
er D
ata
Cha
lleng
es
Enterprise Customer Profiles
Internal Sources
3rd-parties
Consumer720
Enterprise Customer Attributes used for identity
Digital Interactions
Personalized Customer Experience
Page 22 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Insurance business value matrix using HDP
New
Typ
es
of A
naly
tics
New Types of Data
New Types of Data New Analytics Apps
• Sentiment
• Click-stream
• Sensor
• Geographic
• Server Logs
• Unstructured
Existing Data
Exis
ting
Ana
lytic
s
RDBMS
MPP
EDW
• EDW & ETL data & load balancing
• Cost & flexibility • Building new skill sets • Scale out using
commodity hardware
• Single-View of Customer showing full 360-degree profile and history
• Clickstream analysis for Next Best Action with Customers
• Analyzing submission and claims models against larger historical data sets
HDP
HDP
New Historical View
IT Optimization New Data Influencers
• Collecting Sensor/Telematics for Usage Based Insurance
• Sentiment • Enhanced Loss Control /
Prevention Services • Needs based coverage vs.
traditional coverage
HDP
New Analytics Applications • Text Analytics and Link
Analysis for Claim Anomaly and Fraud Analysis/Detection
• Enhance Risk Analysis with Related Party Network Link Analysis
• Enhanced Claim Severity and Frequency Models using “new” predictive data
HDP
Page 23 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Evaluation Biz Value
Awareness & Interest
Evaluation Technical
Enterprise Deployment
Enterprise Production
Point Deployment
Point Production
* Timeline varies by company size. Often smaller or focused online businesses achieve milestones at the shorter end of the range.
1 – 2 months 2-6 months
9-15 months 18-36 months
Start small and grow over time…
1 2 3 4 Potential Operational Strategic Data-Driven
Data Lake
Modern Data Architecture
Industry Leadership
Page 24 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Clarity Solution Group and Hortonworks Background and Focus …how we can help the Insurance Industry
Page 25 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Elegant solutions for insurance business needs
Clarity helps top insurers tackle:
• Telematics & UBI for pricing and risk
management
• 360-Degree Customer Views to improve cross-selling
• Multi-Channel Optimization to measure marketing effectiveness and improve customer experience
• Distribution Channel Analysis to reduce costs, improve retention and drive profitability
• Underwriting Optimization to reduce loss and improve pricing
• Product Development to increase customer satisfaction and target new markets
The largest independent US services firm exclusively focused on data and analytics
Page 26 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Hadoop for the Enterprise: Implement a Modern Data Architecture with HDP
Customer Momentum
• 230+ customers (as of Q3 2014)
Hortonworks Data Platform • Completely open multi-tenant platform for any app & any data. • A centralized architecture of consistent enterprise services for
resource management, security, operations, and governance.
Partner for Customer Success • Open source community leadership focus on enterprise needs • Unrivaled world class support
• Founded in 2011 • Original 24 architects, developers,
operators of Hadoop from Yahoo! • 600+ Employees • 800+ Ecosystem Partners
Page 27 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Only HDP delivers a Centralized Architecture for the modern data needs HDP is uniquely built around YARN serving as a data operating system that provides multi-tenant Resource Management, consistent Governance & Security and efficient Operations services across Hadoop applications.
Hortonworks Data Platform
YARN Data Operating System • A centralized architecture of
consistent enterprise services for resource management, security, operations, and governance.
• The versatility to support multiple applications and diverse workloads from batch to interactive to real-time, open source and commercial.
Key Benefits
• Multiple applications on a shared data set with consistent levels of service: a multitenant data platform.
• Provides a shared platform to enable new analytic applications.
• Delivers maximum cost efficiency for cluster resource management. Fewer servers fewer nodes.
Storage
YARN: Data Operating System
Governance Security
Operations
Resource Management
Existing Applications
New Analytics
Partner Applications
Data Access: Batch, Interactive & Real-time
Page 28 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Customer partnerships matter Driving our innovation through Apache Software Foundation Projects
Apache Project Committers PMC Members
Hadoop 27 21
Pig 5 5
Hive 18 6
Tez 16 15
HBase 6 4
Phoenix 4 4
Accumulo 2 2
Storm 3 2
Slider 11 11
Falcon 5 3
Flume 1 1
Sqoop 1 1
Ambari 34 27
Oozie 3 2
Zookeeper 2 1
Knox 13 3
Ranger 10 n/a
TOTAL 161 108 Source: Apache Software Foundation. As of 11/7/2014.
Hortonworkers are the architects and engineers that lead development of open source Apache Hadoop at the ASF
• Expertise Uniquely capable to solve the most complex issues & ensure success with latest features
• Connection Provide customers & partners direct input into the community roadmap
• Partnership We partner with customers with subscription offering. Our success is predicated on yours.
27
Cloudera: 11
Facebook: 5
LinkedIn: 2
IBM: 2
Others: 23
Yahoo 10
Page 29 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Q&A: Please use the Q/A panel to ask your questions! More Information: Clarity Solution Group - clarity-us.com Hortonworks - hortonworks.com Industry email Updates A Modern Data Architecture Whitepaper The Rise of the Data First Enterprise
Speaker Contact Information: Tripp Smith, CTO Clarity Solution Group: [email protected] Cindy Maike, GM-Insurance Hortonworks: [email protected]
Page 30 © Hortonworks Inc. 2011 – 2014. All Rights Reserved © 2015 Clarity Solution Group, LLC
Our Missions: Clarity Solution Group: We help businesses decrease time to market and reduce costs by providing data and analytics solutions to discover hidden trends and find value in the data they already have.
Hortonworks: To enable Apache Hadoop to be the enterprise data platform that powers the modern data architecture and process half the worlds data