Upload
sygroup
View
12
Download
3
Tags:
Embed Size (px)
Citation preview
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DATA DISRUPTION!AND INNOVATION"
1
Closing Keynote"DBTA Data Summit – May 13, 2015""
John O’Brien | Radiant Advisors " @obrienjw [email protected]"!
!
@obrienjw [email protected]
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DIGITAL BUSINESS TRANSFORMATION"Data Disruption and Innovation"
The world around us "
has changed "
to consumer-centric and data-centric economy"
where companies "
must transform "
to remain competitive "
…and survive"
2
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
3 STEPS TO FOLLOW"Data Disruption and Innovation"
3
Business Disruption" Platform of Speed"Capability by Design"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
THE POWER OF DISRUPTION"Business Disruption"
•! Understand digital life"
•! Understand new consumers"
•! Experiment and move fast"
•! Think like a startup"
•! Competitors are moving"
•! Innovate then optimize"
•! Practice Creative Destruction"
4
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
FAMILIAR STORIES?Business Disruption"
5
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
“Advantages”"• Ready to compete on analytic ability• Deeper understanding of your customers• Deeper understanding of your products usage
“Data Centric”"• Treat data as an asset – Treat data as oxygen• Willing to give services for free• Willing to buy data
DIFFERENT MINDSET NEEDEDBusiness Disruption"
6
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
CUSTOMER LIFETIME VALUE"Business Disruption"
7
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
BRAND-TO-CUSTOMER"Business Disruption"
8
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
PRODUCT RELATIONSHIPS"Business Disruption"
9
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
“Brand Analytics”• Sentiment analysis
• Partnerships
• Influence
“Customer Analytics”• Identify potential valuable customers
• Nurture to customers
• Engage lifelong customers
BRAND-TO-CUSTOMER FOCUS"Business Disruption"
10
Brand Loyalty
Becomes
Customer Loyalty
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
“Mobile”• Individually centered – Me, Here, Now
• Generation “C” Connected
• Opportunity for more data
“Engaged”• Engaging experiences
• Throughout their day
• Platform invitation
CONNECTED CUSTOMER"Business Disruption"
11
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
ALWAYS ON THE GO"Business Disruption"
12
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
OPPORTUNITY TO INFLUENCE"Business Disruption"
13
RT Gr8 #Breakfast at @FavCasino #GoBuffet #Reno
Gr8 #datasummit #keynote w/ @obrienjw #GoBigData
#Empowered
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
EXPERIENCES CAPTURED"Business Disruption"
14
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
BE A PART OF SHARING"Business Disruption"
15
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
WHAT’S NEXT?"Business Disruption""
16
• What would it take for you to have wearable technology?
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
CAPABILITY BY DESIGN"Data Disruption and Innovation"
•! Discovery is iterative"
•! Experiment / Fail Fast"
•! Exploratory; can be unclear"
•! New paths; new ideas"
•! Driven by unknowns"
•! Foundation for discovery"
17
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
FOUR ELEMENTS OF DESIGN"Capability by Design"
18
Self-Sufficiency"
Visualization"
Collaboration"
Mobile"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DESIGNED FOR SELF-SERVICE"Capability by Design"
•! Self-sufficiency"
•! From access to ability"
•! Partners in discovery"
•! Greater autonomy"
•! Remove barriers"
19
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DESIGNED FOR COLLABORATION"Capability by Design"
•! Investment strategy"
•! Fosters collective innovation"
•! Closes the business/IT gap"
•! Establishes transparency"
•! Data governance and security"
20
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DESIGNED FOR VISUALIZATION"Capability by Design"
•! Cognitive hardwiring"
•! Makes analytics approachable"
•! Understanding to insight"
•! Curated images and color"
•! Visual discovery"
21
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
DESIGNED FOR MOBILITY"Capability by Design"
•! More than consumption"
•! Visually interact with data"
•! Portable discovery"
•! Personalized experience"
22
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
BI Casual Users - Consumers "• Represent 80% of BI Users
• Reporting and Monitoring generations of BI
Power Users and Business Analysts – Data Provisioners "• Represent 20% of BI Users
• Analysis and Prediction generations of BI
Data Scientists and Analytic Modelers"• A few professionals in the organization•! Creators of statistical models"
INCREASING VALUE GENERATION"Capability by Design"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
ITERATIVE “FRICTIONLESS” DISCOVERY"Capability by Design"
24
Get Data Access
Explore Data
Agilely Blend Data Discover
Collaborate 99x "
for each"Discovery"
“I need to work with new data”"“I want to check something”"
“Is this data usable?”"“What is this data?”"
“Lets mix with other data”"“Does this help my data set?”"
“Is this what I’m looking for?”"“This is interesting…”"
“What do you think of this?” “How can we use this?”
“That helped, need more.”"“Nope, keep looking…”"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
“FRICTIONLESS” DATA INGESTION"Capability by Design"
1 hour steps"x 99"= 495 hours"
= 62 days"""
1 minute steps"x 99"= 495 minutes"
= 8.25 hours"""
25
Get Data Access
Explore Data
Agilely Blend Data Discover
Collaborate
Self-Sufficiency"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Start with business goal, then imagine the data needed
Get external public and purchased data – Big Data
• Facebook, Pintrest, Twitter, Weather, demographics
Create new ways to collect the data you want
• Mobile apps, wearables, sensors, “Internet of Things”
Current systems are the feedback loop
1. COLLECT DATA FROM EVERYWHERECapability by Design
26
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Customer Insights
Product Insights
Optimize Innovate
2. ANALYTICS CULTURECapability by Design
27
Customers,Their NetworkTheir Influence
Business Functions &Processes
New and Disruptive
Business
InteractionsRelationships
Usage
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
• Ongoing A/B experiments
• Continuously improving analytic models
• Tracking prediction and recommendation engines
• Lean product development and interaction
3. ACT ON ANALYTICSCapability by Design
28
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Governing People, Roles, Responsibilities Define roles and responsibilities for accessing and working with data Define ownership and delegation for semantic context discovered
Governing Discovery Results Governing semantic context after discovery
Operationalizing and monitoring analytic models
Modern Data Platforms Managing data abstraction and mixed workloads
Defining discovery or analytic sandboxes as dev, prod or both
Design for Capability!GOVERNED DATA DISCOVERY
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
PLATFORM OF SPEED"Data Disruption and Innovation"
•! Built to Change"
•! Polyglot Persistence approach"
•! Data unification is key"
•! Platform oriented and not a bunch of database silos"
•! Requires data governance"
•! Enabling platform for analytics and discovery"
!
30
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Op
UX
RDBMS
eCom
UX
Doc
Mobile
UX
Graph
DW/DM
BI Apps
UX
MPP
BI Apps
UX
HDFS
YARN
UX
Appl
icatio
n Da
ta S
ilos
Polyglot Persistence Principle
Op
UX
RDBMS
eCom
MPP
Mobile
UX
Hadoop
Doc
Graph
Relational SQL NoSQL
Optimized Columnar In-Memory
MPP OLAP
Specialized Key-Value Document Distributed
Graph
Transactions Analytics Behavioral Sensors
POLYGLOT PERSISTENCE PRINCIPLE Platform for Speed
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
32
Platform for Speed!DATA TECHNOLOGIES"
Key Value Store (Hadoop) Discovery Oriented
Highest Scalability
Lowest Cost Schema-less
Without Context
Analytic Database
Technologies
EDW RDBMS
Accessibility: Programming -> SQL SQL, MDX, UDF SQL Access
Workload: Flexible, Scalable Analytic Optimized Reference Data Mgmt
Maturity: Emerging Accepted Mature
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
33
Platform for Speed!MODERN DATA PLATFORM"
Flexible Class"
Data Warehouses"
ç"
Analytic Optimized Class" Reference Class"
ç"
Master Reference"Data"
Discovery, Scalable, Programmable" Stable, Context, SQL"Discovery and Analytics Oriented"
Hortonworks"
Cloudera"
MapR"
Cassandra"
MongoDB"
ç"
askdjfl"kasjdfl"iuyuiio"
Highly Specialized for BI"
Highly Optimized for BI"
In-memory"
ç"Columnar" MOLAP"
Graphs"Document"
Stores"Text "
Analytics"
MPP"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Platform for Speed!ARCHITECTED FOR DISCOVERY"
ç ç ç ç
3rd Party
Web
Mobile Mobile
Digital Servers Digital Servers
BI Tool
BI Tool
Social Networks
Replication Technology
ç ç ç ç
ç ç ç ç
ç ç ç ç
Excel
T
Modern Data Platform
T Integration Discovery T Institutional Integrations
Cache
Website Activity
Mobile Activity
IoT Activity
Social Activity
Report User
Users
User
User data
Exte
rnal
Dat
a
Data Lake
T ç ç ç ç
SAS
Analytic Tool
Connect
Connect
VoC Command Ctr
SAS Server
Connect
Inte
rnal
Ope
ratio
nal S
yste
ms
Operational Reporting Tier
ç ç ç ç T
STG EDW
Analytics Optimized Tier
Big Data Discovery Tier
Managed File Transfers
Partner Partner
Beha
vior
al D
ata
Essbase
Analytics Database
ç Analytics ç Analytics Database ç Analytics ç Analytics Database Analytics Database Analytics Database Analytics Analytics Analytics ç Database Database Database Database Database ç
Analytics DB
ç Analytics ç DB Analytics Analytics
ç Analytics Analytics ç Analytics DB Analytics Analytics DB Analytics Analytics Analytics DB DB ç DB DB DB DB DB ç
Statistics
Social Media Aggregator
Data Acquisition Data Unification (SQL/Data Services)
ç ç ç ç
ç ç ç ç T ç
ç
T
ç ç ç ç
ç ç ç ç T ç
ç ç
T ç ç ç
Cache Cache Cache
Tableau Server
Tableau Server
Tableau Server W
Business Objects Business Objects Business Objects U
T
Tableau
Tableau Mobile
Tableau Server
Tableau Server
Tableau Server
ç ç
Governed Discovery
Proxy Server
T
T Local Discovery
Governed Acquisitions
Business Reporting
Business Discovery
Analysts
Social Response
Executive
Data Science
MPP Col
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
35
Platform for Speed!KEY SQL CONSIDERATIONS"
Evaluation Criteria"
SQL Capability"•!Tools Compatibility"•!ANSI SQL"•!Analytic SQL"•!User Defined Functions"
Scalability"•!How many nodes max?"•!All nodes in cluster?"•!Subset of cluster?"•!Data duplication?"
Speed"•!Response time"•!Ad-hoc workloads"•!Without caching"•!Concurrency"
Architecture"•!YARN compatible"•!Data file formats"•!Data Lake strategy"•!Semantic Layer"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
*Vor
tex"
PIG"
Hive
-QL"
MapReduce"
""
Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS
Hadoop v1"
Map
"Re
duce
"
PIG"
Hive
0.
13"
YARN"
""
Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS
Hadoop v2"
PIG"
Hive"
*Vor
tex
M/R"
YARN"
""
Hadoop HDFS"Hadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFSHadoop HDFS
Impa
la, H
AWQ
, Pr
esto"
MPP
Eng
ines"
Hadoop HDFSHadoop HDFS
Platform for Speed!EVOLVING ARCHITECTURE FOR SQL"
Tez"
Tez"
Batch-oriented SQL" Interactive SQL" Architectural SQL"
Hadoop v2"with more SQL options"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Flexibility Class"
37
Enterprise"Data
Warehouses"
ç"
ç"Master
Reference"Data"
Discovery, Scalable, Programs" Stable, Context, SQL"Discovery & Analytics Oriented"
Apache Hadoop "
ç"
Highly Optimized for Analytics"
In-memory" MOLAP"MPP"
Optimized Class" Reference Class"
Gen
erat
e"
Hive
SQ
L"
askdjfl"kasjdfl"iuyuiio"
Highly Specialized for Analytics"
Graphs"Document"
Stores"Text "
Analytics"
P
IG /
Hive"
Map
Red
uce"
Ope
ratio
nal S
yste
ms,
Big
Dat
a, S
tream
s"
HDFS
"
ç"Columnar"
Extending SQL Access to Big Data and Hadoop via Hive and other HDFS SQL engines"
Platform for Speed!MODERN DATA PLATFORM UNIFIED SQL"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Business Demands! Hadoop Generation!
Internet and e-Commerce companies needed to cost-e"ectively work with big data !(both operationally and analytically)!
Hadoop v1 inspired by Google papers and from Apache projects delivered a"ordable scalability and a parallel programming framework!
Business needed a more interactive environment for users and apps with improved resource management and multi-tenancy!
Hadoop v2 improved the two-tier architecture with YARN for managing resources and reorganized for YARN applications !
HADOOP EVOLVES FOR BUSINESS"Platform for Speed"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
Business Demand! Hadoop Generations!
E#ciently integrating big data, BI and operational data apps in a singular data architecture meeting data management requirements!
Hadoop v2x focus on improvements in operational capabilities: SQL, performance, security, governance while Data Lake provides a strategy!
Long term “data operating system” architecture for e!ciently and e"ectively manage data workloads, reusability, and leveraging cloud#
Future Hadoop will focus on IT ecosystem integration, more Hadoop applications with improved distributed file systems#
HADOOP EVOLVES FOR BUSINESS"Platform for Speed"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
YARN Applications M
apRe
duce
(Bat
ch)
UX
Hive
(SQ
L)
UX O
nline
(HBa
se)
UX
Data
Ope
ratin
g Sy
stem
=== === === === ===
=== === === === ===
=== === === === ===
=== === === === ===
HDFS
Sear
ch
(Sol
r)
Tez
(Inte
ract
ive)
Stre
aming
(S
torm
)
In-m
emor
y (S
park
)
ISV
Engi
nes
Data Engines on YARN
Hadoop Engines
ISV Engines
Future Engines
=== === === === ===
=== === === === ===
=== === === === ===
=== === === === ===
Optimized Distributed File Systems
Apps Mobile Machine
Cloud On-Premises
Hybrid Inter-Cloud
Data
Reu
sabi
lity
Avoi
d Da
ta D
uplic
atio
n Ce
ntra
lize
Info
Mgm
t
FUTURE DATA OPERATING SYSTEM Data Disruption and Innovation
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
3 STEPS TO FOLLOW"Data Disruption and Innovation"
41
Business Disruption" Platform of Speed"Capability by Design"
© Copyright 2015 Radiant Advisors. All Rights Reserved v1.10.000
THANK YOU!"
For more information""www.RadiantAdvisors.com ""Twitter: " "@RadiantAdvisors "
"Email: " "[email protected]"LinkedIn: " "www.linkedin.com/company/radiant-advisors "
42