SAP, Big Data, and Telco

Use this title slide only with an image

SAP & Big Data

@timoelliott

Timo ElliottInnovation EvangelistJune 2016

Technology Priorities for 2016 and beyond

Rank Technology Trend

1 BI/Analytics2 Cloud3 Mobile4 Digitalization / Digital Marketing5 Infrastructure & Data Center6 ERP7 Security8 Industry-Specific Applications9 Customer Relationships

10 Networking, Voice, and Data Comms

Nine out ofeleven years2006-2016

ANALYTICS

#1

© 2016 SAP SE or an SAP affiliate company. All rights reserved. 3

Analytics Takes Over The World…

@timoelliott


By 2020, information will be used to

reinvent, digitalize, or

eliminate 80%of business processes and products

from a decade earlier.

From The Back Office To The Business Models of Future

”

“

@timoelliott

5

Digital Business and the Rise of the CDOs

@timoelliott


Bimodal IT…

“CIOs practicing bimodal IT are making a strategic mistake.”

—Forrester

“Customer experiences aren’t confined to a small subset of systems. Even simple purchases needs to reach back into fulfillment and billing systems.”

Live Business


Are you a BI-nosaur?


Culture Change

From Power to Empower

From Collection to Connection

From Control to Trust

“So we don’t need a centralized truth?!”

“Absolutely. Never worked, doesn’t work, will not work”

@frankbuytendijk

“BICCs are Dead”

Data-Driven Approach

Push:• From IT• Data-Driven• Data to Insight• Technology-Centric

Value-Driven Approach

Pull:• From LOB• Outcome-Driven• Insight to Data• Use-Case-Centric

Combination Approach

Push:• From IT• Data-Driven• Data to Insight• Technology-Centric

Pull:• From LOB• Outcome-Driven• Insight to Data• Use-Case-Centric

Use this title slide only with an image

Advanced Analytics@timoelliott

15© 2015 SAP SE or an SAP affiliate company. All rights reserved.

Big Data Discovery =

Big DataData DiscoveryData Science

Gartner Strategic Planning Assumption: By 2017, Big Data Discovery Will Evolve Into a Distinct Market Category


Big Data Discovery

• Volume, velocity, or variety of data

• Potential business impact

• Difficult to implement• Potentially expensive• Lack of skills available

• Ease of use• Agility and flexibility• Time-to-results• Installed user base

• Complexity of analysis

• Potential impact• Range of tools• Smart algorithms• Difficult to implement• Slow and complex• Narrow focus of

analysis

• Limited depth of information exploration

• Low complexity of analysis

BIGDATA

DATASCIENCE

DATADISCOVERY


Big Data Discovery

• Simpler to use than data science

• Accessible to a wider range of users

• Broad range of data manipulation features

• Able to handle new types of data sources

• With adequate performance for big data

BIG DATA

DISCOVERY


Potential impact per user

Potential user base

The Rise of the Citizen Data Scientist?

Business analyst

Data scientist

Citizen data scientist


SAP’s Opportunity

Big Data

Discovery

SAP HANA SAP IQ

Vora / Spark / Hadoop

SAP Predictive Analytics 3.0

SAP Lumira


Bringing it All Together

ETL DW Q&R, Data

Discovery

Predictive Planning Other (spatial,

etc.)

Data Visualization

Operational Reporting

Enterprise Data Big Data


The New Multi-Polar World of Big Data Architectures

Data Warehouse

Hybrid Transaction/

Analytical Processing

Hadoop,MongoDB,Spark, etc Personal

Data / BI

Where does data arrive?When does it need to move?Where does modeling happen?What can users do themselves?What governance is required?

Big Data Architectures got complicated

What we want — consistent, seamless solution


Your BI Data ISN’T In Your Corporate System

“We found, on average, that 45% of the data business people use resides outside of the enterprise BI environments.

An astonishingly miniscule 2% of business decision-makers reported using solely enterprise BI applications.

This is undoubtedly connected to 76% of business respondents indicating they continue to resort to spreadsheets and other homegrown BI applications to analyze BI data. ”

Source: Forrester

55%

45%

In enterprise systemsNot in enterprise system

“Intricate calculations of sales by territories will appear as if by magic in the digital age ahead”


Decision Cockpits



SAP BusinessObjects Cloud


Cloud Analytics

@timoelliott

It’s not Pie in the Sky!

Data gravity: BI in the cloud when the data’s in the cloud

Securely Connect to Your Data In The Cloud AND On-Premise

SAP Cloud for AnalyticsWeb Client running in browser

S/4HANA

Public Cloud

BW

HANA

External data via HANA Smart Data Access

Public Cloud data sources

Data Connector

On-Premise System

Local Network

SAP Cloud for AnalyticsPlatform and Content Repository



SAP Lumira for Data Discovery


Health indicators across fleet


Lufthansa Systems LIDO

Lufthansa integrateddispatch operations


Mercy Health

Mercy — one of US Most Wired for 12th Year in a row!

• $4.48 billion revenue• 40K employees• > 8M patients/year

“It is mind-blowing how versatile and nimble our data warehouse is on SAP HANA.”

Agile self-service with SAP HANA and SAP Lumira. 9 years of data, structured & unstructured


What Pipes?

Type 1

Type 2?



Centerpoint Energy

New Business Models

Kaeser Compressor, a global leader in air compressors

≈€500 million, 4,800 employees, 50 countries, partners in additional 60 countries

@timoelliott


Modeling Example

E.g. Total energy consumption

• Aggregation of 10 sec values

• Calculation of typical consumption patterns

• Pattern associated with each compressor and day

Repeat for temperature, pressure, vibration, etc.

@timoelliott


Predictive Examples

Model combines sensor readings and ERP data (location, type of usage, last service, etc.)• Status alerts: “Oil change / oil analyze / no action”• Predict machine failure 24 hours in advance

@timoelliott


High-Level Technical View

Predictive Model(in-memory)

Long-term disk storage

User Interfaces

CRMERP

Event Stream Processing

all sampled

Customer Field Svs Sales R&D

DW

@timoelliott


Benefits

Customers• Less downtime• Decreased time to resolution• Optimal longevity and performance

Kaeser• More efficient use of spare parts, etc• New sales opportunities• Better product development

“We are seeing improved uptime of equipment, decreased time to resolution, reduced operational risks and accelerated innovation cycles.

Most importantly, we have been able to align our products and services more closely with our customers’ needs.” �

Kaeser CIOFalko Lameter

@timoelliott


© 2014 SAP AG. All rights reserved. 50

Hadoop + Hana


Information Ecosystems

51


New Products & Services


Pret A Manger


55

HADOOP is key Part of SAP’s Open Source Development usage

1

10

100

1000

10000

Open source consumption Open source contribution SAP Contributes to over 100 Open Source Projects

http://images.google.com/imgres?imgurl=http://www.hostgatorhosting.net/wp-content/uploads/2009/10/mysql.jpg&imgrefurl=http://www.hostgatorhosting.net/&usg=__MHGsnQJ7dLoZNRIdlC4St5AZKnw=&h=291&w=500&sz=42&hl=de&start=13&um=1&itbs=1&tbnid=kB9XD4caKi9rJM:&tbnh=76&tbnw=130&prev=/images?q=mysql+ab&um=1&hl=de&sa=N&tbs=isch:1

http://images.google.com/imgres?imgurl=http://blogs.zdnet.com/security/images/red_hat_logo_big.jpg&imgrefurl=http://blogs.zdnet.com/security/?p=1784&usg=__Q0A5Wr7to5WJ8CKutVGWnYeJNL4=&h=396&w=360&sz=63&hl=de&start=1&um=1&itbs=1&tbnid=mw3P22rAAis9PM:&tbnh=124&tbnw=113&prev=/images?q=red+hat&um=1&hl=de&sa=N&tbs=isch:1





























SAP Hadoop Partnerships

SAP HANA Platform

The SAP focus: End-to-end value chain

SPATIAL PROCESSING

ANALYTICS, TEXT, GRAPH, PREDICTIVE

ENGINES

CONSUME

COMPUTE

STORAGE

SOURCE

INGEST

Application Development Environment

Transformations & Cleansing

Smart Data IntegrationSmart Data Quality

StreamProcessing

Smart Data Streaming

STREAM PROCESSING

LogsTextOLTP Social Machine GeoERP SensorStore & forward

Mobile applications and BI

Smart Data Access

Virtual Tables

User Defined Functions

101010010101101001110

Dynamic Tiering

Aged datain Disk

In-Memory

Data model& data

Calculation engine

Fastcomputing

Column Storage

High performance analytics

Series Data Storage

Store time-series data

Reporting &Dashboards

High Performance Applications

Data Exploration& Visualization

Adhoc & OLAP Analytics

PredictiveAnalysis

Business Planning & Forecasting Lumira / BI

But there is more work to do…

Hadoop / NoSQL

MapReduce

YARN

HDFS


Our Journey

SP06

SP07

SP09 HDFS

Yarn/MR

HBASEHive

SparkPig

Mahout

Ambari

Hive Added as a Remote Source

ODBC Based Communication

Query Optimization Like Remote Caching and Join Relocation

Reading HDFS Directly

Map Reduce Job Execution

SP10 Spark SQL added as a new Remote SourceAmbari launcher tile in HANA Cockpit


HANA & Hadoop Integration

HANA & Hadoop Integration SQL on Hadoop via SDA (virtual tables) – Hive

(SPS06) Remote caching with Hive (SPS07) Connectivity to Apache Spark using ODBC Execution of MR-Jobs via HANA (Virtual Functions)

and direct access to HDFS (SPS 09) Spark SQL adapter via SDA (SPS10) Join relocation to Hadoop thru SparkRDD Unified Admin thru Ambari integration for Hortonworks

Key Benefits Deep Integration for storage & processing Optimized data access between HANA & Hadoop Data tiering to Hadoop for cold storage


SAP HANA VoraWhat’s Inside and What Does It Do?

DemocratizeData Access

Make PrecisionDecisions

SimplifyBig DataOwnership

SAP HANA Vora is an in-memory query engine which leverages and extends the Apache Spark execution framework to provide enriched interactive analytics on Hadoop. Drill Downs on HDFS

Mashup API EnhancementsCompiled Queries

HANA-Spark AdapterUnified LandscapeOpen Programming

Any Hadoop Clusters


YARN

HDFS

Enable Precision DecisionsWith Contextual Insights In Enterprise Systems

Other Apps

Files Files Files

HANA-Spark Adapter for improved performance between distributed systems

Gain business coherence with business data and big data

Compiled queries enable applications & data analysis to work more efficiently across nodes

Familiar OLAP experience on Hadoop to derive business insights from big data such as drill-down into HDFS data

Compiled Queries

Spark Adapter

Drill Downs

SAP HANA in-memory platform

Vora

Spark

Vora

SparkIn-Memory

Store

Application Services

Database Services

Integration Services

Processing Services

SAP HANA Platform

Vora

SparkHANA-Spark

Adaptor


Democratize Data Access for Data Science Discovery

Extensive programming support for Scala, python, C, C++, R, and Java allow data scientists to use their tool of choice,

Pursue new inquiries without compromise on data and easily integrate these insights with all data

Enable data scientists and developers who prefer Spark R, Spark ML to mash up corporate data with Hadoop/Spark data easily

Optionally, leverage HANA’s multiple data processing engines for developing new insights from business and contextual data.

Mashup Enhancements

Open Programming

Optional Use of SAP HANA for Delegated, multi-engine pre-processing

Spark Data-source API enhancement

In-Memory Store

SAP HANA Platform

YARN

HDFSFiles Files Files

Vora

Spark

Vora

Spark

Vora

Spark

HANA Smart Data Access, UDFs, Others

Application Services

Database Services

Integration Services

Processing Services


SAP HANA VoraWhat’s Inside and What Does It Do?

DemocratizeData Access

Make PrecisionDecisions

SimplifyBig DataOwnership

SAP HANA Vora is an in-memory query engine which leverages and extends the Apache Spark execution framework to provide enriched interactive analytics on Hadoop. Drill Downs on HDFS

Mashup API EnhancementsCompiled Queries

HANA-Spark AdapterUnified LandscapeOpen Programming

Any Hadoop Clusters


Vora SQL Engine

#FEA433

Components

Written FromScratch

Multi Platform

Compressed Columns

Parallel QueryProcessing

In Memory Storage Fast Column Scans

Cache EfficientAlgorithms

Code Generation


SAP HANA Vora: Strategic Point of View

● Add functionality for enterprise applications● Hierarchies● OLAP modeling

● Boost SQL performance● Federate access across HANA and Hadoop● Integrate tooling

SAP HANA


SQL/OLAP on Big Data

• Hierarchical data storage of contextual data supports structured analysis

• Fast drill-down interaction aids in root-cause analysis

• Familiar OLAP tool enables experienced business analysts derive useful insights from contextual data

• Support for HDFS, Parquet and ORC formats

• LLVM/Clang – JIT compilation of query plans and execution Hadoop/NoSQL DATA


ALL IN-MEMORY

In-Memory Data Fabricfor Enterprise + Distributed Compute

Enterprise Compute Distributed Compute

CO

NSU

ME | C

OM

PUTE | STO

RE

HANAOLTP + OLAP

Scale Up

Scale Out+

Massive Scale Out

Appliance | TDI

Vora Vora Vora Vora

Vora Vora Vora Vora

Vora Vora Vora Vora

Distributed File System | Network Storage | Cloud Persistence | Any Hardware

Federated Queries &

Programming Model

+Tiering


SAP Predictive Analytics 3.0

Native Spark Modeling

Standalone or included in SAP HANA

Predictive Factory

Integration with cloud & other apps


Suite

Applications

S/4HANA

DigitalBoardroomIcon

Analytics

C4A

BOBJ

ExtensionsApplicationsIoT

HANA Cloud Platform

(Micro-) Services

IoTPlatform

Identity Management

Business Network

CEC

Platform

HANAEnterprise

Computing Platform

any DB Hadoop

VoraDistributed Computing

Platform

SAP Platform for Digital Transformation


Revenue Assurance

Detection

Investigation

Prevention

Correction

Revenue Assurance Lifecycle


The Big Idea

AR PUM


It’s Complicated

CostTransactional cost on monthly basis

for 3-6 month

SubscriberSubscriber master data

for the last 12 month

RevenueRevenue on monthly basis for 3-6 month

ProductProduct master data

Service ComponentsMaster data on service components that are the core element for building products and tariffs

Margin Margin table generated during data discovery service forms the basis for the analysis

=Billions of records


Seeing Clearly

Customer

Subscription

Cost and Revenue

Cost and Revenue

Classification

Unit Prices for Indirect and Direct Cost

ServicesService Specification

Billing Account

Carrier

Third Party

Product Offering

Inspired by TMForum’s Information Framework, follows ABDR (Analytics Big Data Repository) design principles.

Overview

Typical aggregated view of profitability only

SAP HANA & Lumira

SAP HANA & Infinite Insight

Find Opportunities


Visualization

Tree map: Profitability per level & drill down

Distribution: Lowest Margin Customer Analysis


Visualization

Identify SOC Combinations which make less margin per client

than all the SOC Combinations covering it


Visualization

green - acceptable margin

red - margin decrease

bubble size – number of customer

bubble – combination of service components

arrow - similar but slightly different service component combinations

thickness – similarity of service component combinations


Margin Assurance

Invoice ControlDelta-Calculation

Identification of Trends

&Trend-Alarming

Detection of Fraud and Misuse

Business Application Scientific Approach/ Method

SAP Solution

Identification of Margin Drivers

Analysis of Underperforming

Clients

Statistical Outlier Detection

Time-Series Analysis

Clustering

Network Analysis

Tarif

f & P

rodu

ct P

ortfo

lio M

anag

emen

t Too

ls Aut

omat

ed M

onito

ring

Tool

s

Da

ta-D

riv

en

To

ols

fo

r M

arg

in A

ss

ura

nc

eSAP Lumira

(Data Visualization)

HTML5 – based apps/dashboards

MNO Custom Built Web Applications

PAL(Data

Analytics)

HANA DB

HANA XSWeb

ContentXSJS REST

Services

Stored Procedures

(Automation Logic)

Views and SQL Scripts

R-ServerSAP Infinite

InsightApp-Server (Custom Built Application)

SA

P H

AN

A


Margin Assurance

€100m’s

€10m’s

Revenue LeakageInsight Value

Time

Learning, Prediction & Process Integration Decision

SAP Margin Assurance

Revenue Assurance Today

3-6 months

DataAnalysis


Margin Assurance

“The ability to understand granular customer profitability, specifically which customers are profitable or unprofitable and why, is a game-changer for our industry.”

-- Thomas Holtmanns, Vodafone Director Finance Operations Germany and Global Margin Assurance.


Use Cases

Underperforming Clients

Low margin customers mapped to Post-pay Tariffs & Products.

& Price simulation on static usage profiles.

Discount Control

Identification of low margin customers driven by discounts

e.g. Discount stacking. Or poor combinations.

Negative Margin Clusters

Combinations of Usage behaviour; Tariffs, Products and SoCs with negative margins.

e.g. M2M reverse charging and hotspots.

Profitability Time Analysis

Benchmarking, trend monitoring and alarms (against KPIs).

Identification of positive and negative trends month to month.

MarginDrivers

SoCs, Tariffs, Products, Customer Behaviour or Events, significantly driving positive or negative margin.

Mis-Use OR Fraud

Outlier detection and causal effect analysis

e.g. SMS boxes, breaching T&Cs.

Automated Invoice Control

Benchmarking, trend monitoring and alarms.

Identification of positive and negative trends month to month.

New Offer Performance

Tag & Track of new product OR tariff introductions.

Link with Profitability Time Analysis.

Learning & Prediction

WHAT IF analysis leveraging historical trends – price and usage elasticity.

e.g. Simulation of new propositions; competitor tariffs OR changing regulation

End to End Client Performance

Profitability of ALL Tariffs & Products for all market types & customer segments.

e.g. MMC, MMO / Corporate - VGE Top X….

ShowcasePhase 1 Phase 2

Monthly Data Source Near Real-time


Looking to The Future

Customer Value

(Profitability)

Customer Experience

Network Performance Geospatial ROI & Data

Monetisation

Granular Profitability – Customer, Group (Corporate, Family), Tariff and Product

Single View of Customer & Network (Learning & Prediction)

Customer Experience – Call, Data, Group Behaviour and NPS

Network Performance – Radio, Cell & Data Usage & Errors

Asset Value – ROI & Data Monetisation

Geospatial – Journey, Occupancy, Proximity, Qos, RTOM

Insight to Action Unique Profitable Propositions e.g. SLA Guarantees &

Dynamic Pricing and Policy Management

Investment in the right assets in the right locations (ROI)

Improvement to; or new business processes / workflows Learning & Prediction

Enablement environment for new ‘valuable’ business apps.

© 2014 SAP AG. All rights reserved. 85

Design Thinking

Understand

Ideate

Prototype

Test

Build

Deliver


Thank You!Timo ElliottVP, Global innovation Evangelist

[email protected] @timoelliott