Impact of big data on analytics

Impact of Big Data on Analytics

Mamatha Upadhyaya

2

Big Data & Analytics

Copyright © 2014 Capgemini. All rights reserved.

Impact of Big Data on Analytics | Mamatha Upadhyaya

Big Data and Analytics Summit 2014

The terms Big Data

and Analytics are

used simultaneously

3




However, analytics, predictive modeling, advanced analytics, data science is not new!

4




So what does this mean for Analytics?

Yes, the amount of data that is available to us is exploding

And Big Data Platforms and Commodity Hardware and bringing in additional capabilities

So what does this

mean for Analytics?

Media is rife with Big Data and Analytics

AND The Data Scientist makes it from Nerd to the most cool person!!

…makes it to on top of CIO agenda

Big Data and analytics is touted as the panacea for all problems

5




Data Science perspective

– A Data Science perspective

Big Data and Analytics Impact of

6




A brief history of Data Science

Pre 1800s 1800-1900 1900-1940 1940-1960 1960 1970 1980 1990 2000 2010

Text/ string search

1974 Peter Naur “Concise Survey of Computer

Methods”, Data Science, Datalogy

Knuth – Art of Computer Programming.

1976 – SAS Institute

1977 The International Association for

Statistical Computing (IASC).

Computer Science

Data Technology

Visualization

Mathematics/ OR

Statistics

Probability

Correlation

Bayes Theorem.

Regression, Least

Squares

Time Series.

Theoretical Foundations of Modern Stats

Hypothesis, DOE

Mathematical Statistics.

Bayesian Methods

Time Series Methods (Box Cox,

Survival, etc.)

Stochastic Methods.

Simulation, Markov

Computational Statistics.

Decision Science

Pattern recognition

Machine learning.

Liebniz – Binary Logic. Babbage, Lovelace

Boolean Algebra

Punch cards.

Turing machines

Information Theory

Weiner & Cybernetics

Von Neumann Architecture.

Calculus

Logarithms

Newton-Raphson.

1989 First KDD Workshop

Gregory Piatetsky-Shapiro.

Sort & Search Algorithms –

Dijkstra, Kruskal, Shell Sort, …

Heuristics – Simulated Annealing, …

Graph Algorithms

Multigrid methods

Tree based methods.

Database Marketing

Data Mining, Knowledge Discovery

“Data science, classification, and related methods.”

William Cleveland: Data Science

Leo Breimann: Statistical Modeling: 2 Cultures.

Optimization Methods

Fourier and other transforms

Matrix & Generalizations

Non-euclidean geometries.

Applications to Military,

manufacturing,

Communications.

1962 John W. Tukey, Future of

Data Analysis

Networks

Assignment Problems

Automation

Scheduling.

First IBM

Computers

DBMS.

Removable Disk drives

Relational DBMS. Desktop, floppy

SQL, OOP

High level languages. William Playfair

Charles Minard

Florence Nightingale.

Catrography

Astronomical Charts.

John Tukey

Jacques Bertin. Edward Tufte. Grammar of Graphics

Word Cloud, Tag Cloud.

7




Drivers of change

Data

Availability

Technology

Ability to

Handle

Structured and

unstructured

data

Platform Cost Agility

Business

Expectation

Digital

Experience

Strategic

Initiatives

New Business Models

8




TABLE OF CONTENTS

Data drivers Technology drivers

So what does all of this increased activity

mean to Talent!

9




8 TB data everyday

10 TB data everyday

152 million blogs on the internet

Data availability

Possible

Technically but

very expensive Not efficient enough to

handle the amount &

type of data generated

by newer internet-scale

technologies

Big Data

Internal data

M&A New Tech

adoption

Need to access

Unstructured data

External data

2 billion internet users

* Hortonworks CEO Rob Bearden

Digital Customer

IOT

Legacy data management system are not designed to handle heavy

demand of data consumption “85% of that data is coming from net-new data

sources.” – mobile, social media, and web- and

machine-generated data*… and this will increase.

RFID

10




Improving traditional analytics

Understand Act Measure

Change in Market Share

Campaign Effectiveness

Revenue/ profit metrics/

reports

Customer Churn/

retention reports

Customer Next

Best Offer

Cross Sell/ Up

Sell Opportunities

CRM (Customer

Info, billing, etc.)

Subscription and

Usage Summary

External data

(Demography, etc.)

Customer Profiling

Demographic Segments

Behavior (usage/ profit/

satisfaction/ etc.) based

profiles.

Product Association and Product Mix

Customer Profitability/ Life Time Value

Product Purchase

Propensity Score

Targeted

Retentions

Strategies

Existing Customer Analytics Insights

CDR, IPDR data

Customer Service

Data

Network Data

Usage Based

Profiling Customer Links/

Network Analysis

Drivers of

Satisfaction Network

Performance and

Service Levels

New Analytics from Big Data

Social Media Data and Analysis

Social Media and

Web Data

Sentiment

Analysis Social Media

Influence Analysis Drivers of

Sentiment

Churn Scores

11




…but it is all about business value

KS Statistic

Accuracy Ratio

ROC Curve

Gini Coefficient

Implemented Cut-Off

Cut-off neighborhood

Shift

Baseline Population

Current Population

New

Applic

ants

PSI:

Distributional Shift

Scorecard Score

Strategic KPIs

Reduce Costs

Regulations

Compliance

Increase

Revenue

12




Engineering analytics: Makes it a reality with Big Data + Data Science on top of traditional mathematical models

Various Data

Usage Pattern, Health Monitoring

Alarms & Control, Benchmarking

Predictive Rules for optimal recommendation from connected

Assets

Failure Prediction & Root Cause Analysis

Resource Scheduling.

Design for Reliability

Benchmark data/content Mining

Reduced Order Modeling for high volume Simulation

and Testing

Supplier Risk Modeling

Weight & Cost analytics.

Design, Testing & Production Operation Service

Manufacturer User Service Provider

Analytics

Engines

Mfg & other Guideline

Specification &

Performance

Benchmark Reports

System Topology

Financial

Sensors/Telemetry –

usage, operations

setting, events

/alarms logs, etc.

Failure/ Warranty

Claims

Field/Technical

Inspection Notes

Contract/Service

History

Social Media and

Third Party

Reliability

Testing/Simulation

Supplier/OEM

transactions

Value Chain

First Time Right Product Design Connected Assets, Operations Control &

Predictive Maintenance

Supplier

Medium Volume, Low Speed,

Domain Specific

High Speed, High Volume and

Domain Neutral

Data

Behaviors

13




As more data becomes available to the Data Scientist, so does the complexity…

Product Affinities

Private Label Analysis

Customer Purchase Patterns

Campaign Response Effectiveness

Shrinkage & Productivity Analysis

Category Scorecard & Contribution

to GM Promotion Decomposition

Promotion Mix Optimization

Product Price Optimization

Pricing with Consumer

Perception Analysis

Assortments Optimization for

Market Basket Analysis

Shopper Trip Mission Analysis

Shopper Market Basket

Shopper Brand Sentiment Analysis

Product Behavior Scan

Adjacencies Analysis

Out of Shelf Analytics

Scoring of Stores/ Retail

Chains

Cross-Channel Order

Management

Inventory Optimization

at DCs (SCM)

Demand/ Volume

Forecasting

Social Impact on

Category/ Brand

Consumption

Promotion Halo/

Cannibalization

Pricing Elasticity Analysis

Shopper Segmentation

Shopper Demographics

Shopper Loyalty Base

RFM Analysis

Product/ Brand

Switching

Trial & Repeat

Category

Uniqueness,

Popularity Indices

Category Leakage

Tree

Store Clustering

Category/ Brand

Offer Conversion

Cross-Sell

Up-Sell

Shopper

Assortment

Price

Promotion

Product Competition Category

Tactics

Data

Need

s

+

Other

Consumer +

Survey+

Social Data

+

Household

Panel +

Loyalty/ CRM

Data

+

Syndicated +

Promotions

Data

(IRi/ Nielsen)

+

POS Data

+ Campaign +

Shipments +

Public* Data

Public* Data includes

Weather, Census,

Topography,

Ordinance etc

Maturity Stages

14




Availability of data is changing the way we address some traditional business problem

Pharmaceutical

Companies have used

physical surveys to

identify KOL. Big data

and analytics is

pioneering the way to

use a data driven

objective approach to

identifying and

monitoring KOL

Selection of right KOLs can help in better utilization of these marketing funds

A key success factor for these marketing spends is the correct methodology to

identify KOLs

Managing brand perception for the key Opinion Leaders is crucial for Brand

Management.

15




TABLE OF CONTENTS

Data drivers

Technology drivers So what does all of this increased activity

mean to Talent!

16




Technology

New Technologies are allowing us to

manage this at a fraction of the cost &

faster than ever before.

Tra

ditio

nal

Data

Ware

house

BIG DATA

1/30 of the cost

Data does not have to be isolated in

repressive silos

17




Big Data technologies are enabling a new approach

Response time

Vo

lum

e

Hadoop

Data warehouses

PB

TB

GB

Hour Min Sec SubSec

In-memory

databases

Event

processing

tools

Real-time

Applications

18




Data and technology implications…

Model development

Reduced timelines

• Access all data from a „data lake‟

• Data discovery and visualization tools to

reduce EDA timelines

• In-memory/ in-database/ high performance

analytics and parallelized algorithms

Increased analytical capability

• Implement techniques like Graph/ Network

analysis, ensemble methods, matrix

algorithms at scale

• Analyze structured and unstructured data on

one platform

Improved accuracy

• Analyze much larger data sets

• Ability to personalize for a segment of one, for

e.g. targeting).

Model deployment

Seamless deployment (In-database, PMML)

• Decreases error in deployment

Big data deployment

• Analytics on exabytes, scoring in MB/ sec

Real time deployment

• Response (alert/ recommendations) in

milliseconds or less

Adaptive, machine learning algorithms

• “learn” and respond to recent events

Availability and velocity of data leads to

change in analytical approach

• for e.g. Can move from „complex algorithms

for precision prediction of failure modes‟ to

„real time monitoring, alerts and control

processes‟.

19




Decreased

response time

Customer experience

Information is becoming the

new battleground

Business expectation

20




Analytics is playing an ever important role

Increased Focus on identifying the

customer across all channels

Segmentation to Micro segmentation

to the individual

Personalized Messaging and offers –

Increased Individual Customer Centricity

Gradual evolution of Customer Analytics

Past

Customer segments who are

most likely to respond to

targeted campaigns for new

products offers

Can tailor offers to specific to

each customer segment

Mostly delivered through mass

mail campaigns and in store

promotions.

Now

Micro segmentation

Analyze customer behavior

and buying patterns across

channels

Delivery through email, web,

mass mail campaigns.

Moving toward

Historical individual customer behavior

and buying patterns across channels

Individual customer consumption

pattern

In-store basket analytics

Additional dimensions Location & time

Targeted Strategies to pre-empt

customers from visiting competition

Instantaneous Delivery in store or a

proactive delivery via mobile to bring

the customer to store.

Segment to Individual to Individual @ time, place and behavior

You have purchased Cheese,

here are the offers on Bagels

You are within 2 KMs of a store offering 50% off garden furniture

Do you need coffee?

21




Analytics is only as good as the implementation…

Analytics has long excelled in silos …as

the amount of data and business

expectation increases, this will no longer

be feasible

IT will move from a facilitator role to an

enabler role

Decreased response time will mean end to end integrations – enterprise architecture

teams will need to be involved…

The Data Science team will have to work along with technology teams to effectively

serve the end customer

22




much of which is outside the

organization

Increased availability of data

Analytics as a Service and Data

Monetization

New service models

Decreasing Time value of data!

23




TABLE OF CONTENTS

Data drivers

Technology drivers

So what does all of this

increased activity mean to

Talent !

24




Scalability and industrialization to address skill shortage

Technical skills (Coding, Statistics,

Math)

+ Perseverance

+Creativity

+ Intuition

+Presentation Skills

+Business Savvy

= Great Data Scientist!

Key to a Great Data Scientist

Identified four Data Scientist clusters based

on how data scientists think about

themselves and their work, not

• Years of experience,

• Academic degrees, favorite tools

• Titles, pay scales, org charts.

Most successful data scientists are

those with substantial, deep expertise

in at least one aspect of data science,

be it statistics, big data, or business

communication

T-Shaped Skills.

25




…so can analytics solve all our problems

Help us acquire customers Product Recommendation

engine

Solve World Hunger Crop Sciences

Keep us fit!

Catch the bad guys Numbers

Win FIFA “German national football team uses real

time analytics for a competitive edge”

Get you married! Dating sites, Matchmaking

Analytics in Healthcare

26




So what is it Big Data and Analytics cannot do!!!

The information contained in this presentation is proprietary.


Rightshore® is a trademark belonging to Capgemini.

www.capgemini.com/bim

About Capgemini

With more than 130,000 people in over 40 countries, Capgemini

is one of the world's foremost providers of consulting, technology

and outsourcing services. The Group reported 2013 global

revenues of EUR 10.1 billion.

Together with its clients, Capgemini creates and delivers

business and technology solutions that fit their needs and drive

the results they want. A deeply multicultural organization,

Capgemini has developed its own way of working, the

Collaborative Business Experience™, and draws on Rightshore®,

its worldwide delivery model.

http://www.facebook.com/Capgemini

http://www.linkedin.com/company/capgemini

http://www.twitter.com/capgemini

http://www.youtube.com/capgemini

http://www.slideshare.net/capgemini