NASSCOM Big Data and Analytics Summit 2014 - Predicting, Explaining and the Business Analytics...

Galit ShmuéliSRITNE Chaired

Professor of Data Analytics

Predicting, Explaining and the Business Analytics Toolkit

Business Intelligence

Traditional: Describe the past

State-of-the-Art: Describe the present

Business Analytics

Predictive Analytics: Predict future of individual records

Explanatory Analytics: Explain cause-effect of “average record”

(overall effect)

Today’s Talk

1. Predictive Analytics: The process & applications

2. Prediction is not explanation

3. The Explanatory Analytics toolkit

Will the customer pay?

What causes non-payment?

Past Present Future

Case Studies

Overall Behaviour

“Presonalized” Behaviour

The Predictive Analytics Process

Determine Outcome and Predictors

MeasurementDraw sample,Split into training/holdout

DataData Mining algorithms& Evaluation

Models

Predict New Records;Get More Data;Re-Evaluate

Actions

What to Predict? Why? Implications?

Problem Identification:

5 Examples of Predictive Analytics

Applications

Problem Identification

Outcome: redemptionPredictors: customer, shop & product info

Measurement

From similar past campaign (redeemers and non-redeemers)

Data

Predictive AlgorithmsExpected gain per offer sent

Models & Evaluation

Example 1:Personalized

Offer

Who to target?

Which coupon?

What medium?

Send Offers (or not!) More Data & Re-Evaluation

Actions

Problem Identification

Outcome: performance Predictors: employee & training info

MeasurementFrom past training efforts (successes and failures)

Data

Which employees to train?

Example 2: Employee Training

Send employees for training (or not!) More Data & Re-Evaluation

Actions

Predictive AlgorithmsExpected gain per employee

Models & Evaluation

Problem Identification

MeasurementOutcome: renewal Predictors: customer & membership info

DataPast renewal campaigns (successes and failures)

Which members most likely not to renew?

Example 3: Customer Churn

Send renewal incentive (or not!) More Data & Re-Evaluation

Actions

Predictive AlgorithmsExpected gain per person

Models & Evaluation

Example 4: Product-level demand forecastingProblem Identification

ActionsUpdate Orders, Pricing, PromoGet More Data, Re-Evaluate

Historic infoData

Forecasting;Expected gain

Models & Eval

MeasurementOutcome: month-ahead weekly forecasts of #units purchased, per itemPredictors: past demand for this & related items, special events, economic outlook, social media

Item-level weekly demand forecasts

Problem Identification

Outcome: pay/not Predictors: customer, product, transaction info

MeasurementPast deliveries (payments and non-payments)

Data

Predict payment probability

Example 5: COD Prediction

Reconfirm with suspect deliveriesMore Data & Update Model

Actions

Predictive AlgorithmsExpected gain per delivery

Models & Evaluation

Predictive Analytics: It’s all about correlation, not causation

Algorithms search for correlation between the outcome and inputs

Different algorithms search for different types of structure – lots of predictive algorithms!

Every time they turn on the seatbelt sign it gets bumpy!

Causality?

www.tylervigen.com

The Causal Explanation Process

Determine Outcome and Causes

MeasurementAssign records to treatment(s)Collect data on inputs+output

DataStatistical models& Evaluation of uncertainty

Models & Eval

Make Decisions; Implement Changes Get More Data and Re-Evaluate

Actions

Which Inputs Cause the Output? How? Implications?Inputs under our control, inputs uncontrollable

Problem Identification:

What causes average customer to redeem?

Example 1:Personalized Offer

Change coupon design/typeCollect new data (gender)

Actions

Problem Identification:

Tailor trainingPrepare employeesIncentivize learning

Actions

Example 2: Employee Training

What causes average employee to succeed?

Problem Identification:

Improve serviceChange target market

Actions

What causes average member not to renew?

Example 3:Customer Churn

Problem Identification:

Create flexible designsOpen new locations

Actions

Example 4: Demand

Forecasting

What causes high/low demand?

Problem Identification:

Modify payment policyChange website designTrain delivery staff

Actions

What causes average transaction to result in non-payment?

Example 5: Cash-On-Delivery Prediction

Problem Identification:

Toolkit for Determining Causality

Gold Standard: Controlled, Randomized Experiment

Beyond A/B Testing:Multiple factors andInteractions between factors

Causal Explanation withObservational Data

(not a controlled experiment)

Self Selection

Current PracticeCompare online/offline performance stats

Turns out: online and offline users differ on “awareness”

Awareness of electronic services provided by Government of India

Performance Evaluation:% Using Agent

Naïve Comparison:Online system →Less agents

After correcting for self-selection:Online system → More agents for “unaware” users!

Aware Unaware

Asia Analytics Lab @ ISBfacebook.com/groups/asiaanalytics