A Targeting Marketing

Analytical Modelling StackSCOTTY NELSON

Outline

Business problem/context

Modelling stack:

Sparse data

Optimal learning

ROI measurement

Future directions

Business Context

Client is a loyalty program for small/medium sized businesses

Members shop using a registered credit card at participating

merchants and receive cash-back offers

Targeted email campaigns send offers most relevant to the

members in order to drive business to the merchants

Available member/merchant transaction history is very sparse

Currently modelling ~1.6 million members and ~1500 merchant offers

Business Context

2. Send targeted offers to

members

3. Members incentivized to shop at merchants

4. Measure impact of

campaign on actual

member spend

5. Refine targeting models

1. Identify best offers for

each member

Summary of Modelling

RequirementsIssue What We Need Proposed Solution

Sparse Data Personalized member level

merchant recommendations &

price sensitivities

Hierarchical models

Learning Adjusts recommendations to

reflect uncertainty (don’t put

all your eggs in one basket)

Randomized probability

matching

ROI

Measurement

$ impact of the campaign Weighted inverse

propensity score models

Hierarchical Models

We want individual level merchant

scores but we don’t have a

complete set of data on any one

member

This blows up traditional regression

models scores (there are more

parameters than data)

Hierarchical models solve this issue

by “drawing strength” across similar

members in a statistically valid way

Has connections to kernel smoothing

and collaborative filtering

Prior

Member 1

Sports

REI

Neptune

Restaurants

Southern Sun

Zoe Ma Ma

Member 2

Sports

REI

Neptune

Restaurants

Southern Sun

Zoe Ma Ma

Randomized Probability Matching

Motivating example:

Let’s say we have a slot machine with 5 different arms(A, B, C, D, E)

We play D and it wins Does that mean we should always play D?

Randomized probability matching (Scott, 2010)

Samples strategies proportional to their posterior probability of being ‘the best’

Balances exploitation vs exploration of the (unknown) payoff distribution for the different strategies

Source: “Optimal Learning”, Powell & Ryzhov

A B C D E

ROI Measurement

How to measure ROI?

Run the campaign compare before/after sales?

What about people who would have bought anyways?

Dealing with selection bias: propensity score matching

Solution: Create a synthetic control group by re-weighting observations according to a estimated probability of treatment

This creates a balanced panel appropriate for comparing those who received the offer, and those who didn’t

Summary of Modelling Stack

Issue Proposed Solution What It Gives Us Why it Works

Sparse Data Hierarchical models Personalized member

level merchant

recommendations &

price sensitivities

Blends individual and pooled

information in a statistically valid

manner

Learning Randomized

probability matching

Adjusts

recommendations to

reflect uncertainty

(don’t put all your eggs

in one basket)

Balances exploitation of proven

recommendations, with

accumulation of more

information on underperforming

recommendations

ROI

Measurement

Weighted inverse

propensity score

models

Outputs the $ impact of

the campaign

Controls for confounding

selection bias

Questions?

Contact me at scotty@convergenceanalytics.co