The Future is Now: Winning Papers Cognitive Analytics: Machine Learning for ROI Measurement

Kyle Gerhart VP of Delivery and Analytics

Leslie Wood CRO

Is my advertising working?

Who is it working for?

Who is it not working for?

Is the response in line with my strategy?

How can I do better in the future?

What’s my ROAS? How am I changing

consumer behavior?

Who is my creative resonating with?

Did it drive incremental sales?

How can I replicate success?

Single Source Data matches what consumers watch on TV with what they buy.

TV Watch Data •  Nielsen Media Data •  Set Top Box Data

Buy Data •  Catalina Frequent

Shopper Card Data

•  Nielsen Homescan All-Outlet Data

Anonymous Single Source Households

ANCOVA Methodology

Test & Control

Exposed Unexposed

ANCOVA Methodology

Exposed Unexposed

Campaign Measurement Window

Challenges of ANCOVA

Exposed

Unexposed

Campaign Measurement Window

window window window window window window window

Exposed

Unexposed

New Tool:

Cognitive Analytics

Purchase Occasions Influenced by the Advertising

Purchase Occasions Not Influenced by the Advertising

Exposed Purchase

Occasions

Unexposed Purchase

Occasions

$ 28 Days Prior

28-Day Exposure Window: Accounts for Reach

28-Day Exposure Window: Accounts for Attribution

$28 Days Prior

exposure window after the trip more than 28 days ago

Exposure Model

Purchase Model

Chicken or Egg? HH Attributes and Purchase Behavior are Interdependent

•  Purchase Behavior •  Demographics •  Media

Consumption •  Location •  Timing Of Visit

1. Exposure Model The advertiser may have made a conscious decision to expose only certain types of households, based upon demographics and purchase history

2. Model Purchase Influence On a trip, a household chose whether or not to buy the brand; this decision was both influenced by the household's purchase history + demographics, and the household's exposure or non-exposure to the ad

Cognitive Analytics Solves for Targeting Bias

Sales Lift Accounting

For Targeting

Relationship Universe + Demographics + Purchase History + Media Consumed + Timing of Visit + Geolocation

Model A

Cognitive Analytics – The Data Picks the Model

Model C

Model B

Model D

Purchase Behavior

Exposure

Timing of Visit

Brand Purchase

Media Consumption

Category Purchase

Location

Demographics

Model E

Model G

Model F

New Tool: What Does Advertising Impact?

A household's likelihood to choose the category?

✔

1

A household's likelihood to choose the brand, given they chose the category?

✔

2

The amount purchased of the brand, given they chose the brand?

✔

3

Cognitive Analytics - Validation ü NCS has delivered over 3,000 measurement studies, primarily test vs.

control methodology

ü Cognitive Analytics has been tested against a broad group of TV campaigns

ü When compared to traditional test vs. control methodology:

§  No systematic variations results; differences are distributed around 0

§  61% of results have a difference of $0.01 on incremental dollars return per exposed HH basis

§  High degree (78%) of intersection in confidence intervals

§  Where differences exist, they can be explained by CA’s additional rigor

Why is Cognitive Analytics Important?

1.   High Reach Campaigns Can be measured

2.   Solves Targeting Bias For the problem of targeting being confounded with advertising impact

3.   Factors Recency Into Analysis Using an “exposure window prior to purchase” vs. a household-only basis

The Right Tool for Every Job

Thank you.

Learn More: http://ncsolutions.com Twitter: @NCSolutions