Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 1 May 1, 2014
The Marketing Mix Crisis, Attribution Models & The Experimental Design Renaissance
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 2
How Do I Make Sense of All the Ways to “Attribute”?
Audience Verification
Behavioral Analysis
Brand Lift
Cross Media Modeling
Sales Lift/ROI Analysis
Marketing Mix Models
Attribution Modeling
Cross Media Reach Analysis
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 3
How Do I Make Sense of All the Ways to “Attribute”?
Use the ABCS of Marketing Effectiveness Funnel as an easy way to organize the measurement ecosystem in your brain!
Audience Verification*
Behavioral Analysis Brand Lift/ CMM
Sales Lift/ROI Analysis Marketing Mix Models
Cross Media Reach*
Attribution Modeling
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 4
How Do I Make Sense of All the Ways to “Attribute”?
*Audience Verification and Cross Media Reach neither models nor experiments
Experiment Design Econometric Modeling ! Methodology "
Sales Lift/ROI Analysis Marketing Mix Models
Attribution Modeling
Cross Media Reach* Audience Verification*
Behavioral Analysis Brand Lift/ CMM
The two methods for attributing:
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 5
The Marketing Mix Crisis
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 6
The Marketing Mix Crisis
Or NOT….
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 7
Growing Concerns Old and New
• What about long term effects?
• Who has time to wait a year?
• How do I measure digital media ROI in an MMM?
• Display
• Video
• Mobile
• Tablet
• Owned (website, email)
• Social
• Search
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 8
The Marketing Mix Crisis
Or NOT….
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 9
Correlation is Not Causation
“A statistical relationship, however strong and however suggestive, can never establish causal connection: our ideas of causation must come from outside statistics, ultimately from some theory or other.”
M. G. Kendall and A. Stuart – The Advanced Theory of Statistics
“To ascribe causality, one must appeal to a priori or theoretical considerations.”
D. N. Gujarati- page 21 Basic Econometrics
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 10
Top Criticisms of Marketing Mix Modeling
• Correlation is Not Causation
• Proper MMM overcomes this by drawing from Economic/Econometric Theory
• And scientific cross validation with experimental design
• Not always apples to apples comparisons of channels
• Price, promo, in-store favored vs. all media (Long Term Effects)
• Measures only one return objective of advertising (Sales)
• Supplement with other techniques if you seek to optimize
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 11
Marketing Mix is Not in Crisis but it will Change
• MMM remains an excellent tool for marketers, the tires have been kicked for years
• MMM is increasingly in need of supplemental analysis especially for understanding of the interactions with the Digital Media Mix, as well as understanding what’s happening earlier in the consumer journey
• MMM is being disrupted by “Big Data” that is single source consumer level which will lead to a hybrid of econometric, existing frameworks and experimental design techniques
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 12
So What About Attribution Models?
Top of the food chain? Bottom Feeding?
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 13
“Attribution Models”
• What exactly are attribution models?
• What is the difference between attribution models and mix models?
A) “Marketing mix models are top down approaches and attribution models are bottom up”
B) Attribution models are mix models that haven’t been properly validated
C) There is no difference
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 14
What is Attribution Modeling?
• “Digital attribution — the measurement of the value of each digital marketing contact that contributed to a desired outcome.” IAB
• “The measurement of the value of each digital marketing contact or exposure that contributed to an outcome.” Forrester Consulting !
How$is$that$anything$different$than$marke3ng$mix$or$other$techniques?$
• “Marketing Mix Modeling is a top down approach, while Attribution Modeling is a bottom up approach.” Conference on Attribution Modeling
NOT$TRUE…$Consumer$Level$Marke3ng$Mix$Models$have$existed$for$over$a$decade!$
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 15
Attribution Modeling…It’s better than last click
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 16
Attribution Modeling… But Its Very Assumption Driven
• What would you like to assume? Take your pick!
1. Last Interaction/Last Click Attribution model. 2. Last Non-Direct Click Attribution Model. 3. Last AdWords Click Attribution Model. 4. First Interaction/First Click Attribution Model. 5. Linear Attribution Model. 6. Time Decay Attribution Model. 7. Position Based Attribution Model. 8. Customized/Personalized Attribution Model.
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 17
Attribution Modeling: Assumption Driven
“Pre-determined weighted systems lead to arbitrary insights… Because simple attribution models use
predetermined rules to assign credit to every touch points, even advanced systems offer somewhat arbitrary conclusions….
Data-driven attribution ensures impartiality.”
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 18
The Bigger Issue with Attribution Modeling
Standard Approach to Measuring the “individual” using cookie level data
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 19
Consumer Reality
Home$Devices$
Browsers$Cookies$
Work$
The$Individual$
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 20
The History of Data Needs
0 5 10 15 20 25 30
National Monthly Data (1950)
National Weekly Data (1960)
Market Level Weekly Data (1970)
Store Level Weekly Data (1980)
Household Weekly Data (1990)
Household Daily Data (1995)
Consumer Level Data (2011)*
Cookie Level Microsecond Data (1997)
Granularity (Log Space and Indexed to National Monthly Data)
A R
ecen
t His
tory
of D
ata
Col
lect
ion
Granularity (log space)
**Dates based on my memory of events #
Data Nerds got what they asked for: For the first time in the history of data it is too granular.
The Code of Hammurabi (1772 BC)
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 21
Misattribution on Audience Verification
1.9
3.3 4.1
1.9 2.5 2.3 2.4
3.7 3.1
2.3 3.0
10.5
12.3
3.2
4.6 3.3 3.5
7.9
4.8 5.7
3.3
11.0
17.3
4.0
7.7 8.0
4.2
9.9
5.0 6.1
Campaign 1 Campaign 2 Campaign 3 Campaign 4 Campaign 5 Campaign 6 Campaign 7 Campaign 8 Campaign 9 Campaign 10
Online Campaign Impression Frequency by ID Technology Browser Cookie Device Level/Metered/LSO InsightExpress Universal ID
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 22
Misattribution from Cookie Level Attribution
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Test Control Test Control Test Control
B-Cookie Device Level Universal ID
% E
xpos
ure
Advertising Effectiveness Analysis Contamination
(Illustrative)
The Majority of Attribution Modeling is done on Cookie Level Data not Consumer Level Data…
* Illustrative Example
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 23
Thoughts on Attribution
It is difficult to distinguish a meaningful methodological difference between most “Attribution Models” and Marketing Mix Modeling.
What’s referred to in the marketplace as Attribution Modeling tends to be more dashboard driven, and faster than MMM, but be careful what you ask for:
• Real-time ROI may sound like a good idea, but the problem is that ROI takes time (Often years: See seminal IRI Study “How Advertising Works”)
• Many Attribution Modeling approaches in the marketplace can provide deceivingly plausible insights that may be “arbitrary insights”
• Things to watch out for:
• Fine print assumptions?
• How is it validated?
• Is it cookie level or consumer level?
• How is the software constraining?
• How is sales data being integrated? Consumer level or aggregate?
• The best attribution modeling techniques are actually converging on and rediscovering long standing Marketing Mix Econometric Methods and Experimental Design methods
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 24
The Experimental Design Renaissance
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 25
The Experimental Design Renaissance
History: Before there were Mix Models and Attribution Models the business of attribution was done using Test vs. Control Experimental Design (AKA the Scientific Method)
• Historic Weakness(in marketing research):
• Cost
• Scale (can’t measure when always on everywhere: TV)
• But the weaknesses are going away:
• Automation, cost of data acquisition
• Big Data – Shift from Market Level to Consumer Level data
• The big advantage of Experimental Design is that it lets the data speak for itself:
• Far fewer assumptions, and its Causal
• Not black box => very transparent
• Easy to understand
• Experimental Design measurement is now re-emerging for virtually every marketing tactic!
Why Assume When You Can Know!
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 26
Closing Thoughts
• The marketplace accepted best ways to do proper attribution remain Econometric and Experimental Design driven methodologies
• Experimental Design is the scientific gold standard for data driven proper attribution. It requires the fewest possible assumptions. “When Life is on the Line use Experimental Design”
• Proper Econometric methods like Marketing Mix Modeling are great and generally been validated, often using experimental designs, which is one of the reasons they are trusted by the market
• “Attribution Models” are certainly better than last click attribution, but be careful; what’s under the hood matters and methodology remains the wild west.
Confidential and Proprietary – Not for Public Distribution – Do Not Copy © 2014 InsightExpress 27 May 1, 2014
Thank You Jerome Shimizu SVP, Chief Research Officer (203) 406-3218 [email protected]