Embed Size (px)
Citation preview
© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Matt ZilliSenior Product Marketing ManagerMarketo
[email protected]@mattzilli
Page 2© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Page 3© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Agenda
1 2 3Intro to A/B Testing
Best & WorstPractices
A/B Testing in Marketo
Page 5© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Why aren’t more marketers actively testing?
Page 6© 2013 Marketo, Inc. Marketo Proprietary and Confidential
My favorite excuses
1. I have no time for this
2. The results are rarely conclusive
3. It’s too hard!
4. I already know theanswer
Page 7© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Which Subject Won?Test email subjects from the registration email for the2014 Marketing Sherpa Email Summit:
Your emails don’t work (We’ll tell you why)
What happens in Vegas…will improve your emails! Save $300 now.
(Open BEFORE Christmas) Email Summit 2014: discount ends Jan 9th
Sonia Simone, Copyblogger, Jan 2014
Page 8© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Which Subject Won?Test email subjects from the registration email for the2014 Marketing Sherpa Email Summit:
Your emails don’t work (We’ll tell you why)
Middle
What happens in Vegas…will improve your emails! Save $300 now.
Loser
(Open BEFORE Christmas) Email Summit 2014: discount ends Jan 9th
Winner
Sonia Simone, Copyblogger, Jan 2014
Page 9© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Things to test: Landing Pages
Page 10© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Things to test: Website
Page 11© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Things to test: Emails
From name/address
SubjectsWholeEmails
Date/Time
Page 12© 2013 Marketo, Inc. Marketo Proprietary and Confidential
7% Better than B
30% Better than B
The loser
A.
B.
C.
Jason Fried, signalvnoise.com, Jan 2009
Which headline won?
Page 13© 2013 Marketo, Inc. Marketo Proprietary and Confidential
7% Better than B
30% Better than B
The loser
A.
B.
C.
Jason Fried, signalvnoise.com, Jan 2009
Which headline won?
Page 14© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Testing Best Practices
Page 15© 2013 Marketo, Inc. Marketo Proprietary and Confidential
#1: Send to a large sample size,test a single variable
Page 16© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Page 17© 2013 Marketo, Inc. Marketo Proprietary and Confidential
#2: Check for statistical significance
Calculator: www.zettasphere.com/abcalculator
Page 18© 2013 Marketo, Inc. Marketo Proprietary and Confidential
#3: Over time,test everything
Subject lines, from names/address, content, text, templates, CTAs, creative, button colors, link order….
Page 19© 2013 Marketo, Inc. Marketo Proprietary and Confidential
#4: Whenever you can,measure conversions,
not opens/clicks
dmix.ca Blog, 2011
21% Better
dmix.ca Blog, 2011
Page 22© 2013 Marketo, Inc. Marketo Proprietary and Confidential
New things• Test Deeper CTAs
Chris Hexton, MarketingLand
Page 23© 2013 Marketo, Inc. Marketo Proprietary and Confidential
New things• Test “Mad Libs” Forms – 25-40% Increases
Luke Wroblewski, LukeW Ideation & Design, Feb 2010
Page 24© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Things to Avoid:
1) Relying on test results that are insignificant
2) Missing opportunities to test & learn
3) Unstructured tests
Page 25© 2013 Marketo, Inc. Marketo Proprietary and Confidential Miles Gotcher, Marketo Blog, Oct. 2013
Page 26© 2013 Marketo, Inc. Marketo Proprietary and Confidential
12% HigherClick-to-Open
Miles Gotcher, Marketo Blog, Oct. 2013
Page 27© 2013 Marketo, Inc. Marketo Proprietary and Confidential
A/B Testing in Marketo
Page 28© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Page 29© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Page 30© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Page 31© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Page 32© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Fills out Form
Visits web page
Forwarded to a Friend
Shared on Social Networks
Page 33© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Batch Email A/B Test
Page 34© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Testing in Triggered & Engagement Campaigns
Page 35© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Testing in Triggered & Engagement Campaigns
Page 36© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Testing in Triggered & Engagement Campaigns
Page 37© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Landing Page A/B Test
Page 38© 2013 Marketo, Inc. Marketo Proprietary and Confidential
Takeaways
1. Test whenever you can
2. Check for statistical significance
3. Go beyond batch tests
4. Test whole funnel, not just opens/clicks
