Marketing Mashup: Top takeaways from Web Opt Summit 2014

Ask questions and tell us what you learned on Twitter!

#WebOpt14

Marketing Mashup: Top Takeaways from Web Opt Summit 2014

#WebOpt14

Allison BankoReporterMECLABS@AllisonBanko

Daniel BursteinDirector of Editorial ContentMECLABS@DanielBurstein

Today’s speakers

twitter.com/allisonbanko

twitter.com/danielburstein

Ask questions and tell us what you learned on Twitter!

#WebOpt14

Related resources

• Web Optimization: Can you repeat your test results?

• Customer-centric Marketing: How transparency translates into trust

• Web Optimization: Ancestry.com improves conversion 20% by reducing choice barriers

• Web Optimization: VacationRoost implements 2 testing methodologies to boost total conversion rates by 12%

• The Boston Globe: Discovering and optimizing a value proposition for content

http://sherpablog.marketingsherpa.com/marketing/web-optimization-repeat-test-results/

http://sherpablog.marketingsherpa.com/online-marketing/transparency-trust-customer-centric/

http://www.marketingsherpa.com/article/case-study/reducing-choice-barriers-web-optimization

http://www.marketingsherpa.com/article/case-study/2-testing-methodologies-boost-conversion

http://www.marketingsherpa.com/video/boston-globe-optimization-summit-replay

Capture transferable insights

Web & Product Marketing Manager Accellos

Kait Vinson


Web & Product Marketing Manager Accellos

Kait Vinson

Need headline with value

Intensify the value of copy

Images of product

CTA needs more value

Need to focus the two CTAs

“Register Now”

implies no value

Original


OriginalNew Design


OriginalNew Design


56%Increase in conversion

Testing and optimization cycle boosts conversion

Source: MarketingSherpa.com/ecommerce

Poll: How do you want this webinar to go?A. Snapshot: Let’s go fast and get through a lot

B. Take it Slow: Let’s dive deep into a few key discoveries

Michael Norton

Harvard Business School

Trust Through Transparency

The submerged state

Transparency

• Expressed a desire for larger government with more services

• Perceived the government to be doing a better job

• And even: Government often does a better job than it’s given credit for

Use transparency to:

• Get more credit from your partner

• Get more credit from your coworkers

• Help your customers understand all of the services you provide them

Takeaway: Times haven’t changed that much since third grade math class –show your work to get full credit.

Ryan HutchingsDirector of MarketingVacationRoost

How one company implements an entire testing methodology every day

The Nuts and Bolts

What’s a good goal for bounce rates?

> 50%

40%-50% 30%-40%

< 30%

What’s a good goal for CTR to the next page or step?

> 60%

50%-60%30%-50%

< 30%

Determining a “good” conversion rate

Source: MarketingSherpa.com/ecommerce

Conversion rate was most frequently less than 15%, with

many responses below 1%

Conversion goal results

Control Treatment

Large Test: Lead forms 1

Security seals

Control

Fail – no lift.


Treatment

Control Treatment


Shorter form

Control

Fail – no lift.


Treatment

Control Treatment


Option to call

Control Treatment

Fail – no lift.


Large Test: Lead forms



19% Lift

Small Test: PPC landing pages

39


40


41

How can we reduce friction and

anxiety on this page?


42


43


44

427% increase in CTR

Takeaway: Testing your webpages is like golf – you wonder why you put yourself through all of the “no lifts” until you finally get that win.

Test Snapshot: VMware

Background: SMB Solutions Page features multiple products.

Objective: To increase Try, Buy and engagement clicks.

Primary Research Question: Will elevating vSphere Essentials and vCenterLog Insight above the fold with CTAs increase engagement?

Test Design: A/B test (Control vs. Variant B vs. Variant C) 33/33/33 split

Presenter: Cindy LuTitle: Sr. Marketing Manager, Digital Marketing StrategyCompany: VMware

SMB: Identify and capitalize on quick winsControl Variant B: vSphere Essentials Hero

Variant C: vCenter Log Insight Hero

Experiment 3: Results

What You Need to Understand: For a quick win, start with thinking about where low-hanging fruit opportunities can be found.

Buy Online CTRConversion

RateRelative

DifferenceLevel of

Confidence

Control Page 1.56% - -

Treatment A – vSphere Essentials “Get Pricing” CTA 16.50% 956.16% 98%

Treatment B – vCenter Log Insight “Buy Online” CTA 6.73 330.49%

956% increase in clickthrough Moving the CTA above the fold increased CTR 956%.

From one-size-fits-all to customized experience

Offer Page Transformation

Emily TitcombSr. Manager, Product MarketingAncestry.com

Julia BabiarzSr. Interactive Art DirectorAncestry.com

12

13

Self-selectors Interrupted Browsers

21

Self-selectors Interrupted Browsers

Experiment 2New Template Design: Background

Objective: Replace the current high-performing Free Trial Deny page design with a template-based design, so that we can then target, customize and optimize.

Primary Research Question: Can we succeed with a total redesign of the Free Trial Deny page?

Test Design: A/B test – old design vs. new design

Research Notes: Since our “Interrupted Browsers” group is the largest, we decided to test the Free Trial Deny page first.

24

Control Treatment: Template Format

!


What You Need to Understand: We failed on our first attempt at a template design, but couldn’t isolate why.

Recipes Conversion % Lift

Control 2.57% -

Treatment 1 – New template design 2.38% -7.5%

7.5% decrease in conversion

Experiment 3New Template Design Take 2: Background

Objective: Examine three different factors of the new template design in an attempt to isolate exactly why the new template design failed.

Primary Research Question: Can we figure out what factors matter most for conversion?

Test Design: 9-cell full-factorial test. Default vs. new template with three factors: button size, price placement and imagery placement

Research Notes:

27

27

Button size:Standard vs. large

27


27


Price placement:Near CTA vs. far from CTA

27



27



Imagery placement:Side vs. top

27



Imagery placement:Side vs. top

!


What You Need to Understand: Price placement was the most influential element on conversion.

Recipes Conversion

Control 1.93%

Standard Button, Price Top, Images Left 1.73%

Standard Button, Price Below, Images Left 1.94%

Standard Button, Price Below, Images Top 1.83%

Standard Button, Price Top, Images Top 1.66%

Large Button, Price Top, Images Left 1.69%

Large Button, Price Top, Images Top 1.66%

Large Button, Price Below, Images Top 1.77%

Large Button, Price Below, Images Left 1.80%

Conversion matched

!


What You Need to Understand: Price placement was the most influential element on conversion.

Recipes Conversion

Control 1.93%

Standard Button, Price Top, Images Left 1.73%

Standard Button, Price Below, Images Left 1.94%

Standard Button, Price Below, Images Top 1.83%

Standard Button, Price Top, Images Top 1.66%

Large Button, Price Top, Images Left 1.69%

Large Button, Price Top, Images Top 1.66%

Large Button, Price Below, Images Top 1.77%

Large Button, Price Below, Images Left 1.80%

Conversion matched

29

Deny Page Consistency and Template Format: The winning treatment

29

Interrupted Browsers:Free Trial Deny


29

Interrupted PastFree Trialers:Hard Offer Deny


29

Upgraders:Upgrade Deny


29

Upgraders:Upgrade Deny


10% lift

Takeaway: Test multiple elements on your page to appeal to the multiple persona types visiting your site – never stop discovering things about your visitors.

Test Snapshot: Publishers Clearing House

Objective: To convert unengaged visitors into engaged customers.

Primary Research Question: Will a simple, but attention-grabbing, header convince unengaged visitors to play a game?

Test Design: A/B split test

Presenter: Michael ZaneTitle: Sr. Director of Online MarketingCompany: Publishers Clearing House

Research Notes:

Test Design: Runs for 7 days

Unengaged repeat visitor

Message drops from top after 2-second delay

PCH.com: Control

Hey there, Speedy! Don’t leave yet! We have winners every day, MichaelB – Play now and you could be one of them!

Unengaged message variations

Hey there, Speedy! Don’t leave yet! We have winners every day, [username] – play now and you could be one of them!

FREE Money! Daily WINNERS Guaranteed! [username], how can you say no?

Is it a Bird? Is it a Plane? No, it’s [username] coming and going SO FAST! Slow down & play awhile – it could win you BIG CASH!

Money! Money! Money! We’re giving it away, [username] – Play Now!

We saw you left without playing last time, [username]. Don’t you want to win?

Don’t leave without playing our $2,500.00 Instant Win Scratch Cards, [username]! A BIG WIN could be just a few clicks away!

36% lift in engaged usersCompared to the control

Engaging the Unengaged: Results

Initial test shows strong results, but they are only valuable if it can be repeated.

Size % Engaged

Control – No real-time messaging 152,634 14%

Treatment – Real-time messaging with in-line banner 273,198 19%

Lift 36%

!

Experiment 1.2: Engaging with pop-up

Objective: To convert unengaged visitors into engaged customers using a different presentation of the messaging.

Primary Research Question: Can we improve on results of round 1 by using a pop-up instead of a banner?

Test Design: A/B split test

Experiment Research Notes:

Test Design: Runs for 7 days

Unengaged repeat visitor

Message pops up in the center after 2-second delay

-1.3% decrease in engaged usersCompared to the control

Experiment 1.2 Results

Experiment 1.2 wasn’t working, so it was stopped after four days.

Size % Engaged

Control – No real-time messaging 94,270 20.8%

Treatment – Real-time messaging with lightbox pop-up 118,977 20.5%

Lift -1.3%

!

Test Snapshot: One Call Now

Presenter: Jacob BaldwinTitle: Digital Marketing ManagerCompany: OneCallNow.com

Background: My curiosity led me to test the Request a Quote button copy on our homepage.

Objective: To increase the number of people clicking the Request a Quote button.

Primary Research Question: Which button copy will produce the desired outcome?

Test Design: Sequential

Research Notes:

Request For Quote: Control

84

Request For Quote: Treatment

85

Side by side

86

Original Treatment

vs.

!

Experiment: Results

Having the right imagery matters!

Test Page Button Click Rate

Control – Stock image of older phone 1.98%

Treatment – Stock image of modern phone 3.88%

Relative Difference 95.96%

95% increase in conversion initiationLed to an 82% increase in number of quote requests.

Peter DoucetteVP Consumer Sales & MarketingThe Boston Globe

How The Boston Globe moved beyond the landing page

Optimizing the Entire Business

Optimization beyond the landing page –Beyond marketing Reporting

Printing

Delivery

Live Newsroom Headline Test

2 versions were tested

Did School Run by State Contractor Fail to Report Rape?

Live Headline Test

OR go to MarketingSherpa.com/LivePoll

Police Allege Sex Assault in Walpole Not Reported

Did School Run by State Contractor Fail to Report Rape?

Text GlobeB to 22-333

Text GlobeA to 22-333

My every intention is to push the kind of boldness and investment that will

make The Globe a laboratoryfor major newspapers across the country.

http://www.bostonglobe.com/opinion/2013/10/27/why-bought-globe/QmFHhvRGFajQh1oMcAJ64M/story.html

Digital access price testing

Prices tested:• Control price point = $3.99

• T1 price point = $0.99




Each treatment featured a 99¢ trial offer for four weeks stepping to the full rate noted in the treatments.

Digital Access Price Testing: Early results

36% increase in digital access subscriptionsThe $0.99 price point drove 36% more conversions than the control.

Treatment Conversion Samples Successes Failures

Control $3.99 1.65% 17,788 327 17,461

Treatment 1 $0.99 2.18% 18,061 453 17,608

Treatment 2 $1.99 1.95% 17,946 400 17,546

Treatment 3 $2.99 1.80% 17,988 367 17,621

Treatment 4 $4.99 1.47% 18,017 301 17,716

Digital Access Price Testing: 52 week results

62% decrease in digital access revenueThe $0.99 price point drove 62% less conversions than the control.

Treatment Conversion Cancel Rate 52 Week Revenue Rev/Subscriber Rev/Visit

Control $3.99 1.84% 57% $45,189.15 $112.13 $2.09

Treatment 1 $0.99 2.51% 41% $17,172.54 $36.46 $0.77

Treatment 2 $1.99 2.23% 51% $26,048.01 $61.73 $1.18

Treatment 3 $2.99 2.04% 56% $31,611.53 $82.54 $1.44

Treatment 4 $4.99 1.67% 60% $39,386.81 $130.42 $1.80

Digital access accordion checkout testing


An accordion checkoutwas successful for

ASSUMPTION



35% decrease in digital access revenueThe accordion checkout drove 35% less conversions than the control.

That everything we’ve done has beensuccessful is not reality. Some thingsdid not meet our expectations.

and we apply to the next test.But we learn every time

What our audience thought:

What our audience thought:

At the end of the 30 minutes, B was leading!

Takeaway: Optimization transcends marketing webpages – company-wide optimization is a paradigm shift toward a culture of learning and constant improvement.

Test Snapshot: Panel

Presenter: Lauren PitchfordTitle: Sr. Optimization ManagerCompany: MECLABS

Background: Home delivery subscription has multiple delivery options such as daily, weekend, weekday, Sunday only.

Goal: To determine if reordering the delivery options would have an effect on the subscription mix and revenue.

Research Question: Which order of subscription options will yield the most revenue?

Test Design: A/B/C split test

High2Low

Low2High

Popularity

Which price order generated the most revenue?

Results

! What You Need to Understand: By simply reordering the options displayed, we were able to significantly affect the product mix.

Design Revenue per Visit

Price high to low .4

Price low to high .5

Popularity .4

Relative Difference 20%

20% increase in revenue per visit (93% LOC)

Treatment 1 increased revenue per visit by 20%.

Micahel Lykke AagaardSenior Conversion Optimization Consultant, Founder of ContentVerve.com

ContentVerve ApS/Atcore ApS

How, When and Why Minor Changes Have a Major Impact on Conversion

“It’s not about making extensive changes – it’s about making the right changes.”

- When they address critical friction in the conversion process

- When they are made strategically to prominent and/or mission-critical elements that directly impact the decision-making process of the prospect

114

Small changes have a major impact

When do small changes have little or no impact?

Hell no! Low motivation Hell yes! High motivationHmmm, maybe …

Friction

Landing Page

Email

PPC SEO

Banner

38.46% more sign-ups

During the campaign, the treatment copy sold 38.46% more memberships.

Mission-critical elements

23.88% higher open rateStatistical confidence: 95%

Control

Treatment

Control

31.95% more sales via product pagesStatistical confidence: 98%

Treatment

Control

213.16% higher CTR Statistical confidence: 99%

“It is not the magnitude of change on the ‘page’ that impacts conversion; it is the magnitude of change in

the ‘mind’ of the prospect.”

– Flint McGlaughlin, MECLABS

Takeaway: Big impacts can come in the form of small changes, but these changes must be made to the right elements.

One more test from Michael Aagaard …

100% privacy - we will never spam you!

Variant Conv. Rate Rel. Diff. Confidence

Control 2.03%

Treatment 1.65% -18.70% 96%

18.70% less sign-upsThe treatment with privacy policy decreased sign-ups.

Sample: 16,152 visitors

Conversions: 297

We guarantee 100% privacy. Your information will not be shared.

19.47% sign-upsStatistical confidence: 96%

Selling the Intangible

Tim KachuriakChief Innovation & Optimization Officer

Next After

How e-commerce and subscription-based companies can learn to sell intangible value like nonprofit organizations

The conference that changed my life.

Experiment: My very first experiment

Experiment ID: NA00001Location: Next After Research LibraryTest Protocol: GWBPC01

Background: Email donor acquisition campaign for the George W. Bush Presidential Center.

Objective: To increase clickthrough, increase revenue.

Primary Research Question: Does this MECLABS stuff really work?

Test Design: Single-factorial A/B split test

Research Notes:

Version A - Control Version B - Treatment

Version A - Control

141% increase in clickthrough rate38% increase in revenue

Version B - Treatment

Optimized

Original

141% increase in clickthrough

38% increase in revenue

!

Experiment: Results

What You Need to Understand: Small changes to the right elements can produce dramatic results. By increasing the force of the value proposition, we inspired more clicks and received more donations.

Email Version CTR

Control – “Make a tax-deductible contribution” 0.8%

Treatment – “Become a charter member” 1.9%


141% increase in clickthrough rate38% increase in revenue.

Strategic framework

When it comes to increasing online revenue, there are three primary strategies:

– Increase traffic

• Get more of the (right) people to show up

– Increase conversion rate

• Get more people to say, “Yes!”

– Increase average order value

• Get more people to say, “Heck yes!”

You already sell intangible value every day.

Experiment: The effect of value proposition on average donation

Experiment ID: NA01011Location: Next After Research LibraryTest Protocol: THF024

Background: The Heritage Foundation is a think tank in Washington, D.C.

Objective: To increase donation conversion rate.

Primary Research Question: How does value proposition effect conversion rate?

Test Design: Radical redesign A/B split test

Research Notes:

TreatmentControl

74% increase in donor conversion189% increase in average gift274% increase in revenue

TreatmentControl

74% increase in donor conversion rate189% increase in average gift274% increase in revenue

• Headline: Conveys value and vision

• Bulleted Copy: Provides reasons why you should give

• Embedded Quotes: Bolsters credibility

• Call-to-action and Donation Form: Introduced at the end of thought sequence when value has been clearly communicated

!

Experiment: Results

What You Need to Understand: By increasing the force of the value proposition, not only did we get more people to say, “Yes,” and give a gift, but they said, “HECK YES!” and gave at a much higher level.

Email Version Avg. Gift

Control – Two column, donation form above the fold $37.50

Treatment – Single column, long form, donation below fold $108.30


189% increase in average donation74% increase in donation conversion rate.

Takeaway: Optimization doesn’t always result in the most beautiful pages, but it does result in the most user-friendly experiences.

Allison BankoReporterMECLABS@AllisonBanko

Daniel BursteinDirector of Editorial ContentMECLABS@DanielBurstein

Thank you!

twitter.com/allisonbanko

twitter.com/danielburstein