Web Analytics 101  OPTIMIZATION

The power of A/B and Multivariate testing.

Optimization is the practice of using hypothesis testing and experimentation to continuouslyimprove site and campaign performance.

A/B TestingRandomized experiment with two variants.

Multivariate TestingRandomized experiment with multiple variants.

Benefits

Sustained performance improvement.

Predictable, controlled routine.

Lower risk from changes.

Optimization TestingExamples

Mobile vs. OnlinePre-header clicks

A

B

“Mobile”

“Online”

A

“Mobile”

Version A received a 173% increase in unique clicks.

Test was conducted after learning 25% of members read email on their

mobile devices.

Sub-NavigationLeads

A BWithout Sub-Navigation With Sub-Navigation

B With Sub-Navigation

Version B increased leads by

39%.

Noticed a correlation between

leads and visitors seeing 5

specific pages on the site.

Good site analytics leads to

strong hypotheses!

ImagesAccount sign-ups

A

B

Sarah

Blair

A

ASarah

A

Sarah drove 36% more account signups than Blair.

TipsTo keep in mind

Get agreement up front on the

Evaluation Criteria with Business

and IT decision-makers.

Targeting increases the power of

optimization.

Be aware of confounding factors

such as site changes that occur

independently of your experiments.

They can pollute your metrics and

lead to incorrect conclusions.

TricksTo remember

Optimization should be an ongoing

program – you will continue to get

benefit as long as you do it.

As you get more sophisticated, you

can run many experiments at once.

But don’t get sloppy with the

samples.

Be patient – some tests (especially

UX changes) take many months to

show results.

SOCIETY CONSULTING Presented by

Connect With Us

SOCIETY CONSULTING

@SOCIETY_CONSULT

SOCIETY CONSULTING

SOCIETYCONSULTING.COM