Embed Size (px)
Citation preview
#opticon2015
Scaling The Next Web’s Testing Program to Maximum ImpactMartijn ScheijbelerLead Growth, SEO & Analytics - The Next Web
@martij[email protected]
Yearly 5.000 attendees, 75.000 products sold. Monthly 6 million unique users, 12 million pageviews.
The Next Web: 2006 > 2015
What’s happened in the last 12 months?
Scaling our testing program
That clearly wasn’t good enough
1 test a month :(
Currently 5-6 tests a week, how did we get there?
15-20 tests a month :)
What did we learn along the way?
• Swim lanes, KPIs, Goals.
• Segments: traffic sources, etc.
• Idea creation: backlog + prioritisation.
• Scaling > Analysing results.
• Do it live!
Unique Users, Page Views, Emails & Revenue.Goals: UU, PV, Emails & $
Related stories, pagination, sharing, email signups.
KPI: CTR, Engagement, Funnels
New v.s. Returning - Europe v.s. USA - Facebook v.s. SEO - Mobile v.s. Desktop
Type, Regions, Source, Device
Idea creation v.s. Planning
• Different templates + segments.
• How much time per test? Run tests for a fixed predefined time.
• The maximum of your program.
• Take away: create a backlog!
The part that we can’t scale yet
Analysing > Scaling!
Well, this is both the most boring and most exciting part!
1-2 main KPIs, 15 secondary KPIs
Rails, Google Analytics API Built with love by the TNW Marketing team.
TNW Analytics
TNW Analytics was born!
Adding a new idea
Schedule
Test
Saving 20% of time per test!
Take away: Be able to push code!
What’s next?
Scaling our testing program to the next level?• Even more templates, across more projects.
• 25+ tests a month with current growth of traffic.
• Bigger team? We can’t keep up with coding ;-)
• More automation? Push test variant to GTM.
• Mix it up with personalisation/ behaviour.
• Push our experiments to Slack for sharing.
• Create a better style guide + Bootstrap guidelines.
Summary & Take aways
• Know your KPI framework, expectations and limitations.
• Know your segments, CR% so you know how to prioritise.
• Follow your planning, priorities.
• Scale = 40% - 50% automation: analyse fast!
• Be able to push code live!
#opticon2015
Have fun & Thank you!
Martijn ScheijbelerLead Growth , SEO & Analytics - TNW
@martij[email protected]
