Embed Size (px)
Citation preview
#opticon2015
Optimization at OptimizelyAsh AlhashimDirector of Sales & Market Development, Optimizely
Amanda SwanCommunity Manager, Optimizely
Adam AvramescuHead of Customer Education, Optimizely
@avramescu
We optimize a lot. Today, let’s focus on one slice of the pie.
When experimenting, there can be many
sources of complexityWhen testing, there can be many
sources of complexity
A few examples…
#opticon2015
Optimizing Optiverse
Amanda SwanCommunity Manager, Optimizely
Adam AvramescuHead of Customer Education, Optimizely
What is Optiverse?Optimizely’s Community, Knowledge Base, & Academy
What you’ll get out of this portion
• Trying to optimize your support site or community?
• Testing search or nav structures?
• What do you do with low sample size (low conversions)?
• How do you optimize without clear conversion funnels?
Effective Communication Channels are Crucial
4 Steps
Expose themes through
qualitative feedback
Test assumptions through
conceptual validation
Analyze results for
directions & preferences
Loop back to qualitative validation and
expand the hypothesis
Optiverse Search
Hypothesis
By removing non-essential filters, and making the essential ones more prominent, we can increase filter usage, improving the relevance of results.Are “content type” filters actually useful and relevant to visitors?
Exposing themes through qualitative feedback
Testing assumptions through conceptual validation
Testing assumptions through conceptual validation
Analyze results for directions & preferences
Analyze results for directions & preferences
Support Clicks > Academy & Community Clicks
Loop back to qualitative validation & expand the hypothesis
Loop back to qualitative validation & expand the hypothesis
Loop back to qualitative validation & expand the hypothesis
The Knowledge Base
Hypothesis
The existing search experience and relevant sidebar content are less helpful for visitors to find the right support content.If we create search-first and FAQ-first experiences, will visitors engage with these options more?
Exposing themes through qualitative feedbackSearch
SidebarDrawers
Testing assumptions through conceptual validation
Search
Testing assumptions through conceptual validation
Sidebar
Testing assumptions through conceptual validation
Sidebar
Search
Analyze results for directions & preferences
Analyze results for directions & preferences
Analyze results for directions & preferences
Analyze results for directions & preferences
Analyze results for directions & preferences
• Implement prominent search
• Search-first vs. search AND sidebar
• Different types of FAQ content
• What would this look like without the drawers?
Loop back to qualitative validation & expand the hypothesis
Summing Up the 4 Steps
• Expose themes through qualitative feedback
• Test assumptions through conceptual validation
• Analyze results for directions & preferences
• Loop back to qualitative validation & expand the hypothesis
LEARN from every experiment
#opticon2015
Optimizing for Reductions& Operational EfficienciesAsh Alhashim
Director of Sales & MarketDevelopment
Background
[email protected] [email protected]
[email protected]@poop.com
[email protected]@gmail.com
Background
How can we reduce errors without negatively impacting lead gen numbers?
Adding email verification to our lead form will increase the overall lead quality, and allow Sales
Development Representatives to spend their time on more revenue generating activity.
Hypothesis
All non-signed-in visitors to the homepage that click on “Test it Out” will enter the experiment
Audience
Original Variation #1
The Recipe
Ingredients
Ingredients
Body text goes here.
Body text goes here.
Body text goes here.
Ingredients
Body text goes here.
Body text goes here.
The Recipe: Preparation
The Recipe: Preparation
• Test launched using conditional activation mode • Stopping Condition: 33,570 visitors • Double blind experiment
Goals
Goals
Primary Goal: • Sales Accepted Leads
Goals
Primary Goal: • Sales Accepted Leads
Secondary Goals:• Lead/Account Creation• Avg. Infer Lead Score• Opportunity Creation• Customer Creation
Results
Body text goes here.
Results
Statistical
Key Takeaways• Most experiments result in a
statistical tie – plan accordingly• Define success before you run an
experiment• Test deep into your funnel, we have
the technology• Don’t only optimize for CR lift; test for
quality as well
#opticon2015
Ash AlhashimDirector of Sales & Market Development, Optimizely
Amanda SwanCommunity Manager, Optimizely
Adam AvramescuHead of Customer Education, Optimizely
Thank you!
