Upload
john-oppenheimer
View
88
Download
2
Embed Size (px)
Citation preview
COMPANY NAME www.nicecompany.com
Best practices for building a testing culture
Presented by : Mari Ju and John Oppenheimer Optimizely User Group - August 2015
Product Marketing Manager Mari Ju
About US
[email protected] linkedin.com/in/mariju @mari4ju
Marketing Operations Analyst John oppenheimer
[email protected] linkedin.com/in/oppenheimerjohn
about Autodesk
Mission: Help people imagine, design, and create a better world
Est. 1982
12+ million users
Autodesk
Mission: Unlock the creativity in everyone
Est. 2011
170+ million users
Autodesk Consumer and 3d printing
Testing at Autodesk
Spring ‘14 Fall ‘14 Spring ‘15 Fall ‘15
Start small
Build trust
GENERATE Excitement
Advance & REPEAT
Training
• Onsite demo
• Team sessions
• Dedicated champions
Pick a starter property
• Simple first test as proof of
concept
Phase 1: Start Small
vs.
Original Variation
Results: Variation showed 24% more subscriptions compared to the Original
Address potential concerns:
• Test proposals to larger team
• Allow for plenty of QA
• Connect developers with Optimizely
support team
• Have developers launch test
• One test at a time, simple tests only
Phase 2: Build trust
Spread testing throughout the organization: • Add more products/domains • Get more team members involved Improve testing program: • Develop testing workflow:
Data à User Testing à Hypothesis à A/B Test
• Advanced tests (multi-page, redirects, cross-platform) that leverage integrations
• Establish testing as essential when updating the site
Share results (both wins and losses): • Distribute reports via email and internal blog • Celebrate team wins (Optie finalist)
PhaSE 3: Generate excitement
Continuously advance our testing program: • Standardized idea backlog and prioritization rubric
• Create an archive of previous tests that’s easy to share and search
• Work closely with designers and developers to launch more technical tests
• Start experimenting with personalization • Launch mobile app A/B testing program Evangelize benefits of testing:
• Internally – other products, departments, office locations, and more
• Externally – with partners and user groups
Phase 4: advance
Original Variation
1 2
3 4
Demonstrate potential impact of testing with a simple test
1. Start small
Increase complexity and frequency of tests for greater impact, and share results
3. Generate excitement
Bring all stakeholders into the process to gain trust, and eventually autonomy
2.Build trust
Continue to evaluate and improve testing program
4. ADVANCE
Steps for building a testing culture
COMPANY NAME www.nicecompany.com
Questions?
Mari Ju – [email protected]
John Oppenheimer – [email protected]