Cultivating a Culture of Experimentation

Speakers

Cultivating a Culture of Experimentation

Bryan Berger, Product Design Lead, General Assembly

Taylor Gilbert, Partnerships & Alliances, Optimizely

Today’s Speakers

Bryan Berger

Taylor Gilbert

Housekeeping

• We’re recording!• Slides and recording will be

emailed to you after the webinar is complete

• There will be time for questions at the end

Agenda

• ‘Culture’ of Experimentation

• General Assembly’s Approach

• How to Energize Your Team

• Some Things We Tested

‘Culture’What does that

mean?

Turns out… culture is a

big deal.

of Fortune 500 companies from 1955 are still in business today12%

¼ companies fell off the list in the last year alone

The Organization of the Future…. Arriving Now

1. Culture has never been more important, experimentation is the vehicle.

2. Your executives are struggling and they need your

help. They are looking for someone to step up and lead/orchestrate.

General Assembly (GA) is a global educational company. Focusing on the most relevant and in demand skills across data, design, business and technology.

General Assembly is empowering a global community to pursue work they love through best-in-class instruction and access to opportunities.

Experimentation at

General Assembly

A Little About Me

● I manage our Product Design Team, whose focus is on designing an

ecosystem to support online and in-person learning experiences.

● I also work quite extensively with our Marketing and Website CRO team to

improve the positioning of our Products.

● One of my many goals is to foster a fun and impactful culture of

experimentation deeply rooted in the design process.

Why Experiment?● Progress over perfection. Experimentation allows our team to ideate multiple

solutions to a problem and not paralyze the process in favor of “the one”.

● Expand potential. Anyone with a hypothesis can partake. Inclusion has become

an ally in bringing down silos among our product teams.

● Gain insights. To develop a better understanding of our data, and how we can

affect the levers of improvement.

● Build consensus. Save time and money by testing your assumptions.

Experimentation at General Assembly

● Started to hit the limits of our current solutions.

● Our business was scaling quickly and along with it, our growth targets.

● We needed to start questioning things that have been “established”.

● Our testing program has been running for about a year.

● Prior to that we had no repeatable testing process in place.

How to Energize Your Team

Make Testing Tangible

Get Creative

Really Creative...

Make It Known

Highlight and Acknowledge Success

Our Approach

Getting Started: Finding a ProcessWhere we all start

Where some give up

The goal, a smoother repeatable process

Our Initial Process

Our Current Process

Finding Great Ideas● All ideas must contain:

○ Problem definition○ Hypothesis○ Audience Type (Persona)○ Page url(s) affected

● All ideas are added to a backlog list○ Tickets are created or updated once ideas are vetted for impact, time to significance, and

resource allocation.

● Hold weekly A/B testing meeting to: ○ Discuss how a specific test would be implemented ○ Review cost / benefits for different possible solutions

○ Amount of traffic needed○ Length of the experiment○ Next steps post test

Prioritizing Test Ideas

Website Tests are prioritized by our Marketing Team & Director of SEO/CRO

● We define quarterly Objectives & Key Results (OKRs) and roll up each test result toward one of those objectives

● Test key routes (purchase flow, wayfinding, and value prop improvement)● As of today, all tests aim to improve conversion and engagement at key steps of our funnel

Stay Organized

Share Results

Clear communication, acknowledgement, and collaboration.

● #CRO Slack channel for asking questions, and posting results.

● Announces test results and shoutouts.● Big wins are also sent to an email group.● Slack is effective. Everyone is engaged in it,

from engineering to execs.

Program GoalsOur current benchmark is to have at least one live test on each of the most critical pages in our conversion funnel (to keep momentum):

○ New user signup modal○ Homepage○ Navigation○ Product pages○ Wayfinding systems○ Discovery & catalog pages

Some Things We Tested

25% Off Class & Workshop Offer

Modal Incentive Copy Test

Original Variation

Drive more urgency to the next course start dateCourse Start Date

Original Variation

Copy and styling tests in Navigation menu Copy and Style in Navigation

Original Variation

Show the customer potential ROI of our courses

No original variation

Adding Sections

Original Variation

Looking Forward

● As we scale our experimentation program we’ll likely bring in an agency to help build, design, & deploy tests in Optimizely.

● With their support, we’ll be able to move more quickly on our backlog of 80+ test ideas.

● Continue testing to inform design direction for our new site.● Penetration into our Product Apps and onboarding flows.

Key Takeaways● Become the champion of your experimentation program● Make your testing program known, give it a physical presence● Start backlogging and organizing ideas● Establish a process to vet, prioritize and execute those ideas● Set up a staging environment to act as your laboratory● Hold yourselves accountable for stagnation● Beware of vanity metrics. ● Demonstrate how testing can be inclusive, impactful and FUN!

Questions?

Thank You!Feel free to reach out to continue the conversation:

@bryanberger @ga