How Validation and Vision Co-exist by Andres Glusman

How Validation and Vision Co-exist

Andres Glusman Chief Product-Revenue Officer

Meetup

DEC 2013

2

3

BEST THING ABOUT LEAN:

“Let’s test it!”

WORST THING ABOUT LEAN:

“Let’s test it!”

8

MILLIONS OF WAYS TO FAIL

9

3 KINDS OF WINS

10

11

SMALL WINS TIED

TO LIFE SOURCE

12

13

BREAKTHROUGH WINS

14

BREAKTHROUGH WINS

15

MEANINGLESS WINS

16

Hunter Walk

LOW EFFORT HIGH EFFORT

BIG WIN

SMALL WIN

Dogs

Stars

???

???

Junk Food

Worthy Bets

18

OUR LESSONS AT MEETUP

Craft a design-driven

vision

Validate & iterate using usability lab

Break into iterative tests to (in)validate

vision

Repeat

19

20

21

WE UX TESTING

On pace to do 400 - 500 sessions a year

Everything user-facing gets tested (at different stages)

Test 5 days a week (mo, tu, wed, th, fri)

Set up tests prior to knowing what will be tested

Recruit participants from community and through craigslist

Anyone can request to have something tested

Follow each session with discussion & notes/video

COSTS AS MUCH AS 2 OUTSOURCED USABILITY

STUDIES

“I thought there would be more

colors““I thought there would be more

colors“

“I thought there would be more

colors“

26

IM from recent usability test

Shorten the time between someone at Meetup wondering how someone would react to something

and

actually watching someone react to something

MALKOVICH BIAS:THE TENDENCY TO BELIEVE THAT EVERYONE USES TECHNOLOGY LIKE

YOU DO

Links to our playbook:glusman.com

30

WE LIVE TESTING

32

1

2 3

33

34

35

WHAT ABOUT SMALL WINS?

BEWARE THE SMALL WIN

“This will only take a day or so”

“We saw a 12% lift in adoption from this flow”

37

FOCUS ON THE PRIZE

Run tests in core UX that could move vital metrics

Keep them super low-cost

38

SPLIT TEST BANDIT

The Bandit automatically optimizes tests in real-time

Allows us to run many variants on smaller drivers with minimal human intervention & debate

Go after big visions with systematic validation

Ditch meaningless wins

Take some of the small wins that supports the cause

41

Andres Glusman

andres@meetup.com

@glusman