20121106 Smidig Keynote

8/13/2019 20121106 Smidig Keynote

http://slidepdf.com/reader/full/20121106-smidig-keynote 1/89

8/13/2019 20121106 Smidig Keynote


About Myself & Spark59Founded 2 tech startups,worked with many others

8/13/2019 20121106 Smidig Keynote


Innovation in anAgile Environment

8/13/2019 20121106 Smidig Keynote


Waterfall

Release!

Risk

Time

B U I L D

B U I L D

B U I L D

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


Are we making our

product

better or worse?

8/13/2019 20121106 Smidig Keynote


Are we efficient?vs

Are we effective?

8/13/2019 20121106 Smidig Keynote


The Definition of Done

should include testing

customer behaviour

8/13/2019 20121106 Smidig Keynote


Validated Learning

8/13/2019 20121106 Smidig Keynote


IDEAS

PRODUCTDATA

MEASURE

BUILDLEARN

Experiment

8/13/2019 20121106 Smidig Keynote


“State, before you do the work,this is what I believe will happen,

and when you do the the worksee whether it happens or not.”

- Jabe Bloom

8/13/2019 20121106 Smidig Keynote


Hypotheses

instead of

Requirements

8/13/2019 20121106 Smidig Keynote


Requirement:

Users can create 3D avatars.

8/13/2019 20121106 Smidig Keynote


We believe that users will create a 3Davatar and spend more time on other

users profiles.

Hypothesis:

8/13/2019 20121106 Smidig Keynote


50% of newly signed up users will create a 3D avatar,

Avg time spent on user profiles will go from 30s to 1min+

Falsifiable Hypothesis:

8/13/2019 20121106 Smidig Keynote


Be closer to reality,

and avoid

bias & comfort zones

8/13/2019 20121106 Smidig Keynote


User Stories != Experiments

Missing the “Measure” Stage

No Way to Validate the Learning

8/13/2019 20121106 Smidig Keynote


Deploy != Launchif $rollout.active?(:chat, current_user) ...

else ...end

8/13/2019 20121106 Smidig Keynote


Deploy Every Commit

Release To A Small Set of CustomersLaunch The New Feature When Ready

8/13/2019 20121106 Smidig Keynote


Experiments need to be

shown to actual customers

Not just to your team!

8/13/2019 20121106 Smidig Keynote


Lean Stack:Experiment Reports

T o o l s & T a c t i c s

8/13/2019 20121106 Smidig Keynote


Background

EXPERIMENT REPORT Author:Title: Created:

Experiment Scope

Minimum Build Plan

Lean Stack by Spark59.com

Falsifiable Hypothesis

Validated Learning

Results

Next Action

What are you trying to learn or achieve?

List the build plan step by step to get your measurement and experiment.

How many and/or how long will the experiment last?

[DATE][NAME][TITLE]

Your list of statements on the expected outcome of the experiment.

Use this format:[Specific Repeatable Action] will [Expected Measurable Outcome]

Enter the data.

Summarize your learning from the experiment.

VALIDATED or INVALIDATED

What!s the next experiment?

http://spark59.com/

http://spark59.com/

http://spark59.com/

8/13/2019 20121106 Smidig Keynote


Background

EXPERIMENT REPORT Author:Title: Created:

Experiment Scope

Minimum Build Plan



Validated Learning

Results

Next Action




[DATE][NAME][TITLE]



Enter the data.




http://spark59.com/

http://spark59.com/

http://spark59.com/

8/13/2019 20121106 Smidig Keynote


Give Context.Why is this relevant?

Background


8/13/2019 20121106 Smidig Keynote


Quantify Your Goals


Your list of statements on the expected outcome ofthe experiment.

Use this format:[Specific Repeatable Action] will[Expected Measurable Outcome]

8/13/2019 20121106 Smidig Keynote


Set Scope & Timebox

Experiment Scope

How long will the experiment last?

Minimum Build Plan

List the build plan step by step to get yourmeasurement and experiment.

8/13/2019 20121106 Smidig Keynote


1. Context - Why is this relevant?

2. Quantify Your Goals3. Set Scope & Timebox

Planning an Experiment

8/13/2019 20121106 Smidig Keynote


Test your

Value and your Growth

Hypothesis first

8/13/2019 20121106 Smidig Keynote


Validated Learning:

Define Hypothesis,Build & Launch Experiment,

Learn from the Results

8/13/2019 20121106 Smidig Keynote


Concepts for making this work:1. Product Ownership

2. Shared Understanding

3. Accountability4. Scoping & Timeboxing

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


Cross-Functional

Team

Product

Owner

Customers

Centralized Product Ownership

8/13/2019 20121106 Smidig Keynote


Cross-Functional

Team

Business

Stakeholder

Customers

Distributed Product Ownership

8/13/2019 20121106 Smidig Keynote


Product Ownership is about

Defining Customer Value,not Aesthetics

8/13/2019 20121106 Smidig Keynote


Business Experiments

instead of (just)Development Tasks

8/13/2019 20121106 Smidig Keynote


Experiments are

Owned ByA Team Member

8/13/2019 20121106 Smidig Keynote


Draws ExpertResources as needed

(e.g. through pairing)

8/13/2019 20121106 Smidig Keynote


1. Product Ownership

2. Shared Understanding3. Accountability

4. Scoping & Timeboxing

8/13/2019 20121106 Smidig Keynote


Why?are we doing

what we’re doing

8/13/2019 20121106 Smidig Keynote


Team needs to

Understand The Impact

of their work

8/13/2019 20121106 Smidig Keynote


“Shared understanding is aboutgetting everyone to walk in the

other person’s shoes a little.”Bill Scott - UX at Netflix, PayPal

This includes the business owner’s shoes!

8/13/2019 20121106 Smidig Keynote


Are we making progresstowards our business goal?

8/13/2019 20121106 Smidig Keynote


Company-WideMetrics Dashboard


8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


Actionable

AccessibleAuditable

8/13/2019 20121106 Smidig Keynote


ActionableTie specific repeatable actions toobserved results.

8/13/2019 20121106 Smidig Keynote


AccessibleLet everyone in the companysee and understand whats going on.

8/13/2019 20121106 Smidig Keynote


AuditableTie the numbers back to actualcustomers, so you can verify the metricand believe in it.

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


Unit Economics

instead of Gross Revenue

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


ACQUISITION

ACTIVATION

RETENTION

REVENUE

REFERRAL

Value MetricsHow do users find you?

Do users have a great first experience?

Do users come back?

How do you make money?

Do users tell others?

8/13/2019 20121106 Smidig Keynote


ACQUISITION

ACTIVATION

RETENTION

REVENUE

REFERRAL

How do users find you?

Do users have a great first experience?

Do users come back?

How do you make money?

Do users tell others?

Growth Metrics

8/13/2019 20121106 Smidig Keynote


Cross-FunctionalPairing


8/13/2019 20121106 Smidig Keynote


Weekly Strategy MeetingPart 1: Ideation


8/13/2019 20121106 Smidig Keynote


Avoid Brainstorming.

It causesPremature Convergence

Around One Idea.

8/13/2019 20121106 Smidig Keynote


http://www.lostgarden.com/2010/08/visualizing-creative-process.html

By Daniel Cook




8/13/2019 20121106 Smidig Keynote


Design Studioby Will Evans

@semanticwill

8/13/2019 20121106 Smidig Keynote


Observe & ExplainTo The Group

IdeateAlone

DiscussIn the Group

Sketch &

Write down!

8/13/2019 20121106 Smidig Keynote


Review ExperimentsOnly In Weekly Meeting

Focus On Efficiency

In Daily Stand-ups

8/13/2019 20121106 Smidig Keynote


1. Product Ownership

2. Shared Understanding

3. Accountability

4. Scoping & Timeboxing

8/13/2019 20121106 Smidig Keynote


Lean Stack:Validated Learning Board


8/13/2019 20121106 Smidig Keynote


IDEAS

PRODUCTDATA

MEASURE

BUILDLEARN

Build Measure Learn

!

!

Kan-ban Board

8/13/2019 20121106 Smidig Keynote


Build Measure Learn!

!

8/13/2019 20121106 Smidig Keynote


1 experimentper team member

8/13/2019 20121106 Smidig Keynote


Separate Task Board

for tracking the details

Swimlane for each Experiment

8/13/2019 20121106 Smidig Keynote


Weekly Strategy MeetingPart 2: Review


8/13/2019 20121106 Smidig Keynote


Review Experiment Results

Challenge the InterpretationKill / Restart overdue Experiments

EXPERIMENT REPORT Author:Title: Created: [DATE][NAME][TITLE]

8/13/2019 20121106 Smidig Keynote


Background

Experiment Scope

Minimum Build Plan



Validated Learning

Results

Next Action






Enter the data.




http://spark59.com/

http://spark59.com/

http://spark59.com/

8/13/2019 20121106 Smidig Keynote


Actual data you Measured,based on your Hypothesis.

Results

Enter the data.

8/13/2019 20121106 Smidig Keynote


ValidatedInvalidatedInconclusive

Validated Learning


+ Why?

8/13/2019 20121106 Smidig Keynote


What will you do nextbased on your learning?

Next Action


8/13/2019 20121106 Smidig Keynote


!Failure is good.

Validated Learning is ourMeasurement of Progress.

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


Timeboxing

8/13/2019 20121106 Smidig Keynote



!

Experiment Scope

8/13/2019 20121106 Smidig Keynote



!

Experiment Scope

8/13/2019 20121106 Smidig Keynote


Startups don’t starve,

they drown.

Open-ended learning is dangerous,always timebox.

8/13/2019 20121106 Smidig Keynote


Interviews/Prototypes: 1-2 WeeksQualitative Tests: 1 Month

Quantitative Tests: 2-3 Months

Recommended Timeboxes:

8/13/2019 20121106 Smidig Keynote


Inside the Building Comfort Zone

Customer Interview

(Paper) Prototype

MVP

“The Perfect Product”

Get-out-of-the-Building Reality

8/13/2019 20121106 Smidig Keynote


Customer Interview

(Paper) Prototype

MVP

“The Perfect Product”

Get out of the Building Reality

8/13/2019 20121106 Smidig Keynote


Justify the Cost of MVPswith Proof

U d t d P bl

8/13/2019 20121106 Smidig Keynote


Understand Problem:

Explorative Customer Interviews

Define Solution:

User Testing With Prototypes

Validate Qualitatively:Release MVP and hand-collect feedback

10s to 100s of customers

Verify Quantitatively:

Partial Rollout or Split Test and measure improvement

100s to 10,000s of customers

8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


8/13/2019 20121106 Smidig Keynote


T l

8/13/2019 20121106 Smidig Keynote


Experiment Reports

Company-wide Dashboard

Validated Learning BoardWeekly Strategy Meeting


8/13/2019 20121106 Smidig Keynote


Thanks!

leanstack.com

[email protected]

mailto:[email protected]



Documents

20121106 Smidig Keynote