View
225
Download
0
Category
Preview:
Citation preview
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 1/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 2/89
About Myself & Spark59Founded 2 tech startups,worked with many others
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 3/89
Innovation in anAgile Environment
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 4/89
Waterfall
Release!
Risk
Time
B U I L D
B U I L D
B U I L D
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 5/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 6/89
Are we making our
product
better or worse?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 7/89
Are we efficient?vs
Are we effective?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 8/89
The Definition of Done
should include testing
customer behaviour
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 9/89
Validated Learning
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 10/89
IDEAS
PRODUCTDATA
MEASURE
BUILDLEARN
Experiment
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 11/89
“State, before you do the work,this is what I believe will happen,
and when you do the the worksee whether it happens or not.”
- Jabe Bloom
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 12/89
Hypotheses
instead of
Requirements
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 13/89
Requirement:
Users can create 3D avatars.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 14/89
We believe that users will create a 3Davatar and spend more time on other
users profiles.
Hypothesis:
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 15/89
50% of newly signed up users will create a 3D avatar,
Avg time spent on user profiles will go from 30s to 1min+
Falsifiable Hypothesis:
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 16/89
Be closer to reality,
and avoid
bias & comfort zones
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 17/89
User Stories != Experiments
Missing the “Measure” Stage
No Way to Validate the Learning
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 18/89
Deploy != Launchif $rollout.active?(:chat, current_user) ...
else ...end
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 19/89
Deploy Every Commit
Release To A Small Set of CustomersLaunch The New Feature When Ready
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 20/89
Experiments need to be
shown to actual customers
Not just to your team!
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 21/89
Lean Stack:Experiment Reports
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 22/89
Background
EXPERIMENT REPORT Author:Title: Created:
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
[DATE][NAME][TITLE]
Your list of statements on the expected outcome of the experiment.
Use this format:[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
What!s the next experiment?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 23/89
Background
EXPERIMENT REPORT Author:Title: Created:
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
[DATE][NAME][TITLE]
Your list of statements on the expected outcome of the experiment.
Use this format:[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
What!s the next experiment?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 24/89
Give Context.Why is this relevant?
Background
What are you trying to learn or achieve?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 25/89
Quantify Your Goals
Falsifiable Hypothesis
Your list of statements on the expected outcome ofthe experiment.
Use this format:[Specific Repeatable Action] will[Expected Measurable Outcome]
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 26/89
Set Scope & Timebox
Experiment Scope
How long will the experiment last?
Minimum Build Plan
List the build plan step by step to get yourmeasurement and experiment.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 27/89
1. Context - Why is this relevant?
2. Quantify Your Goals3. Set Scope & Timebox
Planning an Experiment
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 28/89
Test your
Value and your Growth
Hypothesis first
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 29/89
Validated Learning:
Define Hypothesis,Build & Launch Experiment,
Learn from the Results
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 30/89
Concepts for making this work:1. Product Ownership
2. Shared Understanding
3. Accountability4. Scoping & Timeboxing
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 31/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 32/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 33/89
Cross-Functional
Team
Product
Owner
Customers
Centralized Product Ownership
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 34/89
Cross-Functional
Team
Business
Stakeholder
Customers
Distributed Product Ownership
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 35/89
Product Ownership is about
Defining Customer Value,not Aesthetics
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 36/89
Business Experiments
instead of (just)Development Tasks
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 37/89
Experiments are
Owned ByA Team Member
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 38/89
Draws ExpertResources as needed
(e.g. through pairing)
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 39/89
1. Product Ownership
2. Shared Understanding3. Accountability
4. Scoping & Timeboxing
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 40/89
Why?are we doing
what we’re doing
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 41/89
Team needs to
Understand The Impact
of their work
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 42/89
“Shared understanding is aboutgetting everyone to walk in the
other person’s shoes a little.”Bill Scott - UX at Netflix, PayPal
This includes the business owner’s shoes!
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 43/89
Are we making progresstowards our business goal?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 44/89
Company-WideMetrics Dashboard
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 45/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 46/89
Actionable
AccessibleAuditable
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 47/89
ActionableTie specific repeatable actions toobserved results.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 48/89
AccessibleLet everyone in the companysee and understand whats going on.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 49/89
AuditableTie the numbers back to actualcustomers, so you can verify the metricand believe in it.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 50/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 51/89
Unit Economics
instead of Gross Revenue
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 52/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 53/89
ACQUISITION
ACTIVATION
RETENTION
REVENUE
REFERRAL
Value MetricsHow do users find you?
Do users have a great first experience?
Do users come back?
How do you make money?
Do users tell others?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 54/89
ACQUISITION
ACTIVATION
RETENTION
REVENUE
REFERRAL
How do users find you?
Do users have a great first experience?
Do users come back?
How do you make money?
Do users tell others?
Growth Metrics
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 55/89
Cross-FunctionalPairing
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 56/89
Weekly Strategy MeetingPart 1: Ideation
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 57/89
Avoid Brainstorming.
It causesPremature Convergence
Around One Idea.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 58/89
http://www.lostgarden.com/2010/08/visualizing-creative-process.html
By Daniel Cook
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 59/89
Design Studioby Will Evans
@semanticwill
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 60/89
Observe & ExplainTo The Group
IdeateAlone
DiscussIn the Group
Sketch &
Write down!
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 61/89
Review ExperimentsOnly In Weekly Meeting
Focus On Efficiency
In Daily Stand-ups
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 62/89
1. Product Ownership
2. Shared Understanding
3. Accountability
4. Scoping & Timeboxing
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 63/89
Lean Stack:Validated Learning Board
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 64/89
IDEAS
PRODUCTDATA
MEASURE
BUILDLEARN
Build Measure Learn
!
!
Kan-ban Board
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 65/89
Build Measure Learn!
!
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 66/89
1 experimentper team member
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 67/89
Separate Task Board
for tracking the details
Swimlane for each Experiment
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 68/89
Weekly Strategy MeetingPart 2: Review
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 69/89
Review Experiment Results
Challenge the InterpretationKill / Restart overdue Experiments
EXPERIMENT REPORT Author:Title: Created: [DATE][NAME][TITLE]
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 70/89
Background
Experiment Scope
Minimum Build Plan
Lean Stack by Spark59.com
Falsifiable Hypothesis
Validated Learning
Results
Next Action
What are you trying to learn or achieve?
List the build plan step by step to get your measurement and experiment.
How many and/or how long will the experiment last?
Your list of statements on the expected outcome of the experiment.
Use this format:[Specific Repeatable Action] will [Expected Measurable Outcome]
Enter the data.
Summarize your learning from the experiment.
VALIDATED or INVALIDATED
What!s the next experiment?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 71/89
Actual data you Measured,based on your Hypothesis.
Results
Enter the data.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 72/89
ValidatedInvalidatedInconclusive
Validated Learning
Summarize your learning from the experiment.
+ Why?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 73/89
What will you do nextbased on your learning?
Next Action
What!s the next experiment?
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 74/89
!Failure is good.
Validated Learning is ourMeasurement of Progress.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 75/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 76/89
Timeboxing
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 77/89
Build Measure Learn!
!
Experiment Scope
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 78/89
Build Measure Learn!
!
Experiment Scope
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 79/89
Startups don’t starve,
they drown.
Open-ended learning is dangerous,always timebox.
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 80/89
Interviews/Prototypes: 1-2 WeeksQualitative Tests: 1 Month
Quantitative Tests: 2-3 Months
Recommended Timeboxes:
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 81/89
Inside the Building Comfort Zone
Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”
Get-out-of-the-Building Reality
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 82/89
Customer Interview
(Paper) Prototype
MVP
“The Perfect Product”
Get out of the Building Reality
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 83/89
Justify the Cost of MVPswith Proof
U d t d P bl
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 84/89
Understand Problem:
Explorative Customer Interviews
Define Solution:
User Testing With Prototypes
Validate Qualitatively:Release MVP and hand-collect feedback
10s to 100s of customers
Verify Quantitatively:
Partial Rollout or Split Test and measure improvement
100s to 10,000s of customers
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 85/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 86/89
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 87/89
T l
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 88/89
Experiment Reports
Company-wide Dashboard
Validated Learning BoardWeekly Strategy Meeting
T o o l s & T a c t i c s
8/13/2019 20121106 Smidig Keynote
http://slidepdf.com/reader/full/20121106-smidig-keynote 89/89
Thanks!
leanstack.com
lukas@spark59.com
Recommended