How do you know you are delivering value?

How do we know we’re delivering value or having the intended impact?

Presented by Kevin Burns1-27-2017

kburns@sagesw.com, @kevinbburns

Open Discussion

Who’s measuring value, outcomes, and/or impacts today?

How are you measuring them?

If you’re not measuring them, why not?

Who’s measuring cost?

How are you measuring cost?

kburns@sagesw.com, @kevinbburns 2

The 1st Agile Principle

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

(should we change valuable to beneficial impact?)

How do we define value (impact) and how do we measure it?

Not all Projects (or Features) are created equal.

Is value determined by delivery on time, on budget, and on scope?

Is the scope delighting the customer?Are they using everything we delivered?

In a survey of 4 products, 65% of the features were rarely or never used.

How much money could have been saved if we never built them?

In the Waterfall project world, we have to ask for everything we can think of because capital will end at the end of the project. Instead we should be asking what has the most value in terms of the business outcome and/or impact and how are we going to measure it.

Assumptions Challenged

• 3 things we wish were true

• Customer knows what they want

• Developers know how to build it

• Nothing will change along the way

• 3 things we have to live with

• Impact isn’t known until software is used in production

• Developers discover how to build it

• Many things change along the way

Marty Cagan Quotes

• Customers don’t know what they want. It’s very hard to envision the solution you want without actually seeing it.

• At least 2/3 of our ideas are never going to work. The other 1/3 will take 3-4 iterations to get right.

• The role of the product manager is to discover a product that is valuable, usable, and feasible. Product, design, and engineering work together to arrive at optimal solution.

What we measure is changing

Business CustomerPO, SM, BL

Software EngineeringAD, DD, DA

UserUX, BA, QA, SME

BusinessValuable

Design Usable

TechnicallyFeasible

INNOVATIVESOLUTION

Lean Startup

• Are we asking what are Minimum Viable (Valuable) Product and how do we know when we’ve delivered it?

• Use a scientific method to measure, learn and pivot or preserver.

• Use meaningful quantitative objective measure to evaluate impact.

• Can you use A/B testing?

MVPInnovation

UserUX, BA, QA, SME

Business Valuable

Design Usable

Software EngineeringAD, DD, DA

Business CustomerPO, SM, BL

Use scientific method (measurable) to learn

and discovery your Minimum Viable

(Valuable) Product (MVP)

Technically Feasible

MVP innovations emerge from Conversations

Impact-Drive Development

Value and/or Impact driven culture

• Are we measuring the Cost vs Benefit at all levels of our work items?• Portfolio

• Program

• Project

• Feature/Capability

• Story/Requirement

• Tasks/Test

• Are we measuring the Impact our features have on our customers?

• The act of sizing helps us define done and what the really valuable work is

• Use story telling and test statements create understanding of value and DoD

Multi-month

Monthly

2-weeks

Leadership T-Shirt Sizing

X-S 1 SprintS <1 monthM 1-3 monthsL 3-9 monthsX-L >9 months

Team Planning-Poker Fibonacci Sizing

(1,2,3,5,8,13,20,40,100)

Team task hours to capacity (2,4,6)

Solution Decomposition Sizing Pattern

Sizing our CostThe act of sizing helps us

understand what’s valuable to deliver.

Scope doesn’t grow, our understanding does.

kburns@sagesw.com, @kevinbburns

Deliver 100% of 10% of Project

• Can we incrementally deliver value and test it’s impact?

• Can we create incremental release plans to deliver 100% of 10% of project?

• What constraints do we have in working this way?

• Can we overcome or work within these constraints and still deliver incrementally?

• What/who is preventing this approach?

Create Faster Feedback

• When queues and batch sizes are large feedback is slow

• Slow feedback hurts quality, efficiency, and cycle time

• Feedback speed has enormous economic leverage in product development, but it is rarely explicitly managed

The Front-Loaded Lottery

• A lottery ticket pays $3000 to winning three digit number

• You can pick the number in two ways:• Pay $3 to select all three digits at once

• Pay $1 for the first digit, find out if it is correct, then choose if you wish to pay $1 for the second digit, and then choose if you wish to pay $1 for the third digit.

Value of Feedback

Spend $1

Savings = $0.90Savings = $0.99

0 $1 $2 $3

Probabilityof

Occurrence

Cumulative Investmentkburns@sagesw.com, @kevinbburns 22

Sequence Work Correctly (Cost of Delay)

• The sequence in which work is processed is called the queuing discipline

• By changing the queuing discipline we can reduce the cost of a queue without decreasing the size of the queue

• Since manufacturing has homogeneous flows it always uses FIFO (First-In-First-Out)

• For the non-homogeneous flows of product development other approaches have better economics

Use FIFO for Homogeneous Flow

First-In First-Out

Cost of

Delay CostLast-In First-Out

Cost of

Delay 1

Project Duration Cost of Delay

Weighted Shortest Job First (WSJF) for Non-homogenous flow

High Weight First

Cost of

Delay Cost

Low Weight First

Cost of

Project Duration Cost of Delay

Weight = COD/Duration

1 1 10 10

2 3 3 1

3 10 1 0.1

160 796 % Reduction in COD

Paul Ellarby example

1. Create Method for measuring Value

2. Understand the value and cost of each portfolio down to the feature level

3. Allocate Value Points across feature/capabilities

4. Track Value vs Cost for each iteration

https://www.thoughtworks.com/insights/blog/how-do-you-measure-value

How to measure anything – Douglas Hubbard

http://www.howtomeasureanything.com/kburns@sagesw.com, @kevinbburns 27

Measurement Basics

• A measurement is an observation that quantitatively reduces uncertainty. Measurements might not yield precise, certain judgments, but they do reduce your uncertainty.

• A good object of measurement is something that is clearly defined and it’s observable.

• Uncertainty is the lack of certainty: the true outcome/state/value is not known.

• Risk is a state of uncertainty in which some of the possibilities involve a loss.

• Much pessimism about measurement comes from a lack of experience making measurements. Hubbard, who is far more experienced with measurement than his readers, says:• Your problem is not as unique as you think.• You have more data than you think.• You need less data than you think.• An adequate amount of new data is more accessible than you think.

Apply Information Economics for Decision-making

1. Define a decision problem and the relevant variables.

2. Determine what you know.

3. Pick a variable, and compute the value of additional information for that variable.

4. Apply the relevant measurement instrument(s) to the high-information-value variable.

5. Make a decision and act on it.

Selecting a measurement method

To figure out which category of measurement methods are appropriate for a particular case, we must ask several questions:

1. Decomposition: Which parts of the thing are we uncertain about?

2. Secondary research: How has the thing (or its parts) been measured by others?

3. Observation: How do the identified observables lend themselves to measurement?

4. Measure just enough: How much do we need to measure it?

5. Consider the error: How might our observations be misleading?

http://www.howtomeasureanything.com/

Questions & Next Steps

• How many of us know what business/user outcomes and impacts we’re trying to achieve on our projects?

• Do you have metrics in place to evaluation our progress/success outcomes and impacts?

• Who want’s help creating some objective measures?

• Where do we go from here?

How do you know you are delivering value?

Software

You Know What You Write, But Do You Know Your Rights?

“Know from whence you came. If you know whence you came

Know what you don’t know

Things You Should Know - NAVSUP · Things You Should Know. ... Know your Entitlements - weight allowances,

Soybean Physiology: How Well Do You Know Soybeans? · Soybean Station Delivering First Class Information ©2010-11, Purdue University - 1 Soybean Physiology: How Well Do You Know

Saber y conocer To know or to know?. saber sé I know sabemos We know sabes You know sabe He, she, you know saben They, you all know

You can’t know, what you don’t know, that does not make you a dum-dum. – Candee Brakefield 2008 What you don’t know about your benefits – you don’t know!

Do You Know You

Delivering successful retail investment. We know how

Epistemic Logic Kevin: I know. Tom: I know you know. Kevin: I know you know I know

P Opera a You Know More Than You Think You Know

Contents · 2017-08-31 · • Know your audience: plan your communication to suit the listeners’ level of: understanding. • Know your purpose: make it clear whether you are delivering

Delivering application happiness for you!

Delivering online learning: are you ready?

Endless Rewards helps businesses succeed by delivering ... · corner storekeeper, we help you get to know who your customers are so you can meet ... • Establish clear program deliverables

What do you need to know about delivering the IST Courses in … AIEC 2018/AIEC2018... · 2018-10-23 · Bev Jeffreys - Kangan Institute What do you need to know about delivering

You Know Your Client! Does Your Client Know You?

How do you know what you know? How do you know it? What don’t you know? Why don’t you know it?

If You Know JSF, You Know Portals and Portlets

Delivering Business Valued Services - Your IT Support .../media/HDIConf/2016/Files/Speaker-Resources/mcgarahan-shift...Direction / Operational Excellence 1. Know where you are –