View
16
Download
1
Category
Preview:
Citation preview
Agile Metrics - Lessons Learned from my FitBit
Goodhart’s Law
When a measure becomes a target, it ceases to be a good measure.
Goodhart’s Law
The Danger of Targets and Incentives
• They kill intrinsic motivation
• Achievement of the target starts to supersede the overarching goal
• Can lead to gaming of the system
https://www.youtube.com/watch?v=u6XAPnuFjJc
The Hawthorn Effect
The alteration of behavior by the subjects of a study due to their awareness of being observed.
That which is measured, will improve… but at what cost?
Metrics - Good or Bad?
What is velocity?
Velocity – Measure of a Complex System
The Human Body – A Complex System
Measuring your Weight … a Lagging Indicator
Lean – Optimize the Whole
Focus on the Baton
Percent Utilization
Why should we measure things?
Reduce uncertainty Manage our work Provide transparency Answer questions from stakeholders
Metrics are not for management…
They are for the people doing the work!
Management sets the context and strategic objectives
Teams should decide how they will measure themselves, in alignment with strategic objectives.
Our Strategic Objective
Increase Predictability of Delivery
1. Self-organize into Teams 2. Pick a Team Name 3. Identify three metrics your team will use to align with our Strategic Objective
10 Minutes
Activity
Metric Ideas
• Velocity
• Throughput
• Team Member Happiness
• Team Backlog Health
• Planned / Accepted ratio
• Cycle Time
• Queue Size
• Team Turnover
• Scope Change Rate
• …
Activity
Activity
Scaling Metrics
Scoreboard Index
1. Define a set of Qualitative Metrics aligned to Objectives
2. Define a set of Quantitative Metrics aligned to Objectives
3. For each metric, define a lower and upper bound • Lower Bound = Unhappy
• Upper Bound = Very Happy
4. Collect measurements on a consistent, frequent basis
5. Express every measurement as a point between 0 and 100
6. Calculate an index of the metrics
A Basic Scoreboard Index Example
Team Apple Team Orange Team Pear
Kanban Scrum Scrum
Our Strategic Objective
Increase Predictability of Delivery
Metric Team Apple Team Orange Team Pear
Throughput
Cycle Time
Velocity
Scope Change Rate
Planned vs. Accepted
Team member Turnover
Lower Upper Iteration 1 Iteration 2 Iteration 3
Throughput
Cycle Time
Turnover
0
10
2
12
2
0
10
0
4.5
83.3%
68.8%
100%
84.0%
5
7.2
1
71.3% 35.0%
50.0%
42.2%
11
4.3
0
91.7% 41.7%
100%
87.6%
= (Value – Lower) / (Upper – Lower)
= (10 – 0) / (12 – 0)
Lower Upper Iteration 1 Iteration 2 Iteration 3
Cycle Time
Velocity
Scope Change
10
0
100%
2
16
0.0%
8.8
33%
8
15.0%
50.0%
67.0%
44.0%
9
7
40%
87.5% 43.8%
60.0%
38.8%
4.8
14
10%
65.0% 12.5%
90.0%
80.8%
Lower Upper Iteration 1 Iteration 2 Iteration 3
Throughput
Velocity
Planned/Accepted
0
0
0%
9
22
100%
8
89%
20
88.9%
90.9%
89.0%
89.6%
7
14
75%
100% 63.6%
75.0%
72.1%
10
22
100%
111% 77.8%
100%
103.7%
+ + Iteration 1 Iteration 2 Iteration 3
Team Apple 80.0% 42.9% 86.3%
Team Orange 44.0% 38.8% 80.8%
Team Pear 89.6% 72.1% 103.7%
71.2% 51.3% 90.3%
It’s not about the numbers…
There is a story behind every number Focus on the conversations Rely on a Data Informed
approach not a Data Driven approach. Focus on the trends over time
References
• Doc Norton – Agile Metrics – Velocity is NOT the Goal • https://vimeo.com/97505655
• http://www.intelliberg.com/wp-content/uploads/2013/07/7DeadlySins_ofAgileMeasurement.pdf
Use both Leading and Lagging Indicators
Leading
• Team Member Happiness
• Queue Size
• Team Backlog Health
• Team Turnover
Lagging
• Lead Time
• Cycle Time
• Velocity
• Throughput
Use Both Qualitative & Quantitative Metrics
Qualitative
• Interviews
• Focus Groups
Quantitative
• Velocity
• Cycle Time
• Throughput
• Sprint Burn Up/Down
• Cumulative Flow Diagram
• Measuring percent utilization
• Measuring individual team members
• Measuring Developers based on lines of code
• Measuring based on defects logged
Avoid…
• Going after hard to collect data
• Relying on low quality data
• Relying on only one metric
• Tracking too many metrics
• Applying targets and/or incentives to metrics
Avoid…
http://www.intelliberg.com/wp-content/uploads/2013/07/7DeadlySins_ofAgileMeasurement.pdf
Recommended