!!!!!!
MEASURES OF EFFECTIVENESS KPIS & METRICS FOR EFFECTIVE
SOFTWARE DEVELOPMENT
This work is licensed under the Creative Commons
Attribution-NonCommercial-ShareAlike 3.0 Unported License. To view
a copy of this license, visit
http://creativecommons.org/licenses/by-nc-sa/3.0/.
PRINCIPLES • NEVER measure individuals, always
measure teams. • Focus on trends, not individual figures.
• Use Kanban practices and kanban system policies for remedial
work.
FEEDBACK LOOPS • Daily Stand-ups • Weekly Report • Data Driven
Retrospective • Ops Review • Exec Monthly Meeting • Customer
Feedback • Web Analytics & Insight
!A team running a data driven retrospective
QUESTIONS FOR TECH LEADS • How do you define tech quality? • How
do you measure tech quality? • How do you communicate tech quality
findings?
• How do you prioritise remedial work? QUESTIONS FOR DELIVERY
LEADS • How do you define team efficiency? • Do your current tools
allow you to
measure team efficiency? • What are your sources of cycle time
variation?
• How do you track/measure/manage activity in the trade-off
zones?
• How do you ensure a fair balance of tech debt, new features,
incidents?
QUESTIONS FOR COMMERCIAL LEADS • How do you know if you're
developing the right thing? • What commercial metrics are
you
measured against? • What feedback loops do you have
in place to validate if you're developing / developed the right
thing?
• How quickly can you go from
ARE WE BUILDING IT FAST ENOUGH? • Cycle Time – the measure of
how long it takes
for a card to get from the left side of the card wall to the
right. Goal: to reduce variation in cycle time. This is a useful
diagnostic metric for the team. A control chart is used to
visualise Cycle Time.
• Dwell Time – Measure and track waiting time for work in
queues.
• Release rate – aka team throughput. Measures the number of
releases per week/month. Plot on a control chart.
• Cumulative Flow – a useful visualisation to spot
bottlenecks.
UK.LINKEDIN.COM/IN/SOLUTIONEER
@IANCARROLLUK
07968 399947
[email protected]
WWW.IANCARROLL.COM
What problem are you trying to solve? …is the first question you
should ask when implementing metrics. If you attempt to roll out
all of the metrics in this paper for no apparent reason then you
will certainly fail. Some of the metrics are useful as diagnostic
tools to give you insight into problems you didn’t realise you had.
Others are metrics to solve certain systemic problems or specific
problems commonly faced by software development teams. Therefore,
this guide does not attempt to prescribe any or all of the metrics
presented but rather seeks to provoke thought through presenting
some metrics the author has found useful when working with teams.
Some of the metrics are pretty common in the industry already. For
the rest feel free to ‘pick n mix’ to solve specific team
problems.
Are we building the right thing?
Are we building it right?
Are we building it fast enough?
ROI
Quality
Efficiency
Effectiveness
Trade-off zones
ARE WE BUILDING THE RIGHT THING? This circle is more generic
than the other two as all businesses are different. However, all
benefit distils down to three specific themes. Explore how you can
measure and track benefit by using these three themes as a starting
point. • Make money? • Save money? • Protect money? • Investment
Balance This may prove to be the most challenging circle from the
three circles of effectiveness to quantify and measure, but
ultimately it’s the most important. ARE WE BUILDING IT RIGHT?
• Internal Quality
• Test Coverage • Duplication • Coding Rules • Complexity
There are a number of static code analysis tools available on
the market enabling you to automate the collection and analysis of
internal quality metrics. • External Quality
• Live defects – sum per week / month • Incidents – meantime
between • Page load times – and other Non-funcs • Net Promoter
Score
External quality is an inferred measure of user experience and
can be measured by a number of different tools on the market.
TRADE-OFF ZONES These are the zones where I find most teams
spend their lives. These are the zones that demoralise teams, strip
companies of profits, and yet could be the zones offering maximum
growth for companies. • Technical Debt • Deferred defects •
Deferred features • Deferred incidents Trade-off decisions come in
many forms and for many reasons but the main point here is to find
a way to capture these decisions in a controlled fashion and make
them highly visible in the backlog. Combining this with Kanban’s
classes of service you can enable a system of balance across the
three circles.
2 1 5
3 4
1
2
3
4
5
6
7
8
9
10
11
12
6
7 8
9 10
11 12
13 14
15
13
14
15
17
18
19
20
16
17 19
18
20
16
