Definition of Quality (XP Days Ukraine)

Definition of Quality

Natalya Rukol, Quality Lab.

XP days Ukraine Kiev 11/10/2013 Natalya Rukol Definition of Quality

About speaker• 10+ years in IT• 9 years in quality

management• PM, TM, CEO• She is cool!

Introduction


About speech1. Why to

measure2. What to

measure3. How to

measure

Introduction


Don’t measure!Why measure?


No visible failDon’t measure!


Meet your fateDon’t measure!


Make a real job!Don’t measure!


Kill a weak teamDon’t measure!


Leave it:Don’t measure??

Projects are usually dead without measurements


What to measure?


What to measure?Product quality

• Make clear decisions

• Visualize your goals

Project status

• Control• React in time

Team efficacy

• Better planning

• Improvement strategy


Product qualityWhat to measure

How it looks

What’s inside


External qualityProduct quality

• Customers’ satisfaction• Business

results• Refunds (N &

%)• Support &

Hotline statistics


External qualityProduct quality


ExamplesProduct external quality

Metric 6.5 Release 7.0 Release Status

Customers’ satisfaction 4.4 4.2

Refunds 115 109

Refunds % 0,06% 0,11%

Support time per client, hours 6 8

Support tickets amount per month

4112 3640

Support tickets in % (bugs/questions/praises)

27/61/12 19/79/2 ?


ExamplesProduct external quality

Metric 6.5 Release 7.0 Release Status

Customers’ satisfaction 4.4 4.2

Refunds 115 109

Refunds % 0,06% 0,11%

Support time per client, hours 6 8

Support tickets amount per month

4112 3640

Support tickets in % (bugs/questions/praises)

27/61/12 19/79/2 ?

OK, and what???


How to useProduct external quality

• Track changes while it’s not too late• Investigate

WHY problems?• Ask boss for

increase


Too late…Product external quality

Your software is

shit!


Let’s go insideProduct quality

• Functionality (what?)• Non-functional

requirements (how?)• Usability• Performance• Etc.

• Stability (when?)


Measure insideProduct quality

What How to measure

Functionality (What?)

• Requirements accepted

• Users scenarios execution

Non-functional requirements (How?)

• Load results• Performance results• Usability scenarios pass

ratio

Stability (When?) • Tests pass rate• Builds stability statistics• Bugs


Internal qualityProduct quality


ExamplesProduct internal quality

Metric 7-11/09 14-18/09 20-24/09

Tests pass rate (% OK/Fail/Bugs) 70/12/18 68/13/19 69/12/19

Bugs total / criticals 612 / 28 654 / 19 602 / 24

Bugs average severity 3,7 3,2 3,8

Builds stable / unstable 11 / 0 9 / 4 6 / 7

Features accepted 19/21 17/21 20/22

Usability scenarios passed 9/10 n/a 8/10

Load results 18K 18K 6K


Not relevantProduct internal quality

Your software is still shit!

100% pass rate, no

criticals, blah blah


ConsequencesProduct quality

How it looks

What’s inside

Customers satisfaction

• What do they like?• What they use

mostly?• What they don’t like?• Pain features?

• Critical bugs• Testing pass rate• Usability scenarios• Features accepted

Refunds • Why refunds?• Collect critics

• Critical bugs• Features accepted

Support tickets • What bothers most? • Usability scenarios

1. You have to find your own consequences! 2. First find, then prove!


ConsequencesProduct quality

Internal quality• Design• Code• Testing

External quality• Sales, Refunds• Satisfaction• Support cost


Why external first?Product quality

• CLOKs & bugs are useless w/o customers & sales• No way to

forget priorities• Think big!• Tech depends

on business, not vice versa!


How to measure?


Do it right!How to measure

• Pre-analysis• Introduction• Measurement• Control• Continuous

improvements


Pre-analysisHow to measure

• Collaborate with the whole team• Remember The

Great Goal (TGG)• Find your

strategy together


Soft introductionHow to measure

• Don’t force• Share

responsibility• Find enough

time• Provide with

enough resources • Inside out


DisciplineHow to measure

• Automate all the measurements• Show your

respect• Grow a habit• Stand it for 2

months


PDCAHow to measure

• Plan, Do, Check, Act• Post-Mortems• Accept mistakes


ResultsQuality Definition:

• Clear view• Achievement

s• Team

collaboration• Karma

increase


What’s nextQuality Definition:

Contact with any questions:

http://natalyarukol.ru

http://nrukol.moikrug.ru

http://software-testing.ru

Skype: natalya.rukol

mailto: [email protected]

Additional materials:

http://natalyarukol.ru/definition-of-quality

http://xn--e1atebk/

http://xn--e1atebk/