GQM – an example from Fjellanger - Widerøe

Hans J. LiedTor StålhaneIDI / NTNU

Environment• Medium sized company making tailor-made software

systems• Fixed price projects with little room for rework

caused by introduction of errors• Two development projects, same project team and

process model. – Project 1: standard with a given requirements specification– Project 2: prototyping without requirements specification

The projects • Project 1 had

– structure given from the start– customer-defined quality assurance requirements – requirements specification from the customer

• Project 2 – depended on prototyping.– started with an idea from the customer– the requirements specification was a co-

production between customer and developers.

Methods

We used a combination of three methods:• GQM, to decide what and how to

measure• Pareto with a robustness analysis, to

analyse and assess collected data• RCA: brainstorming and Ishikawa

diagrams to identify root causes

GQM method – 1

Analyse for the purpose of

with respect to as seen from in the context of

Focus Q1:Q2:

EnvironmentQa:Qb:Qc:

What do we believeQ1:Q2:

How does the environment influence focusIncreased Qa values will reduce Q1Low Qb values will increase Q2

Comments Interesting info that surfaces during the GQM process but is not directly relevant for the current GQM goal.

GQM abstraction sheet without the lower part

GQM method – 2 The GQM abstraction sheet was used to:• manage and structure the brainstorming

sessions• identify problem areas for the RCA• benefit from its focus on developer

participation to get a set of basic root causes to analyse

GQM method – 3 The GQM method helps us to obtain:• reliable data. The developers are more

interested in obtaining correct data if they feel that they have ownership to the goal and purpose of the data collection

• more interest in the results. Involving the developers from day one => – create interest– keep the improvement program alive.

GQM - process

• Two hour workshop to introduce the method• Establishing the GQM goal:

– Analyse the reported failures in order to understand causes for failures seen from the developers in the X project.

• Produced a GQM abstraction sheet, which was converted to a data collection form

GQM - Data collection formWPnr.: One report pr. failure

Please specify: Data collection form for failure causes in project 1/ project 2

Focus Variation factors

Main cause: Cross: Secondary cause: Cross:

1 Incorrect use of cut-and-paste from old code 1 Time pressure during development2 Incorrect reuse of old code 2 Missing competence in use of3 Incorrect use of language features : 2,1 -language

3,1 -pointers 2,2 -development tools3,2 -char '0' terminating s trings og var. 3 Uncontrollable, external dis turbances - 3,3 -other 3,1 -noise

4 Incorrect use of library components 3,2 -fire fighting old systems5 Incorrect attempts to make the code general 3,3 -support on other products6 Attempted reuse by introducing global variables 4 Errors in development tools7 High component complexity 5 Errors in libraries and subsystems supplied by subcontractors

7,1 -module 6 Lack of user interaction7,2 -application 7 Lacking or miss ing motivation7,3 -design 7,1 -salary

8 Large, unanticipated variation in input data 7,2 -time off9 Incomplete or bad tes t data 8 Incomplete project model, for ins tance too few check points

10 Too large code components11 Missing or incomplete configuration management Failure consequence:12 Incorrect component integration13 Insufficintly detailed requirements specification14 Wrong code logic15 Errors in hardware - software communication

Degree of seriousness: ( 1 is not serious, 5 is very serious )1 2 3 4 5

Amount of person-hours spent correcting the failure:

Pareto – 1

• The 20/80 distribution, known as Pareto’s Law, is applicable in software development

• Pareto diagrams used to visualise the main problem areas

• We used Pareto analysis on our data to identify main areas of improvement

Pareto – 2 Rework

0

5

10

15

20

25

30

3 4 14 7 2 1 6 13

Focus cause

Pe

rso

n h

ou

rs

A

Rework

0

2

4

6

8

10

12

14

16

13 15 14 1

Focus cause

Pe

rso

n h

ou

rs

A

Pareto identified the following main problem areas:

Project1 – standard project:• Incorrect use of language features (3)• Incorrect use of library components (4)• Wrong code logic (14)

Project2 – prototyping :• Incomplete or insufficiently detailed req. spec. (13)• Errors in HW-SW communication (15)• Wrong code logic (14)

RCA - Brainstorm

We presented the development team with theresults from the Pareto analysis and introducedthem to RCA.

We made an Ishikawa diagram, using the categories from the data collection form as a starting point.

The strong point of a brain-storming session is that it maximises the use of available knowledge

The RCA diagram

Incompleterequirements

Incompleteenvironmentdescription

Incomplete QAactivities

Incomplete functionalrequirements

Too complexdescription

Incompletenon-functionalrequirements

Othermisunderstandings

Incompleteunderstanding ofcustomer needs

Insufficientcustomer contact

Missingroutines

Time pressure

Routines notfollowed

Lack ofknowledge

Lack ofoverview

RCA - Results

Improvement actions identified from this session:• The need for sharing competence internally• Checklist on Intranet• Common routines/solutions for standard tasks

(best practice experience)• Common dynamic document templates• Start the planning, design and development of an

experience database

Conclusions

• Measurements gave knowledge of how things really are in the company

• Common understanding of what we learned from this knowledge

• Explicit shared knowledge about the process that the company can reuse