Inspection Tutorial (Short) - agse.cs.uni-kl.de

Slide 1

Inspection Tutorial© Fraunhofer IESE

Slide 3


Small Bug, Great Disaster - Ariane 5 Flight 501 (1996)

Cause

Malfunction in control software reused from Ariane 4 (arithmetic overflow during number conversion)

Effect

Loss of spacecraft

Cost

>> $1 billion

In safety-critical domains, software defects may lead to great disasters

Inspection Tutorial© Fraunhofer IESESlide 4

Typical problems

Actual project costs exceed planned costs

Actual project schedule exceeds planned schedule

40-50% of the effort is avoidable rework

Results: reduced product quality and lowcustomer satisfaction

Slide 5


Chaos Report 1994-2006

16

27

26

28

34

29

35

31

40

28

23

15

18

19

53

33

46

49

51

53

46

0% 20% 40% 60% 80% 100%

1994

1996

1998

2000

2002

2004

2006

Software projects completed on timeProjects cancelled before completionLate and over budget

Slide 6


Which Quality Assurance (QA) Techniques Exist?

Dynamic QA techniques

Evaluation of a product/system with respect to fulfillment of requirements (specification)

Testing

Static QA techniques

Evaluation of documents that are created within the development process, in order to find defects

Inspections/Reviews

Analysis

Design

Implemen-tation

ModuleTest

IntegrationTest

SystemTest

Contract Software

Slide 7


Typical Quality Assurance Strategy - Testing

Testing is often used as the only quality assurance activity

Executable code is needed

Testing is very expensive (up to 50% of the total costs)

Not all defects can be addressed by testing

Late detection can have destructive impact (erosion of software)

Defects in the early phases of development are hard to find and therefore remain in the product throughout later development phases

High concentration of defects at the end of product development

Chaos Zone

Design Code TestRequirements MaintenanceDefect detection

Defect injectionDesign Code TestRequirements Maintenance

Slide 8


Costs for Late Defect Detection Increase Dramatically

0

10

20

30

40

50

60

70

80

90

100

Analysis Design Implementation Testing Maintenance

Costs for correcting one defect (in 1.000 €)Defects detected (in %)Defects introduced (in %)

Source: SQS AG, analysis of 5000 projects

Slide 9


Solution: Use of Reviews / Inspections

Customized reviews / inspections in the early phases of softwaredevelopment in combination with testing

Goal: to find and remove defects in the phase in which they are injected

Focused use of inspections in order to guarantee the desired/required quality of the documents created within the development process

Design Code TestRequirements Maintenance

Defect injectionDesign Code Test MaintenanceRequirements

Defect detection

Slide 10


Comparison of Inspections and Testing

Can be used in early development stages

Can address different aspects of quality

Usually manual performance

Identification of potential defects

Is usually used in late development stages

Requires an executable system/model

Identification of “wrong behavior” debugging is necessary

Very useful for dynamic aspects

TestingInspections

For effective QA, joint use of inspections and testing is required


Definition of Formal Inspections

Definition

Formal inspections are methods of static analysis, used for evaluating quality attributes of software documents, with the goal of identifying defects

Characteristics

Follow a structured and defined process

Team is composed of professionally qualified people

Use of specific reading techniques for the detection of defects

Results are documented

Formal Inspections are the most formal static quality assurance method; most of the other methods can be derived from this one.


The Basic Process of Inspections

Planning

Preparation Problem-list

Meeting

Correction

SoftwareArtifact

Inspections-package

Correctionlist

Organizer

Inspector

ModeratorInspector

Author

Author

Corrected SoftwareArtifactSoftware

Inspection

Defect-list

Overview

Third-hour

Follow-Up

Data Collection

Role

Activity

Document

Optional

Slide 13


Overview: Static Quality Assurance Methods

Different approaches and definitions exist

Depending on the goals, the context of development, and the quality requirements, different methods can be used

Empirical studies show that formal inspections are the most effective method

Inspections typically allow for defect detection ratios of between 30% and 90%

Mostformal

Leastformal

FormalInspection

TeamReview

Walk-through

Peer Deskcheck,Passaround

Ad-hoc

Slide 14


Ideen der verschiedenen Ansätze

Ad-hoc-Review:

Ziele: Fehlerreduzierung, Einholen einer zweiten Meinung

Ablauf: Eine weitere Person (z.B. Kollege) wird bei einem konkreten (häufig „lokalen“) Problem spontan eingebunden

Vorteile

einfach und überall einsetzbar

geringe Kosten

Nachteil

Ergebnis ist abhängig von der Erfahrung der eingebundenen Person

Slide 15



Peer Deskcheck:

Ziele: Fehlerfindung, konstruktive Hinweise, Mentoring / Coaching

Ablauf: Eine Person begutachtet ein Dokument und kommentiert entsprechend

Hinweise

Für nicht-kritische Systemteile zu empfehlen

Weder Prozess noch weitere Hilfsmittel sind vorgegeben, können aber eingesetzt werden

Vorteile

Geringer Aufwand, einfach einzusetzen

Verminderter Druck für den Autor, da nur ein Inspektor das Dokument inspiziert

Nachteile

Ergebnis abhängig von Wissen, Erfahrung und Disziplin der eingebundenen Person

„Gruppeneffekte“ werden nicht genutzt

Autor nicht anwesend

Anmerkung: Ein Deskcheck als solcher entspricht einem „Selbst-Review“

Slide 16



Passaround:

Ziele: Fehlerfindung, konstruktive Hinweise

Ablauf: Mehrere Personen begutachten ein Dokument und fügen Kommentare ein

Hinweis: Für nicht kritische Systemteile zu empfehlen

Vorteile

Geringer Aufwand, einfach einzusetzen

Schwächen des Deskcheck reduziert (Gefahr von keinem Feedback sowie geringwertigem Feedback)

Überbrückung geographischer Differenzen

Nachteile

Widersprüchliche oder identische Vorschläge

Kein oder verspätetes Feedback möglich

Keine Team-Inspiration

Autor nicht anwesend

Slide 17


Walkthrough: Ziele: Präsentation einer Lösungsidee, gemeinsames Verständnis, Diskussionen zu Lösungsideen, Fehlerfindung

Ablauf: Der Autor eines Dokuments präsentiert es weiteren Person, regt eine entsprechende Diskussion an und sammelt Kommentare der Teilnehmer

Vorteile

Teilnehmer erlangen ein besseres Verständnis des Dokumentes

Teamaktivität (z.B. Diskussion, Feedback der Teilnehmer)

Autor ist aktive Rolle

Nachteile

Kein formaler Prozess, keine Rollenzuweisung, keine Datensammlung

keine Hilfestellung zur Bewertung der Qualität

Risiko: Autor stellt Arbeit extrem positiv dar

Risiko: Verständnislücken bei Teilnehmern werden nicht ausgeräumt und dadurch Fehler „maskiert“

Risiko: Kommentare der Teilnehmer werden nicht umgesetzt


Slide 18


Team-Review:

Ziel: Fehlerfindung, Diskussionen zu Lösungsideen

Ablauf: Verschiedene Kollegen begutachten ein Dokument, um potentielle Fehler zu finden

Hinweise

Sehr ähnlich zur Inspektion

Autor darf Moderatorrolle einnehmen

Mehr Spielraum für inhaltliche / konstruktive Diskussionen

Vorteile

Verständnis zum Dokument wird verbessert

Ergebnisse werden in einem Meeting mit dem Autor konsolidiert

Nachteil

Keine definierte Unterstützung bei der Fehlersuche

Metriken spielen eine untergeordnete Rolle


Slide 19


Reading Techniques – Overview (1/2)

Ad-hoc

No support, experience-based

Checklist-based reading

Checklist-based reading (CBR): “one checklist for all”

Focused checklist: Inspectors receive different checklists containing questions that focus on the needs of different perspectives

Guided checklist: Inspectors receive different checklists containing questions that focus on different quality aspects

Slide 20


Reading Techniques – Overview (2/2)

Scenario-based reading

Perspective-based reading (PBR): The Inspector performs defect detection from one perspective. The Inspector receives a scenario that describes the activities that have to be performed in order to identify defects and which focuses on defects that are important for the respective perspective.

Defect-based reading (DBR): The Inspector receives a scenario that describes which defect classes the Inspector has to focus on during inspection as well as how to perform defect detection.

Usage-based reading (UBR): The Inspector receives a set of Use Cases. These will be executed “manually” during the inspection, in order to check if the design/the code implements the Use Cases.

Others

Stepwise abstraction

Slide 21


Reading Techniques – Coverage and overlap

Better coverage of potential defect sources higher effectiveness

Reduction of detection overlapping higher efficiency

Reduction of the required experience and impact of experience

Artifact with defects

Single Checklist

PBR scenariosFocused checklist

Slide 22


Example of a “tester” scenario for requirements inspection:

Introduction: Imagine you are a tester of the requirements. One of your main tasks is to derive acceptance test cases for the various requirements specified in the functional specification and the related user scenarios. To successfully perform your task, it is most important for you that the test cases are easy to derive from the requirements…

Instructions: For each assigned requirement of the functional specification in the document and for each related functional user scenario generate a test case or set of test cases that allow you to ensure that an implementation of the system satisfies the functional specification (set of acceptance cases)...Document all issues you identify while performing the task in the issue list.

Questions: While following these instructions answer the following questions:• Which parts of the requirements are difficult to understand and hard to derive the test cases from, i.e. where is it unclear what the correct behaviour should be?• Which necessary information is missing to identify the item being tested and the test criteria (e.g. where are valid inputs or outputs missing, are acceptance criteria missing)?

perspective task

quality focus

artifact unit activity

questions

Reading Techniques – PBR Example

Slide 23


Literature

Peer Reviews in Software, K. Wiegers, 2002

Software Inspection, T. Gilb, D. Graham, 1993

http://inspection.iese.de Bibliographie

Documents

Inspection Tutorial (Short) - agse.cs.uni-kl.de