How to deal with requirements quality all along the life-cycle · Requirements Quality Management How to deal with requirements quality all along the life-cycle Martin Schleiermacher

Requirements Quality Management How to deal with requirements quality all

along the life-cycle

REConf, 2016

29.February.2016

José Fuentes (The REUSE Company)

Martin Schleiermacher (XTRONIC)

2 © The REUSE Company, 2016

Content

Impact of low quality requirements

The Systems and Requirements Engineering life-cycles

Requirements authoring

The Requirements Quality Suite

Live demo

Q&A

Requirements Quality Management


The impact of low quality requirements


The impact of low quality

requirements










In some cases:

But clearly, in every case:



Chaos report and some other surveys (e.g. PMI: Pulse of the

profession study):

[“40%-70% of defects in the projects are related to requirements. ”]

Capers Jones:

[“The average rework is over 40%-50%.”]


Project Success Factors % of Responses 1. User Involvement 15.9%

2. Executive Management Support 13.9%

3. Clear Statement of Requirements 13.0%

4. Proper Planning 9.6%

5. Realistic Expectations 8.2%

6. Smaller Project Milestones 7.7%

7. Competent Staff 7.2%

8. Ownership 5.3%

9. Clear Vision & Objectives 2.9%

10. Hard-Working, Focused Staff 2.4%

Other 13.9%

Source:

Chaos Report 2004




50%

29%

21%

30%

52%

18%

20%

22%

58%

Requirements Engineering Capability

Pro

ject P

erf

orm

an

ce

s

Correlation between Project Performances and

Requirement Engineering Capability

Source: Report on SE

Effectiveness Survey NDIA-IEEE-

SEI/CMU-INCOSE Nov 2012



The Systems and Requirements Engineering

life-cycle: where should we put the focus?


Systems and Requirements Engineering life-cycles

Where should we put the focus?:


95%

85%

70%

Time

Cu

mu

lati

ve p

erce

nta

ge

Life

cylc

e C

ost

Operations through Disposal

100% Production

and test

50%

8% Design

15%

20%

Concept

Commited Costs

3-6x

500-1000x

20-100x

Development Source:

INCOSE

Handbook




CONOPS

Stakeholders

Requirements

System

Requirements

System

Design

Equipment

Requirements

Equipment

Design Equipment

Verification

System

Equipment

System Verification

Product

Product Verification

http://www.google.es/url?sa=i&source=images&cd=&ved=0CAgQjRw&url=http://ebgconsulting.com/blog/category/ellen-gottesdiener/&ei=Hik1VcG_EIbJPOSJgaAG&psig=AFQjCNEdp91ciOQP6sbSpD1wMk-n6OcdhQ&ust=1429633694334726

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://www.diyalarms.net/system_design.html&ei=ZSs1VYPAFcnbPI63gNAP&psig=AFQjCNFLB94RGp-VWAc8TdxFV1KcmIhbag&ust=1429634277421348

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://globe-views.com/dreams/aircraft.html&ei=XzI1Ve2DNIH7PNjEgZAO&psig=AFQjCNFgZ_GnAffKepduZEvVNoO4MPTNOQ&ust=1429636063927303

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://en.wikipedia.org/wiki/Aircraft&ei=nzI1VYexFYeqOvapgPgB&psig=AFQjCNHphBW9uKFpY7zGF1lb4lZURvPDjw&ust=1429636127467650




Elicitation Analysis Specification Validation

close gaps clarify

rewrite

re-evaluate

confirm and correct

Source: Karl Wiegers




Elicitation Analysis Specification Verification Validation

Management

close gaps clarify

rewrite

re-evaluate

fix errors

confirm and correct

create baseline

support

stakeholder

input

stakeholder

feedback

Adapted from: Karl Wiegers




CONOPS

Stakeholders

Requirements

System

Requirements

System

Design

Equipment

Requirements

Equipment

Design Equipment

Verification

System

Equipment

System Verification

Product

Product Verification

Requirements

Verification

Requirements

Verification

Requirements

Verification

Requirements

Validation

http://www.google.es/url?sa=i&source=images&cd=&ved=0CAgQjRw&url=http://ebgconsulting.com/blog/category/ellen-gottesdiener/&ei=Hik1VcG_EIbJPOSJgaAG&psig=AFQjCNEdp91ciOQP6sbSpD1wMk-n6OcdhQ&ust=1429633694334726

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://www.diyalarms.net/system_design.html&ei=ZSs1VYPAFcnbPI63gNAP&psig=AFQjCNFLB94RGp-VWAc8TdxFV1KcmIhbag&ust=1429634277421348

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://globe-views.com/dreams/aircraft.html&ei=XzI1Ve2DNIH7PNjEgZAO&psig=AFQjCNFgZ_GnAffKepduZEvVNoO4MPTNOQ&ust=1429636063927303

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://en.wikipedia.org/wiki/Aircraft&ei=nzI1VYexFYeqOvapgPgB&psig=AFQjCNHphBW9uKFpY7zGF1lb4lZURvPDjw&ust=1429636127467650



Requirements Authoring


Requirements Authoring: quality checking

The cost of Requirements errors is very high

Fixing requirements after delivery may cost up to 100 times more than fixing in

the requirements definition stage

Training, best practices and verifying requirements by reviewers can

help to get SMART requirements:

But the process is still costly and time consuming

Introducing quality analysis during the authoring activity:

Reduce the number of iterations between System Engineers and sub-

contractors and leverage the verification activities




Experiences showed that about 25% of system Requirements are critical

and can be grammatically improved

No Shall: 8 to 10%

Forbidden words: 10 to 15%

Subject, multiple objects, design: 15%

Incorrect grammar: 50%

…


Source:

AFIS & Gauthier Fanmuy

RAMP Project



Well-known requirements quality characteristics


IEEE Std. 830:

Correct

Unambiguous

Complete

Consistent

Ranked

Verifiable

Modifiable

Traceable

ESA PSS-05,

ISO/IEC 29148, others:

Pretty much the same characteristics

SMART:

Specific

Measurable

Achievable

Relevant

Traceable

"I believe that this nation should commit

itself to achieving the goal, before this

decade is out, of landing a man on the

Moon and returning him safely to Earth"



Good characteristics to check but…

…can we measure how correct, how complete, how consistent, how

measurable… a specification is??

Are those characteristics really SMART?

Are they specific?

Easy to measure? From a objective point of view?

Is it realistic to ask for those characteristics?




Our approach:

Similar to INCOSE Guide for Writing Requirements

High-level quality characteristics

And Low-level set of metrics addressing these characteristics:

Easy to understand

Easy to measure

Easy even for a tool!!


Characteristic Cxx – Characteristic name

Rationale: xxxx

Strategy: xxxx

Rules that help establish this characteristic:

Rxx - /Section/Rule name

Avoid xxxx

Ryy - /Section/Rule name

Avoid yyy


Requirements Authoring: on-the-fly metrics

Correctness verification:

Metrics based on information coming from the RMS:

Attributes, links, versions…

Metrics based on patterns:

Compliance with different types of requirements patterns

Detection of specific structures within the requirements

Metrics based on the conformance with models:

Concepts in your requirements coming from PBS, FBS…

Metrics based on linguistic algorithms:

Detection of passive voice, imperative tense…

Text length, misspelling….

Metrics based on lists of terms:

Forbidden: ambiguous…

Restricted: negations, pronouns…

Mandatory: ‘shall’



Requirements Authoring: on-the-fly metrics

Consistency verification:

Use of consistent vocabulary

Consistency within a requirements document

Consistency among different levels of requirements

Consistency between requirements and models

Detection of duplicated requirements

…

Completeness verification:

Detection of the current gaps between:

Specifications and models

Different levels of requirements

Current project vs. other previous reference projects

…



Requirements Authoring: ontologies

Ontologies as the driving element for requirements quality

Enable a set of tools to enhance performance and reduce defects


Terminology layer

Conceptual model layer

Patterns layer

Formalization layer

Inference layer


Ontology : Example


A380 A350 System Operate Temperature Environment Pressure Controlled

Vocabulary

A380 A350

<<Aircraft>> “ Greater than (>) “

Operate Work

<<Operation>>

<<Aircraft>> Shall <Operation> <<Minimum>> At Environment Of [MEASUREMENT

UNIT] NUMBER

temperature “ Greater than (>) “

ºC -70

Patterns

Temperature Pressure

Environment

Temperature [-60ºC , +60ºC] “ Operation Range “

Inference

Rules NUMBER “ Lower than (<) “ -60º NUMBER “ Greater than (>) “ +60º ||

Thesaurus

Formalizations The aircraft shall be able to operate

at a minimum temperature of -70º C

If ºC ºC

“ Lower than (<) “

Shall

At a minimum

<<Minimum>>

At a minimum Of


Requirements Authoring: requirements patterns

To provide consistent grammar

To provide consistent vocabulary

To allow completeness and consistency checking

To identify requirements in unstructured sources



Requirements Authoring: semantic search engine

To promote reuse

To detect duplicates

To provide quick access to related requirements



Requirements Authoring: the process


On-the-fly check of quality guidelines

Boilerplate based support

Communication author - V&V team

Centralized feedback repository

Centralized approval

Communication with ontology manager


Requirements V&V: the process

Many other issues involved in this V&V process, still not covered in

the automatic verification during the authoring stage:

Proper allocation

Is the verification method adequate?

Is the specification conforming to standards, regulations…?

However:

Proven reduction of re-work

Reduced number of loops between OEM and contractors

Faster, accurate and objective V&V

Increased overall quality





Introduction to RQS




The Requirements Quality Suite (RQS) intends to tackle requirements quality management by offering a set of tools and processes

Automatic measurement of requirements quality metric

Support to Requirements Authoring

RQS models requirements quality metrics using the CCC approach (Correctness, Consistency and Completeness)

Requirements Quality Analyzer (RQA):

to setup, check and manage the quality of a

requirements specification.

Requirement Authoring Tool (RAT):

to assist authors while they are creating or

editing requirements.

Knowledge Manager (KM):

to manage knowledge around a requirements

specification: the ontology it is based on, the

structure of the requirements to be used in

the project, the communication between

authors and domain architects.


1990

2000

2005

Present


The ages of Requirements Engineering


Requirements traceability

Requirements management

Requirements exchange (RIF, ReqIF)

Core of methods and best practices

Mandatory in the relationship OEM/Supplier

First professional tools

First methods

Semantic approaches

Requirements Quality Analysis

Requirements Authoring

Requirements Reuse

Requirements Product Lines

Requirements Interoperability


The Requirements Quality Suite: connectors


IBM DOORS ©

PTC Integrity ©

CATIA Reqtify ©

VISURE Requirements ©

OSLC

Microsoft Excel ©

XML file

Near future:

Microsoft Word ©

Siemens Teamcenter ©

https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=QsDpGlkai9R4FM&tbnid=ra3RwYKyt1hIhM:&ved=0CAUQjRw&url=https://help.rallydev.com/ibm-doors&ei=OMPhU8L-Jais0QX-iIG4Dg&bvm=bv.72197243,d.d2k&psig=AFQjCNHbwoatil1oOCZbKzYlsM5nrV1LDA&ust=1407390896885195


The Requirements Quality Suite: natural language


RQS is highly dependent of the language of the requirements

Languages supported so far:

Other languages on the way:



Live demo.

But if you want more?

Visit us: Booth PA2


Q&A


Thank you!!!


http://www.reusecompany.com

@ReuseCompany

[email protected]

Margarita Salas, 16 2nd Floor

Innovation Center

LEGATEC Technology Park

28919 Leganés – Madrid

SPAIN – EU

Tel: (+34) 912 17 25 96

Fax: (+34) 916 80 98 26

Herrenberger Straße 56

71034 Böblingen

http://www.xtronic.de

Tel: +49 7031 20947 0

Fax: +49 7031 20947 101

[email protected]

mailto:[email protected]

Documents

How to deal with requirements quality all along the life-cycle · Requirements Quality Management How to deal with requirements quality all along the life-cycle Martin Schleiermacher