Introduction to Verification and Test of Embedded Systems ... · Formal speciﬁcation and veriﬁcation of embedded hardware (lecture + exercise) 3/53 Ulrich Kühne 26/11/2018. Plan

Introduction to Verificationand Test of EmbeddedSystemsSE767: Vérification & Test

Ulrich Kü[email protected]/11/2018

Objectives of this Course

Understanding the role of test & verification in thedevelopment processApplying Test-Driven Design to embedded softwareGetting in touch with formal methodsUnderstanding and writing a formal specification for ahardware module

2/53 Ulrich Kühne 26/11/2018

Course Structure

1. Introduction(this lecture)

2. Test-Driven Design of embedded software(lecture + exercise)

3. Embedded systems modeling(video lecture + exercise)

4. Introduction to formal methods(lecture + exercise)

5. Formal specification and verification of embeddedhardware(lecture + exercise)

3/53 Ulrich Kühne 26/11/2018

Plan

Motivation

Basic Validation MethodologyObjectivesDevelopment Cycle

Test CoverageSoftware Coverage MetricsTesting Requirements in the Railway DomainCoverage with gcc and lcov

Testing Embedded Systems

Hardware Verification & Test

4/53 Ulrich Kühne 26/11/2018

Why do we need to test and verify?

5/53 Ulrich Kühne 26/11/2018

Ariane 5

Ariane 5 rocket explodesshortly after liftoff (1996)Integer overflow causingexceptionLegacy code (Ariane 4)used in flight control

6/53 Ulrich Kühne 26/11/2018

Mars Pathfinder

Rover gets stuck on Mars(1997)Scheduling problem leadsto permanent restartsBug fixed on identical copyon earth, live update overthe air

7/53 Ulrich Kühne 26/11/2018

Therac-25

Therac-25 radiationtherapy machine (1982)Software bug leads to raceconditionPatients exposed to lethalradiation dose

8/53 Ulrich Kühne 26/11/2018

Zune 30

Zune-30 music playersbricked during December31, 2008Infinite loop in time servicein loop (!) yearsSeveral New Year’s Eveparties without music. . .

9/53 Ulrich Kühne 26/11/2018

Plan

Motivation





10/53 Ulrich Kühne 26/11/2018

Goals of Testing & Verification

1. Find and eliminate bugs2. Improve design quality3. Reduce risk of user4. Reduce risk of enterprise5. Fulfill certification requirements6. Improve performance7. . . . ?

11/53 Ulrich Kühne 26/11/2018

Test & Verification Techniques

Plug & prayNon-regression testsUnit testingTest-driven designModel-based designFormal methodsMathematical proofs

12/53 Ulrich Kühne 26/11/2018



Easy

Hard

12/53 Ulrich Kühne 26/11/2018



Easy

Hard

Sloppy

Complete

12/53 Ulrich Kühne 26/11/2018

Validation in the Development Cycle

When should validation take place??

As early as possibleAs often as necessary

13/53 Ulrich Kühne 26/11/2018

Validation in the Development Cycle

When should validation take place??

As early as possibleAs often as necessary

13/53 Ulrich Kühne 26/11/2018

The V-Model

Requirementsanalysis

Softwarespecification

Softwarearchitecture

Moduledesign

Implemen-tation

Unittests

Moduleintegration

Systemintegration

Acceptancetests

Deployment andmaintenance

14/53 Ulrich Kühne 26/11/2018

The V-Model

Requirementsanalysis

Softwarespecification

Softwarearchitecture

Moduledesign

Implemen-tation

Unittests

Moduleintegration

Systemintegration

Acceptancetests

Deployment andmaintenance

14/53 Ulrich Kühne 26/11/2018

Agile Design

Continuous integrationBreak long validation cyclesOne feature (functionality) at a time

Featuredefinition

Implemen-tation

Vali-dation

Featuredefinition

Implemen-tation

Vali-dation

. . .

15/53 Ulrich Kühne 26/11/2018

Agile Design


Featuredefinition

Implemen-tation

Vali-dation

Featuredefinition

Implemen-tation

Vali-dation

. . .

15/53 Ulrich Kühne 26/11/2018

Agile Design


Featuredefinition

Implemen-tation

Vali-dation

Featuredefinition

Implemen-tation

Vali-dation

. . .

15/53 Ulrich Kühne 26/11/2018

What to Test?

How to define our test cases??

Depends on abstraction level:Module tests, integration tests, acceptance tests, . . .Depends on test strategy:Functional vs. structural testingDepends on test objectives:Bug hunting, coverage, performance, stress testing, . . .Depends on context:Certification and safety norms (e.g. EN 50128)

16/53 Ulrich Kühne 26/11/2018

What to Test?

How to define our test cases??

Depends on abstraction level:Module tests, integration tests, acceptance tests, . . .Depends on test strategy:Functional vs. structural testingDepends on test objectives:Bug hunting, coverage, performance, stress testing, . . .Depends on context:Certification and safety norms (e.g. EN 50128)

16/53 Ulrich Kühne 26/11/2018

Functional vs. Structural Testing

Functional Testing

Black box testingDriven by specificationGoal: Cover all specifiedfunctionality

Structural Testing

White box testingDriven by code structureGoal: Achieve structuralcoverage metrics

17/53 Ulrich Kühne 26/11/2018

Success Criteria

When do we stop testing??

Boss says to stop(time budget)Bug rate belowthresholdCoverage abovethreshold

bug

rate

time

cove

rage

time

18/53 Ulrich Kühne 26/11/2018

Success Criteria

When do we stop testing??

Boss says to stop(time budget)Bug rate belowthresholdCoverage abovethreshold

bug

rate

time

cove

rage

time

18/53 Ulrich Kühne 26/11/2018

Plan

Motivation





19/53 Ulrich Kühne 26/11/2018

Test Coverage

DUT

Input Coverage

Full coverageBoundary valuesEquivalence classes

Code Coverage

Statement coverageBranch coveragePath coverage

20/53 Ulrich Kühne 26/11/2018

Test Coverage

DUT

Testbench

inpu

tstim

uli

outputchecks

Input Coverage


Code Coverage


20/53 Ulrich Kühne 26/11/2018

Test Coverage

DUT

Testbench

inpu

tstim

uli

outputchecks

Input Coverage


Code Coverage


20/53 Ulrich Kühne 26/11/2018

Test Coverage

DUT

Testbench

inpu

tstim

uli

outputchecks

Input Coverage


Code Coverage


20/53 Ulrich Kühne 26/11/2018

Input Coverage: Equivalence Partitioning

Black box approachCover all interesting cases ofinput stimuliDetermine equivalence classesthat should entail the samebehaviorSelect one test case for eachclass

Are the equivalence classes consistent with the code??

21/53 Ulrich Kühne 26/11/2018




21/53 Ulrich Kühne 26/11/2018




21/53 Ulrich Kühne 26/11/2018




21/53 Ulrich Kühne 26/11/2018

Boundary Conditions

Test values close to and onboundariesReveals bugs due to off-by-onemistakesTest negative (i.e. failing) results

22/53 Ulrich Kühne 26/11/2018

Boundary Conditions


22/53 Ulrich Kühne 26/11/2018

Boundary Conditions


22/53 Ulrich Kühne 26/11/2018

Code Coverage

Statement coverageBranch coverage

• Decision coverage• Condition coverage• Condition/decision coverage• Modified condition/decision coverage (MC/DC)• Compound condition coverage

Path coverage

100% path cov. ⇒ 100% decision cov. ⇒ 100% statement cov.

23/53 Ulrich Kühne 26/11/2018

Control Flow Graph

01: int square(int x){

02: if (x <= 0)

03: x = -x;

04: int s = 0, a = 1;

05: for (int i=0; i<x; ++i){

06: s += a;

07: a += 2;

08: }

09: return s;

10: }

2

3

4

5

67

9

24/53 Ulrich Kühne 26/11/2018

Control Flow Graph


02: if (x <= 0)

03: x = -x;

04: int s = 0, a = 1;

05: for (int i=0; i<x; ++i){

06: s += a;

07: a += 2;

08: }

09: return s;

10: }

2

3

4

5

67

9

24/53 Ulrich Kühne 26/11/2018

Control Flow Graph


02: if (x <= 0)

03: x = -x;

04: int s = 0, a = 1;

05: for (int i=0; i<x; ++i){

06: s += a;

07: a += 2;

08: }

09: return s;

10: }

2

3

4

5

67

9

24/53 Ulrich Kühne 26/11/2018

Branch Coverage

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Decision coverage Every decision has taken all possible values atleast once.

Condition coverage Every condition has taken all possible values atleast once.

Condition/decision coverage Combination of decision and conditioncoverage.

Modified condition/decision coverage Condition/decision coverageplus every condition has been shown to independentlyaffect the decision’s outcome.

25/53 Ulrich Kühne 26/11/2018

Branch Coverage

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Decision





25/53 Ulrich Kühne 26/11/2018

Branch Coverage

1: x = 0;

2: if (a && (b || c))

3: x = 1;

DecisionConditions





25/53 Ulrich Kühne 26/11/2018

Branch Coverage

1: x = 0;

2: if (a && (b || c))

3: x = 1;

DecisionConditions





25/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc}{abc, abc}{abc, abc}{abc, abc}{abc, abc, abc, abc}

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc}{abc, abc}{abc, abc}{abc, abc, abc, abc}

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc} yes yes no no no{abc, abc}{abc, abc}{abc, abc, abc, abc}

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc} yes yes no no no{abc, abc} no no yes no no{abc, abc}{abc, abc, abc, abc}

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc} yes yes no no no{abc, abc} no no yes no no{abc, abc} yes yes yes yes no{abc, abc, abc, abc}

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc} yes yes no no no{abc, abc} no no yes no no{abc, abc} yes yes yes yes no{abc, abc, abc, abc} yes yes yes yes yes

26/53 Ulrich Kühne 26/11/2018

Let’s Cover. . .

1: x = 0;

2: if (a && (b || c))

3: x = 1;

Tests Statement Decision Condition C/D MC/DC{abc} yes no no no no{abc, abc} yes yes no no no{abc, abc} no no yes no no{abc, abc} yes yes yes yes no{abc, abc, abc, abc} yes yes yes yes yes

26/53 Ulrich Kühne 26/11/2018

Path Coverage

Every execution path of theprogram has beenexplored

Execute loops 0,1, morethan 1 time (loop coverage)Exponential number ofpaths in generalInfeasible paths

2

3

4

5

67

9

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

×1

27/53 Ulrich Kühne 26/11/2018

Path Coverage



2

3

4

5

67

9

×2

27/53 Ulrich Kühne 26/11/2018

Path Coverage

Every execution path of theprogram has beenexploredExecute loops 0,1, morethan 1 time (loop coverage)

Exponential number ofpaths in generalInfeasible paths

2

3

4

5

67

9

×n

27/53 Ulrich Kühne 26/11/2018

Path Coverage

Every execution path of theprogram has beenexploredExecute loops 0,1, morethan 1 time (loop coverage)Exponential number ofpaths in generalInfeasible paths

2

3

4

5

67

9

×n

27/53 Ulrich Kühne 26/11/2018

Example: Railway Systems

Safety-critical embedded systemsStrict regulation by European and national agenciesEuropean Train Control System (ETCS)

28/53 Ulrich Kühne 26/11/2018

European Norm CENELEC EN 50128

Development & validationprocess for railwaysystemsSafety integrity levels SIL0up to SIL4Organizational structureDevelopment cycle (Vmodel)Validation activities andreports for each projectphase

29/53 Ulrich Kühne 26/11/2018

Extracts from EN 50128

Table A.18 – Performance Testing

TECHNIQUE/MEASURE Ref SIL 0 SIL 1 SIL 2 SIL 3 SIL 4

1. Avalanche/Stress Testing D.3 - R R HR HR

2. Response Timing and Memory Constraints D.45 - HR HR HR HR

3. Performance Requirements D.40 - HR HR HR HR

Table A.19 – Static Analysis

TECHNIQUE/MEASURE Ref SIL 0 SIL 1 SIL 2 SIL 3 SIL 4

1. Boundary Value Analysis D.4 - R R HR HR

2. Checklists D.7 - R R R R

3. Control Flow Analysis D.8 - HR HR HR HR

4. Data Flow Analysis D.10 - HR HR HR HR

5. Error Guessing D.20 - R R R R

6. Walkthroughs/Design Reviews D.56 HR HR HR HR HR

30/53 Ulrich Kühne 26/11/2018

Extracts from EN 50128Table A.21 – Test Coverage for Code

Test coverage criterion Ref SIL 0 SIL 1 SIL 2 SIL 3 SIL 4

1. Statement D.50 R HR HR HR HR

2. Branch D.50 - R R HR HR

3. Compound Condition D.50 - R R HR HR

4. Data flow D.50 - R R HR HR

5. Path D.50 - R R HR HR

Requirements:

1) For every SIL, a quantified measure of coverage shall be developed for the test undertaken. This can support the judgment on the confidence gained in testing and the necessity for additional techniques.

2) For SIL 3 or 4 test coverage at component level should be measured according to the following:

- 2 and 3; or

- 2 and 4; or

- 5

or test coverage at integration level should be measured according to one or more of 2, 3, 4 or 5.

3) Other test coverage criteria can be used, given that this can be justified. These criteria depend on the software architecture (see Table A.3) and the programming language (see Table A.15 and Table A.16).

4) Any code which it is not practicable to test shall be demonstrated to be correct using a suitable technique, e.g. static analysis from Table A.19.

31/53 Ulrich Kühne 26/11/2018

Practical Code Coverage with gcc and gcov

gcc has a magic option --coverage

Instrumentation of binary codeCount execution of each basic blockCount branches taken/untakenGenerate coverage report using gcov (or lcov)Integration into build system

32/53 Ulrich Kühne 26/11/2018

Coverage Tool Flow with gcov

main.c gcc

a.out

main.gcno

main.gcda

gcov main.c.gcov

Let’s do it. . .

33/53 Ulrich Kühne 26/11/2018

Coverage Tool Flow with gcov

main.c gcc

a.out

main.gcno

main.gcda

gcov main.c.gcov

Let’s do it. . .

33/53 Ulrich Kühne 26/11/2018

Test Coverage (Again)

DUT

Testbench

inpu

tstim

uli

outputchecks

Input coverage Code coverage

34/53 Ulrich Kühne 26/11/2018


DUT

Testbench

inpu

tstim

uli

outputchecks

Input coverage Code coverage

What am I missing here. . . ??

34/53 Ulrich Kühne 26/11/2018


DUT

Testbench

inpu

tstim

uli

outputchecks

Input coverage Code coverage Output coverage?

What am I missing here. . . ??

34/53 Ulrich Kühne 26/11/2018

Mutation Coverage

Did we check the right things at the output??

How to assess test bench quality??

Mutation coverage(aka error seeding)Randomly insert errors into thecode (mutants)Check if the test bench captures(kills) themCompute ratio of killed mutants

35/53 Ulrich Kühne 26/11/2018

Mutation Coverage




35/53 Ulrich Kühne 26/11/2018

Mutation Coverage




35/53 Ulrich Kühne 26/11/2018

Mutation Coverage: Rationale

Mutations should mimic typical mistakes• Loop condition off by one• Replace operators such as < vs ≤• Modify constants• . . .

A test bench not detecting these mistakes should beimprovedMutation coverage approximates ratio of real bugs found

Number of mutants killedTotal number of mutants

≈ Number of real bugs foundTotal number of real bugs

36/53 Ulrich Kühne 26/11/2018

Mutation Coverage: Rationale

Mutations should mimic typical mistakes• Loop condition off by one• Replace operators such as < vs ≤• Modify constants• . . .

A test bench not detecting these mistakes should beimprovedMutation coverage approximates ratio of real bugs found

Number of mutants killedTotal number of mutants

≈ Number of real bugs foundTotal number of real bugs

36/53 Ulrich Kühne 26/11/2018

Summary Coverage

Coverage metrics measure test qualityWidely used in embedded industryStrong requirements for railway, aerospace, andautomobile domainsSimple coverage with gcc and gcov

Comes at virtually no cost ⇒ Use it!

37/53 Ulrich Kühne 26/11/2018

Plan

Motivation





38/53 Ulrich Kühne 26/11/2018

Testing Embedded Software

What’s so special about embedded software testing??

Runs on dedicated (expensive,scarce, buggy, . . . ) hardwareHardware not available (yet)Limited memoryLimited debug capabilitiesReal-timeComplex interactions withphysical worldLong build and upload times

39/53 Ulrich Kühne 26/11/2018

Testing Embedded Software

What’s so special about embedded software testing??

Runs on dedicated (expensive,scarce, buggy, . . . ) hardwareHardware not available (yet)Limited memoryLimited debug capabilitiesReal-timeComplex interactions withphysical worldLong build and upload times

39/53 Ulrich Kühne 26/11/2018

Embedded Testing Techniques

Testing on target hardwareHigh confidence in test resultsLong test cycles, hardware might not be available (yet)

Emulating the target hardware (FPGA)Test results close to the real targetDifficult to set up, HDL sources needed

Testing on a virtual platform (SystemC, qemu, . . . )High performance, no dedicated hardwareHigh development effort

Purely C-based shallow test harnessShort test cycles, easy to set upDifficult for real-time systems

SE743

SE744

SE747

Here!

40/53 Ulrich Kühne 26/11/2018






SE743

SE744

SE747

Here!

40/53 Ulrich Kühne 26/11/2018






SE743

SE744

SE747

Here!

40/53 Ulrich Kühne 26/11/2018






SE743

SE744

SE747

Here!

40/53 Ulrich Kühne 26/11/2018






SE743

SE744

SE747

Here!

40/53 Ulrich Kühne 26/11/2018

Alternative: Model-based Testing

Developmentand Test Model

Test EngineIntegrated HW/SW System

TestProcedures

SystemCode

Generatedfrom model

Generated ormanually

developed

HW/SW IntegrationTests

41/53 Ulrich Kühne 26/11/2018

Alternative: Model-based Testing

DevelopmentModel

TestModel

Test EngineIntegrated HW/SW System

TestProcedures

SystemCode

Generatedfrom model

Generated ormanually

developed

HW/SW IntegrationTests

41/53 Ulrich Kühne 26/11/2018

Principles of Model-Based Testing

Use of well-founded models(e.g. SysML state charts)Models serve for documentationand reviewEnables testing duringdevelopmentAutomated generation of testcasesAutomated requirementstraceability

Requirements

Test Model

TestProcedures

Test Engine

TestPlan

42/53 Ulrich Kühne 26/11/2018

Example from Railway DomainCSM_ON

CSM_ON

OVERSPEED

/entry OpaqueBehavior

SpeedSupervisionStatus = OverspeedStatus;

currentSpeed = SimulatedTrainSpeed;

/do OpaqueBehavior

permittedSpeed= V_mrsp;

displayPermittedSpeed=true;

WARNING


SpeedSupervisionStatus =

WarningStatus;


/do OpaqueBehavior permittedSpeed

SERVICE_BRAKE



InterventionStatus;EmergencyBrakeC

ommand= !

sbCmd;ServiceBrakeCommand=

sbCmd;

EMER_BRAKE



InterventionStatus;EmergencyBrakeCommand= 1;ServiceBrakeCommand= 0;


CSM_INIT

NORMAL/entry OpaqueBehavior

SpeedSupervisionStatus = NormalStatus;EmergencyBrakeCommand=0;

ServiceBrakeCommand=0;currentSpeed = SimulatedTrainSpeed;

(SimulatedTrainSpeed == 0)]

[SimulatedTrainSpeed > V_mrsp+dV_warning]

[SimulatedTrainSpeed <= V_mrsp]

[SimulatedTrainSpeed > V_mrsp]

[SimulatedTrainSpeed > V_mrsp+dV_sbi]



[EmergencyBrakeCommand != 1 && ServiceBrakeCommand == 1]

ServiceBrakeCommand != 1]

[EmergencyBrakeCommand != 1 &&

[SimulatedTrainSpeed > V_mrsp+dV_ebi]

[EmergencyBrakeCommand == 1]

[(SimulatedTrainSpeed <=V_mrsp && RevocationEmergencyBrake) ||

[Source: OpenETCS project, Cécile Braunstein]

43/53 Ulrich Kühne 26/11/2018

Other Testing Aspects

Mechanical testing• Vibrations• Shock• Standardized stress

Environmental conditions• Temperature• Pressure• Humidity• Radiation

Ageing

[Source: Institute of Space Systems, DLR]

44/53 Ulrich Kühne 26/11/2018

https://www.dlr.de/irs/en/desktopdefault.aspx/tabid-11374/

Plan

Motivation





45/53 Ulrich Kühne 26/11/2018

Hardware Design Flow

Specification

Electr. System Lvl.

Transaction Lvl.

Register Transfer Lvl.

Netlist

Layout

Chip

Natural language

UML, SysML, Matlab, . . .

C, C++, SystemC, . . .

VHDL, Verilog, . . .

Gate models

Geometric, electr. models

Silicon

Req. eng.,modeling

Design Spaceexpl., partitioning

Implementation,refinement

Synthesis

Place & route

Manufacturing

46/53 Ulrich Kühne 26/11/2018

Hardware Design Flow

Specification

Electr. System Lvl.

Transaction Lvl.

Register Transfer Lvl.

Netlist

Layout

Chip

Verification

Equiv. Checking

Test

Req. eng.,modeling

Design Spaceexpl., partitioning

Implementation,refinement

Synthesis

Place & route

Manufacturing

46/53 Ulrich Kühne 26/11/2018

Hardware Verification vs Test

Verification

Detect design bugsExtract properties fromrequirementsApplied on RTL codeHigh manual effort

Test

Detect physical defectsTest generation from netlistaccording to fault modelApplied on fabricated chipsHigh automation

47/53 Ulrich Kühne 26/11/2018

Physical Defects

[Source: IEEE Spectrum “The Art of Failure”]

48/53 Ulrich Kühne 26/11/2018

Stuck-at Fault Model

ab

c

d

a b c d0 0 0 00 0 1 00 1 0 10 1 1 01 0 0 11 0 1 01 1 0 11 1 1 0

〈000〉 is a test vector for the shown stuck-at-1 fault{〈010〉, 〈100〉, 〈110〉} are test vectors for the stuck-at-0 fault

49/53 Ulrich Kühne 26/11/2018


ab

c

d

sa-1

a b c d0 0 0 0/10 0 1 00 1 0 10 1 1 01 0 0 11 0 1 01 1 0 11 1 1 0

〈000〉 is a test vector for the shown stuck-at-1 fault

{〈010〉, 〈100〉, 〈110〉} are test vectors for the stuck-at-0 fault

49/53 Ulrich Kühne 26/11/2018


ab

c

d

sa-1

sa-0

a b c d0 0 0 0/10 0 1 00 1 0 1/00 1 1 01 0 0 1/01 0 1 01 1 0 1/01 1 1 0

〈000〉 is a test vector for the shown stuck-at-1 fault{〈010〉, 〈100〉, 〈110〉} are test vectors for the stuck-at-0 fault

49/53 Ulrich Kühne 26/11/2018

Automatic Test Pattern Generation

ATPG

Create a list of all possible(stuck-at) faultsFor each fault:

• Find a test pattern• Drop all other faults

detected by this pattern

Untestable faults?Hard to test faults?Sequential tests?Test compression?

50/53 Ulrich Kühne 26/11/2018

Summary

Validation on all levels of abstraction> 50% of overall costsCrucial for project success and product qualityVarious techniques

• Dynamic testing• Static verification• Model-based design

Integration into development cycle

51/53 Ulrich Kühne 26/11/2018

Outlook

Test-Driven Design of embedded softwareIntroduction to formal methodsFormal specification and verification of embeddedhardware

52/53 Ulrich Kühne 26/11/2018

Preparation for Exercises

Log in to GitLab:https://gitlab.telecom-paristech.fr

Go to the GitLab group:https://gitlab.telecom-paristech.fr/MSSE/TestVerif/2018

Request access to the group

53/53 Ulrich Kühne 26/11/2018

https://gitlab.telecom-paristech.fr

https://gitlab.telecom-paristech.fr/MSSE/TestVerif/2018

References I

DO-178B: Software Considerations in Airborne Systems and EquipmentCertification, 1982.

EN 50128 - Railway applications - Communication, signalling and processingsystems - Software for railway control and protection systems.Technical report, European Commitee for Electrotechnical Standardization, 2001.

James W. Grenning.Test Driven Development for Embedded C.Pragmatic Bookshelf, Raleigh, N.C, 1st edition, May 2011.

Kelly Hayhurst, Dan S. Veerhusen, John J. Chilenski, and Leanna K. Rierson.A practical tutorial on modified condition/decision coverage, 2001.

54/53 Ulrich Kühne 26/11/2018

Documents

Introduction to Verification and Test of Embedded Systems ... · Formal speciﬁcation and veriﬁcation of embedded hardware (lecture + exercise) 3/53 Ulrich Kühne 26/11/2018. Plan