Embed Size (px)
Citation preview
Chapter 2.11
Program
Validation
Reliable System
=Reliable Hardware
AND
Reliable Software
AND
Compatible Hardware and Software
Reliable Hardware
Good Design
Diagnostic Circuits
Build-in
Redundancy
Diagnostic Programs
Reliable Software
should be
ADEQUATE(accomplishes the tasks it is used
for)
and
ROBUST(survives all abnormal inputs)
Software Validation
Before being used
a program should be
VALIDATED
for
ADEQUACY and ROBUSTNESS
Software Validation
WARNING !
Validation can never improve
the quality of software,
it can only prevent the use of
erroneous programs
Program Errors
• Syntax errors– Detected by the compiler
• Run-time errors– The program stops with an error message– Special recovery software is activated
• Erroneous results– Consequences can be catastrophic !!!
Software engineering tries to push errors upwards in the list
Mandatory Declarations
VAR LOAD, Force : REAL;BEGIN... L0AD := 1000.Undeclared:L0AD... Force := LOAD * ...
BEGIN... L0AD := 1000.... Force:= LOAD * ......
Cross reference: .. . L0AD : 15 LOAD : 20,27,39,44
Normal executionErroneous value of Force
No executionCompilation error
Static vs. Dynamic Validation
• Static validation:– based on the analysis of the program – source code required– could guarantee correctness
• Dynamic validation (testing):– based on the execution of the program– only the object code is required– can only prove the presence of errors,
never their absence
Bottom-up Validation
• Start with small building blocks
• Finish with the entire system
• Natural and intuitively attractive
• Major incompatibilities between building blocks discovered last
• For dynamic validation:– Test programs required for each building block
Top-down Validation
• Start with the entire system, ignoring details.
• Finish with the small building blocks
• Somewhat counter-intuitive
• Major incompatibilities between building blocks discovered first
• For dynamic validation:– Dummy substitutes for all building blocks have to
be made for high-level tests.
Static Validation Techniques
• Syntax checking by the compiler
• Manual code checking– Desk checking– Walk through– Inspection
• Data flow analysis
• Correctness proofs
Static Validation Techniques
• Syntax checking by the compiler
• Manual code checking– Desk checking– Walk through– Inspection
• Data flow analysis
• Correctness proofs
Static Validation Techniques
• Syntax checking by the compiler
• Manual code checking– Desk checking– Walk through– Inspection
• Data flow analysis
• Correctness proofs
Manual code checking
• Desk checking– Informal code reading– preferably by somebody else
• Walk through– Formal meeting– Simulated execution of code on blackboard
• Inspection– Formal meeting– Contradictory discussion of code and
programming style
Static Validation Techniques
• Syntax checking by the compiler
• Manual code checking– Desk checking– Walk through– Inspection
• Data flow analysis
• Correctness proofs
Example : Second
order equation
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Dataflow of a :
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Dataflow of x1 :
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Uninitialized variable
?
x := ...
:= X
Useless statement
Write x
x := exp1
x := exp2
Why this statement ???
Static Validation Techniques
• Syntax checking by the compiler
• Manual code checking– Desk checking– Walk through– Inspection
• Data flow analysis
• Correctness proofs
Correctness Proof
Inputs Outputs
Specifications
Program
EQ
UIV
AL
EN
CE
Correctness Proof
• Formal proofs can be long and complex– Impossible for large systems– Very error prone– Could be automated
• Doesn't help with specification errors…
• Useful for well specified, short, but complex, algorithms.
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
TESTING
"Testing can proof the presence of errors but not
their absence"
E.W.Dijkstra
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
Test data
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Tests should exercise all paths through a program
a b c d1 3 2 11 2 1 01 1 1 -3
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Test dataTests should exercise exceptional values
a b c d0 1 1 11 0 1 -41 1 0 1
d > 0 ?
Read a,b,c
Write x1, x2
No
d:=b 2 - 4ac
d = 0 ?x1 := …
x2 := … x := …
Write x Write "no roots"
No
Test dataTests should exercise very large and very small values
a b c1 10000000001 10000000000
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
Dynamic Validation Techniques
• Testing– No guarantee !!! – Black box vs. white box testing.– Test data
• Debugging– The program state– Control flow breakpoints– Data flow breakpoints– Real time considerations
