IMPLEMENTATION

Implementation

Process of translating the detailed design into code

Real-life products are generally too large to be implemented by a single programmer

“programming-in-the-many” – a team working at the same time on different components of the product

Choice of Programming LanguageThe programming language is usually specified in the contract

But what if the contract specifies thatThe product is to be implemented in the “most suitable” programming language

What language should be chosen?

Choice of Programming Language

How to choose a programming languageCost–benefit analysis as well as risk analysis should

be performedCompute costs and benefits of all relevant languages

Which is the most appropriate object-oriented language?C++ is C-likeThus, every classical C program is automatically a C+

+ programJava enforces the object-oriented paradigmTraining in the object-oriented paradigm is essential

before adopting any object-oriented language

Good programming practiceVariable names should be consistent and

meaningfulAs program will be studied by many

people – especially during maintenanceIf frequency, freq refer to the same item then

should only use one name If freq_max is being used then should use

freq_min instead of min_freg

The Issue of Self-Documenting CodeProgram does not need comment because it is carefully written with proper

variable names, structured etcSelf-documenting code (the program is written so well that it reads like a

document) is exceedingly rare

The key issue: Can the code artifact (program ) be understood easily and unambiguously by The SQA (software quality assurance) teamMaintenance programmersAll others who have to read the codeEven the original programmers after couple of years

A prologue comments should be included in every program

Prologue Comments

Minimal prologue comments for a code artifact

Figure 14.2

Nested if StatementsExample

A map consists of two squares. Write code to determine whether a point on the Earth’s surface lies in map_square_1 or map_square_2, or is not on the map

Figure 14.2

Nested if StatementsWell-formatted, badly constructed – too many ‘if’

Figure 14.4

Nested if Statements (contd)

Solution 3. Acceptably nested

Figure 14.5

Nested if Statements (contd)

A combination of if-if and if-else-if

statements is usually difficult to readSimplify: The if-if combination

if <condition1>if <condition2>

is frequently equivalent to the single condition

if <condition1> && <condition2>

Nested if Statements (contd)

Rule of thumb if statements nested to a depth of greater than

three should be avoided as poor programming practice

Programming StandardsSetting rules for the program writing – such as

“each module will consist of between 35 and 50 executable statements”

Aim of coding standards is to make maintenance easier.

Flexibility should be considered when designing the rules, example“Programmers should consult their managers before

constructing a module with fewer than 35 or more than 50 executable statements”

Remarks on Programming StandardsNo standard can ever be universally applicable

Developing different systems may require different rules

Standards imposed from above will be ignored Rules should be discussed and agreed by the

programmersStandard must be checkable by machine

How can you check if rules are being followed?

Examples of Good Programming Standards

“Nesting of if statements should not exceed a depth of 3, except with prior approval from the team leader”

“Modules should consist of between 35 and 50 statements, except with prior approval from the team leader”

“Use of goto should be avoided. However, with prior approval from the team leader, a forward

goto (force exit) may be used for error handling”

IntegrationThe approach up to now:

Implementation followed by integrationThis is a poor approach Better:

Combine implementation and integration methodicallyCode and test each code artifact separatelyLink all artifacts together, test the product as a whole

Product with 13 Modules

Figure 14.6

Drivers and StubsTo test artifact a, artifacts b,c,d must be stubs (a short

remaining piece) An empty artifact, orPrints a message ("Procedure radarCalc called"), orReturns predefined values from preplanned test cases

To test artifact h on its own requires a driver, which calls it Once, or Several times, or Many times, each time checking the value returned

Testing artifact d requires a driver and two stubs

Implementation, Then IntegrationProblem 1

Stubs and drivers must be written, then thrown away after unit testing is complete

Problem 2Lack of fault isolationA fault could lie in any of the 13 artifacts or 13 interfacesIn a large product with, say, 103 artifacts and 108

interfaces, there are 211 places where a fault might lieSolution to both problems

Combine unit and integration testing

Top-down Integration

If code artifact mAbove sends a message to artifact mBelow, then mAbove is implemented and integrated before mBelow

One possible top-down ordering is a,b,c,d,e,f,g,h,i,j,k,l,m

Figure 14.6 (again)

Top-down Integration (contd)Another possible top-down

ordering is a[a] b,e,h[a] c,d,f,i[a,d] g,j,k,l,m

The above integration sequence cannot executed in parallel Figure 14.6 (again)

Top-down Integration (contd)

Advantage 1: Fault isolationA previously successful test case fails when mNew is added to what has been

tested so farThe fault must lie in mNew or the interface(s) between mNew and the rest of the product

Advantage 2: Stubs are not wastedEach stub is expanded into the corresponding complete artifact at the

appropriate step

Top-down Integration (contd)Advantage 3: Major design flaws show up earlyLogic artifacts include the decision-making flow

of controlIn the example, artifacts a,b,c,d,g,j

Operational artifacts perform the actual operations of the productIn the example, artifacts e,f,h,i,k,l,m

The logic artifacts are developed before the operational artifacts

Top-down Integration (contd) Problem 1

Reusable artifacts are not properly tested Lower level (operational) artifacts are not tested frequently

(because you start at the top-level!) Defensive programming (fault shielding)

Example in the driver:if (x >= 0)

y = computeSquareRoot (x, errorFlag); computeSquareRoot is never tested with x < 0 This has implications for reuse because testing is not thorough

Bottom-up IntegrationIf code artifact mAbove

calls code artifact mBelow, then mBelow is implemented and integrated before mAbove

One possible bottom-up ordering isl,m,h,i,j,k,e,f,g,b,c,d,a

Figure 14.6 (again)

14.6.2 Bottom-up Integration

Another possible bottom-up ordering is

h,e,bi,f,c,dl,m,j,k,g [d]a[b,c,d]

Figure 14.6 (again)

Bottom-up Integration (contd)

Advantage 1Operational artifacts are thoroughly tested

Advantage 2Operational artifacts are tested with drivers, not

by fault shielding, defensively programmed artifacts

Advantage 3Fault isolation

Bottom-up Integration (contd)

Difficulty 1Major design faults are detected late because design faults

usually embedded in the logic modules

SolutionCombine top-down and bottom-up strategies making use of their

strengths and minimizing their weaknesses

Sandwich IntegrationLogic artifacts are

integrated top-downOperational artifacts

are integrated bottom-up

Finally, the interfaces between the two groups are tested

Figure 14.7

Sandwich Integration (contd)

Advantage 1Major design faults are caught early

Advantage 2Operational artifacts are thoroughly testedThey may be reused with confidence

Advantage 3There is fault isolation at all times

Summary

Figure 14.8

Integration of Object-Oriented ProductsObject-oriented implementation and integration

Almost always sandwich implementation and integrationObjects are integrated bottom-up –assuming objects usually

represent the operationsOther artifacts are integrated top-down

14.7 The Implementation Workflow

The aim of the implementation workflow is to implement the target software product

A large product is partitioned into subsystemsImplemented in parallel by coding teams

Subsystems consist of components or code artifacts

The Implementation Workflow (contd)

Once the programmer has implemented an artifact, he or she unit tests it

Then the module is passed on to the SQA group for further testingThis testing is part of the test workflow

The Test Workflow: ImplementationA number of different types of testing have to be performed

during the implementation workflow:Unit testing, integration testing, product testing etc

Unit testingInformal unit testing by the programmerMethodical (systematic) unit testing by the SQA group

There are two types of methodical unit testingNon-execution-based testingExecution-based testing

Test Case Selection

In order to test, proper test cases must be chosenWorst way — random testing

There is no time to test all but the tiniest fraction of all possible test cases, totaling perhaps 10100 or more

We need a systematic way to construct test cases

Testing to Specifications versus Testing to CodeThere are two extremes to testingTest to specifications (also called black-box,

data-driven, functional, or input/output driven testing)Ignore the code — use the specifications to select

test casesTest to code (also called glass-box, logic-

driven, structured, or path-oriented testing)Ignore the specifications — use the code to select

test cases

Feasibility of Testing to SpecificationsExample:

The specifications for a data processing product include 5 types of commission and 7 types of discount

35 test cases

We cannot say that commission and discount are computed in two entirely separate artifacts; and therefore should be tested separately (test cases are not 5+7)

The structure is irrelevant

Feasibility of Testing to Specifications (contd)Suppose the specifications include 20 factors

(input types), each taking on 4 valuesThere are 420 or 1.1 1012 test casesIf each takes 30 seconds to run, running all test cases

takes more than 1 million yearsThe combinatorial explosion (the many

combinations) makes testing to specifications impossible

14.10.3 Feasibility of Testing to CodeEach path through an artifact must be

executed at least onceCombinatorial explosion still a problem

Feasibility of Testing to Code (contd)Code example:

Figure 14.9

Feasibility of Testing to Code (contd)

The flowchart has over 1012 different paths (why??)

How can you do an exhaustive test?

Figure 14.10

Testing to code is not reliable

We can exercise every path without detecting every fault (such as x=2, y=1, z=3)

Feasibility of Testing to Code (contd)

A path can be tested only if it is present

In case (b), path is not obvious

A programmer who omits the test for d = 0 in the code probably is unaware of the possible danger

Feasibility of Testing to Code (contd)

Figure 14.12

Black-Box Unit-testing Techniques

Neither exhaustive testing to specifications nor exhaustive testing to code is feasible

The art of testing:Select a small, manageable set of test cases to Maximize the chances of detecting a fault, while Minimizing the chances of wasting a test case

Every test case must detect a previously undetected fault

Black-Box Unit-testing Techniques

We need a method that will highlight as many faults as possibleFirst black-box test cases (testing to specifications)Then glass-box methods (testing to code)

Equivalence testing and boundary value analysis is a form of black-box testing technique

Equivalence Testing and Boundary Value Analysis

ExampleThe specifications for a DBMS state that the product

must handle any number of records between 1 and 16,383 (214 – 1)

If the system can handle 34 records and 14,870 records, then it probably will work fine for 8,252 records

If the system works for any one test case in the range (1..16,383), then it will probably work for any other test case in the rangeRange (1..16,383) constitutes an equivalence class

Equivalence Testing

Any one member of an equivalence class is as good a test case as any other member of the equivalence class

Range (1..16,383) defines three different equivalence classes:Equivalence Class 1: Fewer than 1 recordEquivalence Class 2: Between 1 and 16,383

recordsEquivalence Class 3: More than 16,383 records

Database Example (contd)Select test cases on or just to one side of the boundary of

equivalence classes This greatly increases the probability of detecting a fault

Test case 1: 0 records Member of equivalence class 1 and adjacent to boundary value

Test case 2: 1 record Boundary valueTest case 3: 2 records Adjacent to boundary

value Test case 4: 723 records Member of equivalence

class 2

Database Example (contd)

Test case 5: 16,382 records Adjacent to boundary value

Test case 6: 16,383 records Boundary valueTest case 7: 16,384 records Member of

equivalence class 3 and adjacent to boundary value

Equivalence Testing of Output SpecificationsWe also need to perform equivalence

testing of the output specificationsExample:

In 2009, the maximum tax return is $6000Test cases must include input data that should

result in return of exactly $0.00 and exactly $6000 Also, test data that might result in return of less

than $0.00 or more than $6000

Overall Strategy

Equivalence classes together with boundary value analysis to test both input specifications and output specificationsThis approach generates a small set of test

data with the potential of uncovering a large number of faults

Exercise

Q2 in 08/09 exam

Functional Testing

An alternative form of black-box testing for classical softwareWe base the test data on the functionality of the code

artifactsEach item of functionality or function is identifiedTest data are devised to test each (lower-level) function

separatelyThen, higher-level functions composed of these lower-

level functions are tested

Functional Testing (contd) In practice, however

Higher-level functions are not always neatly constructed out of lower-level functions using the constructs of structured programming

Instead, the lower-level functions are often intertwinedFor example, consider a complex number division

how this can be done?

Functional testing

Also, functionality boundaries do not always coincide with code artifact boundariesThe distinction between unit testing and integration

testing becomes blurredThis problem also can arise in the object-oriented

paradigm when messages are passed between objects

14.13 Glass-Box Unit-Testing TechniquesGlass-box testing is based on the

examination of the code We will examine

Statement coverageBranch coveragePath coverageLinear code sequencesAll-definition-use path coverage

Structural Testing: Statement, Branch, and Path Coverage

Statement coverage: Running a set of test cases in

which every statement is executed at least once

Difficult to achieve manuallyWeakness

Branch statementsBoth statements can be

executed without the fault showing up Figure 14.15

Example Int returnInput (int x, boolean cnd1, bool cnd2, bool cnd3){

if (cnd1) {x++;}

if (cnd2) {x--;}

if(cnd3) {x=x;}

return x;}

Exercise

Test case for statement coverageX = 0, cnd1=cnd2=cnd3=T

Structural Testing: Branch Coverage

Running a set of test cases in which every branch is executed at least once (as well as all statements)

A branch is the outcome of a decisionThis solves the problem on the previous slideAgain, a CASE tool is needed as there may be

many cases to test manually

Branch coverage

Test casesX = 0, cnd1=cnd2=cnd3 = TX = 0, cnd1=cnd2=cnd3 = F

Structural Testing: Path Coverage

Running a set of test cases in which every path is executed at least once (as well as all statements)

Problem:The number of paths may be very large

We want a weaker condition than all paths but that shows up more faults than branch coverage

Path coverage

Form a basis path coverage based on the cyclomatic complexity

Test cases derived to test the basis path independently

Each new basis path “changes” exactly one previously executed decision leaving all other executed branches unchanged

Exercise

Based on the example used in the branch coverage determine test cases for path coverage

Linear Code Sequences and jump

Identify the set of points L from which control flow may jump, plus entry and exit points

Restrict test cases to paths that begin and end with elements of L

This uncovers many faults without testing every path

Linear Code Sequences and jump

the start of the linear sequence (basic block) of executable statements

the end of the linear sequencethe target line to which control flow is

transferred at the end of the linear sequence.

1. #include <stdlib.h>2.#include <string.h> 3.#include <math.h> 5.#define MAXCOLUMNS 26 6.#define MAXROW 20 7.#define MAXCOUNT 90 8.#define ITERATIONS 750 10.int main (void) •{ 2.int count = 0, totals[MAXCOLUMNS], val = 0; 3.memset (totals, 0, MAXCOLUMNS * sizeof(int)); 5.count = 0; 6.while ( count < ITERATIONS ) 7.{ 8.val = abs(rand()) % MAXCOLUMNS; 10.totals[val] += 1; •if ( totals[val] > MAXCOUNT ) 3.{ 4. totals[val] = MAXCOUNT; 5.} 6.count++; 7.} 8.return (0); 10.}

LCSAJ Number Start Line Finish Line Jump To Line

1 10 17 28

2 10 21 25

3 10 26 17

4 17 17 28

5 17 21 25

6 17 26 17

7 25 26 17

8 28 28 -1

test data is derived so that the execution of each of these LCSAJs at least once.

All-Definition-Use-Path Coverage

Each occurrence of variable, zz say, is labeled either as The definition of a variable

zz = 1 or read (zz) or the use of variable

y = zz + 3 or if (zz < 9) errorB ()

Identify all paths from the definition of a variable to the use of that definition This can be done by an automatic tool

A test case is set up for each such path

1 i = get_a_number_from_console();2 j = get_a_number_from_console();3 if (a<30) {4 i= get_a_number_from_console();5 } else {6 j=5;7 }8 if (a>b) {9 i=8; j=6;11 } else {12 printf("%d\n",j);13 }14 if (b<30) {15 printf("%d\n",i);16 j=6;17 } else {18 printf("%d\n",i);19 }

Definition occurrences of variables i: 1, 4, 9

Use occurrences of variable i: 15, 18

There are 6 definition use pairs

1-18 (a=30, b= 32)

1-15 (a=28, b=29)

4-15 (a=25, b=24)

4-18 (a=25, b=34)

9-15 (a=29, b=25)9-18 (a=35, b=34)

All-Definition-Use-Path Coverage

Disadvantage:Upper bound on number of paths is 2d, where d is the number

of branchesIn practice:

The actual number of paths is proportional to d not 2d

This is therefore a practical test case selection technique

Infeasible Code

It may not be possible to test a specific statement We may have an

infeasible path (“dead code”) in the artifact

Frequently this is evidence of a fault

Figure 14.16

14.13.2 Complexity Metrics

A quality assurance approach to glass-box testing

Artifact m1 is more “complex” than artifact m2Intuitively, m1 is more likely to have faults than artifact m2

If the complexity is unreasonably high, redesign and then reimplement that code artifactThis is cheaper and faster than trying to debug a fault-

prone code artifact

Lines of Code

The simplest measure of complexityUnderlying assumption: There is a constant

probability p that a line of code contains a fault Example

The tester believes each line of code has a 2% chance of containing a fault.

If the artifact under test is 100 lines long, then it is expected to contain 2 faults

The number of faults is indeed related to the size of the product as a whole

Other Measures of Complexity

Cyclomatic complexity M (McCabe)Essentially the number of decisions (branches)

in the artifact Easy to computeA surprisingly good measure of faults (but see

next slide)In one experiment, artifacts with M > 10

were shown to have statistically more errors per line of code

Problem with Complexity Metrics

Complexity metrics, as especially cyclomatic complexity, have been strongly challenged onTheoretical groundsExperimental grounds, andTheir high correlation with LOC

Essentially we are measuring lines of code, not complexity

Code Walkthroughs and Inspections

Code reviews lead to rapid and thorough fault detectionUp to 95% reduction in maintenance costs

14.15 Comparison of Unit-Testing TechniquesExperiments comparing

Black-box testingGlass-box testingReviews - inspection

[Myers, 1978] 59 highly experienced programmersAll three methods were equally effective in finding

faultsCode inspections were less cost-effective

[Hwang, 1981] All three methods were equally effective

Cleanroom

A different approach to software development

Incorporates a number of different software development techniques:An incremental process modelFormal techniquesReviews

With “cleanroom” approach, a software module is not compiled until it has passed inspection (clean!!)

Cleanroom (contd)

Prototype automated documentation system for the U.S. Naval Underwater Systems Center

1820 lines of FoxBASE18 faults were detected by “functional verification”Informal proofs were used19 faults were detected in walkthroughs before

compilationThere were NO compilation errorsThere were NO execution-time failures

Cleanroom (contd)

Testing fault rate counting procedures differ:

Usual paradigms:Count faults after informal testing is complete (once

SQA starts)

CleanroomCount faults after inspections are complete (once

compilation starts)

Report on 17 Cleanroom Products

Operating system350,000 LOCDeveloped in only 18 monthsBy a team of 70The testing fault rate was only 1.0 faults per KLOC

Various products totaling 1 million LOCWeighted average testing fault rate: 2.3 faults per KLOC

“[R]emarkable quality achievement”

14.18 Management Aspects of Unit TestingWe need to know when to stop testing

A number of different techniques can be usedCost–benefit analysisRisk analysisStatistical techniques

When a code artifact has too many faultsIt is cheaper to redesign, then recode

The risk and cost of further faults are too great

14.19 When to Rewrite Rather Than Debug

Figure 14.18

Fault Distribution in Modules Is Not Uniform [Myers, 1979]

47% of the faults in OS/370 were in only 4% of the modules [Endres, 1975]

512 faults in 202 modules of DOS/VS (Release 28) 112 of the modules had only one fault There were modules with 14, 15, 19 and 28 faults, respectively The latter three were the largest modules in the product, with

over 3000 lines of DOS macro assembler language The module with 14 faults was relatively small, and very

unstable A prime candidate for discarding, redesigning, recoding

When to Rewrite Rather Than Debug (contd) For every artifact, management must predetermine the maximum

allowed number of faults during testing

If this number is reached Discard Redesign Recode

The maximum number of faults allowed after delivery is ZERO

14.21 Product Testing

Product testing for COTS (commercial off-the-shelf ) softwareAlpha, beta testing

Product testing for custom softwareThe SQA group must ensure that the product passes the

acceptance test (performed by the client)Failing an acceptance test has bad consequences for the

development organization

Product Testing for Custom SoftwareThe SQA team must try to approximate the

acceptance testBlack box test cases for the product as a wholeRobustness of product as a whole

Stress testing (under peak load)Volume testing (e.g., can it handle large input files?)

All constraints must be checkedAll documentation must be

Checked for correctnessChecked for conformity with standardsVerified against the current version of the product

Product Testing for Custom Software (contd)The product (code plus documentation) is

now handed over to the client organization for acceptance testing

14. 22 Acceptance Testing

The client determines whether the product satisfies its specifications

Acceptance testing is performed byThe client organization, or The SQA team in the presence of client

representatives, or An independent SQA team hired by the client

Acceptance Testing (contd)

The four major components of acceptance testing areCorrectnessRobustnessPerformanceDocumentation

These are precisely what was tested by the developer during product testing

Acceptance Testing (contd)

The key difference between product testing and acceptance testing isAcceptance testing is performed on actual dataProduct testing is preformed on test data, which

can never be real, by definition

14.25 Metrics for the Implementation WorkflowComplexity metrics, fault per KLOC

Fault statistics are importantNumber of test casesPercentage of test cases that resulted in failureTotal number of faults, by types

The fault data are incorporated into checklists for code inspections

14.26 Challenges of the Implementation WorkflowManagement issues are paramount here

Appropriate CASE toolsTest case planningCommunicating changes to all personnelDeciding when to stop testing

Challenges of the Implementation Workflow (contd)Code reuse can reduce time and cost during

implementation, butCode reuse needs to be built into the product from

the very beginningThe software project management plan must incorporate

reuseImplementation is technically straightforward if

previous phases such as requirements, analysis were carried out satisfactorily.

Challenges of the Implementation Phase (contd)Make-or-break issues (成敗的關鍵 )

include:Use of appropriate CASE toolsTest planning as soon as the client has signed

off the specificationsEnsuring that changes are communicated to all

relevant personnelDeciding when to stop testing