CEN 5070 - Software V & V Unit Testing Concepts

  • Published on
    12-Jan-2016

  • View
    28

  • Download
    0

Embed Size (px)

DESCRIPTION

CEN 5070 - Software V & V Unit Testing Concepts. Purpose. This module presents the basic concepts of black-box and white-box testing for unit testing. A systematic approach is shown for deriving functional, boundary and white-box test cases. - PowerPoint PPT Presentation

Transcript

  • CEN 5070 - Software V & V

    Unit Testing Concepts

    Unit Test Concepts

  • PurposeThis module presents the basic concepts of black-box and white-box testing for unit testing. A systematic approach is shown for deriving functional, boundary and white-box test cases." arbitrarily selected test set ... results in inefficient testing, leavingsome functions untested while performing redundant testing of others."Darlene Mackay, Quality Consultants Unlimited

    Unit Test Concepts

  • AgendaWhat Is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All Together

    Unit Test Concepts

  • WHAT IS UNIT TESTING?Executing a software element to determine whether it meets its specificationExecuting a software element to discover defects or anomaliesInspecting software element code to discover defects or anomalies.

    Unit Test Concepts

  • WHAT IS A UNIT?Named software element Separately invokablePerforms single functionExamplesSubprogram or scriptField with validationDatabase stored procedureJava class method

    Unit Test Concepts

  • SPRAE: A Model for Testing PracticeSpecification -- basis for software testingPremeditation -- testing requires planning, forethought Repeatability -- process, results independent of testerAccountability -- testing artifacts maintainedEconomy in the use of human, time and computing resources

    Unit Test Concepts

  • A TESTING LIFECYCLEAnalysisDesign

    Unit Test Concepts

  • AgendaWhat is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All Together

    Unit Test Concepts

  • BLACK-BOX TESTINGTesting based on the specification rather than the implementation. Specification defines the expected response(s) to stimuliStimuliResponse(s)

    Unit Test Concepts

  • BLACK-BOX TECHNIQUESFunctional testing -- tests the behavior of the software. Boundary testing -- tests behavior at the lower/upper bounds of input valuesRandom testing -- tests using randomly generated stimuli (load testing)Intuitive (ad hoc) testing -- error guessing

    Unit Test Concepts

  • FUNCTIONAL TEST DESIGN METHODOLOGYSpecificationIdentify behaviors Develop test casesWrite test script

    Unit Test Concepts

  • EXAMPLE A(1) SpecificationCompute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.

    Unit Test Concepts

  • EXAMPLE A(2) Identify BehaviorsCase 1: No overtime (Hours 40)Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)

    Unit Test Concepts

  • EXAMPLE A(3) Create Test CasesCase 1: No overtime (Hours 40)Use Rate = 10, Hours = 50 Expect Pay = 40*Rate+1.5*Rate*(Hours - 40) = 550

    Unit Test Concepts

  • EXAMPLE A(4) Write Test Script

    Unit Test Concepts

  • A MORE COMPLEX EXAMPLE (B)Increased number of behaviorsUse of decision table to document behaviorsTest case generation from decision table

    Unit Test Concepts

  • EXAMPLE B(1) SpecificationCompute pay for employee, given the number of hours worked and the hourly pay rate. For hourly employees (rate < 30), compute overtime at 1.5 times hourly rate for hours in excess of 40. Salaried employees (rate >= 30) are paid for exactly 40 hours.

    Unit Test Concepts

  • EXAMPLE B(2) Identify BehaviorsCase 1: Hourly AND No overtime (Rate < 30) & (Hours 40)Expect Pay = 40*Rate+1.5*Rate*(Hours - 40)Case 3: Salaried (Rate >= 30) Expect Pay = 40 * Rate

    Unit Test Concepts

  • DECISION TABLEColumns defineBehaviors

    Unit Test Concepts

  • EXAMPLE B(3) Create Test CasesOne test case per column of decision tableCase 1: Hourly, No OvertimeCase 2: Hourly, OvertimeCase 3: Salaried, No Extra HoursCase 4: Salaried, Extra HoursOrder the test cases by column

    Unit Test Concepts

  • EXAMPLE B(4) Write Test Script

    Unit Test Concepts

  • RULES -- DECISION TABLES Use 'Y', 'N', '-' or space

    Unit Test Concepts

  • Your Turn -- Problem P1(1) SpecificationCompute the dosage of drug X for patient, given the patient's Age and Weight. For patients 12 and under, the dosage is 1 pill. For patients over 65, the dosage is 2 pills. For all other patients, the dosage is 2 pills plus an extra pill for each 50 pounds above 120. The drug can not be given to patients over 300 pounds or over the age of 80.

    Unit Test Concepts

  • Your Turn(2a) Identify BehaviorsCaseExpected Stimulus Description#Pills 123456

    Unit Test Concepts

  • Your Turn (2b) Decision Table c1: Age 65 |c3: Age > 80 |c4: Weight > 300 |c5: Weight > 120 |a1: Pills = 0 |a2: Pills = 1 |a3: Pills = 2 |a4: Pills = 2+(W-120)/50 |

    Unit Test Concepts

  • Your Turn(3) Create Test Cases Case 1 2 3 4 5 6 7 8 9Age ___ ___ ___ ___ ___ ___ ___ ___ ___ Weight ___ ___ ___ ___ ___ ___ ___ ___ ___Pills ___ ___ ___ ___ ___ ___ ___ ___ ___

    Unit Test Concepts

  • Your Turn(4) Write Test ScriptStepStimuliPills= Age7891012Weight11

    Unit Test Concepts

  • SCALING UP

    The heart of the approach is to use a decision table as a thinking tool. The most critical task in this process is to identify all the stimuli and responses. When there are many logical combinations of stimuli, the decision table can become large, indicating that the unit is probably too complex.

    Unit Test Concepts

  • IDENTIFYING BEHAVIORApproachesWork backwardsIdentify each responseIdentify conditions that provoke responseIdentify separate stimuliWork forwardIdentify each stimulusIdentify how stimulus influences what unit doesSpecify the responseTreat stimuli combinations

    Unit Test Concepts

  • IDENTIFYING STIMULIArguments passed upon invocationInteractive user inputsInternal, secondary dataglobal or class variablesExternal data (sources)file or database status variablesfile or database dataExceptions

    Unit Test Concepts

  • IT PAYS TO BE A GOOD STIMULUS DETECTIVE Failure to identify stimuli results in an incomplete, possibly misleading test caseThe search for stimuli exposesinterface assumptions -- a major source of integration problemsincomplete design of unitinadequate provision for exception handling

    Unit Test Concepts

  • IDENTIFYING RESPONSESArguments/Results passed back on exitInteractive user outputsInternal, secondary dataupdated global or class variablesExternal data (sinks)output file or database status variablesoutput file or database dataExceptions

    Unit Test Concepts

  • IT PAYS TO BE A GOOD RESPONSE DETECTIVE Failure to identify responses results in incomplete understanding of the software under testshallow test casesincomplete expected resultsincomplete test "success" verification -- certain effects not checkedTo test, one must know all the effects

    Unit Test Concepts

  • A SKETCHING TOOL Black-Box SchematicStimulus TypeResponse TypeSoftwareunderTestArgumentInputsGlobalsDatabaseExceptionArgumentOutputsGlobalsDatabaseException

    Unit Test Concepts

  • BEFORE CONTINUTING

    Much of the discussion so far involves how to identify what software does. We have introduced thinking tools for systematically capturing our findings. These thought processes and tools can be used anywhere in the lifecycle, e.g., in software design!One Stone for Two Birds!!

    Unit Test Concepts

  • BOUNDARY TESTING DESIGN METHODOLOGYSpecificationIdentify elementary boundary conditions Identify boundary pointsGenerate boundary test casesUpdate test script (add boundary cases).

    Unit Test Concepts

  • (1) SpecificationCompute pay for an hourly employee, given the number of hours worked and the hourly pay rate. Compute overtime at 1.5 times hourly rate for hours in excess of 40.HoursPayRate

    Unit Test Concepts

  • (2) Identify Boundary ConditionsCondition 1 (bc1): Hours
  • (3) Identify Boundary Pointsbc1 (Hrs
  • (3a) BP Generalization bc x > y has TWO boundary pointsbp1: Just INside: x = y + precisionbp2: Just OUTside: x = ybc x = y has THREE boundary points:bp1: OUTlo: x = y - precisionbp2: OUThi: x = y + precisionbp3: AT: x = y

    Unit Test Concepts

  • (3b) BP Generalization bc x != y has THREE boundary points:bp1: INlo: x = y - precisionbp2: INhi: x = y + precisionbp3: OUT: x = y

    Unit Test Concepts

  • (4) Generate Test CasesCombine Hours boundary points with RateCase 1 (AT): Hours = 40, Rate = 10, Pay=400Case 2 (IN): Hours = 39, Rate = 10, Pay=390Case 3: (OUT): Hours = 41, Rate=10, Pay=415Observations:Test each boundary point individuallyThen consider pair-wise boundary points

    Unit Test Concepts

  • (5) Update Test ScriptStepStimuliExpected Response HoursRatePay =1230501010300550340104004391039054110415

    Unit Test Concepts

  • Your TurnBoundary Testing Decision Table:

    Unit Test Concepts

  • Your Turn (2-3) Boundary Conditions/PointsBoundary ConditionBoundary Point (BP)ATINbc1:OUTbc2:bc3:bc4:bc5:

    Unit Test Concepts

  • Your Turn(4) Generate Boundary Test CasesTo create a test case, you must pair an Age with a WeightWeight boundary point + NOMINAL AgeAge boundary point + NOMINAL WeightOUT Age + OUT Weight A nominal value is one that is not close to a boundary point. For simplicity, use the same nominal value in all test cases.

    Unit Test Concepts

  • Your Turn(4) Boundary-Nominal Test CasesCondition btc BC Weight Age Expect Weight>300 1 IN 301 21 0 2 OUT 300 21 Age 65 6 IN 7 OUT Age > 80 9 IN 10 OUT Weight>120 11 IN 12 OUT

    Unit Test Concepts

  • Your Turn(4) Out-Out Boundary Test Cases Condition OUT BPWeight Age btc Weight Age Expect >300 65 14 >80 15

    >120 65 17 >80 18

    Unit Test Concepts

  • OBSERVATIONSFunctional testing defines a minimal number of test casesBoundary testing adds a large number of test cases, but are EASY to createBoundary testing finds lots of errors!

    Unit Test Concepts

  • BOUNDARIES EXIST FOROptional fields: (present, missing).Variable length data: null string, max lengthDatabase tables: emptySearching: first/last positionFile: closed, empty

    Unit Test Concepts

  • RANDOM TESTINGBeyond scope of this courseGenerally used to bombard software with inputsNo effort to identify expected resultsAppropriate for automated load testing, where concern is for capacity/volume.

    Unit Test Concepts

  • INTUITIVE (AD HOC) TESTINGMost common type of informal testingOften, no specification!!No scriptsNot repeatableNot systematicVery effectiveDoes not guarantee thorough testing

    Unit Test Concepts

  • INTUITIVE TESTINGAd hoc, exploratory testing, ideal for destructive testingGoal is to break the software via unexpected stimuli or sequences of stimuliBenefitsFlushes out holes in the specification .. What really should happen when the input is X? Forces treatment of error/exception handling

    Unit Test Concepts

  • MAIN POINTSBLACK-BOX TESTINGBlack-box = spec-based testingTests intentionsKey knowledge = stimuli and responsesDecision table organizes S/R conditionsBoundary testing flushes coding errorsBlack-box test case design can drive software design -- same issues addressed

    Unit Test Concepts

  • AgendaWhat is Unit TestingBlack-Box TestingWhite-Box TestingPutting It All Together

    Unit Test Concepts

  • WHITE-BOX TESTINGStimuliResponse(s)Testing to ensure that software does not do what is not supposed to do. Test ALL of it!Tick -- Tick -- TickNo-Surprise Software!!

    Unit Test Concepts

  • WHITE-BOX TESTINGFocus is thorough execution of program elements during the testing process.Warning: Tests only what is built, not what was intended!StimuliResponse(s)

    Unit Test Concepts

  • WHITE-BOX TESTINGConcept of coverage. Numeric measure of thoroughness of testing, relative to StatementsBranchesConditionsPaths

    Unit Test Concepts

  • CONTROL FLOW GRAPHDefines the flow of control through unit. 12453

    Unit Test Concepts

  • CONTROL FLOW GRAPH TERMINOLOGY5 NODESsequential blocks of code terminated by a branch3 PATHS: [1,2,3,5], [1,2,5], [1,4,5] 2 BRANCHES1 and 2 are decision nodes

    Unit Test Concepts

  • CONTROL FLOW GRAPH COVERAGEOne test case forces execution of one path (red)Paths are determined by branches (decision nodes)A thorough test set forces execution of all paths (red, green, blue).

    Unit Test Concepts

  • COVERAGE LEVELS (%)Statement -- a statement has been executed at least once during testingBranch -- each outcome of a branch has been performed at least once during testingPath -- a path through the code has been executed at least once during during testingCondition -- a condition has evaluated to true and to false at least once during testing

    Unit Test Concepts

  • CONTROL FLOW GRAPH COVERAGE MEASUREMENTFor 2 test cases (red, green)Node (statement) cov = 4/5Branch cov = 1/2 [2]Path cov = 2/3Acceptable coverage levelsStatement cov = 90%Branch cov = 80%Path cov = 70%12453

    Unit Test Concepts

  • BRANCH vs CONDITION COVERAGECode example 124100% Branch coverage [1] (x=0,y=2), (x=1,y=2) [TT,FT]But not 100% Condition coverage Need case TF (x=0,y=1)if (x1) x = x + y;else y = y - x;

    Unit Test Concepts

  • THE PROBLEM WITH COMPOUND CONDITIONSMakes complex logic appear simpler than it really isTest cases may be omittedLogic results in 3 paths, not 2!!if (x1) x=x+y; else y=y-x;} else y=y-x;

    Unit Test Concepts

  • THE PROBLEM WITH PATH COVERAGENot all paths are feasibleNo test case can force path [1,2,3,4,5], since consecutive decisions mutually exclusive.if (x= 1) y=3;z=y;

    Unit Test Concepts

  • Measuring Path Testing DifficultyMcCabe metric -- logical code complexity Formula: 1 + #decisions in control flow graphTest Significance: #basis paths through codeDesign use: complexity of codeTest use: min #test cases for 100% path coverageMcCabe measures test (development) difficulty

    Unit Test Concepts

  • How To Design White Box TestsTest cases must execute different pathsDecision tables Rows -- elementary conditions in the codeColumns -- combinations of conditions in the codeColumn based test case forces flow through different logic paths in the codeDecision table built from code reflects what was built versus intended (ie, from the spec)Decision analysis for white-box testing.

    Unit Test Concepts

  • WHITE-BOX TESTING TOOLSTools instrument source code and gathers coverage data when tests Compiled languagesScript languages -- coming to marketSome provide test case design assistanceList of paths not coveredData conditions to force branch/pathGraphic depiction of graph coverage

    Unit Test Concepts

  • Problem P1Code (v1) & Decision Table

    if (Age>80 || Weight>300) return 0; if (Age 65) return 2;if (Weight < 120) return 2else return 2+(Weight-120)/50;McCabe = 6

    Unit Test Concepts

  • Problem P1Code (v2) & Decision Table

    if (Age>80 || Weight>300)return 0; if (Age 65 || (Age

  • Your Turn -- White-Box Testing(1) Construct Decision Table

    pills=0;if (Age < 80 && Weight = 65) pills=2; else if (Age > 12) pills=2+(Weight-120)/50;}return pills;

    Unit Test Concepts

  • Your Turn -- P1(2) Derive White-Box Test Cases Case 1 2 3 4 5 6 7 8 9Age ___ ___ ___ ___ ___ ___ ___ ___ ___ Weight ___ ___ ___ ___ ___ ___ ___ ___ ___Pills ___ ___ ___ ___ ___ ___ ___ ___ ___

    Unit Test Concepts

  • OBSERVATIONS -- WHITE-BOX TEST CASESCode may be incomplete with respect to input combinations from the specification Decision table constructed from code is simpler -- subset of black-box tableClaim: bl...

Recommended

View more >