1

MISSION: Create effective, grounded, timely materials tosupport the teaching and self-study of software testing,software reliability, and quality-related software metrics.

Cem Kaner, J.D., Ph.D.Director

Pat Bond, Ph.D.

Scott Tilley, Ph.D.

Mike Andrews, Ph.D.

James Whittaker, Ph.D.

2

3

Testing?Testing?

• Testing is often taught as a routine activity.– This misses the point

• Testing is a technical investigation of a product– Complete testing would require infinite time

– Instead, we have to apply judgment & skill

• to develop the optimal set of tests and test documentation

• within the project’s time, cost and reporting constraints

• THIS is a much more challenging task,– to learn

– to teach, and

– to do.

4

Testing Techniques?Testing Techniques?• Testing courses are often focused on

– a miscellaneous collection of test techniques, plus– rules for doing the paperwork

• Any technique addresses one or a few of these six issues:– Risk (what bugs we’re looking for)– Activity (how we do the testing)– Tester (who does the testing)– Coverage (how much of what gets tested)– Evaluation (how to tell whether the program passed the test)– Reporting requirements (needed outputs)

• Our task is to apply the right techniques toserve the specific needs of the project at hand

• For that, we need to understand– techniques– context (the circumstances surrounding the project)– project requirements (how testing serves the project)

5

6

Black Box Testing CourseBlack Box Testing Course

• Professional version in development since 1992• Taught at wide range of companies, including Hewlett-Packard,

Microsoft, Intel, Symantec, Postalsoft, Avid, BEA Systems,Cognos, California Dept of Transportation, Iomega, Peoplesoft,Rational, Cigital, Software AG, Safeco, many others.

• Techniques taught / practiced in context of a sample application• Customizable

– At your site (1-5 days)• Core learning objectives• Selection of topics and examples• Balance of lab vs lecture• (co-taught with your technical manager)

– At Florida Tech (4.5 days typical)• Selection of topics• To lesser extent, lab vs lecture

7

Black Box Testing Course MaterialsBlack Box Testing Course Materials• Oracles

• Impossibility of completetesting

• Techniques

– Function testing

– Domain testing

– Risk-based testing

• Bug taxonomies

• Attacks

– Scenario testing

– Specification-based

– User testing

– Stress testing

– Multivariable tests

– Regression testing

– Stochastic testing & high-volume automation

• Exploratory testing– Scripting problems

– Questioning strategies

– Ad hoc models

– paired testing

• Testability

• Bug reporting– effective advocacy

– quality/cost analysis

• Test documentation– doc components

– analyzing requirementsfor testdocs

• Test planning

• Outsourcing

• Legal issues

• Role of test groups /mission of testing

• Adapting testing todifferent lifecycles

• Test-related metrics andmeasurement theory

• Recruiting

• Career planning

• Evaluating staffperformance

8

Possibilities for the FuturePossibilities for the Future

We’re glad to organize these if there is enthusiasm:

• Homebrew test automation Available NOW– Bret Pettichord– Using / customizing the new generation of free software test tools

• Test-driven development course Potentially now– project-driven, paired programming– extensive personal practice, feedback

• Testing boot camp Summer 2005? Interest?– 3 weeks– High teacher : student ratio– Mix of academic and practitioner teachers– Extensive personal practice/coaching

• Lecture (a.m)• Coached practice (p.m.)• Group assignments (evening)

9