11

Software Software TestingTesting

Does your software product perform as it is Does your software product perform as it is intended to perform?intended to perform?

Presented by:Presented by:

Andy BoutsAndy Bouts

John KinuthiaJohn Kinuthia

22

ContentContent

Introduction Introduction Background on software testingBackground on software testing Modern views of software testingModern views of software testing Key TerminologyKey Terminology Categories of software testingCategories of software testing OO software testing environmentOO software testing environment Lessons Learned; Object oriented testingLessons Learned; Object oriented testing ConclusionConclusion

33

What is Software Testing?What is Software Testing?

Process of examining a software product or part of Process of examining a software product or part of a software product to determine whether a a software product to determine whether a software performs as it is expected to performsoftware performs as it is expected to perform

Software testing is a part of quality assuranceSoftware testing is a part of quality assurance

Quality assurance: takes into account all business Quality assurance: takes into account all business processes and not just testingprocesses and not just testing

44

Testing outcomesTesting outcomes

Desired results are / are not achievedDesired results are / are not achieved

Non – desired results are / are not achievedNon – desired results are / are not achieved

Users accept / do not accept productUsers accept / do not accept product

55

History of Software TestingHistory of Software Testing

There are four distinct periods of time that mark the There are four distinct periods of time that mark the development of software testing during the twentieth development of software testing during the twentieth century.century.

During these periods, software testing has changed During these periods, software testing has changed from its earliest days of continuously debugging to the from its earliest days of continuously debugging to the more proactive approaches of today.more proactive approaches of today.

66

Four Periods of DevelopmentFour Periods of Development

1957-1978: Engineers distinguish between testing and 1957-1978: Engineers distinguish between testing and debugging software. This is the first time they were debugging software. This is the first time they were able to prove that a software system can meet able to prove that a software system can meet requirements.requirements.

1979-1982: Engineers began trying to break the 1979-1982: Engineers began trying to break the system to find errors, instead of just debugging the system to find errors, instead of just debugging the known errors.known errors.

77

Four Periods of DevelopmentFour Periods of Development

1983-1987: Engineers began offering ‘observational 1983-1987: Engineers began offering ‘observational periods’ on software, today these periods as called periods’ on software, today these periods as called beta programs.beta programs.

1988-Present: As software projects are more closely 1988-Present: As software projects are more closely managed in terms of financials, risk, and timing, the managed in terms of financials, risk, and timing, the focus has shifted toward detecting errors as soon as focus has shifted toward detecting errors as soon as possible. This is because early detection leads to possible. This is because early detection leads to better products which are often delivered on time and better products which are often delivered on time and with less risk to manage.with less risk to manage.

88

Software Testing: Modern ViewSoftware Testing: Modern View

Software should be tested during different stages of Software should be tested during different stages of system development life cycle.system development life cycle.

Almost all software is developed using a type of Almost all software is developed using a type of iterative beta program where programmers, testers, iterative beta program where programmers, testers, developers, and users are encouraged to break the developers, and users are encouraged to break the software or find software functionality that does not software or find software functionality that does not meet desired results.meet desired results.

It has been proven that it is cheaper to fix defects if It has been proven that it is cheaper to fix defects if detected early in development stages.detected early in development stages.

99

Key Terms: Black Box TestingKey Terms: Black Box Testing

Black, White, or Grey Box Testing refers to the point Black, White, or Grey Box Testing refers to the point of view taken while designing test cases and plans.of view taken while designing test cases and plans.

Black Box Testing (BBT) takes an external view of the Black Box Testing (BBT) takes an external view of the test object.test object.

BBT focuses on inputs and outputs of the system, BBT focuses on inputs and outputs of the system, user functionality of the system, and the higher level user functionality of the system, and the higher level workings of the system.workings of the system.

BBT is often automated and is usually one of the final BBT is often automated and is usually one of the final steps before software is released.steps before software is released.

1010

Key Terms: White Box TestingKey Terms: White Box Testing

White Box Testing (WBT) is focused on the internal White Box Testing (WBT) is focused on the internal aspects of the system.aspects of the system.

WBT test cases and test plans are usually developed WBT test cases and test plans are usually developed by programmers and engineers who understand and by programmers and engineers who understand and want to examine the inner workings of the system.want to examine the inner workings of the system.

WBT is usually among the earliest tests performed in WBT is usually among the earliest tests performed in software development.software development.

WBT can also be used to evaluate and fix system WBT can also be used to evaluate and fix system errors for software that is already in production.errors for software that is already in production.

1111

Key Terms: Grey Box TestingKey Terms: Grey Box Testing

Grey Box Testing (GBT) is a hybrid of the black and Grey Box Testing (GBT) is a hybrid of the black and white box testing.white box testing.

GBT is usually focused around databases and GBT is usually focused around databases and connectivity.connectivity.

GBT often requires the tester to populate a GBT often requires the tester to populate a development database with sample data to evaluate development database with sample data to evaluate the proper workings of the data store.the proper workings of the data store.

Testers for GBT are often proficient with SQL, XML, or Testers for GBT are often proficient with SQL, XML, or other data centric languages / technologies.other data centric languages / technologies.

1212

Three Types of TestingThree Types of Testing

““Component testing tests the minimal software item Component testing tests the minimal software item that can be tested.” that can be tested.” 11

““Component integration testing exposes defects in the Component integration testing exposes defects in the interfaces and interaction between integrated interfaces and interaction between integrated components.” components.” 22

““System testing tests an integrated system to verify System testing tests an integrated system to verify that it meets its requirements.” that it meets its requirements.” 33

A team’s failed integration test…A team’s failed integration test… 1,2,3: 1,2,3: http://en.wikipedia.org/wiki/Software_testinghttp://en.wikipedia.org/wiki/Software_testing

1313

Acceptance TestingAcceptance Testing

““Acceptance testing - allows the end-user or customer Acceptance testing - allows the end-user or customer to decide whether or not to accept the product.” to decide whether or not to accept the product.” 11

““Alpha testingAlpha testing is simulated or actual operational is simulated or actual operational testing by potential users/customers or an testing by potential users/customers or an independent test team at the developers’ site.” independent test team at the developers’ site.” 22

““Beta testingBeta testing comes after alpha testing. Versions of comes after alpha testing. Versions of the software, known as beta versions, are released the software, known as beta versions, are released to a limited audience outside of the company.” to a limited audience outside of the company.” 33

1,2,3: 1,2,3: http://en.wikipedia.org/wiki/Software_testinghttp://en.wikipedia.org/wiki/Software_testing

1414

Regression TestingRegression Testing

Regression testing is often used to re-test functionality Regression testing is often used to re-test functionality that has previously passed system or acceptance that has previously passed system or acceptance testing.testing.

It is being re-tested because of changes to the It is being re-tested because of changes to the software code to ensure that the system still works software code to ensure that the system still works properly and that the changes have not introduced properly and that the changes have not introduced unintentional regression of previously achieved results.unintentional regression of previously achieved results.

A vendor’s failure to thoroughly regression test…A vendor’s failure to thoroughly regression test…

1515

OO Testing Environment … Unique OO Testing Environment … Unique ChallengesChallenges

In an Object Oriented (OO) programming environment, In an Object Oriented (OO) programming environment, individual parts (objects) work together as a system.individual parts (objects) work together as a system.

An object has a set of attribute values that define a An object has a set of attribute values that define a state of the object and the actions the object can do. state of the object and the actions the object can do.

Because objects are the basic building blocks of a Because objects are the basic building blocks of a system, any change to their attribute values system, any change to their attribute values functionality, or accessibility may cause unintended functionality, or accessibility may cause unintended changes downstream in the environment.changes downstream in the environment.

1616

OO testing environment: So what’s OO testing environment: So what’s the problem?the problem?

In the paper, In the paper, Developing an Object-Oriented Software Developing an Object-Oriented Software Testing and Maintenance Environment, Testing and Maintenance Environment, the main the main problems seen during OO testing areproblems seen during OO testing are11:: The understanding problemThe understanding problem Complex interdependencyComplex interdependency Object state behavior testing problemObject state behavior testing problem The tool support problemThe tool support problem

1: 1: Article CitationArticle Citation

1717

The Understanding Problem The Understanding Problem 11

It becomes difficult to decipher to process / data flow It becomes difficult to decipher to process / data flow when encapsulation and information hiding are used.when encapsulation and information hiding are used.

Often multiple objects are used with multiple classes Often multiple objects are used with multiple classes creating a system that can use an exponentially large creating a system that can use an exponentially large number of object combinations. number of object combinations.

1,2: 1,2: Article CitationArticle Citation

1818

The Understanding Problem The Understanding Problem 11

Understanding the relationships between objects is Understanding the relationships between objects is important for a successful test.important for a successful test.

““Although each of the member functions is very small Although each of the member functions is very small (a total of 15 lines of code, or an average of 5 lines per (a total of 15 lines of code, or an average of 5 lines per member function), it took, on average, 0.95, 0.93, and member function), it took, on average, 0.95, 0.93, and 0.79 person-hours to prepare the test data, test driver, 0.79 person-hours to prepare the test data, test driver, and test stub, respectively.” and test stub, respectively.” 22

1,2: 1,2: Article CitationArticle Citation

1919

Complex Dependency Problem Complex Dependency Problem 11

Analysts and testers spend large amounts of time Analysts and testers spend large amounts of time trying to understand classes in large OO program and trying to understand classes in large OO program and determine if the classes depend on other classes.determine if the classes depend on other classes.

““Without sufficient insight, a tester may not know where Without sufficient insight, a tester may not know where to start testing an OO library.” to start testing an OO library.” 22

1,2: 1,2: Article CitationArticle Citation

2020

Complex Dependency Problem Complex Dependency Problem 11

““It is difficult to identify and test the effect of It is difficult to identify and test the effect of polymorphism and dynamic binding.” polymorphism and dynamic binding.” 22

““It is difficult to identify change impact in OO It is difficult to identify change impact in OO maintenance, since the impact may ripple throughout maintenance, since the impact may ripple throughout the OO program through the complex dependencies.” the OO program through the complex dependencies.” 33

1,2,3: 1,2,3: Article CitationArticle Citation

2121

The State Behavior The State Behavior Testing Problem Testing Problem 11

What an object returns is very dependant on the state What an object returns is very dependant on the state of the object when it is told to act.of the object when it is told to act.

It is erroneous to assume a product works fine by just It is erroneous to assume a product works fine by just testing individual operations without evaluating the testing individual operations without evaluating the object’s state.object’s state.

Integration testing doesn’t solve this problem; it will just Integration testing doesn’t solve this problem; it will just test that individual member functions are interfacing test that individual member functions are interfacing correctly.correctly. 1: 1: Article CitationArticle Citation

2222

The Tool Support Problem The Tool Support Problem 11

Conventional testing methods used by commercially Conventional testing methods used by commercially available software may not necessarily deal with available software may not necessarily deal with problems identified in object oriented testing.problems identified in object oriented testing.

It is better to have a formal specification language that It is better to have a formal specification language that will be used by a testing tool to derive test cases will be used by a testing tool to derive test cases customized to meet software specifications.customized to meet software specifications. 1: 1: Article CitationArticle Citation

2323

An OO Testing SituationAn OO Testing Situation

““Central to the environment is a mathematically defined Central to the environment is a mathematically defined test model, consisting of three types of diagrams:” test model, consisting of three types of diagrams:” 11

ORD: Object relation diagram - represents inheritance, ORD: Object relation diagram - represents inheritance, aggregation, association aggregation, association 22

BBD: block branch diagram - represents the control BBD: block branch diagram - represents the control structure of a member function and its interface to structure of a member function and its interface to other member functions other member functions 33

OSD: Object state diagram - represents the state OSD: Object state diagram - represents the state behavior of an object class behavior of an object class 44

1,2,3,4: Article Citation 1,2,3,4: Article Citation

2424

Advantages of OO Test ModelAdvantages of OO Test Model

It uses graphical models to explain the relationships It uses graphical models to explain the relationships between classes and objects. between classes and objects.

It accelerates preparations by helping the tester / It accelerates preparations by helping the tester / analyst work more efficient.analyst work more efficient.

It assists with defining and analyzing OO testing specs.It assists with defining and analyzing OO testing specs. ““It facilitates the development of the various algorithms It facilitates the development of the various algorithms

and capabilities for OOTM.” and capabilities for OOTM.” 11

1: Article Citation1: Article Citation

2525

Software Testing as an IndustrySoftware Testing as an Industry

Software testing has grown and is continuing to Software testing has grown and is continuing to develop as an industry.develop as an industry.

Larger companies have teams and / or departments Larger companies have teams and / or departments that are solely focused on testing.that are solely focused on testing.

Often firms utilize automated testing tools to Often firms utilize automated testing tools to supplement human testing.supplement human testing.

2626

Testing Industry: CompaniesTesting Industry: Companies

http://www-306.ibm.com/software/rational/offerings/testhttp://www-306.ibm.com/software/rational/offerings/testing.htmling.html

http://www-128.ibm.com/developerworks/rational/kits/tehttp://www-128.ibm.com/developerworks/rational/kits/tester/?S_TACT=105AGX23&S_CMP=SWQster/?S_TACT=105AGX23&S_CMP=SWQ

IBM will provide consulting or testing packages.IBM will provide consulting or testing packages.

2727

Automated Test SoftwareAutomated Test Software

http://www.surgient.com/solutions/devtest/?gclid=CL-Phttp://www.surgient.com/solutions/devtest/?gclid=CL-P2f73sIgCFQ5_UAodB1er3g2f73sIgCFQ5_UAodB1er3g

http://www.seapine.com/products.htmlhttp://www.seapine.com/products.html

Seapine offers a full suite of software development Seapine offers a full suite of software development tools supporting the entire development life cycle.tools supporting the entire development life cycle.

2828

ConclusionsConclusions

Test early and test oftenTest early and test often Integrate the application development and testing life cycle. Integrate the application development and testing life cycle.

You'll get better results and you won't have to mediate You'll get better results and you won't have to mediate between two armed camps in your IT shop.between two armed camps in your IT shop.

Formalize a Formalize a testingtesting methodology; you'll test everything the methodology; you'll test everything the Same way and you'll get uniform results. Same way and you'll get uniform results.

Develop a comprehensive test plan; it forms the basis for Develop a comprehensive test plan; it forms the basis for the vesting methodology. the vesting methodology.

Use both static and dynamic Use both static and dynamic testingtesting. .

2929

ConclusionsConclusions

Define your expected results. Define your expected results. Understand the business reason behind the Understand the business reason behind the

application. You'll write a better application and application. You'll write a better application and better better testingtesting scripts. scripts.

Review and inspect the work; it will lower costs. Review and inspect the work; it will lower costs. Use multiple levels and types of Use multiple levels and types of testingtesting

(regression, systems, integration, load). (regression, systems, integration, load). Don't let your programmers check their own work; Don't let your programmers check their own work;

they'll miss their own errorsthey'll miss their own errors

3030

QuestionsQuestions

Do you have any questions?Do you have any questions?

3131