Software testing techniques Automated software testing

  • Published on

  • View

  • Download

Embed Size (px)


Software testing techniques Automated software testing. Presentation on the seminar Kaunas University of Technology. Topics. 2. Automated software testing. What? Why? Automated vs. Manual Common types of automated testing methods Automated software testing framework Summary. - PowerPoint PPT Presentation


<ul><li><p>Software testing techniques</p><p>Automated software testingPresentation on the seminar</p><p>Kaunas University of Technology</p><p>Testing based on specifications</p></li><li><p>TopicsAutomated software testing. What? Why?Automated vs. ManualCommon types of automated testing methodsAutomated software testing frameworkSummary*</p><p>Testing based on specifications</p></li><li><p>What is automated software testing?A process which covers:the use of software to control the execution of teststhe comparison of actual outcomes to predicted outcomesthe setting up of test preconditionsother test control and test reporting functions. *</p><p>Testing based on specifications</p></li><li><p>Why testing is critical?*</p><p>Testing based on specifications</p></li><li><p>Why automated testing?A time savings translates directly into cost savingsImproves testing productivityImproves accuracyIncreases test coverageDoes what manual testing cannot do</p><p>*</p><p>Testing based on specifications</p></li><li><p>Effort and CostLets assume 6 test casesEffort required to run all 6 manually =&gt; 10 minEffort required to write unit tests for all 6 cases =&gt; 10 minEffort required to run unit tests for all 6 cases =&gt; &lt; 1 minNumber of testing iterations =&gt; 5Total manual testing time =&gt; 50 minTotal unit testing time =&gt; 10 min</p><p>*</p><p>ReleaseManual TestAuto TestManual Test Cumulative1101010210020310030410040510050</p><p>Testing based on specifications</p></li><li><p>Automated vs. Manual testing</p><p>Not all tests can be automated and most times is difficult to decide what to automate and what to manually test. </p><p>*</p><p>Testing based on specifications</p></li><li><p>Advantages*</p><p>Automated TestingManual Testing Need to run a set of tests repeatedly Test cases have to be run a small number of times Helps performing "compatibility testing" (on different configurations and platforms) Allows the tester to perform more specific testsLong term costs are reduced Short term testing costs are reduced Possible to run regressions on a code that is continuously changing and in shorter time The more time tester spends testing a module the grater chance to find real bugs</p><p>Testing based on specifications</p></li><li><p>Disadvantages &amp; other factors*</p><p>Automated TestingManual Testing It's more expensive to automate (bigger initial investments) Manual tests can be very time consuming You cannot automate everything, some tests still have to be done manually For every release you must rerun the same set of tests which can be tiresome</p><p>Other Factors The performance of your test tools The knowledge level of your testing team The continuous growth of software to be tested</p><p>Testing based on specifications</p></li><li><p>Common types of automated testing methodsMonkey testingCapture / playbackCode-based (Unit) testingIntelligent test automation*</p><p>Testing based on specifications</p></li><li><p>Monkey testingRandomly selecting inputs from a large range of values and monitoring if exceptions are thrown.For example, a monkey test can enter random strings into text boxes to ensure handling of all possible user input.It applies not only for GUI or WEB testing, but also for Unit testing.*</p><p>Testing based on specifications</p></li><li><p>Tools example Monkey Android testing tools. The simplest way to use the monkey is with the following command, which will launch the application and send 500 pseudo-random events to it.</p><p>$ adb shell monkey -v -p 500*</p><p>Testing based on specifications</p></li><li><p>Capture / playbackIts a set of software programs that capture user inputs and stores it into a script for later replay.Repeated testing can be performed quickly.Does not require programming skills.When the GUI changes, input sequences previously recorded may no longer be valid.Difficult to determine location of bugs.*</p><p>Testing based on specifications</p></li><li><p>Tools exampleAutohotkey - Free keyboard macro program. Supports hotkeys for keyboard, mouse. </p><p>*</p><p>Testing based on specifications</p></li><li><p>Code-based (Unit) testingIndividual units of source code are tested to determine if they fit for use.Ideally, each test case is independent from the othersBugs can be found in early development stageEasy to test boundary casesNot effective for the integrated system testing*</p><p>Testing based on specifications</p></li><li><p>Unit test-case generation methodRandom</p><p>Genetic</p><p>Other*</p><p>Testing based on specifications</p></li><li><p>RandomInput values generated randomlyRun with generated inputsUser checks output values (pass/fail) and saves them for later automatic checking (Oracle).</p><p>*</p><p>Testing based on specifications</p></li><li><p>GeneticThis methods approach is to combine JML (JAVA model language) and genetic algorithms. JUnit is used as a test execution platform.*</p><p>Testing based on specifications</p></li><li><p>Tools examplesParasoft Jtest is an automated Java testing and static code analysis product</p><p>TestGen is a collection of open-source tools that automatically generates unit test cases.</p><p>*</p><p>Testing based on specifications</p></li><li><p>Intelligent test automationA model-based method for generating tests from a description of an applications behavior (UML)Models are simple to modify, so model-based tests are economical to maintain over the life of a project. Most of these methods are researched only theoretically.</p><p>*</p><p>Testing based on specifications</p></li><li><p>OCL + UML Inputs generatorUML + OCLTested softwareOracleReportInputsConstraintsConstraintsOutputspass / fail*</p><p>Testing based on specifications</p></li><li><p>Automated testing frameworkA framework is an integrated system that sets the rules of automation of a specific product. This system integrates the function libraries, test data sources, object details and various reusable modules. The framework provides the basis of test automation and simplifies the automation effort.*</p><p>Testing based on specifications</p></li><li><p>Common types of frameworksData-driven testing</p><p>Keyword-driven testing</p><p>Hybrid testing*</p><p>Testing based on specifications</p></li><li><p>Data-driven testingData-focused automationUser defines just data sets to run tests withData is defined in external data source and de-coupled from scriptGood fit for a single application testing, featuring multi-environment, big datasets, and rarely changing Test Cases*</p><p>+-Good usability and re-usabilityQuality and coverage strongly depends on the automation skills of implementing personGood test flow coverageContinuous maintenance issues due to a big amount of the codeMulti-environment supportManual validation is required</p><p>Testing based on specifications</p></li><li><p>ExampleData setTest script*</p><p>Testing based on specifications</p></li><li><p>Keyword-driven testingGood fit for a multi-application testing featuring multi-environment, big datasets, and a big number of short and straight Test CasesUsers de-compose test cases into reusable action keywordsNon-coders are able to author automated test cases with action keywords*</p><p>+-Test development doesnt require programming skillsDue to keywords limit doesnt allow covering very complicated Test CasesGood test flow coverageMeta-language specific training is required for staffCompact codeManual validation is required</p><p>Testing based on specifications</p></li><li><p>Example*</p><p>Testing based on specifications</p></li><li><p>Hybrid testingCombines the best of data-driven and keyword-driven testing:User defines data sets to run tests withUser also defines flow control through action keywordsData is separated from script and stored in external data source with action keywords*</p><p>+-Test Plan / Test Scenario creation doesnt require programming or training on the Testing ToolRequires initial investment for design and implementation of the frameworkCompact code and scalable architectureBuilt-in strict validation</p><p>Testing based on specifications</p></li><li><p>Framework componentsTest generation (data and script generators).Tools might create specialized data such as randomized email messages, or populate databases etc.System configuration. Tools might preserve or reproduce system parameters.Test execution. Tools might operate the software itself, either simulating a user working through the GUI and using an alternative testable interface.Oracles. An oracle is any mechanism by which we detect failure or success. Tools might automatically detect certain kinds of error conditions in a product. Activity recording &amp; coverage analysis. Tools might watch testing as it happens and retrospectively report what was and was not tested. Test management. Tools might record test results, organize test ideas or metrics.*</p><p>Testing based on specifications</p></li><li><p>General framework structure*</p><p>Testing based on specifications</p></li><li><p>Common functional automated test tools*</p><p>Testing based on specifications</p></li><li><p>Tools examples*</p><p>Testing based on specifications</p></li><li><p>SummaryAutomated testing improves productivity, accuracy, increases test coverage, saves time and costNeed to decide what to automate and what to manually testThe framework provides the basis of test automation and simplifies the automation effort.</p><p>*</p><p>Testing based on specifications</p></li><li><p>More information</p><p>*</p><p>Testing based on specifications</p></li><li><p>Thank you!*</p><p>Testing based on specifications</p></li><li><p>Question ???What is the benefit of automated testing?What are the common types of automated testing methods?What are the common types of frameworks?What components come to framework?What are the examples of automated testing tools?*</p><p>Testing based on specifications</p><p>**</p></li></ul>