31
Edgars Diebelis Prof. Dr. sc. comp. Jānis Bičevskis This work has been supported by the European Social Fund within the project «Support for Doctoral Studies at University of Latvia».

Software Self-Testing

  • Upload
    ranit

  • View
    21

  • Download
    0

Embed Size (px)

DESCRIPTION

Software Self-Testing. This work has been supported by the European Social Fund within the project « Support for Doctoral Studies at University of Latvia ». Edgars Diebelis Prof. Dr. sc. comp. Jānis Bičevskis. Self-Testing. - PowerPoint PPT Presentation

Citation preview

Page 1: Software Self-Testing

Edgars DiebelisProf. Dr. sc. comp. Jānis Bičevskis

This work has been supported by the European Social Fund within the project «Support for Doctoral Studies at University of Latvia».

Page 2: Software Self-Testing

Ability to execute stored test examples to ensure that software functionality is correct.

Testing approach◦Manual testing;◦Automated testing with testing tools;◦Self-Testing.

2

Page 3: Software Self-Testing

Problem statement. Concept of Self-Testing. Implementation of Self-Testing. Comparison of the Self-Testing Concept with

Conventional Testing Support Tools. Efficiency Measurements of Self-testing. Conclusions.

3

Page 4: Software Self-Testing

„Computing systems’ complexity appears to be approaching the limits of human capability”

(Kephart, J., O., Chess, D., M.). IBM Autonomic Computing Manifest, 2001

◦ outlined four key features that characterise autonomic computing.

Smart Technology approach, 2007◦ identified seven types of the smart technology;◦ self-testing is one of the type of the smart

technology.

4

Page 5: Software Self-Testing
Page 6: Software Self-Testing

Self-testing contains two components:

◦ Test cases of system’s critical functionality to check functions, which are substantial for system using;

◦ Built-in mechanism (software component) for automated software testing (regression testing) that provides automated running of test cases and comparing test results with standard values.

6

Page 7: Software Self-Testing

Test storage file (XML)

7

Test storage mode

Page 8: Software Self-Testing

Test storage file (XML)

Self-Testing tool

Test storage file (XML)

8

Self-Testing mode

Page 9: Software Self-Testing

Test point is a programming language command in the software text.

Test point ensures that:◦ particular actions and field values are saved when

storing tests;◦ the software execution outcome is registered

when tests are executed repeatedly. By using test points, it is possible to repeat

the execution of system events.

9

Test Points

Page 10: Software Self-Testing

Test point types:◦ input field test point;◦ comparable value test point;◦ system message test point;◦ SQL query result test point;◦ application event test point;◦ test execution criteria test point;◦ self-testing test points;◦ etc.

10

Test Point Types

Page 11: Software Self-Testing
Page 12: Software Self-Testing

Self-Testing tool modules:◦ Self-Testing module;◦ Self-testing test management module.

12

Build of Self-Testing

Page 13: Software Self-Testing

13

Self-Testing in use

Page 14: Software Self-Testing

14

Page 15: Software Self-Testing
Page 16: Software Self-Testing

Automated Testing Institute (ATI) opinion. “ATI Automation Honors ” awards. Since May 2009, the ATI has been publishing

its magazine Automate Software Testing. Website has a list of 716 automated testing

tools. ATI annual conference on automated testing

(Verify/ATI). ~8000 registred users.

16

Testing Tools Selection

Page 17: Software Self-Testing

TestComplete the best commercial automated functional testing tool in 2010

FitNesse the best open source code automated functional testing tool in the NET

sub-category in 2010 Ranorex

the best commercial automated functional testing tool in the NET and Flash/Flex sub-categories in 2010

T-Plan Robot the best open source code automated functional testing tool in the Java

sub-category in 2010 Rational Functional Tester

in 2009 and 2010, it was a finalist among the best commercial automated functional and performance testing tools

HP Unified Functional Testing Software the best commercial automated functional testing tool in 2009

Selenium the best open source code automated functional testing tool in 2009 and

2010

17

Testing Tools

Page 18: Software Self-Testing

Testing method (TM) Test automation approach (TAA) Test automation framework (TAF) Testing level Test recording and playback Desktop applications testing Web applications testing Services testing Database testing Testing in production environment System user can create tests Simultaneous running of several

tests

Performing simultaneous actions Identifying the tested object Test result analysis Test editing Screenshots Control points Object validation Object browser Test log Test schedule planner Identification of the end of

command execution Plug-ins and extensions etc.

18

Criteria for Comparison

Page 19: Software Self-Testing

19

Comparison

Page 20: Software Self-Testing

20

Comparison II

Page 21: Software Self-Testing

New Test automation frameworks. Test editor and log. Object validation and object browser. Load, stress and other testing levels. Web applications and services testing. Additional platforms. Plug-ins and extensions. Integration with external environment

testing.

21

Self-Testing further development

Page 22: Software Self-Testing

White box testing. Testing in production environment. Users without in-depth IT knowledge to

define and run test cases. Testing external interfaces. Perform system testing without specific

preparation for running the test.

22

Self-Testing advantages

Page 23: Software Self-Testing
Page 24: Software Self-Testing

Retrospective analysis of incident notifications in a real project.

It is not possible to apply and compare two different concepts in the same conditions.

Analysis of incident notifications (1,171 in total) in the CSAS in the period from July 2003 to 23 August 2011.

Subjective opinion; however, the high number of incident notifications and the statistics do reflect trends.

24

Efficiency Measurements

Page 25: Software Self-Testing

25

Statistics of Incident NotificationsType of Incident Quantity % of total Hours % of total

Duplicate 68 5.81 23.16 0.47

User error 43 3.67 67.46 1.37

Unidentifiable bug 178 15.2 1011.96 20.52

Identifiable bug 736 62.85 3293.74 66.79

Improvement 102 8.71 241.36 4.89

Consultation 44 3.76 293.92 5.96

Total: 1171 100 4931.6 100

Page 26: Software Self-Testing

26

Bugs Unidentifiable by the Self-testingBug type Quantity % of total

External interface bug 5 2.81

Computer configuration bug 12 6.74

Data type bug 7 3.93

User interface bug 25 14.04

Simultaneous actions by users 5 2.81

Requirement interpretation bug 41 23.03

Specific event 83 46.63

Total: 178 100

Page 27: Software Self-Testing

27

Bugs Identifiable by the Self-testingTest point

Quantity % of total Hours % of total

File result test point 59 8.02 150.03 4.56

Input field test point 146 19.84 827.14 25.11

Application event test point 105 14.27 364.24 11.06

Comparable value test point 28 3.8 93.53 2.84

System message test point 11 1.49 58.84 1.79

SQL query result test point 387 52.58 1799.96 54.65

Kopā: 736 100 3293.74 100

Page 28: Software Self-Testing
Page 29: Software Self-Testing

self-testing approach offers not only options equal to those offered by other globally recognized testing support tools; moreover, self-testing additionally offers options that other testing tools do not possess:◦ testing external interfaces;◦ testing in production environment;◦ testing with the white-box method;◦ possibility for users without IT knowledge to capture tests.

Testing support is part of systems developed and it is available throughout the entire life cycle of software.

29

Conclusion

Page 30: Software Self-Testing

Self-testing changes the testing process by considerably broaden the role of the developer in software testing.

Self-testing requires additional work to include the self-testing functionality in the software and to develop critical functionality tests.

Self-testing saves time by repeated (regression) testing of the existing functionality.

Implementation of the self-testing functionality is useful in incremental development models, in particular in systems that are gradually improved and maintained for many years.

30

Conclusion II

Page 31: Software Self-Testing

31