e Babenskas Automated Software Testing

7/23/2019 e Babenskas Automated Software Testing

1/36

Automated

Automatedsoftwaresoftware

testingtestingEgidijusBabenskas,IFM-0/2

1


2/36

Toi!s

0Automated software testing" #$at%#$&%

0Automated 's" Manua(0)ommon t&es of automated testing

met$ods

0Automated software testing framework0*ummar&

2


3/36

#$at is automated software

testing%A ro!ess w$i!$ !o'ers+0t$e use of software to !ontro( t$eee!ution of tests

0t$e !omarison of a!tua( out!omes toredi!ted out!omes

0t$e setting u of test re!onditions0ot$er test !ontro( and test reortingfun!tions"


4/36

#$& testing is !riti!a(%

.


5/36

#$& automated testing%

0A time sa'ings trans(ates dire!t(& into !ostsa'ings

0Imro'es testing rodu!ti'it&

0Imro'es a!!ura!&

0In!reases test !o'erage

0oes w$at manua( testing !annot do


6/36

Eort and )ost0

et3s assume 4 test !ases0 Eort re5uired to run a(( 4 manua((& 67 10 min

0 Eort re5uired to write unit tests for a(( 4 !ases 67 10 min

0 Eort re5uired to run unit tests for a(( 4 !ases 67 8 1 min

0 9umber of testing iterations 67

0Tota( manua( testing time 67 0 min0Tota( unit testing time 67 10 min

Release Manual Test Auto Test

Manual Test

Cumulative

1 10 10 10

2 10 0 20

3 10 0 30

4 10 0 40

5 10 0 50

4


7/36

Automated 's" Manua(

testing

09ot a(( tests !an be automated andmost times is di:!u(t to de!ide w$atto automate and w$at to manua((&test"

;


8/36

Ad'antagesAutomated Testing Manual Testing

< 9eed to run a set of testsreeated(&

< Test !ases $a'e to be runa sma(( number of times

< =e(s erforming

>!omatibi(it& testing> ?ondierent !on@gurations and(atforms

< A((ows t$e tester to

erform more se!i@! tests


9/36

isad'antages D ot$er

fa!torsAutomated Testing Manual Testing< Its more eensi'e toautomate ?bigger initia(in'estments

< Manua( tests !an be'er& time !onsuming

< ou !annot automate

e'er&t$ing, some testssti(( $a'e to be donemanua((&

< For e'er& re(ease &ou

must rerun t$e same setof tests w$i!$ !an betiresome

Other Factors

< T$e erforman!e of &our test too(s< T$e know(edge (e'e( of &our testing team< T$e !ontinuous growt$ of software to betested

G


10/36

)ommon t&es of automated

testing met$ods0Monke& testing

0)ature / (a&ba!k

0)ode-based ?Hnit testing

0Inte((igent test automation

10


11/36

Monke& testing

0andom(& se(e!ting inuts from a (arge rangeof 'a(ues and monitoring if e!etions aret$rown"

0For eam(e, a monke& test !an enterrandom strings into tet boes to ensure$and(ing of a(( ossib(e user inut"

0 It a(ies not on(& for JHI or #EB testing, buta(so for Hnit testing"

11


12/36

Too(s eam(e

MonkeyK Android testing too(s"

T$e sim(est wa& to use t$e monke&

is wit$ t$e fo((owing !ommand, w$i!$wi(( (aun!$ t$e a(i!ation and send 00seudo-random e'ents to it"

L adb s$e(( monke& -' - &our"a!kage"name00

12


13/36

)ature / (a&ba!k

0It3s a set of software rograms t$at !atureuser inuts and stores it into a s!rit for(ater re(a&"

eeated testing !an be erformed 5ui!k(&"

oes not re5uire rogramming ski((s"

- #$en t$e JHI !$anges, inut se5uen!es

re'ious(& re!orded ma& no (onger be 'a(id"- i:!u(t to determine (o!ation of bugs"

1


14/36

Too(s eam(e

Autohotkey - Free ke&board ma!ro rogram"*uorts $otke&s for ke&board, mouse"

1.


15/36

)ode-based ?Hnit testing

0Indi'idua( units of sour!e !ode are tested todetermine if t$e& @t for use"

0Idea((&, ea!$ test !ase is indeendent fromt$e ot$ers

Bugs !an be found in ear(& de'e(omentstage

Eas& to test boundar& !ases- 9ot ee!ti'e for t$e integrated s&stem

testing

1


16/36

Hnit test-!ase generation

met$od0andom

0Jeneti!

0Nt$erO

14


17/36

andom

0Inut 'a(ues generated random(&

0un wit$ generated inuts

0Hser !$e!ks outut 'a(ues ?ass/fai(and sa'es t$em for (ater automati!!$e!king ?Nra!(e"

1;


18/36

Jeneti!T$is met$ods aroa!$ is to !ombine PM ?PAQAmode( (anguage and geneti! a(gorit$ms" PHnitis used as a test ee!ution (atform"

1C


19/36

Too(s eam(es

Parasoft Jtest

is anautomated Pa'atesting and stati!!ode ana(&sisrodu!t

TestGenis a !o((e!tion of oen-sour!e

too(s t$at automati!a((& generates unittest !ases"

1G


20/36

Inte((igent test

automation0A mode(-based met$od for generating tests from

a des!rition of an a(i!ation3s be$a'ior ?HM

Mode(s are sim(e to modif&, so mode(-basedtests are e!onomi!a( to maintain o'er t$e (ife ofa roje!t"

- Most of t$ese met$ods are resear!$ed on(&

t$eoreti!a((&"

20


21/36

N) HM

Inutsgenerator

HM

N)

Tested

software

Nra!(e

eort

Inuts )onstraints

)onstraintsNututsass /fai(

21


22/36

Automated testing

framework0A framework is an integrated s&stem

t$at sets t$e ru(es of automation of ase!i@! rodu!t"

0T$is s&stem integrates t$e fun!tion(ibraries, test data sour!es, obje!t detai(sand 'arious reusab(e modu(es"

0T$e framework ro'ides t$e basis of testautomation and sim(i@es t$eautomation eort"

22


23/36

)ommon t&es of

frameworks0ata-dri'en testing

0Re&word-dri'en testing

0=&brid testing

2


24/36

ata-dri'en testing

0ata-fo!used automation0Hser de@nes just data sets to run tests wit$

0ata is de@ned in eterna( data sour!e andde-!ou(ed from s!rit

0Jood @t for a sing(e a(i!ation testing,featuring mu(ti-en'ironment, big datasets,and rare(& !$anging Test )ases

+ -

Jood usabi(it& and re-usabi(it&

Sua(it& and !o'erage strong(& deends ont$e automation ski((s of im(ementing erson

Jood test ow !o'erage )ontinuous maintenan!e issues due to a bigamount of t$e !ode

Mu(ti-en'ironmentsuort Manua( 'a(idation is re5uired2.


25/36

Eam(eata set

Tests!rit

2


26/36

Re&word-dri'en testing

0Jood @t for a mu(ti-a(i!ation testingfeaturing mu(ti-en'ironment, big datasets, anda big number of s$ort and straig$t Test )ases

0Hsers de-!omose test !ases into reusab(e

a!tion ke&words09on-!oders are ab(e to aut$or automated test

!ases wit$ a!tion ke&words

+ -

Test de'e(oment doesn3tre5uire rogramming ski((s

ue to ke&words (imit doesn3t a((ow!o'ering 'er& !om(i!ated Test )ases

Jood test ow !o'erage Meta-(anguage se!i@! training isre5uired for sta

)oma!t !ode Manua( 'a(idation is re5uired 24


27/36

Eam(e

2;


28/36

=&brid testing0

)ombines t$e best of data-dri'en and ke&word-dri'en testing+0Hser de@nes data sets to run tests wit$

0Hser a(so de@nes ow !ontro( t$roug$ a!tion ke&words

0ata is searated from s!rit and stored in eterna(data sour!e wit$ a!tion ke&words

+ -

Test (an / Test *!enario !reation doesn3tre5uire rogramming or training on t$e

Testing Too(

e5uires initia( in'estmentfor design andim(ementation of t$eframework

)oma!t !ode and s!a(ab(e ar!$ite!ture

Bui(t-in stri!t 'a(idation2C


29/36

Framework !omonents0 Test generation (data and script generators!Too(s mig$t

!reate se!ia(iUed data su!$ as randomiUed emai( messages, orou(ate databases et!"

0 "ystem con#guration!Too(s mig$t reser'e or rerodu!es&stem arameters"

0 Test e$ecution!Too(s mig$t oerate t$e software itse(f, eit$ersimu(ating a user working t$roug$ t$e JHI and using ana(ternati'e testab(e interfa!e"

0 Oracles! An ora!(e is an& me!$anism b& w$i!$ we dete!tfai(ure or su!!ess" Too(s mig$t automati!a((& dete!t !ertainkinds of error !onditions in a rodu!t"

0 Acti%ity recording & co%erage analysis!Too(s mig$t wat!$testing as it $aens and retrose!ti'e(& reort w$at was andwas not tested"

0 Test management!Too(s mig$t re!ord test resu(ts, organiUetest ideas or metri!s"

2G

J ( f k


30/36

Jenera( frameworkstru!ture

0

)ommon fun!tiona( automated test


31/36

)ommon fun!tiona( automated testtoo(s

Tool + -

IBM /

ationa(Fun!tiona(

Tester?FT

Bui(t as E!(ise (ug-in wit$ fu((

IE, Pa'a suort and sour!emanagement

*uorts #eb-baseda(i!ations, "9et, Pa'a, *iebe(,*AV, owerBui(der, APAW, AdobeF(e, Adobe F do!uments

Fu(( JHI Nbje!t Ma reositor&

i!ense !ost

= /Mer!ur&Sui!k Testro ?ST

*uorts #eb 2"0, J#T, ojo,HI, *i('er(ig$t, F(e, Aja, PAQA,"9ET a(i!ations

Fu(( JHI Nbje!t Ma reositor&

*eam(ess integration wit$Sua(it&)enter

9o IE?!$anging inre(ease 10

i!ense !ost

Qisua(Basi!s!riting is (imited

*e(enium) D IE

Jood browser suortJood (anguage suort ?PAQA,

ub&, )X)an be easi(& etended as PHnit

suite -

9o JHI Nbje!treositor&

Nn(& web-baseda(i!ation suort

Too( suort is1

T ( (


32/36

Too(s eam(es

2


33/36

*ummar&

0Automated testing imro'es rodu!ti'it&,a!!ura!&, in!reases test !o'erage, sa'estime and !ost

09eed to de!ide w$at to automate and w$atto manua((& test

0T$e framework ro'ides t$e basis of test

automation and sim(i@es t$e automationeort"


34/36

More information0 $tt+//www"s(ides$are"net/(@ngerman/test-automation-best-r

!ati!es-wit$-soa-test-aroa!$0 $tt+//ountia"!s"ute"edu/utjm(/utest"$tm(

0 $tt+//www"s(ides$are"net/&ogindernat$/understanding-of-automation-framework-resentation

0 $tt+//www"ibm"!om/de'e(oerworks/rationa(/(ibrar&/G1"$tm(

0 $tt+//www"satis@!e"!om/resentations/agi(eauto"df

0 $tt+//automation-be&ond"!om/200G/0/0/test-automation-ra!ti!es-re!ord(a&ba!k/

0 $tt+//en"wikiedia"org/wiki/TestYautomation0 $tt+//www"s(ides$are"net/dire!ti/automated-testing-'s-

manua(-testing-G1121

.
http://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.ibm.com/developerworks/rational/library/591.htmlhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://www.slideshare.net/yogindernath/understanding-of-automation-framework-presentationhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.htmlhttp://opuntia.cs.utep.edu/utjml/utest.html


35/36

T$ank &ouZT$ank &ouZ


36/36

Suestion %%%Suestion %%%

0#$at is t$e bene@t of automated testing%

0#$at are t$e !ommon t&es of

automated testing met$ods%0#$at are t$e !ommon t&es of

frameworks%

0#$at !omonents !ome to framework%

0#$at are t$e eam(es of automatedtesting too(s%

4

Documents

e Babenskas Automated Software Testing