2012 10 23_3013_rational_integration_tester_fo

PerformanceTestingwithIBMRationalIntegrationTester

Note

Beforeusingthisinformationandtheproductitsupports,readtheinformationin“LegalNotices”onpage35.

©CopyrightIBMCorporation2001,2012.

Page1of36©IBMCorporation2001,2012

1 INTRODUCTION ............................................................................................................................. 3

2 BACKGROUND .............................................................................................................................. 4

3 PERFORMANCE TEST INFRASTRUCTURE ......................................................................................... 5

3.1 INTRODUCTION ....................................................................................................................... 5

3.2 ENGINES ................................................................................................................................ 6

3.3 PROBES ................................................................................................................................ 7

3.4 AGENTS ................................................................................................................................. 7

4 ARCHITECTURE SCHOOL ............................................................................................................... 9

4.1 INTRODUCTION ....................................................................................................................... 9

4.2 BASIC SYSTEM SETUP ............................................................................................................ 9

4.3 AGENT & ENGINE SETUP ....................................................................................................... 10

4.4 PROBE SETUP ...................................................................................................................... 10

5 CREATING THE LOAD GENERATING TEST ...................................................................................... 12

5.1 RE-USING FUNCTIONAL TEST RESOURCES ............................................................................. 12

5.2 BASIC SETUP ....................................................................................................................... 13

5.3 TIMED SECTIONS .................................................................................................................. 14

6 CREATING PERFORMANCE TESTS ................................................................................................ 16

6.1 INTRODUCTION ..................................................................................................................... 16

6.2 INITIAL SETUP ...................................................................................................................... 16

6.3 ADDING TESTS ..................................................................................................................... 17

6.4 ENGINE SETTINGS ................................................................................................................ 18

6.5 MANAGING PROBES .............................................................................................................. 18

7 RUNNING PERFORMANCE TESTS AND ANALYZING RESULTS ........................................................... 20

7.1 RUNNING THE TEST .............................................................................................................. 20

7.2 VIEWING RESULTS ................................................................................................................ 21

7.3 MULTIPLE DATA SETS ........................................................................................................... 23

8 DATA DRIVING PERFORMANCE TESTS .......................................................................................... 25

8.1 DIFFERENCES FROM FUNCTIONAL TESTS ............................................................................... 25

8.2 DRIVING A LOAD GENERATING TEST WITH EXTERNAL DATA ..................................................... 25

9 LOAD PROFILES ......................................................................................................................... 27

9.1 PERFORMANCE TESTING SCENARIOS ..................................................................................... 27

9.2 CONSTANT GROWTH ............................................................................................................ 28

9.3 EXTERNALLY DEFINED LOAD PROFILES .................................................................................. 28

10 ADVANCED TOPICS .................................................................................................................. 30


10.1 BACKGROUND TESTS ........................................................................................................ 30

10.2 LOG MEASUREMENT .......................................................................................................... 30

10.3 CREATING THE MEASUREMENT TEST ................................................................................... 31

10.4 ADDING THE MEASUREMENTS TO A PERFORMANCE TEST ..................................................... 34

11 LEGAL NOTICES ...................................................................................................................... 35


1 Introduction

ThisdocumentservesasatrainingmanualtohelpfamiliarizetheuserwiththeperformancetestingcapabilitiesavailableinIBM®Rational®IntegrationTester.ItisexpectedthatthereaderhasalreadybeenthroughthebasicRationalIntegrationTestertraining,andunderstandstheworkflowofRationalIntegrationTester.

Inthiscoursewewill:

Createperformancetests

Setupagents,probes,andenginestoexecuteandmonitorperformancetests

Analyzeresultsofperformancetests

Managetheamountofloaddrivenbyaperformancetestovertime

Datadriveperformancetests


2 Background

Whentestingaserviceorientedarchitecture(SOA),therewillbetimeswhensimplyverifyingfunctionalrequirementswillnotbeenough.Manysystemswillneedtocomewithservicelevelagreements(SLAs)thatwillstateaminimumlevelofperformancethatmustbesatisfied.

Thislevelofperformancemayhaveanumberofcomponents.Inparticular,systemuptimesandmessageresponsetimeswillbeimportant.However,itwillnotbeenoughtotestthesystemtocheckthatitcanrespondtoasinglemessagewithinagivenamountoftime–thesystemwillneedtoholdupunderacertainamountofloadaswell.Thisloadmaytaketheformofalargenumberofmessages,extrememessagerates,orlargeamountsofdata.Inaddition,accuratelymodelingtheloadonthesystemmayrequireustogeneratemessagerequestsfromanumberofdifferentsources.

Forexperiencedperformancetesters,thiswillallbefairlyfamiliar.However,SOAenvironmentsbringchallengesontopofthetraditionalclient‐servermodel.Forexample,servicesareoftensharedamongseveralapplicationsandfailurecanoccuranywherealongthetransactionpath.Consideringboththenumberofservicesinplaceandthemanypointsatwhichtheyintersect—anyoneofwhichmaynotbeperformant—howcanweensurethatperformancelevelssatisfythenonfunctionalrequirements?

Inaddition,thereisafundamentaldifferencebetweenSOAperformancetestingandatraditional,client‐serverapproach.Performancetesterswhoarefamiliarwiththetraditionalapproachtendtotalkintermsofthenumberofusersor“virtualusers”thatarerequiredtogeneratetheload.Theyalsotendtobeconcernedwithend‐to‐endresponsetimes–theresponsetimeexperiencedbyanenduser.Thisend‐to‐endperformancetestingistypicallyexecutedagainstafunctionallyproven,completesystem.

SOAperformancetestersarestillinterestedinresponsetimesbutwouldbemoreinterestedinthevolumeofmessagessentbetweencomponents–thereisnorequirementtowaituntilthesystemhascompletedassemblyorforafrontendGUIinterfacetobecreated.Hence,theSOAperformancetestercanbegintestingmuchearlier.

Whenrunningperformancetests,youwillnormallybefacedwiththefollowingquestions:

1. Doesthesystem’sperformancesatisfytherequirementsorSLAs?

2. Atwhichpointwilltheperformancedegrade?

3. Canthesystemhandlesuddenincreasesintrafficwithoutcompromisingresponsetime,reliability,andaccuracy?

4. Wherearethesystembottlenecks?

5. Whatisthesystem’sbreakpoint?

6. Willthesystemrecover(andwhen)?

7. Doesthesystemperformancedegradeifrunforanextendedperiodatrelativelylowlevelsofload?

8. Arethereanycapacityissuesthatcomefromprocessinglargeamountsofdata?


3 Performance Test Infrastructure

3.1 Introduction

Beforecreatingperformancetests,wewillneedtorevisehowwecreatetheinfrastructureofRationalIntegrationTester.Asinregularfunctionaltests,wehavetheRationalIntegrationTesterGUI,andtheprojectdatabase.However,thesewillworkslightlydifferentlyinthecontextofaperformancetest.

Whileinaregularfunctionaltest,theGUIandthetestarenormallyrunfromthesamemachine,aperformancetestmayberunfromanothermachine,ormaybedistributedacrossanumberofothermachines.ThismeansthattheRationalIntegrationTestersoftware,aspresentedbytheGUI,alsoprovidesatestcontroller,tomanageanyremotesystemsinvolvedintheperformancetest.

Inaddition,theprojectdatabase,whichisoptionalforafunctionaltest,becomesmandatoryforperformancetests.Thisisduetothehighervolumeofdatathatisrecordedduringaperformancetest–itcannotbeeasilypresentedinasimpleconsolewindow,butwillneedtobesummarized,andpossiblymanipulated.

BesidestheGUI,testcontroller,andprojectdatabase,therearealsothreenewitemsinourinfrastructure:engines,probes,andagents.Theyfittogetherinaframeworktoruntestsandmonitorperformanceacrossanumberofdifferentsystems.

Inthischapter,youwill:

Lookatthedistributednatureofaperformancetestinfrastructure.

Seehowenginesareusedtoexecuteactionswithinaperformancetest

Examinehowdataneedstoberecordedfromthesystemundertest,andhowthiscanbedonewithprobes

Learnhowperformancetestlicensingishandled


3.2 Engines

AnengineistheprocessthatactuallyrunsatestinRationalIntegrationTester.WhencarryingoutfunctionaltestingusingRationalIntegrationTester,anengineexistsunderneaththesurface,andrunsthetestsonbehalfonthecontrollinginstanceofRationalIntegrationTester(ie,theinstanceofRationalIntegrationTesterthatisrunningthemainperformancetest),ontheuser’smachine.

Whenperformancetesting,theengineisseparatedfromthecontrollinginstanceofRationalIntegrationTester.TheenginecanbeonthesamemachineasRationalIntegrationTester,oritcanbeonanothermachine.Infact,theremaybemorethanoneengine,spreadacrossmultiplemachines.

Ifthereismorethanoneengine,testiterationsarespreadacrosstheavailableengines.Forexample,inaperformancetestwhichisexecuting40testspersecondwith2engines,eachenginewouldberunning20testspersecond.ThedistributionofthetestsishandledbythecontrollinginstanceofRationalIntegrationTester.

Usingmultipleenginesletsussolveanumberofproblems.Mostsimply,ifonemachineisnotcapableofgeneratingahighenoughloadforaperformancetest,theloadcanbesplitacrossmultiplemachines.

Secondly,multipleenginesgiveusthecapabilitytodistributetheloadacrossmultipleendpoints.Forexample,ifweneedtosimulaterequestsarrivingfromdifferentpartsoftheworld,orfromdifferentnetworks,wecansetupenginesinsuchlocationsastosatisfythedemandsoftheperformancetest.


3.3 Probes

Withsuchcomplexandheterogeneousplatforms,itcandifficulttounderstandwhattomeasureapartfromtransactionresponsetimes.Itwillbeimpossibletomeasureeverything.

ProbesarethetoolsusedbyRationalIntegrationTestertogatherstatisticsfromthesystemundertest.Thereareavarietyofprobesavailabletotheuser:

SystemStatisticsProbe

WindowsPerformanceMonitorProbe

TIBCOBusinessWorksProbe

TIBCORendezvousProbes

TIBCOEMSProbe

SonicMQProbe

webMethodsBrokerProbe

webMethodsIntegrationServerProbe

JMXProbe

Theprobeswillbedeployedonthesystemsthatyouwanttomeasure,andmultipleprobescanco‐existontheonesystem.

RecordingstatisticswithRationalIntegrationTester’sprobesgivesusaccesstomuchmoreinformationthanjustthetransactionresponsetimes.Thiscanaidustodeterminethecauseofpoorperformance–ifresponsetimesarebecomingtoolongaftergoingpastacertainnumberofrequestspersecond,wecanuseprobestoseeifitisduetoloadontheCPU,excessmemoryusage,messagequeuesgrowinglarger,oranothercause.

Whicheverprobesyouchoose,theywillrecordstatisticsduringtheperformancetest,andsendthosestatisticstothecontroller,forwritingtotheprojectdatabase.Thesewritesaresetupasalow‐prioritytask,sothattheycauseassmallanimpactaspossibleonsystemperformance.

3.4 Agents

Enginesdrivethetests,andprobesmonitorthem.However,bothneedahostcontrollingthem,talkingtotheinstanceofRationalIntegrationTestercontrollingtheperformancetest.ThisroleisplayedbytheRationalIntegrationTesterAgent.

Theagentrunsoneachmachinethathasanengineoraprobe,andhandlesthecommunicationswithRationalIntegrationTester.Theagentcanhostanengine,aprobe,orbothatthesametime.Infact,itcanalsohandlemultipleprobesorengineswithintheoneagent.

TheagentisinstalledwiththeRationalTestPerformanceServer(RTPS),ortheRationalTestVirtualizationServer(RTVS).Itcanberunbyhand,orsetupasaserviceonthesystemit’srunningon.Duetothis,eachmachinethatrequiresanagentrequiresaninstallationofRTPSorRTVS.FollowingtheinstallationofRTPSorRTVS,theagentwillneedtobeconfiguredintheLibraryManager.ThisconfigurationfollowsthesameprocedureasthatofRationalIntegrationTesteritself,andsowillnotbecoveredinthistrainingcourse.


Note:Ifyouarerunningthroughthistrainingmaterialonacloudinstanceorvirtualmachine,allpartsofthesystemwillbeonasinglemachine.Thisispurelyforeaseofconfiguration,anddoesnotreflectareal‐worldscenario.


4 Architecture School

4.1 Introduction

Creatingamodelofthesystemundertestwillbeverysimilarforperformanceteststofunctionaltests.However,inadditiontomodelingthesystemundertest,theArchitectureSchoolperspectivewillalsobeusedtoprovideconfigurationdatafortheagents,engines,andprobesinthesystem.

AddingthisinformationtoyourRationalIntegrationTesterprojectshouldbedoneafterthenormalprocessofmodelingthesystemundertest;configurationfortheperformancetestingcomponentsisthencarriedoutinthePhysicalViewofArchitectureSchool.Notethatasitisconfiguredonaphysicalbasis,youmayneedtoconfigurenewcomponentswhensettingupnewenvironments.

4.2 BasicSystemSetup

Inthisexample,wewillbetestingawebservice–asimpleLoginservicethattakesausernameandpassword,andreturnsalogintoken.Wewillfirststarttheserviceonourlocalmachine,thensynchronizewiththeWSDLprovidedbytheservice.

1. OntheDesktopofthecloudinstance,thereisafoldercalledWebServices.Openthisfolder,andexecuteRunLoginService.bat.

2. Thiswillpopupawindow–keepitopen,butminimizeit.

3. OpenupRationalIntegrationTester,andstartanewproject.Notethatyouwillneedtouseaprojectdatabase–oneisalreadyspecifiedonthecloudinstancebydefault,soyoucankeepusingthis,butusetheTestConnectionoptiontomakesurethatitisworkingcorrectly.Ifyouarenotusingacloudinstance,pleaseaskyourinstructorfortheprojectdatabasesettings.

4. OnceRationalIntegrationTesterisopen,switchtotheSynchronizationviewwithinArchitectureSchool.

5. Pressthe buttoninthetoolbaratthetopoftheSynchronizationview,andselectWSDL.

6. TheCreateanewExternalResourcedialogwillappear.PresstheNew…button.

7. TheNewWSDLdialogwillappear.PresstheSelect…buttontoselectanewlocation.


ConfigureRationalIntegrationTestertoconnecttothesystemundertest

Setupanenginetoruntheperformancetest,withanagenttohostit

Configureaprobetomeasuretheperformanceofthesystemduringtesting


8. OncetheSelectLocationdialogappears,switchtotheURLtab.InordertogettheURLoftheWSDL,copyitfromthewindowthatpoppedupwhenyourantheloginservice.

9. PressOKtoclosetheSelectLocationandNewWSDLdialogs.

10. ClickNext,andrunthroughtherestofthesynchronizationprocessasnormal.

4.3 Agent&EngineSetup

1. Insomecases,theagentmightbeexecutedmanually;however,inthisexample,theagentisrunningasaWindowsserviceonthelocalhost.Thismeansthattheagentcanbefoundatlocalhost:4476.However,inordertoenteritsdetailsproperly,weneedtoknowthenameofthelocalhost.Executethecommandhostnameatacommandprompt.

2. SwitchbacktoRationalIntegrationTester,andgotothePhysicalViewofArchitectureSchool.

3. Pressthe buttonattheleftofthePhysicalViewToolbar,andselecttheAgentoption.

4. IntheHostfield,enterthehostnameprovidedinstep1.ForthePortnumber,makesurethatthedefaultsettingis4476.

5. Anenginecalleddefaultisautomaticallyattachedtotheengine–leavethisas‐is,andpressOKtoclosethedialogandcompletetheagentconfiguration.

4.4 ProbeSetup

We’renowgoingtosetuptheprobethatwewanttorunonthesamemachineastheagent.Rememberthateachprobewillneedtoberunningonanagent,ortestsusingthatprobewillfail.Also,probescanbesetuponindividualhosts,oronservicesrunningonthosehosts–forexample,theSystemStatisticsprobewillrunonaparticularhost,butmostofthetechnology‐specificprobeswillneedtobeattachedtoaparticularprocessonthathost.Ifyouneedtousethoseparticularprobesinthefuture,theycanbeconfiguredbyeditingthepropertiesofthatphysicalcomponent,similartothewaywewilledittheprobeonthehostmachineinthisexercise.

1. InthePhysicalView,eachphysicalcomponentwillbevisibleinatreeunderneathaSubnetandaHost.Doubleclickonthehost(whichshouldhavethehostnameweusedinthepreviousexercise)tobringupitsproperties.Thescreenshotbelowshowswheretofindthehost,thoughyourhostnameandIPwillbedifferent.

2. Oncethepropertiesdialogforyourhosthasappeared,switchtotheProbestab.

3. Forourtests,we’regoingtousetheWindowsPerformanceMonitorprobe.Selectitsothatitcanbeconfigured.


4. Thefirst,andmostimportant,settingtonoteistheHostingAgentattheverybottomofthedialog–ittellsuswhichagentisrunningthisprobe.Currently,weonlyhaveoneagenttodealwith,butmakesurethattheagentforthisprobeissettotheagentcreatedinthepreviousexercise.Iftheagentisnotsethere,thenanyperformanceteststhatattempttousethisprobewillfail.

5. Asfortheothersettings,theprobeshouldbesettoCollectStatisticsEvery1second.

Followingthat,weneedtospecifywhatstatisticsweneedtocollectfromtheWindowsPerformanceMonitor.

6. Pressthe buttontoaddanewcounter.TheAddCountersdialogwillappear.

7. Forourfirstcounters,we’llexaminememorystatistics.Todothis,selectMemoryinthePerformanceObjectfield.UnderCounter,selectAvailable Mbytes,thenpresstheAddbutton.

8. RepeatthisforthePage Faults/seccounter,andanyotherdatayouareinterestedin.

9. NowselectProcessorinthePerformanceObjectfield,andaddthe% Processor Timecounter,alongwithanyothercountersthatareofinterest.

10. PressDonetoreturntotheconfigurationoftheprobe.

11. SelectalloftheCounters,thenpressthe buttontovalidatethateachoneisworking.Thedialogshouldnowappearlikeso:

12. PressOKtoclosethedialog.Oursystemisnowsetuptogatherstatisticsduringperformance

tests.


5 Creating the Load Generating Test

5.1 Re‐usingFunctionalTestResources

OneoftheadvantagesinusingRationalIntegrationTesterforSOAtestingisthatfunctionaltestscaneasilyberefactoredtoberunwithinaperformancescenario.

Thisisimportantbecausewhenevaluatingtheperformanceofthesystem,itisinsufficienttojustsendarequestandmeasurethetimeittakesforaresponsetoarrive.Forexample,ifawebserviceoperationrejectsinputandsendsbackaSOAPFaultmessage,thetimeittakesmaybesignificantlydifferentfromthetimeittakestoproperlyprocessarequestandreturnavalidresponse.Ifatestdoesnottrulyvalidatetheoutcomeofanoperation,itwillprovideaninaccurateviewofthetruesystemperformance.

Inthiscase,wewillcreateasimplefunctionaltestthatwewilluseasthebasisforourperformancetests,toillustratehowthisworks.Thisfunctionaltestwillbeusedasaloadgeneratingtestwithinourmainperformancetest.

Wheneditingtheloadgeneratingtest,thereareseveralnewactionsthatmaybeused.Theseactionswillbeignoredwhenrunningthetestasafunctionaltest–theywillonlybeexecutedwhenitisrunaspartofaperformancetest.

PerformanceActions

BeginTimedSection:Markthebeginningofatimedsectionforaperformancetest.

EndTimedSection:Marktheendofatimedsectionforaperformancetest.

LogMeasurement:Logdatatotheprojectdatabaseduringaperformancetest.


Createafunctionaltestthatcanbeusedasaloadgeneratingtest

Encounterthetestactionscreatedforusewithinperformancetests

Createatimedsectionwithinatesttocapturetimingandstatusinformation


Note:TheInitialise,TestSteps,andTearDownsectionsbecomemoreimportantinperformancetests.Whenwearerunningmultipleiterationsofaloadgeneratingtest,theInitialisepartofthetestwillonlyberunonceatthebeginning,andtheTearDownsectiononceattheend.OnlytheTestStepswillberunforeachiterationofanyloadgeneratingtestusedintheperformancetest.Thismeanswecan,forexample,setupandcleanupadatabasewithintheInitialiseandTearDownsectionswithoutimpactingthedatathatweareactuallyinterestedin.

5.2 BasicSetup

1. Beforewecancreateaperformancetest,weneedtoprovidealoadgeneratingtestthatwillcontaintheactionscarriedoutduringeachiteration.Todothis,we’llcreateanormaltest,andaddatimedsectiontoit.GotoTestFactory,andrightclickontheLoginoperationtobringupthecontextmenu.SelectNew>Tests>TestUsingMEP.

2. TheCreatedialogwillappear–presstheOptionsbuttontobringupaSettingsdialog.

3. OntheMessageSettingstab,makesuretheoptionIncludeOptionalFieldsisselected,andthenpressOKtoreturntotheCreatedialog.

4. CallthetestloginBase,andpressOK.Atestwillbecreated.

5. OpenuptheSendRequestmessage,andfillinausername,password,andapplicationintherelevantfields.Thecontentsdon’tmatterforthisexample–ourloginservicewillacceptanyinputforthesefields.

6. Wenowneedtosetupthevalidationforthismessage.Forourpurposes,itwillbeenoughtocheckthatalogintokenisreturned,andthatitcontainshexadecimaldigitsbrokenupwithhyphens.OpenuptheReceiveReplyaction,andfindtheTokenfield.Doubleclickonthe(Text)sectionbelowthattobringuptheFieldEditor.

7. InthetophalfoftheFieldEditor,makesurethattheEqualityvalidationisselected,aswewillbereplacingthisvalidationwitharegularexpression:

8. ChangetheActionTypeoptionjustbelowfromEqualitytoRegex.

9. Entertheregularexpression^[a‐f0‐9\‐]*$

10. Totestthatitisworking,enter44ef‐2ab7‐573dintotheDocumentfield,andpressTest.TheResultfieldshouldupdatetosaytrue(ifitdoesn’t,makesurethatyouhaven’taccidentallyincludedaspacecharacterattheendofthestring).


11. Nowadd–y8rrtotheendoftheDocumentfield,giving44ef‐2ab7‐573d–y8rr,andpressTest

again.Thisshouldfail.

12. PressOKtoclosetheFieldEditor,andOKagaintoreturntothetest.

13. Savethetest,andthenrunitinTestLabtomakesurethatitpasses.Ifthereareanyproblems,fixthembeforemovingon.

5.3 TimedSections

Timedsections,markedbytheBeginTimedSectionandEndTimedSectionactions,allowustotimetheexecutionofdifferentpartsofthetestbeingexecuted.Asinglefunctionaltestcancontainmultipletimedsections,whichmayoverlap,orcontainothertimedsections.

Foreachtimedsection,RationalIntegrationTesterwilllogdataintotheprojectdatabasewhileaperformancetestisrunning.Thiswillincludenotonlythetimetakenforthetimedsectiontoexecute,butalsothestatusofthesection–whetheritpassed,failed,ortimedout.

Ifnotimedsectionsareaddedtothetest,RationalIntegrationTesterwillstillrecordthelengthoftimetakentoexecuteeachiterationoftheentiretest,andthestatusattheendofthatiteration.

1. StartbyreturningtotheTestFactory.

2. Addtwonewactionstothetest–aBeginTimedSection ,andanEndTimedSection .TheBeginTimedSectionshouldgobeforethetwomessagingactions,whiletheEndTimedSectionshouldgoafterwards.

3. OpentheBeginTimedSectionaction.

4. Thetimedsectionwillneedaname–callitS1.

5. Belowthenameofthetimedsection,thereisanoptiontodeterminehowthistimedsectionwillberecorded(Pass/Fail/Timeout).Wecantakethestatusofthetestattheendofthesection,orwecantakethestatusofthetestattheendofthatiterationofthetest.Inthisparticularcase,sincethetimedsectioncoverstheentiretest,itwillnotmakeanydifferencewhichofthetwooptionswechoose.

6. ClosetheBeginTimedSectionaction,andopentheEndTimedSectionaction.Thishasonlyonesetting–thenameofthetimedsection.MatchittoS1,thesectionwestartedwiththeBeginTimedSectionaction,andclosethedialog.


Ourloadgeneratingtestisnowcomplete.However,weneedtostatehowmanyiterationswewillcarryout,atwhatrate,andsoon.Wewilldothisseparately,inaperformancetest.


6 Creating Performance Tests

6.1 Introduction

Oncewehaveafunctionaltestthatwillcreatealoadonthesystem,wecanstartputtingtogetheraperformancetest.PerformancetestsarecreatedintheTestFactory,andarecontainedwithinthesametreestructureasothertestresources.

Ourfirstperformancetestwillbefairlybasic,runningasingleloadgeneratingtestat1iterationpersecond.Laterperformancetestswilllookatchangingtheloadonthesystem,andvaryingitovertime.

6.2 InitialSetup

1. IntheTestFactoryTree,rightclickontheLoginoperation,andselectNew>Tests>PerformanceTest.CallthetestsimplePerformance.

2. TheinitialscreenofthePerformanceTestEditorwillappear.

3. Clickonthetext onthelefthandside.

4. Therighthandsidewillaltertoshowsettingsfortheperformancetest.MakesurethatyouareontheExecutiontab.


Createasimpleperformancetest

Addaloadgeneratingtesttoaperformancetest

Configureaperformancetesttouseselectedenginesandprobes


5. Mostsettingscanbeleftattheirdefaults,butchangethelengthofthetestphaseto30seconds,asinthescreenshotbelow.

6.3 AddingTests

1. Aperformancetestonitsowndoesnothing–itrequiresloadgeneratingorbackgroundtestsinordertotesttheperformanceofyoursystem.WewillnowaddtheloginBasetestasourloadgeneratingtest.Rightclickonthe textonthelefthandside–twooptionswillappear:

2. ClickonAddLoadGeneratingTest.

3. Theloadgeneratingtestwillappear,andshouldbeselectedonthelefthandsideoftheeditor.Configurationinformationfortheloadgeneratingtestwillappearontherighthandside.Thefirstthingweneedtodoistochoosewhichtestwillbeusedforloadgeneration.Todothis,makesureyouareontheExecutiontab,andfindtheTestPathfield.PresstheBrowsebuttonnexttothatfield,andselecttheloginBasetestfromthedialogthatappears.

4. We’llleavetheotherexecutionoptionsattheirdefaultsettingsforthemoment,asshownbelow.


5. Theloadgeneratingtestisnearlyreadytogo–butfirst,wewillneedtosaywhatengine(or

engines)willbeexecutingthistest.

6.4 EngineSettings

Ourtestcanberunononeormoreengines.Forthepurposesofthismanual,wewillonlybeusingasingleenginerunningonasingleagent,butinmorecomplextests,multipleenginescanbesetupindifferentlocations,splittinguptheloadbetweendifferentmachines.Regardlessofhowmanyenginesarebeingused,rememberthattheenginesareallmanagedbyasinglecontroller‐theinstanceofRationalIntegrationTesterthatisrunningtheperformancetest.

1. SwitchfromtheExecutiontabtotheEnginestab.

2. PresstheAdd…buttonatthebottomofthescreen.

3. ASelectdialogshouldappear.Inthiscase,therewillbeonlyoneengineavailable–thedefaultengineattachedtotheagentwecreatedinArchitectureSchool.Selectthedefaultengine,andpressOK.

4. Ifmultipleengineswereavailable,wecouldselectmoreofthembypressingtheAdd…buttonagain,andselectingotherengines,butforthisexample,ourtestisnowreadytoberun–wejustneedtostatehowwewillbemonitoringit.

6.5 ManagingProbes

Eachperformancetestcanchoosewhichprobesitwantstousetogatherdata.Differenttestsmaybemeasuringdifferentdata.Forexample,onetestmaybegatheringsystemdata,whileanothermaygatherstatisticsfromthemiddlewarelayer.

Regardlessofwhichprobesarebeingrequestedhere,theymuststillbesetupinArchitectureSchool.Forthisexample,wewillusetheWindowsPerformanceMonitorprobe,assetupinthepreviousexercises.

1. OnthelefthandsideofthePerformanceTestEditor,clickonthePerformanceTesttoswitchbacktoitssettings.

2. ClickontheProbestab.


3. Wecannowselectfromtheavailableprobes.AswehaveonlysetuptheWindowsPerformanceMonitor,checktheboxforthatprobe,andleavetheothersblank.

4. SavethesimplePerformancetest.


7 Running Performance Tests and Analyzing Results

7.1 RunningtheTest

Theprocedureforrunningaperformancetestismuchthesameasforafunctionaltest–simplyusetheRun buttonintheTestLab,ordoubleclickonthetestinthetree.Whiletheperformancetestisrunning,asummaryofthedatabeinggatheredwillbedisplayedintheconsole.Forfullreporting,we’llneedtogototheResultsGallery,aswe’llseeinthefollowingexercise.

1. SwitchtoTestLab.

2. RunthesimplePerformancetest.

3. Watchtheconsoleresults–youwillnoticethattheprobeswillbestarted15secondsbeforetheloadgeneratingtestsarerun,asperthesettingsintheperformancetest.

4. Oncetheloadgeneratingtestsarebeingrun,youwillseecountersforthenumbersoftestsstarted,passed,failed,andthenumberofpendingdatabasewrites.Thesearedefinedinthetablebelow:


Executeaperformancetestandviewthestatisticsshownatruntime.

ViewtheresultsofaperformancetestintheResultsGallery

Compareresultsofmultipleexecutionsofaperformancetest


Started

Iterationsstartedinthereportinterval(Thesettingis'Collectstatisticsevery'inPerformanceTestStatisticstab).Thedefaultintervalis5seconds,soifthetestissetfor10TPSthiswouldshowatotalof50eachtime.

Passed Iterationspassedsofarduringtheperformancetest.

TimedOutIterationswheremessagereceiversdidnotgetaresponsewithintheirconfiguredtimeout.

Failed Iterationsfailedsofarduringtheperformancetest.

PendingDBWrites

Databasewritesqueuedontheresultsdatabase.Largenumbersindicatethatdatabaseaccessisslowerthanrequiredandmaybearesultofaslownetworkconnection.Notethatthewritesarebufferedanddonotslowdownthetestrate.

Note:Aperformancetestmayrunonlongerthanthespecifiedtimewhileremainingtestinstancescompleteanddatabasewritesareflushed.Inthiscaseyouwillseethe'started'figureaszeroforthoseintervalssincethegivennumberofiterationshasalreadybeenstarted.

7.2 ViewingResults

1. TheTestLabdoesn’tshowmuchinthewayofresultsbesidesstatisticsforhowmanytimedsectionspassed/failed,etc.Togetthisinformation,weneedtogototheResultsGallery.Switchtothatperspectivenow.

2. IntheResultsGallery,youwillseeasinglelinedescribingbasicinformationaboutyourtest–start/endtimes,numberofiterations,etc.Selectthis,andthenclicktheAnalyseResultsbuttonatthebottomofthescreen.

3. Anemptychartwillnowappear.We’regoingtopopulatethiswithsomeofthedatawehavecollectedwithourprobes.Thisisavailabletotheleftofthechart,sortedintodifferentcategories.ExpandthisouttonavigatetoPerformance Test Sections (Based on Start Times) > Average Pass Section Duration > simplePerformance / S1.

4. Acheckboxshouldnowbeavailable–tickit.


5. Achartshouldnowappearonthelefthandside.Experimentwithaddingotherinformationgatheredbyyourprobestothechart.Inparticular,lookattheinformationrecordedbytheWindowsPerformanceMonitorProbe.Notethatchartscanberemovedsimplybyuncheckingthemagain.

6. Asyouexperiment,youmayfindasituationwheretheaxesofchartsdonotmatch.Forexample,ifyoulookattheWindowsPerformanceMonitorprobe,andselectbothpiecesofdataforthememory,youmightseesomethinglikethis:

ThenumberofMBavailableischanging,butthescaleofthechartforthenumberofpagefaultsispreventingusfromseeingthatinformation.

7. Inordertofixthis,wecaneditthepropertiesofoneofthechartssothatitisdisplayedonaseparateaxis.Gototheleftofthechartdisplay,whereyouhaveselectedyourdata,anddoubleclickononeofthecolouredlinesthathasappearednexttothecheckbox.

8. AChooseStyledialogwillappear.OntheStyletab,youcanchangehowthedataisdisplayed(colouroftheline,typeofchart,etc.).Changeanysettingsherethatareofinterest.


9. SwitchtotheDatatab,andsettheAxisto2.Closethedialog,andthechartsshouldupdate:

10. Atthisstage,wecaneditthechartandgiveitanameandsomenotes,inthetextfieldsbelow

thechart.Wecanalsosavethechartforlaterreference,orusethe buttontoexportdatatoaCSVfile.

7.3 MultipleDataSets

Sofar,we’vejustlookedatresultsforasingleperformancetest.Itisalsopossibletocompareresultsofmultipleperformancetests,runatdifferenttimes.Thisallowsustoseehowchangesmadetoourloadgeneratingtest,orchangesmadetothesystem,haveaffectedtheperformanceofthesystem.

1. Closethechartforthemoment,andreturntotheTestLab.

2. RunthesimplePerformancetestagain.

3. Onceitiscomplete,gobacktotheResultsGallery,andchooseAnalyseResultsforthemostrecenttestrun.

4. Achartwillappear,asbefore.However,thistimeanotheroptionisopentous:wecancomparethistestruntoanyprevioustestrun.SwitchtotheDataSetstab,andpresstheAddbutton.

5. Selectyourprevioustestrunhere.


6. ReturntotheCounterstab.Foreachcounter,twochartswillnowbeavailable,andyoucanusethistocomparethecurrentresultstothepreviousresults.


8 Data Driving Performance Tests

8.1 DifferencesfromFunctionalTests

WhencreatingfunctionaltestsinRationalIntegrationTester,wecansimplyusetheIterateTestDataactiontorunthroughadataset,andtestthesystemwiththatparticulardataset.However,justimaginethatwehadatestsimilartotheonewehavecreatedalready,thatsendsasinglemessage,andreceivesasinglemessage.

Asanexample,ifweweretousetheIterateTestDataaction(oranyotherIteratetestaction)withthattest,tosend10messages,andourperformancetesthadalsospecified10iterationspersecond,wecouldendupsendinganywherebetween10and100messagespersecond,withnoprecisecontrolovertheloadonthesystem.Inaddition,testdurationandstatusdatawillbelimitedintheirusefulness–ratherthanmeasuringthemessagingtimes,wewouldbemeasuringthetimetoexecute10messages;similarly,wewouldberecordingthepass/failstatusofagroupof10messages,ratherthanasinglemessage.

Forthesereasons,itisgenerallyadvisednottousetheIterateTestDataactionwithinaperformancetest.Instead,wecanmapadatasourcetoatestusingtheInputMappingstabforeachLoadGeneratingorBackgroundTest.

8.2 DrivingaLoadGeneratingTestwithExternalData

1. CreateacopyoftheloginBasetest,andcallitloginDataDriven.

2. OpentheSendRequestactionoftheloginDataDriventest,andgototheConfigtab.

3. QuickTagtheUserNameandPasswordfields.


Learnthelimitationsofusingtheiterateactionswithinaloadgeneratingtest.

UsetheInputMappingssettingsintheperformancetesttodatadrivealoadgeneratingtest.


4. SavetheloginDataDriventest.

5. NowmakeacopyofthesimplePerformancetest,andcallitdataDrivenPerformance.

6. OpenthedataDrivenPerformancetest,andgotothesettingsfortheLoadGeneratingTest.

7. FindtheTestPathsetting–itshouldcurrentlybesettotheloginBasetest.UsetheBrowsebuttontochangethistotheloginDataDriventest.

8. Wenowneedadatasource,soleaveRationalIntegrationTestertocreateone.CreateabrandnewCSVfileorExcelspreadsheetwiththefollowingdata(orsomeofyourowninvention–justremembertoaddalinewithheadings):

User,Pass

Jim,gr33nhat

Steve,ght3st3r

Monica,perf0rmance

Karen,eng1n3s

Ben,pr0b3s

9. SaveyourCSV/Excelfile,andreturntoRationalIntegrationTester.CreateaFileDataSourceorExcelDataSourcetoconnecttoyourdata.RemembertousetheRefreshbuttontocheckthatthedatahasloadedproperly,andSavethedatasource.

10. GobacktothedataDrivenPerformancetest,andgototheInputMappingstab.Underneaththis,youwillseethreenewtabsappear–Config,Filter,andStore.

11. IntheConfigtab,usetheBrowsebuttontochooseyourtestdataset.

12. Ifwewantedtofiltertheincomingdata,wecoulddothatontheFiltertab.Inthiscase,we’llusetheentiredatasetsupplied,sogototheStoretab.

13. MapthetagsintheloginDataDriventesttothecolumnsinyourdatasourcehere,andsavethedataDrivenPerformancetest.

14. Runthetestandanalyzetheresults.


9 Load Profiles

9.1 PerformanceTestingScenarios

Sofar,ourtestshavebeenmodelingaverysimplescenario,runningourloadgeneratingtestat1iterationpersecond.However,thereareanumberofscenarioswherewemightliketousemorecomplexperformancetests.We’lllookatafewexamplescenarios,andhowRationalIntegrationTestercandealwiththem.

LoadTesting

Aloadtestattemptstorepresentaperiodintheworkingday,andtesthowthesystemwillrespondtoasimilarload.Forexample,itmaybeanticipatedthatthegreatestriskofnon‐performancemaybeatthestartoftheworkingday.Thescenariowillmodeltherampupfromtheminimumnumberofuserstothepeakloginperiodduringthefirstfewhoursofbusiness.

StressTesting

Astresstestscenarioisusedtoidentifythebreakpointofthesystem.Thebreakpointwillthusbeidentifiedasaspecifiedload(andramp‐up)andwillbeusedtoidentifyperformanceweaknessesinthedistributedsystem.Thismaybemodeledwithasimplelinearincreaseintheloadonthesystem.

Stresstestscanalsobeusefulinprovingtherecoverabilityofasystem–howdoesthesystembreakandhowgracefullycanitrecoverunderextremeloads?Inthiscase,abellcurvecouldbeusedtoviewhowthesystembreaksasitapproachesapointidentifiedasabreakpoint,andthentoallowthesystemsomeroomtorecover.

SoakTesting

Asoaktestscenariowillberunagainstaconstantlow‐levelloadthatmayrunforhoursordays.Thisscenariocouldbeamodelinganiterationrunonceperminute,orless.Runningthetestforanextendedperiodoftimewillidentifyanyissuesthatmaymanifestthemselvesoveralongerperiod,suchasmemoryleaks.


Encounterstandardperformancetestingscenarios,includingload,stress,andsoaktests.

UsetheConstantGrowthsettingsinaperformancetesttoprovideanincreaseintheloadonthesystem.

DatadrivetheloadonthesystemusingtheExternallyDrivensettingsinaperformancetest.


HighIntensityScenarios

Highintensityscenarios,whethertheyareloadtestsorstresstests,willrequirealotofdata.Somebasicmathwillberequiredtounderstandtheoveralldatarequirementsintermsofthedatatodrivethetestsandthedatathatisrequiredtobepresentinthesystemforreferenceandexecution.Ifyouareexpectedtorunat100transactionspersecondforover3hoursthenyouwillrequire3x60x60x100rowsofdata–over1millionrowsofdata!Whetherornotyoucanuserepeatingdatainthesemillionrowsofdatawillbedeterminedbythenatureofthesystemundertest.Similarissuesmayariseinparticularlylengthysoaktests.

Inaddition,modelingthecorrectnumberofrequestsinahighintensityscenariomaynotbepossiblewithasingletestengine.Youmayneedtocreatemultipletestenginesinordertobeabletocreateasufficientloadonthesystem.

9.2 ConstantGrowth

Movingbeyondasimple,constantloadonthesystem,thesimplestwaytovarytheloadinaperformancetestistoincreasetheloadovertime.Todothis,wesplittheperformancetestupintomultiplephasesofagivenduration,increasingthenumberofiterationsforeachnewphase.Thiswillgiveasimpledemonstrationofhowthesystemhandlesanincreasingamountofload.

1. ReturntotheTestFactory,andcreateacopyofthesimplePerformancetest.

2. Renamethenewtest,andcallitthreePhaseTest.

3. OpenthethreePhaseTest,andontheExecutiontab,changetheNumberoftestphasesto3.

4. SwitchtotheLoadGeneratingTestonthelefthandside,andgototheExecutiontab.

5. ChangetheInitialtargetiterationssettingto5persecond.

6. SettheIncrementperphaseto5.Wewillnowhave3phases,5iterationspersecond,10iterationspersecond,and15iterationspersecond.

7. Savetheperformancetest,andrunitfromtheTestLab.Youshouldseeeachphaseexecuteintheconsole;noticethateachphaseisrunningmoreandmoretestspersecond.

8. GototheResultsGallerytoviewthetests,andviewthedatainachart.

9.3 ExternallyDefinedLoadProfiles

Usinganexternaldatasourcegivesusmuchmorecontrolovertheamountofloadonthesystem.Thelengthofeachphasecanbevariedasrequired.Inaddition,aconstantlevelofgrowthisnolongerrequired–theexactnumberofiterationspersecond/minute/hourcanbesetforeachindividualphase.

1. MinimizeRationalIntegrationTester,andcreateanewCSVorExcelfilewiththefollowingdata:Period,Iterations 10,10 20,30 30,10


Wewillusethefirstcolumnofthedatatospecifythelengthofeachphasethatwewillrun,whilethesecondsuppliesthenumberofiterationsineachphase.Aswewillspecifyourunitsassecondsintheperformancetest(minutesandhoursarealsopossible),wewillhave10secondsatthebeginningofthetestwherewerun10iterationspersecond,20secondswherewerampitupto30iterationspersecond,andthen30secondswhereweallowthesystemtogobacktotheoriginal10iterationspersecond.

2. ReturntoRationalIntegrationTester,andcreateaFileDataSourceorExcelDataSourcetolinktoyourdata.RemembertousetheRefreshbuttontocheckthatthedataloadscorrectly.

3. Savethenewdatasource.

4. CreateacopyofthesimplePerformanceTest,andcallitexternalPhaseTest.

5. OpentheexternalPhaseTest,andgototheExecutiontabofthePerformanceTestsettings.

6. ChangetheLoadProfiletoExternallyDefined.

7. NexttoDatasetforloadphases,presstheBrowsebuttontofindandselectthedatasourcecontainingthephasedata.

8. LeavetheExecutetestphasesfieldblank,andmakesurethatthePhasedurationreadfromcolumnsettingissettoPeriod.

9. SwitchtothesettingsfortheLoadGeneratingTest,andgototheExecutiontab.

10. MakesurethatthenumberofiterationsisreadfromtheIterationscolumn.

11. SavetheexternalPhaseTest,andrunitintheTestLab.

12. GototheResultsGallery,andviewthetestresults.Youmaynoticethatthestatisticsfortheminimum,average,andmaximumpasssectiondurationsspikeduringthetest,indicatingthattheloadwasslowingdowntheperformanceofthesystem.


10 Advanced Topics

10.1 BackgroundTests

Sofar,wehaveusedaloadgeneratingtesttoprovideapre‐definedloaduponthesystem.Multipleloadgeneratingtestscouldbeused,ifrequired.However,insomecases,youmaywanttoprovideaconstantstimulusforthesystemwhileusingyourloadgeneratingtest.Theremayalsobesituationswhereyouneedtousestubstosimulatepartofthesystemundertest.Bothofthesesituationscanbehandledbyaddingabackgroundtesttoyourperformancetest.

Abackgroundtestisafunctionaltest(orstub)thatwillberunrepeatedlyforthedurationoftheperformancetest(or,optionally,untilthebackgroundtestfails).Thismeansthatitwillberunconcurrentlywithanyloadgeneratingtestsincludedintheperformancetest.Eachbackgroundtestcanhaveasingleiterationrunningatatime,ormayberunmultipletimesinparallel–unlikeloadgeneratingtests,thisisnotlimitedbytheRationalIntegrationTesterlicensethatisinuse.However,timingandstatusinformationwillnotberecordedforabackgroundtest.

Sincebackgroundtestsarerundifferentlytoloadgeneratingtests,severalthingsshouldbekeptinmind.Firstly,theInitialiseandTearDownphasesofthetestwillberunasnormal.Second,whiletheBeginTimedSectionandEndTimedSectionactionscanstillbeincludedinthefunctionaltest,theywillnothaveanyeffectonwhatisrecordedintotheprojectdatabaseduringtheperformancetest.

10.2 LogMeasurement

TheLogMeasurementactioncanbeusedtologcustomdataintoyourdatabasewhilerunningaperformancetest.Thismaybeusefulinseveralsituations.

Firstly,itmaybeusedwhenrecordingdatafromthesystemundertest,actingasacustomprobe.Thismaybenecessarywheninformationisrequiredthatisnotcoveredbythestandardprobes–forexample,whenqueryingproprietarysystemsforinformation.

AnalternativeuseexistsforsystemswhereamessagegoesthroughseveralprocessesbeforearesponseisreceivedbyRationalIntegrationTester.Inthesecases,itmaybedesirabletomeasurethetimetakenforasingleprocesstoprovidearesponse,ratherthanmeasuringtheentireround‐triptimebetweentheinitialmessagesentfromRationalIntegrationTester,andthateventualresponse.


Encounterbackgroundtests,anddiscusstheiruses.

SeehowtheLogMeasurementactionworks,andhowitcanbeused.

UsetheLogMeasurementactioninabackgroundtesttoprovideacustomprobe.


Inthediagrambelow,RationalIntegrationTesterpublishesamessagetoaqueue,andwaitsforaresponse.Usingthedatanormallygatheredbyaperformancetest,wewouldbetoldhowlongittookforthemessagetobeprocessedbyoperationsA,B,andC.However,iftherewereperformanceissuesasweincreasedtheloadonthesystem,wewouldnotknowifthesecouldbenarroweddowntoasingleservice.

Forexample,wemightsuspectthatserviceBiswheremostofthedelayisoccurring.Inordertoinvestigatethis,wecanaddtimestampstofieldsinthemessageasitpassesthroughthesystem.SubtractingTime2fromTime3wouldthengiveustheamountoftimethatwasspentinsideserviceB.Usingthelogmeasurementaction,thisinformationcouldberecordedintheprojectdatabase,andanalyzedlaterwithrespecttotheloadonthesystem.

WhenusingtheLogMeasurementaction,itisimportanttonotethatitcannotbeusedwithinatimedsection.Thisisbecausewritingtotheprojectdatabasewouldalterthetimetakenduringtheexecutionofthetimedsection,therebyskewingthetiminginformation.

10.3 Creatingthemeasurementtest

Inthisexample,we’lluseabackgroundtestandthelogmeasurementactiontoactasacustomprobeforthesystemundertest.Thisparticularexamplewillbegatheringdataaboutthebytessentandreceivedbythesystem–notethatthiscouldalsobegatheredbytheWindowsPerformanceMonitorprobe.

We’llbeusingabackgroundtestfortworeasons:firstly,itmeanswedon’tneedtochangeourloadgeneratingtest;andsecond,wedon’twanttohaveourprobeconstantlyrunning–we’llgatherourdataeverytwoseconds,ratherthanconstantlypollingthesystemandpossiblyaddingextraunintendedload.

1. CreateanewtestfortheLoginoperation,andcallitbyteMonitor.

2. AddaRunCommandactiontothetest.


3. OntheConfigtab,enterthefollowingcommand:netstat ‐e | find "Bytes"

4. MakesurethattheWaitforcommandexecutiontofinishcheckboxisticked.

5. PresstheTestbutton.Youshouldseeasinglelineofdataforstdout,similartothefollowing:Bytes 12008764 57368668

6. SwitchtotheStoretab,sowecanstorethedataintotags.

7. We’llneedtostorethetwonumbersintoseparatetags,whichwe’llbecallingbytesSentandbytesReceived.Todothis,rightclickonthestdoutfield,andchooseContents>Edit.

8. Makesureyou’relookingattheStoretabwithinthewindowthatappears,andthenpresstheNewbutton.

9. Detailsforthedatatostorewillappearbelow.ThedefaultactiontypeshouldbesettoCopy–changeittoRegularExpression.

10. ChangetheTagtobytesReceived.Youcanalsochangethedescriptionfieldtomatch.

11. IntheExpressionsection,typetheregularexpression\d+tomatchanumber.Belowthat,choosetoExtractInstance1,sothatwe’llbeextractingthefirstnumberfoundinthestring.Intheexamplestringgivenabove,thiswouldstore12008764intothe bytesReceived tag.Youcantestthisoutwithanexamplestringtocheckthatitisworkingcorrectly.

12. Westillneedtostorethenumberofbytessent.PressNewagaintogenerateasecondstoreactionforthestdoutfield,andfollowsteps9‐11again,butthistimesettheTagnametobytesSent,andExtractInstance2.Similartothefirstaction,ifyouweretotestthisoutusingtheexampleabove,youshouldgetaResultof57368668.

13. Onceyou’redone,thetwotagsshouldbeconfiguredasseenbelow:


14. PressOKtoclosetheFieldEditor,andthenOKagaintoclosethetestaction.

15. BeforeweaddtheLogMeasurementaction,we’llcheckthatthisisworkingasweexpect.AddanormalLogaction,andlogthevaluescapturedinthebytesReceivedandbytesSenttagstotheconsole.

16. Runthetest,andcheckthatitworksatthemoment.Ifitdoesn’t,checktheprecedingstepstomakesurethateverythinghasbeenenteredcorrectly.

17. ReturntotheTestFactory,anddeleteordisabletheLogaction.

18. AddanewLogMeasurementactionaftertheRunCommand.

19. SetuptheLogMeasurementactionasshownintheimagebelow.Thiswillgraphthenumberofbytessentandreceivedbylookingupthevaluescapturedearlier.Theattributessectionallowsustographmultiplesetsofdata–inthiscaseweonlyhaveone,butatleastoneattributeisrequiredinorderfortheLogMeasurementactiontorun.


20. PressOKtoclosetheaction.

21. AddaSleepactiontotheendofthetest.Aswe’llberunningthisasabackgroundtest,andbackgroundtestsruncontinuously,we’llwanttopacethistestsoitdoesn’tinterferewithsystemresults.SettheSleepactiontohaveafixeddurationof2000ms.

22. SavethebyteMonitorTest.

10.4 AddingtheMeasurementstoaPerformanceTest

1. CreateacopyoftheexternalPhaseTest,andcallitcustomLogging.

2. EditthecustomLoggingTest,andrightclickonthePerformanceTestlabelonthelefthandsideoftheeditor–you’llseetheoptionsforaddingloadgeneratingandbackgroundtests.Addabackgroundtest.

3. OntheExecutiontabfortheBackgroundTest,selectbyteMonitorfortheTestPathfield.

4. MakesurethatTerminateonfailureisnotchecked.

5. SwitchtotheEnginestab,andclickAdd.Therewillonlybeoneengineavailable,asbefore–selectit.

6. SavethecustomLoggingtest,andrunitintheTestLab.

7. Onceithasrun,youshouldbeabletoviewtheresultsintheResultsGallery.ThecounterwillbefoundintheLog Values section.


11 Legal Notices

ThefollowingparagraphdoesnotapplytotheUnitedKingdomoranyothercountrywheresuchprovisionsareinconsistentwithlocallaw:INTERNATIONALBUSINESSMACHINESCORPORATIONPROVIDESTHISPUBLICATION"ASIS"WITHOUTWARRANTYOFANYKIND,EITHEREXPRESSORIMPLIED,INCLUDING,BUTNOTLIMITEDTO,THEIMPLIEDWARRANTIESOFNON‐INFRINGEMENT,MERCHANTABILITYORFITNESSFORAPARTICULARPURPOSE.Somestatesdonotallowdisclaimerofexpressorimpliedwarrantiesincertaintransactions,therefore,thisstatementmaynotapplytoyou.

Thisinformationcouldincludetechnicalinaccuraciesortypographicalerrors.Changesareperiodicallymadetotheinformationherein;thesechangeswillbeincorporatedinneweditionsofthepublication.IBMmaymakeimprovementsand/orchangesintheproduct(s)and/ortheprogram(s)describedinthispublicationatanytimewithoutnotice.

Ifyouareviewingthisinformationinsoftcopy,thephotographsandcolorillustrationsmaynotappear.

Anyreferencesinthisinformationtonon‐IBMwebsitesareprovidedforconvenienceonlyanddonotinanymannerserveasanendorsementofthosewebsites.ThematerialsatthosewebsitesarenotpartofthematerialsforthisIBMproductanduseofthosewebsitesisatyourownrisk.

Anyperformancedatacontainedhereinwasdeterminedinacontrolledenvironment.Therefore,theresultsobtainedinotheroperatingenvironmentsmayvarysignificantly.Somemeasurementsmayhavebeenmadeondevelopment‐levelsystemsandthereisnoguaranteethatthesemeasurementswillbethesameongenerallyavailablesystems.Furthermore,somemeasurementsmayhavebeenestimatedthroughextrapolation.Actualresultsmayvary.Usersofthisdocumentshouldverifytheapplicabledatafortheirspecificenvironment.

Informationconcerningnon‐IBMproductswasobtainedfromthesuppliersofthoseproducts,theirpublishedannouncementsorotherpubliclyavailablesources.IBMhasnottestedthoseproductsandcannotconfirmtheaccuracyofperformance,compatibilityoranyotherclaimsrelatedtonon‐IBMproducts.Questionsonthecapabilitiesofnon‐IBMproductsshouldbeaddressedtothesuppliersofthoseproducts.

AllstatementsregardingIBM'sfuturedirectionorintentaresubjecttochangeorwithdrawalwithoutnotice,andrepresentgoalsandobjectivesonly.

Thisinformationcontainsexamplesofdataandreportsusedindailybusinessoperations.Toillustratethemascompletelyaspossible,theexamplesincludethenamesofindividuals,companies,brands,andproducts.Allofthesenamesarefictitiousandanysimilaritytothenamesandaddressesusedbyanactualbusinessenterpriseisentirelycoincidental.

Thisinformationcontainssampleapplicationprogramsinsourcelanguage,whichillustrateprogrammingtechniquesonvariousoperatingplatforms.Youmaycopy,modify,anddistributethesesampleprogramsinanyformwithoutpaymenttoIBM,forthepurposesofdeveloping,using,marketingordistributingapplicationprogramsconformingtotheapplicationprogramminginterfacefortheoperatingplatformforwhichthesampleprogramsarewritten.Theseexampleshavenotbeenthoroughlytestedunderallconditions.IBM,therefore,cannotguaranteeorimplyreliability,serviceability,orfunctionoftheseprograms.Thesample


programsareprovided"ASIS",withoutwarrantyofanykind.IBMshallnotbeliableforanydamagesarisingoutofyouruseofthesampleprograms.

Trademarksandservicemarks

IBM,theIBMlogo,andibm.comaretrademarksorregisteredtrademarksofInternationalBusinessMachinesCorp.,registeredinmanyjurisdictionsworldwide.OtherproductandservicenamesmightbetrademarksofIBMorothercompanies.AcurrentlistofIBMtrademarksisavailableonthewebatwww.ibm.com/legal/copytrade.shtml.

MicrosoftandWindowsaretrademarksofMicrosoftCorporationintheUnitedStates,othercountries,orboth.

JavaandallJava‐basedtrademarksandlogosaretrademarksorregisteredtrademarksofOracleand/oritsaffiliates

Othercompany,product,orservicenamesmaybetrademarksorservicemarksofothers.

Documents

2012 10 23_3013_rational_integration_tester_fo