By Ostap Elyashevskyy Automated Performance Testing · PDF fileutomated erformance Testing of...

1Automated Performance Testing of Desktop Applications

Introduction

For the most part, performance testing is associated with Web applications. This area is more or less covered by a lot of articles and videos about how to use JMeter and other tools, which allow load generation and performance metrics capturing.

But how do you test the performance of a Desktop application? Long story short, here are some basic requirements:

•Nosourcesofapplication(blackbox)

•AnyWindowsapplication(usuallytheseareresourceintensiveapplicationssuchasarhivators,backuputilities,etc.)

•Differentoperationswiththeapplicationundertestandsystemperformancetracking(RAM,CPU,andothermetrics)

•Notificationaboutthesystemstatewhentheuserperformsanyaction(forexample,buttonclicks,messageboxpop-out(GUIevents),etc.)

By Ostap Elyashevskyy

Automated Performance Testing of Desktop Applications

2Automated Performance Testing of Desktop Applications

To show how everything works, here is one of the possible approaches to test the performance of Desktop applications.

Tool selection

UI testing toolInordertocaptureperformancedataduringuseractions,weneedtohandletheapplication under test and interact with it, in particular click on controls, emulate mouse and keyboards events, etc.

The selected tool should satisfy the following requirements:

•InteractwithanyDesktopapplication:Java,.NET,nativeWindowsapp,andothers

•Emulateuseractions,clicks,keyboard/mouseevents

•Recognizecontrolsbytext/image(languageandtechnologyagnosticapproach)

•Captureperformanceinformationofactions:whencontrolbecomesavailable,etc.

Afterashortinvestigation,Ihavefoundtwopossibletools:SikuliandAl’exa.Bothareopen source projects, with more detailed features description and comparison to be found here.Asit’scapableofgrabbingperformancedataandintegratingwiththeNagiosmonitoringsystem,Al’exawasselectedasthemaintoolforUIperformancetesting.

Resource monitoring toolToanalyzemetricslikeCPU/RAMusage,I/OHDDoperations,etc.,aresource-monitoringtoolisneeded.Basically,Al’exasupportsintegrationwithNagios,anopensourceresourcemonitoringtoolwhichcanbeinstalledonLinuxbasedOS.IhavetrieditonUbuntuandfoundinstallation/configurationandusageverycumbersome,sinceitrequiresinstallationofNSClient++ontheclientside,alongwithSQL/PHP/WebServerandotherpackagesontheserverside.Ineededtogetresultsfasterandsimplifymonitoringtoolconfigurationanddeployment,soIusedWindowsPerformanceMonitorandintegrateditwithAl’exa.

Integration of UI and resource monitoring toolWindowsPerformanceMonitor(WPM)allowscustomcounterstobecreatedandsetthoughaproxywhichwillsendperformancedatafromAl’exatoWPM.Implementationofsuchaproxyisdescribedlater.

3Automated Performance Testing of Desktop Applications

Solution implementation

ThissectiondwellsuponimplementationofasolutionforDesktopPerformanceTestingbychoosingaspecificDesktopApplicationandtryingtoretrievemetrics.

Ioptedforaninstalled7-zipactivatorthatcangenerateanumberofsystemresources(benchmarkingfeature).Here’sasimpletestcasefor7-zip:

1.Start7-zipFilemanager

2.ClickTools

3.ClickBenchmark

4. Wait for 5 seconds

5.PressStop

6.Closethewindow

The system metrics are measured within the following operations order:

a.Navigatemenuandopenbenchmarkwindow

b.Performbenchmarkingfor10seconds

c.Stopbenchmarking

4Automated Performance Testing of Desktop Applications

Performance Monitor Counters set-upInordertotrackcustomactions,createperformancegroupsandaddperformancecountersforeachuserworkflow.IfyouarenotfamiliarwithWindowsPerformanceMonitor, take a glimpse at this short articledescribingthebasicsofWPM.

OpenPerformanceMonitorandcheckanexampleofMemorycounterswhicharealreadydefined.Inaddition,therearealotofothercountersrelatedtoCPU,HDD,Network,protocols,orspecificprocesses,threads,etc.

Inthiscase,addthecountersrelatedtoaspecifictestcasetoseehowsystemperformancechangesovertimeduringuserinteractionswithapplicationGUI.

Havinganalyzedseveralsolutionsabouthowtosetcustomcounters,Ifoundit’sbettertodoitprogrammaticallyonC#.Therefore,Ihaveimplementedasmallcommand line utility based on this cool article.Downloadthefileshere:C#sources/executable.

Let’srunutilitywiththeseparameters:

> CreatePerfCounters.exe ALEXA open_window benchmark stop_benchmark

Thefirstparametersetsagroupandanotheronedefinescounters.Bylaunching

5Automated Performance Testing of Desktop Applications

this command again, you delete an old group and create a new one with the new counters.Then,openPerformanceMonitorandcheckthenewlycreatedgroupwithcustom counters:

The operation of creating counters was separated for two reasons:

•It’snotnecessarytocreateacounterforeverytestexecution

•Thisisaresource-consumingoperation,soitisfastertocreatecountersseparately

Proxy implementationNowit’stimetosetthecountersprogrammatically.Tosetthecounters’values,implementproxyin.NET(thereareprobablyothersolutionsimplementedonPython,so if you know any libs or better solutions, please share in comments or write me anemail).Fornow,downloadsources and build them or use a compiled library PerfMonitorProxy.dll.

Putthislibraryunderthepath:C:\PerfMonitorProxy.dll

This.NETlibrarywillbecalledbythescriptsinAl’exa(Python),soit’snecessary“tore-call”itintoa.NETcode.Moreinformationaboutbuildingandcallingunmanagedcode can be found here and here.Fornow,it’senoughtodownloadandusethelibrary.

Getting started with Al’exaTestsandinteractionwiththeGUIwillbeperformedbytheAl’exatool.Firstofall,installAl’exa.CreatingscriptsinAl’exaIDEisintuitiveandthelinkedgettingstartedmanual covers this aspect. Let me elaborate on the most important moments and comment on some code snippets.

6Automated Performance Testing of Desktop Applications

Performance monitor configuration and capturing results 1.SetupPerformancemonitortoshowonlyimportantmetrics:

2.Addnextcounters:

3.Setupgrid:

Action->Properties->Graph-> Check Vertical and Horizontal grid:

4.(Optional)ClickonDataandsetWidthforcustomcounterstobold:

5.ClickonMemoryandsetappropriatescalingindex(formy

computerwith8GbofRAM,Iset0,01)

This scaling allows Free Memory to

7Automated Performance Testing of Desktop Applications

bedisplayedonthesamegraph,sincethereisamarkofonly100ontheXaxis,zoomdownMemorycounter.Itmeansthatthereare~35*100=3500MBfree.

6.Trytoexecutethescriptandanalyzetheresultsandcodestructure(youmayneedtochangeimagesforcontrolsintheprojectforyourresolution):

Afterscriptexecution,youshouldbeabletoseeasimilargraph:

•CPUconsumptionstartedincreasingafterweclickedtheBenchmarkbuttonanddroppeddownonlyafterwestoppedit(blueline).

•Beforewestartedthetest,therewasabove3Gbofavailablememoryandafterthetestitdroppedto2.5Gb

•DuringBenchmarking,thehighestloadofCPUwas87%andjustbeforethestopithadincreasedto100%.

8Automated Performance Testing of Desktop Applications

Now,let’sgothroughthecodeandcheckthemostessentialcodesnippets:

proxy.perfmon(“ALEXA”, “open_window”, 50)

Thiscommandexecutesfunctionsfromourproxydlllibraryandsetstheperformancecounteropen_windowwithvalue50(youcansetanyvalueasitisusedtovisualizethestartoftheaction).

Iamcallingtheendaction:

proxy.perfmon(“ALEXA”, <any counter>, 0) to set the counter’s value to 0.

Incaseyouneedtowaituntilthedynamiccontrolsappear,useBindfunction:

performanceValue = ControlToWait.Bind(15) # 15 seconds - wait timeout

9Automated Performance Testing of Desktop Applications

Afterthat,senditsvaluetooutput.Here’showitmaylooklike:

OK: Open Tools time is 0.842000007629 sec.

Recording performance data to log fileThe previous chapter described capturing and showing results in real time mode. Usuallythisisneededtoscheduletestsandcollectresultstoanalyzeitafterwards.

Here’showtorecordperformancetestdatainthefile:

1.RightclickonDataCollectionSets->UserDefined->New->DataCollectorSet

2.Nameit:AlexaMetrics,clickNext

3.ChoseBasic,clickNext

4.ChooselocationforlogsorleaveasisandclickNext

5.ClickFinish

6.Nowaddcounters:

10Automated Performance Testing of Desktop Applications

7.Clickstarttocapturemetrics

8. Launch the script

9.Stoprecordingperformancedata

Nowlet’scheckwhathasbeenrecorded:

1.ClickonPerformanceMonitor

2.Chosepropertiesandchooselogfileyouhavecaptured:

11Automated Performance Testing of Desktop Applications

3.AfterclickingApply,youshouldseesomethingsimilartothis:

Summary

PerformanceWindowsMonitorenablestrackingofalltheimportantmetricsandsystem/processparameters.Inaddition,itcancaptureperformancemetricstoCSV,log,binary,tabulatedfiles,orSQL.

AlsoWPMprovidestheopportunitytoschedulemonitoringexecution,definestopconditions,executealltypesofactionsonspecificconditions,etc.

ThecombinationofAl’exaandWPMresultsinapowerfulflexibletestandmonitoringsystem for Windows desktop applications. When tracking applications from different hosts,optforintegrationwithNagiosorothermonitoringsystems.Scriptsandcodeprovided in this article were created as a proof of concept and can be improved and applied according to your needs.

USAHQTollFree:866-687-3588Tel:+1-512-516-8880

UkraineHQTel:+380-32-240-9090

Bulgaria Tel:+359-2-902-3760

GermanyTel:+49-69-2602-5857

NetherlandsTel:+31-20-262-33-23

PolandTel:+48-71-382-2800

SwedenTel:+46-844-685-084

UKTel:+44-207-544-8414

EMAILinfo@softserveinc.com

WEBSITE:www.softserveinc.com

About Author

OstapElyashevskyyisaTestAutomationExpertatSoftServe. Hehasmorethan7yearsofexperience,2ofwhichareinsoftware development and 5 are in test automation testing. ThemainprojectsOstapworkedonaremobiletestautomationprojectsforbothAndroidandiOSplatforms,DekstopandWebprojectswherevarioustypesoftestingwereimplemented-fromfunctionaltoperformance/loadandstaticanalysis.Ostapbelieves

that automation should be applied not only for software testing, but also other activitieswheretimecanbesavedandautomationbringsbenefits:collectingmetricsof different kinds, implementation of helpful utilities such as bug migration utility or testresultssynchronizationwithtestmanagementsystem,etc.

About SoftServe

SoftServeisaleadingtechnologysolutionscompanyspecializinginsoftwaredevelopmentandconsultancyservices.Since1993we’vebeenpartneringwithorganizationsfromstart-upstolargeenterprisestohelpthemaccelerategrowthandinnovation,transformoperationalefficiency,anddelivernewproductstomarket.

Toachievethiswe’vebuiltastrongteamofthebrightest,mostinquiringmindsinthe industry, and we form close, collaborative relationships with our clients so we canreallyunderstandtheirneedsanddeliverintuitivesoftwarethatexceedstheirexpectations.

OurexperiencestretchesfromCloud,SecurityandUXDesigntoBigAnalyticsandtheInternetofThings,wehaveofficesacrosstheglobeanddevelopmentcentersacrossEasternEurope.Formoreinformationpleasevisitwww.softserveinc.com