Upload
rinky25
View
1.020
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
©2009 Ness Technologies – Proprietary and Confidential
How testing and testers evolved within the time and benefitsof testing tools during mature testing lifecycle.
Testwarez 2009, Krakow
Testing Lifecycle: Don’t be a fool, use a proper tool!
Zdeněk Grössl & Lucie Riedlová
20. 10. 2009
www.ness.com
Our aim: to use the mature Testing Lifecycle
Testing is strategic for us
Over 2.500 testers (18 countries worldwide) sharing their corporate knowledge
Different modes – test labs, consulting, custom development, SAAS
Proof? Satisfied customers around the world• Telco
• IT
• Public and Military
• In CZ customers from TOP 100
Let’s see that “Lifecycle”
Testing LifecycleTesting Lifecycle
www.ness.com
What do we mean by “Lifecycle” :-)What do we mean by “Lifecycle” :-)
www.ness.com
From random clicks to Testing LifecycleFrom random clicks to Testing Lifecycle
test coordinadionprogress of testingconnection to management
test casesRQ coveragetest scripts and test datamanagementautomatization
managing test teamreporting about status
test functionality and requirements before
release bug report
inefficient testingdifficult to reproduce tests
uncoordinatefd action
feedback on test processimputs for next iteration
Tools can help a lot
!How to handle this
?
www.ness.com
Test ManagementTest Management
www.ness.com
Testing tools – Test managementTesting tools – Test managementDynamic changes on smaller projectDynamic changes on smaller project
Project:Financial sectorTesting implemented from scratch, about 12 team membersDynamic development processTesting documentation in Excel sheets
Sudden change: Increase of RQs40% new people hiredIssues with versions of Excel sheetsInformation about changes was not reliableA need to formalize testing process
Solution: Tool for test ManagementEasy to setupOnline accessSuitable for smaller teamsKeeps history of records
SPIRA TEAM was selected as pilot
Benefits:Testing was formalizedHistory and more detailed information about testing processSPIRA extended to other projects
www.ness.com
Testing tools – Test management – screenshotsTesting tools – Test management – screenshots
Example: SPIRA TEAM
Project Home
My Page
www.ness.com
Testing tools – Test managementTesting tools – Test managementToo much extra work at world’s HW leaderToo much extra work at world’s HW leader
Project:UAT testing of world-wide application for Sales (12.000 users)Two successive releases were tested, about 1 year between releasesRole: Regional Test Coordinator (40 testers), reporting about progressComplete Worldwide team: 140 members
First release: Shared Excel filesToo much extra workThree different locations to login into before testing each day Reporting for management: combine excel sheets and reports from Bugzilla
Take care about testers (motivate, coach, inform about bug fixes…)
Extremely challenging!
Second release: HP Mercury QCA pro tool: all in one; no multiple loginsSimple administration of all modulesUser-friendly testing Easy overview of testing progress,Bug tracking and traceabilitySophisticated reports for management.
Benefits:More effective testing on such a large project. Clear overview for managementFocus on testing, not on extra activities!
www.ness.com
Testing tools – Test management – screenshotsTesting tools – Test management – screenshots
Example: Evolution from Excel to Mercury QC
www.ness.com
Testing tools – Test managementTesting tools – Test managementSummarySummary
www.ness.com
Test AnalysisTest Analysis
www.ness.com
Testing tools – Test Analysis and DesignTesting tools – Test Analysis and DesignWe got lost in Test CasesWe got lost in Test Cases
Project:Financial sector (saving bank)Deliver complete QA for new module of client’s internal systemOne customer, three dislocated suppliersTotally about 70 new RQs (about ¼ of the whole system)
All in Excel sheets (RQ, TC, test data)
Excel sheet too complexDifficult to review test coveragePoor reporting optionsIssues with access to filesSeveral errors occurred
Requirements on the toolAchieve and review RQ coverage easilySupports evaluation of TCsUser-friendly environmentQuick and easy setupAccessible worldwideSPIRA TEAM selected
Benefits:Administrative / human errors avoidedGreat overview for test analystModule delivered with remaining 6 minor bugs (totally over 140)Transparent for client and increase of our credit
www.ness.com
Testing tools – Test Analysis – screenshotsTesting tools – Test Analysis – screenshots
Example: SPIRA TEAM
Map TC to RQ
RQ coverage overview
www.ness.com
Testing tools – Test Analysis & DesignTesting tools – Test Analysis & DesignSummarySummary
www.ness.com
Test DesignTest Design
15
www.ness.com
Testing tools – Test Design and Analysis Testing tools – Test Design and Analysis Regression testing under stressRegression testing under stress
Project:SPIRA TEAM as testing tool on the project (Finance)Releases 8x per year, hot-fixes need to be released within 8 hoursMuch time spent on regression testing
Need of change:Regression testing not 100% effective under stressAn error was delivered to the customerMore formalized regression tests were not good solution
We need tool for automated testingOptions: Test Complete, Win RunnerSituation on project did not enable it. Good luck: our SW-coder recommended Selenium – add-on for Firefox
Benefits:Future errors avoidedConnection with SPIRA to automatically record test runsIt is useful to discuss testing issues with developers as well
www.ness.com
Testing tools – Test Testing tools – Test Design Design – – screenshotsscreenshots
Example: SPIRA TEAM together with Selenium► Selenium IDE – write and run tests in Firefox
► Selenium Remote Control – runs tests in multiple browsers and platforms(IE7, Safari, Opera, Firefox, Windows, Linux, Solaris, others)
www.ness.com
Testing tools – Test Design and Analysis Testing tools – Test Design and Analysis More than was requested (by Silvia More than was requested (by Silvia ValientovValientováá))
Project:Biggest insurance company in CZ33 methods, 6 web services were tested separately
What was requested?Test the methods like a processCustomer: Buy us a new testing tool!SoapUI was known tool for customer
Why SoapUIThe old tool has requested functionalityProperty Transfer and Xpath Assertion WizardLink responses with requests Support of E2E testing
Benefits:Client can now easily perform this testingNo extra costs required to purchase licensesEasy upgrade from Integration testing to E2E testingWe found what customer really NEEDS, not what he just REQUESTS
www.ness.com
Testing tools – Test Testing tools – Test Design Design – – screenshotsscreenshots
Example: The SOAP UI Story► Used for Web Service Testing
www.ness.com
Testing tools – Test Design and AnalysisTesting tools – Test Design and AnalysisSometimes just look around for a tool!Sometimes just look around for a tool!
Project:Internetbanking system (Lead Czech saving bank)Load testing was newly requiredTested system was developed by NESS Czech
Requirements on the toolSimulate simultaneous access of 300 users per 10 minutesMinimum time and costs to set up a toolRestriction on client’s side
Why Visual Studio 2005 (for Testers)?Tested application was programmed in this environmentDevelopers were familiar with the toolQuick implementation and results in one week
Benefits:Developers can be very good test designersScope of our services for the client broadened
www.ness.com
Testing tools – Test Design & AnalysisTesting tools – Test Design & AnalysisSummarySummary
automated testing
language & environment independent
connection to other tools
www.ness.com
Test ExecutionTest Execution
22
www.ness.com
Testing tools – Test ExecutionTesting tools – Test ExecutionTesting scope broadened (by Jaroslav StrharskTesting scope broadened (by Jaroslav Strharskýý))
Project:Major Telco company in CZExcel for RQs and TCsOriginally only 2 testers planned
Need of change:25 testers involvedExcel files too complicatedSeveral bugs got misrouted
Testing with HP Mercury Quality CenterTesters complained about difficulty of the tool and level of formalismMonitoring of users was necessaryFormalized and free tests were put in place
Benefits:Inform business testers about fixed bugs they discovered: they will be satisfiedTraining for testers takes approx. the same time regardless of a toolTesting with this tool is as quick as without a tool. But consider the benefits!
www.ness.com
Testing tools – Test Testing tools – Test Execution Execution – – screenshotsscreenshots
HP Mercury Quality Center
www.ness.com
Testing tools – Test ExecutionTesting tools – Test ExecutionNo time for new toolsNo time for new tools
Project:TCs and bugs maintained in Word and ExcelCommunication to developers was face-to-faceBugs were fixed almost immediately
Need of change:Model does not work under stressBig volume of reported bugs (+300%)Management required reporting
Only 3 weeks to deliver the testing!
Why JIRA from Atlassian?JIRA is used in NESS as internal tracking tool (over 750 users)Every employee is familiar with itSkillful administrators
Benefits :No more misunderstandingsReports for managementAll set up in less than 1 day
www.ness.com
Testing tools – Test Testing tools – Test Execution Execution – – screenshotsscreenshots
JIRA by Atlassian
www.ness.com
Testing tools – Test ExecutionTesting tools – Test ExecutionSummarySummary
www.ness.com
Summary? There is no perfect tool!► Every tool suits something
► If the tool is TOO complex and TOO perfect: Is it still easy to maintain?
Is there anything else we should be aware of? Answer these questions about testing tools:► Easy to install?
► Friendly to use?
► Even for non-IT users?
► Any additional features?
• Notifications,
• Attachments, screenshots
• Reporting options
• Upgrades
• …
► Could you customize the tool?
► When you tried it, does it serve you well?
Overall summaryOverall summary
www.ness.com
What do we use?► We – Lucie and Zdenek – like SPIRA (and several our customers too )
We use tools from our partners (Atlassian, HP, IBM, Inflectra, Microsoft…)
Use tool best for the purpose and for the client ► Excel? OK, but educate clients! And remember their maturity!
Which tool should you use? See questions on previous slide!► Summarize your needs
Don’t you know your needs?► Talk to someone who helps you define them
Most powerful tool: OPEN MINDS!
And what about testing at NESS Czech?And what about testing at NESS Czech?
www.ness.com
Complete QA solution in one package
Server and thin-client-based application
Very user-friendly environment
Rich reporting options
Document repository
Option to connect to LDAP
Positive feedback from clients
Even from business (non-IT) people
After one year over 30 projects and over 60 external users
SPIRA TEAM by Inflectra SPIRA TEAM by Inflectra
http://www.inflectra.com
www.ness.com
bug tracking
project management
agile software development
technical support and help-desk
used by more than 10 000 organizations
create any kind of issue in seconds
keep your projects and issues organized
track all activities in your project
designed with both business and technical users in mind
easy to administer and customize
Because you’ve got issuesBecause you’ve got issues
www.ness.com
Our question: What did you take from this presentation?► There is no perfect tool!
► Discuss and ask questions before you decide to use certain tool!
► Ask those who know about this stuff…
We are ready for your questions…
… and for the chat about testing – later :-)
Questions and AnswersQuestions and Answers
It looks like end... but it is rather start