SQ13 IBM Software Group © 2005 IBM Corporation Using Bugs to Bring Developers and Testers Closer Together Michael Kelly Fusion Alliance [email protected]

SQ13 IBM Software Group 2005 IBM Corporation Using Bugs to Bring Developers and Testers Closer Together Michael Kelly Fusion Alliance [email protected]

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Whats the problem? Different pressures and perspectives that guide our work Testers are motivated by management to find and report problems within the system as quickly as possible Developers are motivated to complete code as quickly and accurately as possible in order to move on to the next problem Communication suffers

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. How can we improve communication? Sharing test scripts between teams Distributing the ability to execute smoke tests Performing runtime analysis together Using log files to isolate problems Using defect-tracking systems effectively Speaking face to face

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Strengths Probing Look for errors, faults and failures Know test design techniques that improve the likelihood of finding bugs within a limited period (Perceived) weaknesses Pessimistic, gloomy outlook Limited capability, stunted growth Uncreative, leading to conditioned behaviour 1 st Functional testing 2 nd Regression testing Only then think about the rest Testing skills and mindset Testers start by assuming the software is broken, and look for the problems Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Strengths Creative Innovative Optimistic (Perceived) weaknesses Optimistic Arrogant / Elitist Chaotic Error-prone Narrow focus / perspective Defensive need proof before theyll act Developer skills and mindset Developers start by looking for ways to get something to work Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. How we view ourselves and each other A big anchor that holds back their speedboat Elite intrepid, brilliant creators Information providers Gatekeeper Scapegoat for the work of others Creators of a unfinished go-kart, without brakes or steering testers developers How testers view How developers view testersdevelopers Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Drawing up the battle lines The Timely Testers The Dynamic Developers No mans land Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Symbiotic Relationships In the beginning there was the computer And the computer was without form: so developers were created to write code. Bugs multiplied: so testing was invented to find the bugs Testing was good: so testing was divided from development and the role of tester was formed Software testers depend on developers, and developers need testers if they want to deliver working software Therefore testers and developers have a symbiotic relationship. Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Bridging the gap The goal is to bridge the gap between testers and developers so they can work more productively One way is for the testers to become more technical so they can work more effectively e.g. by Working with developers more effectively Finding technical faults more easily Using appropriate testing techniques Understanding business-specific issues Courtesy of Julian Harty

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Share Test Scripts Between Teams Problems encountered by test scripts can be lengthy or difficult to reproduce If you make all of your test scripts available to the entire team you give developers the ability to: look at the script code look at the script logs re-run the scripts and watch them execute re-run them in local environments with debug information written to logs re-run them in conjunction with other tools

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Share Test Scripts Between Teams Provide the development team with a remote automated test-script execution box: The tools allows for distributed test execution Allows developers to work on your problem while get their own work done

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Share Test Scripts Between Teams Sharing test scripts with developers also enables everyone on the team to use the same tools used to develop the scripts Likely side-effects include developers taking the time to offer improvements to the scripts The more feedback from developers you get on your scripts, the more powerful they'll become The more you collaborate with developers, the more likely it is they will fix your problem

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Distribute the Ability To Execute Smoke Tests Every time the project team creates a build there's potential for something to go wrong. The problem with a bad build is that you won't necessarily know it's bad until you get in there and do some testing. The solution is to create a series of tests that exercise the system from end to end. A smoke test doesn't have to be exhaustive, but it should be capable of exposing major problems.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Distribute the Ability To Execute Smoke Tests If you don't have a smoke test, create one. If it's not automated, automate it.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Distribute the Ability To Execute Smoke Tests Automated smoke tests are particularly powerful: They're used often, possibly many times a day. They provide meaningful information (the system is at an acceptable state for testing and all services are upor not). They provide feedback quickly, typically in a matter of minutes. They're easy to execute and distribute.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Distribute the Ability To Execute Smoke Tests You can make the smoke test available to both testers and developers by using a central interface such as a project web site or simply using TestManager. If everyone has the ability to execute the smoke test and the results are simple to interpret, testers won't be pressured to provide this service for everyone else on the team.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Distribute the Ability To Execute Smoke Tests Most people will prefer not relying on someone else for such a simple task. Distributing the ability to execute smoke tests also increases communication between developers and testers. TestManager Email Etc

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Runtime Analysis Runtime analysis provides information on the following aspects of an application's execution: Execution paths Code coverage Runtime tracing Memory utilization Memory errors and memory leaks in native applications Memory leaks in.NET managed code and Java applications Execution performance Performance bottlenecks Threading problems

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Perform Runtime Analysis I've found that the most effective way to make sure that runtime analysis gets done is to start performing the analysis yourself. As a tester, you don't need to become a runtime analysis expert. In my experience, runtime analysis seems to be the most effective in increasing developer/tester communication (your mileage may vary).

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Perform Runtime Analysis Together By working with developers on runtime analysis, testers can: Learn more about the technologies Learn more about the code Learn about the technical issues that developers face and developers can: Learn more about what risks concern the testers

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Use Log Files To Isolate Problems Leverage log files as a means of capturing bugs and debugging. Often, when a problem happens behind the scenes of an application, you don't see the problem on the user interface. If developers give testers access to the execution log files for the application, the testers can use scripts to parse through the log files, looking for abnormalities and exceptions.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Use Log Files To Isolate Problems Building testability into the application is a partnership between testers and developers. Most developers want to help in any way they can, assuming that they're given time to do so. It's up to testers to let developers know what they need.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Use Defect-Tracking Systems Effectively Developers should tell testers what information is helpful: Steps taken Screenshots Source code Script/test case that can reproduce the bug Relevant log files Try not to be negative Try not to include suggestions

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Use Defect-Tracking Systems Effectively Before assigning priority and severity to defects, testers, developers, and project stakeholders should work out a prioritization scheme. Is it or ? If you're a tester, and you have a problem getting your point across using the bug-tracking tool, go and talk with the developer if possible.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Speak Face to Face There's no substitute for talking face to face with someone. The developer confirms that the problem is not some setting or incorrect configuration on my machine; it's actually a bug. The developer has seen the bug, so it's compellingmuch more so than if he tried and failed to reproduce the problem on his own. The developer can reproduce the problem more easily after seeing the steps performed in detail. The developer may already be aware of the problem and can indicate how it's being handled. The developer can request inclusion of additional information on the ticket to help jog his memory: configuration file, version number, the developer's initial thoughts and reactions, etc. The developer may be able to specify the person best suited to work on this problem. By watching the developer interact with the bug, I may discover other areas I need to test that might not have occurred to me otherwise.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Speak Face to Face The developer needs to be available and willing to help. Engaging the developer directly helps him or her better understand what you're trying to accomplish. Keep in mind that you may be disturbing or distracting the developer during a high- pressure time in the release cycle. If you approach developers frequently, establish some ground rules to avoid annoying the developers, and to avoid reducing their productivity at a critical time in the project.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Next Steps Pick a method and try it. Build a smoke test. If you're good with tools, check out Purify, Quantify, and PureCoverage. If you're less technical, simply start a dialogue with the developers and see what you can do to make life easier for them. Occasionally call a developer over and ask for help, even if you don't need it (just make sure the problem is worthy of help).

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Special Thanks To Julian Harty at www.CommerceTest.com for use of some of the materials from his EuroStar 2004 keynote presentation Technical Testing: Bridging the Gap Between Testers and Developers.

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Questions

IBM Rational Software Development Conference 2005 SQ13 Better Software. Better Business. Michael Kelly [email protected] Thank You

Documents

SQ13 IBM Software Group © 2005 IBM Corporation Using Bugs to Bring Developers and Testers Closer Together Michael Kelly Fusion Alliance [email protected]