Copyright © 2005 QA Insight, Inc. All rights reserved. 1 A Review of Software Inspection Techniques Getting Higher Returns from Your Review Processes Karina

1 Copyright © 2005 QA Insight, Inc. All rights reserved.

A Review of Software Inspection TechniquesA Review of Software Inspection TechniquesA Review of Software Inspection TechniquesA Review of Software Inspection Techniques

Getting Higher Returns from Your Review Processes

Karina Gamble, QA Insight, Inc.

[email protected]

661-799-9279

Presented For SCQAA Inland Empire ChapterJanuary 12, 2006

204/19/23

Copyright © 2005 QA Insight, Inc. All rights reserved.

AgendaAgenda

Introduction Review Types – similarities and differencesBenefits of software inspections/reviewsReasons for not establishing company-wide

review processesRecommendations and solutions

304/19/23


IntroductionsIntroductions Founder of QA Insight, Inc. Specialties: establishing processes, testing, training

and mentoring – soon to launch eMentoring Co-founder of the San Fernando Valley chapter of

SCQAA serving the northern LA county Our SCQAA overall objective to provide

educational talks educating people in various roles or job functions – Testers, PMs, BAs, and yes even developers towards achieving a common goal Improve Software Product Quality

404/19/23


Static AnalysisStatic AnalysisStatic vs. dynamicStatic means visual examination not

examination by executionStatic analysis is sometimes improperly

termed as static testing based on: IEEE std. Glossary 610.12-1990

Static analysis does not have to be 100% manual, in fact you need to use tools to help (not replace) your analysis process

504/19/23


Static Analysis TypesStatic Analysis Types

Inspections

Peer ReviewsPassarounds/Deskchecks

Walkthroughs

Ad Hoc Reviews

Pair*Programming

Most Formal Least Formal

•Can be considered as an informal review type but more of a s/w development style (by Karl E. Weigers)

604/19/23


Fagan’s Inspection Process

Moderator

Planning Overview Preparation Meeting Rework Follow-up

Inspectors

Overview Preparation Meeting Follow-up

Author

Overview Meeting Rework Follow-up

Reader

Overview Meeting Follow-up

Recorder Scribe Defect entry

704/19/23


InspectionsInspections

Most formal – same as formal inspection Developed by Fagan - IBM in 1976 Multiple roles in the review team:

Moderator, Inspector(s), Author, Reader, Recorder Several sequential activities Author is not the moderator and does not lead the

meeting Reader goes over small chunks of work product at a time

in the meeting

804/19/23


Peer ReviewsPeer Reviews

Less formal but still plannedParticipants still get materials to review before

the meetingNo overview or follow-up meetings Author may lead the meeting (unlike

inspections)Not as efficient in finding defects as

inspections areBigger chunks are reviewed

904/19/23


WalkthroughsWalkthroughs

Informal review meeting led by the author Primarily used for education and soliciting feedback Error detection happens during the meeting not

during preparation phase Participants are not expected to be familiar with

code or design, or any other item under review Level of detail reviewed is up to author’s discretion

1004/19/23


Passarounds Passarounds Informal review Good way to start a review culture Find people in your team that you respect and trust to

review Distributing product to review to multiple people at

the same time No meeting is held – just independent review Each reviewer gets to see the comments of others to

minimize redundancy Can end up with very hot debates – Be careful!

1104/19/23


Yet Another Famous Cost of Defects Slide!

1204/19/23


Benefits of Inspections over TestingBenefits of Inspections over Testing

Inspections are better in finding defects than just testing

Symptoms of problems instead of problemsTesting alone cannot tell you how

maintainable or clear the code is

1304/19/23


Benefits of Code InspectionsBenefits of Code Inspections Numerous studies have shown the benefits of static

analysis HP’s inspection program measured an ROI of 10 to 1 Inspection reduced the cost of finding error by factor of 10 at

AT & T Bell labs. Studies have shown the benefits of code inspections

65% errors found from Fagan’s inspections 35% errors found from tests

3.25 errors/unit effort of inspection 0.44 errors/unit effort of testing Fagan inspections are 7.4 times more productive than

testing! (Note: You can use defects founds in inspections to predict defects remaining)

1404/19/23


Benefits of Inspections for TestersBenefits of Inspections for Testers

“Testers can spend more of their time finding more subtle bugs Instead of finding bugs that developers should have found or better yet, should have prevented from introducing into code/design to begin with.”

15 Copyright © 2005 QA Insight, Inc. All rights reserved.

So Why Doesn’t Every software So Why Doesn’t Every software group do Inspections?group do Inspections?

Who Does Software Inspections???

1604/19/23


ReasonsReasons

General lack of knowledge about reviewsNot enough training Cultural inhibitors (attitudes and past

experiences) Improper planning Improper use of review metrics

1704/19/23


Why are Code Inspections often not being done?Why are Code Inspections often not being done?

Very time consuming Code is too complex Reviewers are usually not prepared Manual inspection of OO programs is not easy It is a manual process that you need to rely on the

expertise and experience of your reviewers Inconsistent results Limited on how much code you can inspect

1804/19/23


Our ChallengesOur Challenges

QA teams tend to focus more on testing - not by choice sometimes

Difficult to break territorial and cultural barriers Management does not see the benefits, so no support

for reviews and inspections Lack of knowledge and tools Myths and misconceptions – Management by

opinions rather than by facts – no metrics Status-quo vs. real positive changes

1904/19/23


So What do we do now?So What do we do now?

Training! Training! Training! QA managers justify reviews to your organizations QA managers define and document inspection/review

processes in the QA plan Collect review metrics Summarize defect data and report QA along with the DEV team to communicate the

benefits Do root cause analysis

2004/19/23


Tools and TechniquesTools and Techniques

Static analysis tools and reading techniques Static Analysis tools:

To save time during code inspections and increase productivity run a code checker tool first – check for violations against standards, memory leaks, unhandled exception

These tools still don’t replace manual inspections Reading Techniques are very useful for reviewing

requirements Helps reviewers find more defects efficiently

2104/19/23


Static Analysis ToolsStatic Analysis ToolsAutomate some of the manual checkingCoding standardsMemory leaksUncaught runtime exceptionsRace conditions – different threads accessing

the same variableDeadlock conditionsSecurity vulnerabilitiesExample tools: Jtest, Jlint, PMD

2204/19/23


Static Analysis with JTest by ParasoftStatic Analysis with JTest by Parasoft

Checks for 380 coding standards and automatically corrects the rule violations

Checks for a specific comment structure format validating the comment matches with the code – JContract

Automates unit testing – Black box testing at the unit (class) level

2304/19/23


Static Analysis with JlintStatic Analysis with Jlint Jlint will check your Java code and find bugs,

inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.

Finds unreachable code Threading/lock problems More than just coding standard checking Find bugs that even manual inspections can’t find – not

even by experienced staff! Jlint is extremely fast

2404/19/23


Static Analysis with PMDStatic Analysis with PMD PMD scans Java source code and looks for potential

problems like: Empty try/catch/finally/switch blocks Unused local variables, parameters and private

methods Empty if/while statements Overcomplicated expressions - unnecessary if

statements, for loops that could be while loops Classes with high Cyclomatic Complexity

measurements

2504/19/23


Reading TechniquesReading Techniques Formal software inspections (e.g. Fagan’s)

Focus on structure, frequency of meetings Organizational aspects Not on technical aspects – how to review Use Ad hoc reading techniques

Reading techniques Optimize inspections – find most defects with less efforts Tackle the technical aspects Provide a structured process to help the groups improve

their review process Can help you find out what defects passed thru the process

and what to do to improve the process

2604/19/23


Reading TechniquesReading Techniques Ad Hoc

No structure in place Everyone attempts to look for all classes of defects

Checklist-based Set of questions under each review category One checklist by all inspectors

Scenario-based /Perspective-based – each reviewer gets to review artifact based on his/her own role or based on specific set of usage scenarios

2704/19/23


Scenario/Perspective Based Scenario/Perspective Based

Each reviewer assumes a specific perspective – e.g. tester, designer and customer, maintainer, etc.

Reviewers are also required to produce a high level work products – not passive reading

Specific goals and questions for each perspective or scenario

2804/19/23


Reading Techniques BenefitsReading Techniques Benefits

Systematic FocusedGoal-orientedTransferable via training Inconclusive studies, however, as to

whether or not PBR is significantly better than CBR

2904/19/23


ConclusionConclusion Inspections are better and cheaper in finding defects

than testing alone Earlier detection of defects are possible by

inspections Manual inspections do take a lot of time and may not

catch all defects for complex multi-threaded OO software

Static Analysis tools and Reading Techniques alleviate some of these problems

QA plays a key role in leading the inspection process and educating staff in processes, procedures, static analysis tools and in reading techniques

3004/19/23


LinksLinks www.sourceforge.net www.opensource.org “How Perspective-Based Reading Can Improve Requirements

Inspections” - IEEE Software July 2000 Open source tools: http://www.QAInsight.com/links.htm IEEE Software Engineering BOK: http://www.swebok.org “Peer Reviews in Software, A Practical Guide” – Karl E.

Weigers Automated Requirement Measurement Tool available for

free – see me or send me an email for more details www.scqaa.us (Karina is programs and education chair – we meet

every 1st Wednesday of the month)

Documents

Copyright © 2005 QA Insight, Inc. All rights reserved. 1 A Review of Software Inspection Techniques Getting Higher Returns from Your Review Processes Karina