Upload
kelley-golden
View
221
Download
0
Embed Size (px)
Citation preview
The Tools and The Tools and Techniques of SQATechniques of SQA
SASQAG, February 17, 2000SASQAG, February 17, 2000
SASQAG 2/17/00
Disclaimers
• The views and opinions expressed in this presentation are not necessarily those of my employer.
• Your results may vary.• You are free to use any
slide in your work, but if you do, please include my name and email.
SASQAG 2/17/00
Goals and Objectives
• Some Terms• Peer Reviews• Kickoff Meetings• Audits• Checklist and
Worklists
Sharpen YourSQA Skills
SASQAG 2/17/00
Software Quality Assurance
• Check software products and processes to verify that they comply with the applicable procedures and standards. (Process Checks/Audits)
• Review and measure the quality of software products and processes throughout development. (Peer Reviews)
• Provide software project management (and other appropriate parties) with the results of reviews and process checks.
• Work with the software project during early stages to establish plans, standards, and procedures to keep errors from occurring in the first place.
SASQAG 2/17/00
Validation and Verification
• Validation - Are we building the right product?
• Verification - Are we building the product right?
• Static Verification (of software) - Verification of a program by
examination and analysis of its program elements.
SASQAG 2/17/00
Customer vs. End User
• End User:
- The person who uses the software product in the value stream
• Customer:
- Anyone who is affected by the end product or intermediate software engineering deliverables being produced (reviewed).
SASQAG 2/17/00
Healthy Customer/Supplier Relationship
• Implies product use by customer!
• Implies choice
• Communications centers on expectations.
- How we identify expectations
- How we document and communicate them
- How we measure/improve them
SASQAG 2/17/00
Value vs Quality
Customers are concerned with value not just quality.
• "Value is not overpaying for quality."
• Value is..."the right combination of product quality, fair price, and good service."
* From Business Week, Nov 11, 1991
SASQAG 2/17/00
Low Quality in Software
• Unneeded ComplexityUnneeded Complexity• ReworkRework
-Wrong-Unclear-Missing-Extra
SASQAG 2/17/00
The Project
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
Requirements
FinishedProduct
Specs Design Code Test
Time
ProjectPlan
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
SASQAG 2/17/00
Where is rework introduced?
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
Requirements
FinishedProduct
Specs Design Code Test
Time
ProjectPlan
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
SASQAG 2/17/00
Where is rework found?
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
Requirements
FinishedProduct
Specs Design Code Test
Time
ProjectPlan
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
SASQAG 2/17/00
Where is rework fixed?
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
ENTRY
EXIT
Program ElementCreation
Requirements
FinishedProduct
Specs Design Code Test
Time
ProjectPlan
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
SASQAG 2/17/00
The Quality Lever
Requirements
Design
CodingDevelopment
TestingAcceptance
Testing
ImproveProduct
Production40-100X
30-70X15-40X
10X3-6X
1X
Data from Boehm: Software Engineering Economics, 1981
SASQAG 2/17/00
The Quality Lever
Requirements
Design
CodingDevelopment
TestingAcceptance
Testing
ImproveProduct
Production40-100X
30-70X15-40X
10X3-6X
1X
Data from Boehm: Software Engineering Economics, 1981
HighVisibility/Reward
SASQAG 2/17/00
The Quality Lever
Requirements
Design
CodingDevelopment
TestingAcceptance
Testing
ImproveProduct
Production40-100X
30-70X15-40X
10X3-6X
1X
Data from Boehm: Software Engineering Economics, 1981
HighVisibility/Reward
LowVisibility/Reward
SASQAG 2/17/00
Error/Defect Classification
Number of Errors
Minor Major "ShowStoppers"
ErrorSeverity
SASQAG 2/17/00
Error/Defect Classification
Number of Errors
Minor Major "ShowStoppers"
Find and remove as many Major and Show Stopper errors as early as possible.
ErrorSeverity
SASQAG 2/17/00
Terminology
• Major ErrorMajor Error- Something that is wrong
- High degree of probability that it will cause rework
- Rework later will cost much more if left allowed to migrate downstream.
• Minor ErrorMinor Error An issue or defect that is not major.
The costs of fixing later is not significantly higher than fixing now.
SASQAG 2/17/00
Quality Metric…Rework
• Number of Major Errors / Unit of Number of Major Errors / Unit of WorkWork
- ME/Page
- ME/Drawing Zone
- ME/KLOC
- ME/?
SASQAG 2/17/00
Initial Level
InformalChecking
DoWork
Down-stream
Customers
InputSource
Materials
ErrorsFound
ProcessProduct
EntryCriteriaLooselyDefined
ExitCriteriaForQualityNotDefined
Work Process
ENTRY
EXIT
SASQAG 2/17/00
Initial Level
InformalChecking
DoWork
Down-stream
Customers Re-Work
InputSource
Materials
ErrorsFound
ProcessProduct
EntryCriteriaLooselyDefined
ExitCriteriaForQualityNotDefined
Work Process
ENTRY
EXIT
SASQAG 2/17/00
Measure and Improve Product Quality
DoWork
Down-stream
CustomersRe-
Work
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
SW Inspection
InputSource
Materials
EntryCriteriaLooselyDefined
ExitCriteriaForQualityDefined
ENTRY
EXIT
Work Process
SASQAG 2/17/00
Improve Product Quality and Reduce Variation
DoWork
Down-stream
CustomersRe-
Work
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
InputSource
Materials
EntryCriteriaDefined
ExitCriteriaForQualityDefined
ENTRY
EXIT
Work Process
SW Inspection
SASQAG 2/17/00
To Improve Product AND Process
(EntryCriteria
Met)
DoWork
Down-stream
CustomersRe-
Work
InputSource
Materials
ErrorsFound
ProcessProduct
CrossfunctionalChecklists
ExitCriteriaMet
Statistical DataAnalysis and
Causal Analysis
PD
CA
DemingCycle
(Defect Prevention Process)
Repeatable Work Process
SW Inspection
SASQAG 2/17/00
SW Inspection Process Flow
Planning
Collect andStore Data
AnotherChunk?
Verify andApprove
Authors Moderator Inspection Team Members
OK
Re-Work
YES
NO
Other Inspection andCI Processes
ProduceProduct
Rework
ProductRelease
IndividualPreparation
OverviewMeeting
ErrorLoggingMeeting
7
SASQAG 2/17/00
SDLC
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Program ElementCreation
Program ElementCreation
Time
SASQAG 2/17/00
SW Inspections
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Program ElementCreation
Program ElementCreation
• SW Inspection• SW Inspection • SW Inspection• SW Inspection
SASQAG 2/17/00
Kickoff Meeting
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
Program ElementCreation
Program ElementCreation
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
• SW Inspection• SW Inspection • SW Inspection• SW Inspection
SASQAG 2/17/00
Task Kickoff MeetingTask Kickoff Meeting
• Identify people/organizations affected by errors in the activity
• Identify and document potential major errors (build/modify checklists)
• Confirm task/process entry conditions
• Identify resources
• Forum for learning
Reduce the amount of time needed by the author(s) to produce a deliverable by giving
technical and resource information to do the job.
SASQAG 2/17/00
Kickoff Meeting
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
Program ElementCreation
Program ElementCreation
• SW Inspection• SW Inspection • SW Inspection• SW Inspection
SASQAG 2/17/00
Walkthrough
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
• Walkthrough• Walkthrough
Program ElementCreation
Program ElementCreation
• Walkthrough• Walkthrough
21
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
• SW Inspection• SW Inspection • SW Inspection• SW Inspection
SASQAG 2/17/00
Walkthrough ProcessWalkthrough ProcessReduce the amount of time needed for re-work
by reviewing work completed to date and address questions and concerns.
• Led by author
• Discussion of deliverable style and format
• Examine technical alternatives
• Evaluate checklists, standards, and rules.
• Capture undocumented issues
• Forum for learning
SASQAG 2/17/00
Peer Review Methods
WalkthroughWalkthrough
Methods Typical Goals Typical Attributes
• Minimal overhead• Developer training• Quick turnaround
•Little/no preparation•Informal process•Meetings•No measurement•Led by the author
SW SW InspectionInspection
• Detect and report all defects efficiently and effectively.
• Usage of deliverable
•Formal process•Known Coverage Rate•Moderator•Checklists•Customer Reviewers•Measurement
Desk Desk CheckCheck
• Minimal overhead• Quick turnaround
•Little/no preparation•Informal process•No measurement•No Meetings•Done by the author
SASQAG 2/17/00
SQA Interventions
OutputENTRY
EXIT
Input Program ElementCreation
Start During End
• Kickoff Meeting
SASQAG 2/17/00
SQA Interventions
OutputENTRY
EXIT
Input Program ElementCreation
Start During End
• Kickoff Meeting
• Walkthrough
• Desk Check
SASQAG 2/17/00
SQA Interventions
OutputENTRY
EXIT
Input Program ElementCreation
Start During End
• Kickoff Meeting
• Walkthrough
• Desk Check
• SW Inspection
SASQAG 2/17/00
SQA Interventions
OutputENTRY
EXIT
Input Program ElementCreation
Start During End
• Kickoff Meeting
• Walkthrough
• Desk Check
• SW Inspection• Formal
Walkthrough• Formal Desk
Check
SASQAG 2/17/00
Peer Reviews
ENTRY
EXIT
Output...ENTRY
EXIT
Output
InputInput
Standards,templates, rulesand checklists
Standards,templates, rulesand checklists
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
TaskKickoffMeeting
• Walkthrough• Desk Check
• Walkthrough• Desk Check
Program ElementCreation
Program ElementCreation
• SW Inspection• Formal Walkthrough• Formal Desk Check
• SW Inspection• Formal Walkthrough• Formal Desk Check
• SW Inspection• Formal Walkthrough• Formal Desk Check
• SW Inspection• Formal Walkthrough• Formal Desk Check
• Walkthrough• Desk Check
• Walkthrough• Desk Check
SASQAG 2/17/00
Audit ProcessAudit ProcessSoftware Quality Assurance is to provide management with appropriate visibility into the process being used by the software developers and of the products being built.
• Determine:
• Conformance to policies, procedures, and standards• Adequacy of policies, procedures, and standards• Effectiveness and efficiency of policies, procedures, and
standards• Assess personnel familiarity to requirements and
documentation• Assure availability, use and adherence to software
standards
SASQAG 2/17/00
Audit Process
Un-satisfactory
Report?
OK
NO
YES
ProduceProduct
CorrectiveActions
AuditKickoffMeeting
DevelopAudit
Checklist
ConductAudit
Developers Auditor & Project Manager
Write-upReport &Findings
Auditor
IdentifyRequirements
Follow-upAudit
Re-Work
CloseAudit &
File END
Start
Reviewwith
Manager
SASQAG 2/17/00
Checklists
• Used to identify Major Errors• Usually in the form of questions• When used….
- By the authors to keep from inserting rework into the work stream
- By reviewers to find and report as many Major Errors as possible
- By future authors and reviewers
SASQAG 2/17/00
Generic Document Checklist
• Purpose Has the document satisfied the stated purpose?
• Redundant Is the same information presented elsewhere within the document?
• Acronyms Have all acronyms been defined prior to use?
• Accuracy Is the information wrong?
• Extraneous Is there information outside the stated scope of the document?
• Completeness Is necessary information missing?
• Standard Does the document follow the xyz documentation standard?
SASQAG 2/17/00
Generic Document Worklist• Before creating the deliverable...
- Hold Kickoff Meeting• Invite the right participants
• Plan for a successful document
• Build/Refine Checklists
• While working on the deliverable…- Use the checklists
- Use Walkthroughs and Desk Checks as needed to reduce rework (major errors).
• When finished with the deliverable…- As per SQA plan, perform the appropriate Peer review
- Send peer review data summary to SQA focal
SASQAG 2/17/00
Resources...• SASQAG WEB Site
(sasqag.org)-Past Meetings-Links
• UW Certificate Course• Commercial Courses• SASQAG Training Courses
SASQAG 2/17/00
Goals and Objectives
• Some Terms• Peer Reviews• Kickoff Meetings• Audits• Checklist and
Worklists
Sharpen YourSQA Skills