Upload
xbosoft
View
258
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Do you identify and separate different attributes for different goals? For example, do you use have specific metrics for quality but different ones for runtime, usability, system quality attributes? We welcome you to review Shilpa's insights here. Please visit our Xbosoft blog as well to gain additional insights on how to drive business decisions with software quality.
Citation preview
www.xbosoft.com
1
Webinar March 19 2014 11.00 AM EST
You are at the right place, it’s almost time….10 minute
countdown
2
WELCOME
XBOSOFT Software Quality Knowledge Center is proud to partner with and
introduce
Shilpa DadhwalCEO and Founder
SQE Labs, Inc.
Welcome to the session on ‘Measuring Software Quality –
Its Economic Value’
Shilpa DadhwalFounder & DirectorSQE Labs Inc.
Skype ID : shilpa2dEmail ID : [email protected]
Presented By:
On the callShilpa Dadhwal, Philip Lew and Jan Princen
Moderator: Carolyn Choh
Shilpa Dadhwal,Founder and CEO,SQE Labs Inc
Philip LewCEO, XBOSoft
Jan PrincenManaging Director, XBOSoft
Agenda
• Introduction XBOsoft• Software Quality – Quick review of Day 1• Moving on to Day 2
– Looking at Quality – what attributes to measure?– Implications for Senior Management– Implications for Project Management
• Benefits• QA & Next Sessions at 11 am -12 noon EST• Today’s Meeting March 19th
https://www3.gotomeeting.com/register/197591286
• Tomorrow’s meeting March 20th
https://www3.gotomeeting.com/register/886356774
• Specialize in: Software Quality Improvement
• Quality Process Improvement & Testing
• American Company• Located in: San Francisco, Amsterdam &
Beijing• Over 100 Software Quality Experts
XBOSoft Introduction
8
“Proven Results”
“Our products are very complex. We expected a one year ramp up time, but XBO was able to contribute to our release cycles in just 6 months. ."
- Development Vice President, Hyperion-Oracle
“
"The results are very encouraging. Thanks a lot for all the effort put into this project. We really appreciate your work."
- Manager, Quality Assurance, Autodesk
"Everything you've done is impressive and we are quite pleased with the quality of the test cases, well done!So keep up the good work."
- QA Director, CA
Welcome to the Session 2 on ‘Measuring Software Quality –
Its Economic Value’
Shilpa DadhwalFounder & DirectorSQE Labs Inc.
Skype ID : shilpa2dEmail ID : [email protected]
Presented By:
SQE Labs offers an independent Software Testing and Quality
Assurance services and is committed to improving quality by
providing superior QA solutions to customers
About USA Specialist Independent Software Testing Solutions Provider
Team of Software Quality Engineering professionals having strong
Techno-functional knowledge
Provides services in the areas of Outsourced Software Testing,
Software Testing Consultancy and Software Testing Trainings
Having Offices in Texas (US) and Chandigarh (India)
What is Software Quality?
What is its Economic Value?
How to mange Software Quality?
What are its Attributes?
Why do we need to measure it?
How do we measure it?
Issues while measuring it?
Guidelines for measuring it?
A Specialist Independent Software Testing Solution Provider Questions in Mind
Session 1
Session 2
Session 3
1. Define what quality means to you, for your software? (Give the highest priority
parameter)
‘High Reliability when Operating’
2. How can you measure this factor in your software?
No. of crashes encountered while working on the system for specified no. of
hours
3. Does this require any corrective or preventive action plan?
Yes
4. What action could be taken to improve it?
Testing of Memory Leaks in the application
5. How can you keep track of the same in any future product?
Record the no. of crashes encountered while running the application for
specified no. of hours
Case Study from Session 1
A Specialist Independent Software Testing Solution Provider
1. What is a METRIC?
2. What are DIFFERENT TYPES of Metric?
3. What are the desirable CHARACTERISTICS of a Metric?
4. What are the ATTRIBUTES of Software Quality?
5. How to NORMALIZE Metrics?
A Specialist Independent Software Testing Solution Provider Agenda
“When you can measure what you are speaking
about and express it in numbers you know
something about it;
but when you cannot measure it, when you cannot
express it in numbers, your knowledge is of a
meagre and unsatisfactory kind”.
William Thomson (Lord Kelvin)
QuoteA Specialist Independent Software Testing Solutions Provider
PollA Specialist Independent Software Testing Solutions Provider
How much time will you take to integrate this new module with the existing module?
• Much less
In Project A , 100 defects have been logged but in Project B, 200 defects have been logged. So which Project is better?
• Project B
How is the Software Quality of your product?• Good
Can you as a Manager , make any useful decisions from such subjective information?
1) Yes2) No
Measure: Quantitative indication of the extent, amount, dimension, or size of some attribute of a product or process. A single data point
Metrics: The degree to which a system, component, or process possesses a given attribute. Relates several measures (e.g. average number of errors found per person hour)
Direct Metrics: Immediately measurable attributes (e.g. line of code, execution speed, defects reported)
Indirect Metrics: Aspects that are not immediately quantifiable (e.g. reliability, productivity)
Common Terminology
A Specialist Independent Software Testing Solutions Provider
Products• Quantify the characteristics of the product being developede.g. Size, reliability
Process• Quantify the characteristics of the process being used to
develop the softwaree.g. Efficiency of fault detection, Effectiveness of Defect Removal
Project• Quantify the characteristics of the project and its executione.g. Staffing pattern over the lifecycle of the project, Productivity
Software Metric Classification
A Specialist Independent Software Testing Solutions Provider
Product metrics are measures of the software product at any stage of its development, from requirements to release.
Focus on the quality of deliverables
Product metrics are combined across several projects to produce process metrics
Metrics for the product:o The complexity of the Software designo Size of the final programo Number of pages of documentation producedo Defect densityo Customer Satisfaction index
Product MetricsAn Independent Software Testing Solution Provider
Focus on quality achieved as a consequence of a repeatable or managed process.
Process Metrics are measures of the software development process, such as
o Overall development timeo Average delay of milestone completion
Process metrics are collected across all projects and over long periods of time
Their intent is to provide indicators that lead to long-term software process improvement
It is defined mostly at a Strategic level
Process MetricsAn Independent Software Testing Solution Provider
To improve any process,
Measure specific attributes of the process
Derive meaning metrics from these attributes
Use these metrics to provide indicators
The indicators lead to a strategy for improvement
Process Improvement
A Specialist Independent Software Testing Solutions Provider
Project Metrics are measures of Software project and are used to monitor and control the project
Used by a project manager and software team to adapt project work flow and technical activities:
• Minimize the development schedule by making the necessary adjustments to avoid delays and mitigate problems
• Assess product quality on an ongoing basis and modify the technical approach to improve quality
Used in estimation techniques and other technical work
Project MetricsA Specialist Independent Software Testing Solutions Provider
Metrics collected in the past are used as a basis from which effort and time estimates are made for the current project
Metrics:o Effort or time per SE tasko Errors uncovered per review houro Scheduled vs. actual milestone dateso Number of changes and their characteristicso Distribution of effort on SE tasks
Tactical and Short Term.
Project MetricsAn Independent Software Testing Solution Provider
METRICS should be
– RELIABLE (free from random error)
– REPEATABLE (same entity, same environment, same visitors and same
evaluator)
– REPRODUCIBLE (same entity, same environment, same visitors but
different evaluator)
– AVAILABLE (constraint conditions)
– INDICATIVE OF IMPROVEMENT (of the entity quality)
– CORRECT (objective, impartial and precise)
– MEANINGFUL (about the entity behavior or quality characteristics).
Desirable characteristics of Metrics
An Independent Software Testing Solution Provider
A Metric is a qualitative or quantitative measurement of a specified Software Quality Attribute, for measuring Software Quality
Software Quality Metrics are subset of Software Metrics, that focus on quality aspects of product, process and project
Thus, in order to define Metrics for measuring Software Quality, we must know which Quality Attributes needs to be measured
The Quality Requirement Specification gives quantitative and qualitative values for these attributes that must be satisfied by the executable system
Measuring Software Quality
An Independent Software Testing Solution Provider
Software Quality attributes is a property of a software that can have some
qualitative or quantitative value and can be measured or observed.
Categories of Software Quality Attributes are :
1. Design Quality Attributes
2. Runtime Quality Attributes
3. System Quality Attributes
4. User Quality Attributes
5. Non-runtime Quality Attributes
6. Architecture Quality Attributes
7. Business Quality Attributes
Software Quality Attributes
A Specialist Independent Software Testing Solutions Provider
Conceptual Integrity
• Defines the consistency and coherence of the overall design
• Includes the way the components or modules are designed
Metric – i) Mean Time to Change
Maintainability
• Ability of the system to undergo changes with a degree of ease
Metric – i) Mean Time to Change
Reusability
• Defines the capability for components and subsystems to be suitable for
use in other applications
Metric – i) Implementation time
Design Quality Attributes
An Independent Software Testing Solution Provider
Interoperability
• Ability of a system to operate successfully by communicating and exchanging
information with other external systems written and run by external parties
Metric – i) Modularity
Manageability
• Defines how easy is it for system administrators to manage the applications
Metric – i) Implementation time
Reliability
• Ability of the system to remain operational over time
Metric – i) Mean Time to Failure
Scalability
• Ability of the system to either handle increases in load without impact on the
performance of the system or the ability to be readily enlarged
Metric – i) Horizontal Scalability
RuntimeQuality Attributes
An Independent Software Testing Solution Provider
Performance
• Indication of the responsiveness of a system to execute any action
Metric – i) Throughput
Security
• Capability of a system to prevent malicious or accidental actions outside of the
designed usage
Metric – i) No. of Security defects logged per KLOC
Availability
• Portion of the time the system is functional and working
Metric- i) Total Unavailability = Measured as a percentage of the total system
downtime over total no. of hours of service in a year period
RuntimeQuality Attributes
A Specialist Independent Software Testing Solutions Provider
Supportability
• Ability of a system to provide information helpful for identifying and
resolving issues when it fails to work correctly
Metric – i) Time taken to resolve an issue encountered in the software
Testability
• Measure of how easy it is to create a test criteria for a system and its
componentMetric – i) Modularity
SystemQuality Attributes
An Independent Software Testing Solution Provider
Usability
• Defines how well the application meets the requirements of the user
and consumer by being intuitive
Metric - i) Learnability
UserQuality Attributes
An Independent Software Testing Solution Provider
Portability
• Ability of a system to run under different computing environments
Metric – i) Install ability
Integrity
• Ability to make the separately developed components of the system work
correctly together
Metric – i) Access Control
Reusability
• Degree to which existing applications can be reused in a new applicationMetric- i) Implementation time
Modifiability
• Ease with which a software system can accommodate changes to its software
Non-RuntimeQuality Attributes
A Specialist Independent Software Testing Solutions Provider
Correctness
• Accountability for satisfying all requirements of the system
Metric - Defects per FP/LOC
Conceptual Integrity
• Integrity of the overall structure that is composed from a number of
small architectural structures
ArchitectureQuality Attributes
An Independent Software Testing Solution Provider
Cost
• Cost of a system with respect to time to market, expected project
lifetime
Metric – i) Time to Market
Marketability
• Use of the system with respect to market competitionMetric – i) Price Earning ratio
BusinessQuality Attributes
An Independent Software Testing Solution Provider
In Project A , 100 defects have been logged. But in Project B, 200 defects have been logged.
So which Project is better?
Can we compare the two projects, just by looking at the above data?
Additional information required:KLOC for Project A = 1000KLOC for Project B = 10,000
Defect Density of Project A = 100/1000= 0.1defects/KLOCDefect Density of Project B = 200/10000= 0.02 defects/KLOC
Thus, in Project A more defects have been logged per KLOC as compared to Project B
Normalization of Metrics
An Independent Software Testing Solution Provider
How does an organization combine metrics that come from different individuals or projects?
Depend on the size and complexity of the project
Normalization: compensate for complexity aspects particular to a product
Normalization approaches:Size oriented (lines of code approach)Function oriented (function point approach)
Normalization of Metrics
An Independent Software Testing Solution Provider
Case Study of Metrics Program
An Independent Software Testing Solution Provider
• Motorola followed Goal/Question/Metric paradigm
• Goals were identified, questions were formulated in quantifiable terms and metrics were established
• Few Examples are as follows:• Goals:
• Goal 1: Increase Software Reliability• Goal 2: Reduce Cost of Non-Conformance
• GOAL 1: INCREASE SOFTWARE RELIABILITY• Question: What is the rate of software failures and how does it
change over time?• Metric: Failure Rate = Number of failures/Execution time
• GOAL 2: REDUCE COST OF NON-CONFORMANCE• Question: What is the cost to fix post –release problems during
the month?• Metric: Cost of fixing Problems = Dollar cost associated with
fixing post-release problems within a month
To be continued in Session 3
Next…A Specialist Independent Software Testing Solution Provider
What is Software Quality?
What is its Economic Value?
How to mange Software Quality?
What are its Attributes?
Why do we need to measure it?
How do we measure it?
Issues while measuring it?
Guidelines for measuring it?
A Specialist Independent Software Testing Solution Provider Questions Answered
Session 1
Session 2
Session 3
Quality attributes are measurable or observable characteristics of
products and processes.
The quality requirements specification gives quantitative or
qualitative values for these attributes that must be satisfied by the
executable system.
Software Quality can be measured only if you know which
Software Quality Attribute should be measured and which metric
should be used for measuring the same.
A Specialist Independent Software Testing Solution Provider Few Remarks
Thank YouA Specialist Independent Software Testing Solutions Provider
India OfficeSQE Labs Inc.
116/8, Mohali (Chandigarh)India – 160062
www.sqelabs.com
US OfficeSQE Labs LLC,
6533, Preston Rd, Plano,TX - 75024
Contact me : [email protected]
Thank You!
See you tomorrowTomorrow’s meeting March 20th
https://www3.gotomeeting.com/register/886356774
Shilpa Dadhwal,Founder and CEO,SQE Labs Inc
Philip LewCEO, XBOSoft
Jan PrincenManaging Director, XBOSoft