Upload
others
View
7
Download
0
Embed Size (px)
Citation preview
#PNSQC2020 1
Quality-Focused Software Testing in Critical Infrastructure
© SEL 2020
Zoë OensSchweitzer Engineering Laboratories, Inc.
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 2
Scope
Quality
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 3
Better
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 4
Scope
5
4
3
2
1
1 2 3 4 5 6 7 8 9 10
Defects
Scope
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 5
Ranking Features By Criticality
Decision factors Factors weighedby importance
Featureevaluation
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 6
Kim and Kang Method
Give them weight
Rank each
feature
Calculate testing rank
Assign testing
based on rank
Create decision factors
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 7
Example of Critical Infrastructure
Safety (1.0)If this feature fails, how dangerous would this be?
Reach (0.8)If this feature fails, how many end users will this affect?
Usability (0.7)If this feature fails, will end users lose trust in the product?
Complexity (0.7)How complex is this feature?
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 8
Example of Critical Infrastructure
Decision factorPredetermined
weightRank(0–5)
Weighted rank (weight • rank)
Safety 1.0 5 5
Reach 0.8 3 2.4
Usability 0.7 3 2.1
Complexity 0.7 1 0.7
Feature risk 10.2
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 9
Example of Critical Infrastructure
Tiered testing– Preset quality rank tiers– Test to calculated tier
Percent coverage– Determine quality rank percentage– Test to meet total coverage
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 10
Time
Test processLessons
learned from manufacturing
Automation
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 11
Lessons Learned from Manufacturing
Well-defined processes
Fast issue evaluation
Experienced feedback
Cross training
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 12
Test Process
Specification
Code
Test process Test result
Inputs
Output
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 13
Suppliers and End Users
Trust Understanding Ownership
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 14
Test Process
Write Setup Execute Results creation
Specification
Code
Results
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 15
Test Write Improvements
Automation Better communication
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 16
Test Write for Repeated Software Elements
Machine-readable specification
Consistent testing
Assured quality
Button 1
Button 2
Button 3
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 17
Test Setup Improvements
Prebuilt setups Organization
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 18
Test Execute Improvements
Automation
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 19
Test Results Creation
Consistency Automatic creation
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 20
Hidden Costs of Automation
Training Maintenance Documentation
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 21
Cost of failure Fewer defects Metrics
Cost
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 22
Preemptivemetric
Estimate of improvement
Business Value Delivered
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 23
Test Satisfaction
Use real-timeuser feedback
Survey users after receiving device
or service
Give surveystrackable numbers
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 24
Change Failure Rate
Can only becompleted after
release
Is calculated percentage of
release to failures
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 25
Communication
#PNSQC2020 26
Quality-Focused Software Testing in Critical Infrastructure
© SEL 2020
Zoë OensSchweitzer Engineering Laboratories, Inc.
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens
#PNSQC2020 27
Questions?
Quality Focused Software Testing in Critical Infrastructure, Zoë Oens