Embed Size (px)
Citation preview
Measuring Your Way to Successful Automation
Brought To You By:
QASource Confidential
§ Over 500 engineers § 500+ years collective automation experience
§ 100,000+ automated test cases
§ 100+ automation frameworks
§ 100s of successful engagements
QASource: Automation Experts
Over 500 engineers
100+ automation frameworks 500+ years collective automation experience
100,000+ automated test cases
100s of successful engagements
QASource Confidential
Q1. How large is your QA team?
<5 Engineers
Attendee’s Poll
5-10 Engineers 11-25 Engineers >25 Engineers
Q2. What is the main reason you want to measure your test automation?
Track Progress Utilize Resources Effectively
Plan & Budget for Future
Detect Performers vs Non-performers
Q3. How would you rate your current automation metrics?
Top Notch: Many metrics that measure resource utilization, track
progress and show value
Pretty Good: Basic & consistent
metrics
Needs Improvement:
Some metrics, but they keep changing
No Metrics at All
QASource Confidential
Co-Founder of QASource 20+ Years Experience in QA Rajeev Rai,
CEO, QASource QASource Confidential
QASource Confidential QASource Confidential
Your challenges, if you let them, become your greatest allies.
– Richard Paul Evans
“ “
QASource Confidential QASource Confidential
Measuring Your Way To Successful Automation
QASource Confidential QASource Confidential
What You Will Learn
How to get a BUDGET for automation
How to demonstrate VALUE of automation
How to monitor team performance to OPTIMIZE your automation efforts
QASource Confidential QASource Confidential
Budget
Automation For
QASource Confidential
QASource Confidential
Breakeven in Automation
300 Hours of
Manual Effort Saved
QASource Confidential QASource Confidential
Effort Optimization
QASource Confidential
Effort Optimization
1000 1050
1150
1300
150
350
470 14%
30%
36%
0%
5%
10%
15%
20%
25%
30%
35%
40%
0
200
400
600
800
1000
1200
1400
1600
Sprint 1 Sprint 2 Sprint 3 Sprint 4
Per
cent
age
of M
anua
l Eff
orts
Sav
ed
Tes
t Cas
es
Sprint Cycles
Test Cases Automated Manual Effort Saved Total Test Cases Test Cases Automated Manual Effort Saved
QASource Confidential QASource Confidential
Task Distribution
QASource Confidential
Task Distribution: Sprint 4
QASource Confidential
Task Distribution: Sprint 4 (Drilldown)
QASource Confidential QASource Confidential
Team
Velocity Test Case Automation
QASource Confidential
Team Test Case Automation Velocity
30
32
34
36
38
40
42
44
46
48
50
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Tes
t Cas
es
Week
Expected maximum Expected minimum
36
45
49
Expected Maximum Expected Minimum Test Cases Automated
QASource Confidential QASource Confidential
Team Effort Distribution
QASource Confidential
Team Effort Distribution
Execution Analysis
Review & Fixing
Test Case Automation
Framework Development
Test Suite Maintenance
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Per
cent
age
Eff
ort
Week
QASource Confidential QASource Confidential
Engineer Velocity
Test Case Automation
QASource Confidential
Engineer Test Case Automation Velocity
8
9
10
11
12
13
14
15
16
17
18
19
20
Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8
Tes
t Cas
es
Week
Kevin Kamet John Expected Maximum Expected Minimum
11
18
19
QASource Confidential QASource Confidential
Engineer Effort Distribution
QASource Confidential
Engineer Effort Distribution
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60%
Execution Analysis
Review & Fixing
Test Case Automation
Framework Development
Test Suite Maintenance
Percentage Effort
Tas
ks
Kevin Kamet John
QASource Confidential QASource Confidential
Recap
Breakeven in Automation
Effort Optimization Task Distribution: Sprint 4
Task Distribution: Sprint 4 (Drilldown)
Team Test Case Automation Velocity
Team Effort Distribution
Engineer Test Case Automation Velocity
Engineer Effort Distribution
QASource Confidential QASource Confidential
Key Learnings
How to get a BUDGET for automation
How to demonstrate VALUE of automation
How to monitor team performance to OPTIMIZE your automation efforts
QASource Confidential
Q & A
QASource Confidential
FREE Whitepaper
Want To Learn More About QA Automation?
Visit us at QASOURCE.COM
qasource.com/20-metrics
QASource Confidential
