Embed Size (px)
Citation preview
www.kit.edu
Automatically Improve Software Architecture Models
for Performance, Reliability, and Costs using Evolutionary Algorithms
PerOpteryx
Anne Martens
Karlsruhe Institute of Technology (KIT), Germany
Heiko Koziolek, Steffen Becker, Ralf Reussner
WOSP / SIPEW 2010
An animated version of this slide set (as PowerPoint ppsx)
can be found at http://sdqweb.ipd.uka.de/wiki/PerOpteryx
Software Design and Quality Group, IPD2 12.02.2010
Software Performance Engineering
Anne Martens
C1
A3 sec
Transform Solve
C2
A
Change component and deployment
2.5 secTransform Solve
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD3 12.02.2010
Not only Performance!
Anne Martens
C1
A3 sec
p(fail) 0.01%
$ 5700
Transform Solve
C2
A
Change component and deployment
2.5 sec
p(fail) 0.02%
$ 12000
Transform Solve
Optimise multiple criteria at once
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD4 12.02.2010
Multicriteria Optimisation
Anne Martens
Time
Costs
Architectural Candidate
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
A5s
$40K
Software Design and Quality Group, IPD5 12.02.2010
Multicriteria Optimisation
Anne Martens
Time
Costs
A
Generated & Evaluated
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
5s
$40K
Software Design and Quality Group, IPD6 12.02.2010
Multicriteria Optimisation
Anne Martens
Time
Costs
C
B
A
Pareto-optimal
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
5s
$40K$33K$20K
3s
2s
Software Design and Quality Group, IPD7 12.02.2010
Related Work: Quality Optimisation
Anne Martens
Rule-based approaches: Single quality only
Parsons2008, Cortellessa2009,
PerformanceBooster (Xu&Woodside2008),
ArchE (McGregor2007)
Multicriteria evaluation: No improvement
Bondarev2007, Grunske2007
Optimisation: Limited degrees of freedom
ArcheOpteryx (Aleti2009), Canfora2005,
Kavimandan2009, Sassy (Menascé2010)
Missing: Flexible multicriteria
optimisation at the design level
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD8 12.02.2010
PerOpteryx Approach
Anne Martens
Flexible
degrees
of freedom Multiple
qualitiesMulti-criteria
optimization
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD9 12.02.2010
Degrees of Freedom
Anne Martens
Variation point Which instance to use
for component type C?
Range of options C1, C2, or C3
Degree
of
freedom
Design decision that can still be made
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
C
Software Design and Quality Group, IPD10 12.02.2010
Types of Degrees of Freedom in CBSE
Anne Martens
Deployment
Allocation
Processing Rate
Number of Servers
Software
Component selection
Middleware selection
Component replication
Software configuration
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD11 12.02.2010
Instances of Degrees of Freedom
Anne Martens
C D
Component
selection
for C
Component
selection
for D
Processor speed
of server 1 Allocation of C
Allocation of D
Degree Matching Rule
... ...
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Allocation Each component
Processor speed
Component selection
Each server
Search alternatives
Software Design and Quality Group, IPD12 12.02.2010
Choice
C2
Server1
2 GHz
...
Search Problem
Anne Martens
Degree
Component selection C
Allocation C
Speed server 1
...
transform
candidate model
evaluate
Response in 2.5 s
P(failure) 0.02%
Cost $6000
initial model
evaluate
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD13 12.02.2010
Search Implementation
Anne Martens
[27,50,50,S3,S1,S2,C2]
c0
c2c1 c3 c4
c5c7
[35,40,30,S1,S2,S3,C2]
[35,40,30,S1,S2,S2,C2] [26,45,44,S3,
S2,S1,C2a][36,35,28,S2,S3,S2,C2]
[55,40,30,S1,S2,S3,C2] [35,45,44,S1,
S2,S2,C2a]
[26,45,44,S1,S2,S1,C2a]
[29,40,50,S2,S3,S3,C2a]
c8c6
NSGA-II[Deb2002]
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD14 12.02.2010
Quality evaluation
Anne Martens
Palladio
Component
Model[Becker2007]
PCM2Cost[Martens2010]
Cost
PCM2DTMC[Brosch2009]
Reliability
PCM2LQN[Koziolek2008]
Performance
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD15 12.02.2010
Case Study with PerOpteryx (1/2)
Anne Martens
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD16 12.02.2010
Case Study with PerOpteryx (1/2)
1235 candidates
58 Pareto optimal
8h running time
Anne Martens
Response T
ime
Component allocation
Processing rates
Component selection
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD17 12.02.2010
Case Study with PerOpteryx (2/2)
Anne Martens
0
2
4
6
8
10
12
14
16
18
20
0 50 100 150 200
Avera
ge
Resp
on
se T
ime
(Seco
nd
s)
Costs (K$)
All candidates
Pareto-optimal candidates
Initial Candidate
RT: 2.2 s
POFOD: 6E-4
Cost: 98
RT: 1.34 s
POFOD: 5.2E-4
Cost: 69.83
Only four, but faster
servers
Different Webserver
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD18 12.02.2010
Future Work
Anne Martens
• Performance heuristics
• Requirement support
• More degrees of freedom
Short term
• Handle uncertainty of predictions
• QoS process integration
Long term
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
Software Design and Quality Group, IPD19 12.02.2010
Conclusions
Anne Martens
Flexible
degrees
of freedom Multiple
qualitiesMulti-criteria
Optimization
Motivation – Related Work – Approach – Case Study – Future Work –Conclusion
http://sdqweb.ipd.kit.edu/wiki/PerOpteryx
Automated Architecture Improvement
