Upload
marat-zhanikeev
View
295
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Black Swan is a term used in economics to discuss resilience of complex systems to failures. Black Swans are directly applicable to clouds which are basically large resource economies (where real economies are based on money). This paper develops a framework which can profile Black Swans in clouds at arbitrary levels of aggregation and make placement or migration decisions based on the outcome of profiling.
Citation preview
.
The Black Swan Property
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 2/19...
2/19
.
The Black Swan
• theBlack Swanproperty is well known 01
• how common do you thinkblack swans are around theworld?
• are they common in Japan?
• I have two in a pond near myhome...
01 N.Taleb "The black swan: the impact of highly improbable" Penguin (2008)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 3/19...
3/19
.
The Black Swan : Illustration (trains)
• basically, rareevents have more
impact
Events (ordered by prob.)
Freq
uenc
y =
prob
abilit
y
Accounted for
Un-accountedfor
Scheduling Problem(±10m)
Humanaccident(±40m) Train brakes
down(±4h) 9.11, 3.11,
….(±3d)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 4/19...
4/19
.
The Black Swan : Convex is Bad
.Black Swan Property..
.
When complex systems have convex behavior in their components, they have blackswans.
• convex = Long Tail
• disasters hide in the Long Tail 09
• because disasters are rare, they are normally unaccounted 06
• when disasters happen, we are completely unprepared
09 R.Kennet+1 "Quality, Risk and the Taleb Quadrants" IBM Research (2009)
06 A.Nafday "Consequence-based stuctural design ... for black swan..." Elsevier ...Safety (2011)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 5/19...
5/19
.
Black Swan in Engineering
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 6/19...
6/19
.
Two Ways to Handle Black Swans.Method 1..
.fast response ... when Black Swan is detected, react promptly
• popoular in Black Swan engineering today 05 06
.Method 2..
.
robust design• catch Black Swans before they happen
• this method today!
• not very popular in engineering
• the problem of noticability threshold05 L.McGinty+1 "Black Swans, Gray Cygnets and Other Rare Birds" Springer LNAI vol.5650 (2009)
06 A.Nafday "Consequence-based stuctural design ... for black swan..." Elsevier ...Safety (2011)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 7/19...
7/19
.
Black Swans in Clouds : the Model• each object in clouds (VM, etc.) gets asrankRk
Rk =∑i=1..t
wiFk(v, t) (1)
• risk potential of each object isPk Pk = |Rk,i − Rk,i−1|. (2)
• evaluation of riskE for a given system /collection of objects
E = var({Pk}), (3)
• evolution of evaluation in timeEVO EVO = var({Et}). (4)
• finally, optimization (minimization of risk) andstatistical evaluation of risks are possible
minimize∑i=1,n
∑j=1,m
Pij. (5)
• 2nd order norm and variance L̂ =
∫ K
0(K−x)f(x)dx and σ2
L =
∫ K
0(K−x)2f(x)dx.
(6)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 8/19...
8/19
.
Specific Black Swan Method forClouds
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 9/19...
9/19
.
Black Swan in Clouds: Terminology
• VM : virtual machine◦ VMs can migrate when necessary 04◦ we assume that we can measure the load◦ or study traffic 02◦ ... or use any other method for performance measurement
• PM : physical machine in the a cloud◦ the load is the sum of VM loads
• APP : a multi-VM application
04 myself+1 "VM Migration Avoidance based on Flow Workload Classification" IEICEソサエティ学会 (2013)
02 1+myself "Active Network Measurement: Theory, Methods, and Tools" ITU Japan (2009)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 10/19...
10/19
.
VM Load : Flash Crowd Model
02 1+myself "Active Network Measurement: Theory, Methods, and Tools" ITU Japan (2009)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 11/19...
11/19
.
PM and APP Ranks
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 12/19...
12/19
.
Simulation Setup• methods: do nothing, PM swans, and APP swans
• D : history depth (for E)
• p : per-VM probability of a Flash Crowd
• Lmin, Lmax, La : VM utilization levels and slope
• PMs : number of PMs, fixed at 30 (5x6) grid
• APPs : number of applications, each needs 3 VMs• R : how many fixes per epoch
• PM load to response time
T =1
2[(L− n) +
√(L− n)2 + k
1− L] (7)
◦ load above 0.9 considered dangerous (all VMs are impacted)
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 13/19...
13/19
.
Models
1. do nothing◦ do not try to prevent Black Swans
2. PM swans◦ prevent Black Swans in PMs
3. APP swans◦ prevent Black Swans in APPs
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 14/19...
14/19
.
Results : Snapshot 1 : Do Nothing
• several dark PMs can occur
• ... color is utilization
• raw PDF movie in separate file!
6 0 3 1 4
5 8 1 6 4
1 3 10 7 1
4 5 1 4 8
1 0 5 4 10
2 0 10 6 4Method#do.nothingEPOCH#387
VM LOAD
EVALS
3 JOBS
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 15/19...
15/19
.
Results : Snapshot 2 : PM Swans
• better overall spread
• PDF movie separately!
0 1 8 3 4
4 5 4 3 0
9 6 3 4 1
3 1 4 7 2
6 4 6 4 6
1 1 3 4 6Method#PM.swansEPOCH#480
VM LOAD
EVALS
3 JOBS
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 16/19...
16/19
.
Results : Snapshot 3 : APP Swans
• better overall spread
• PDF movie separately!
1 9 0 8 5
7 1 0 8 7
6 6 1 4 2
2 0 7 5 5
0 6 4 6 7
4 7 2 3 1Method#APP.swansEPOCH#465
VM LOAD
EVALS
3 JOBS
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 17/19...
17/19
.
Results : Catching Swans (red dot!)
4 6 8 10 12 14 16 18 20 22 24 26Number of Group-Failed PMs (0..30)
00.20.40.60.8
11.21.41.61.8
22.22.42.6
log(
runs
bet
wee
n fa
ilure
s)D=10p=0.05APPs=30R=10
do.nothingPM.swansAPP.swans
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 18/19...
18/19
.
That’s all, thank you ...
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 19/19...
19/19
.
[01] N.Taleb (2008)The black swan: the impact of highly improbablePenguin
[02] 1+myself (2009)Active Network Measurement: Theory, Methods, and ToolsITU Japan
[03] myself+1 (2012)Popularity-Based Modeling of Flash Events in Synthetic Packet TracesIEICE CQ研
[04] myself+1 (2013)VM Migration Avoidance based on Flow Workload ClassificationIEICEソサエティ学会
[05] L.McGinty+1 (2009)Black Swans, Gray Cygnets and Other Rare BirdsSpringer LNAI vol.5650
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 19/19...
19/19
.
[06] A.Nafday (2011)Consequence-based stuctural design ... for black swan...Elsevier ...Safety
[07] G.Wright+1 (2009)Decision making and planning ....low predictability......Journal of Forecasting
[08] T.Aven (2009)Identification of safety and security critical systems....Journal of Reliability...
[09] R.Kennet+1 (2009)Quality, Risk and the Taleb QuadrantsIBM Research
M.Zhanikeev -- [email protected] -- Black Swan Based VM Placement and Migration Optimizations -- http://tinyurl.com/kyutech131018 --- 19/19...
19/19