Embed Size (px)
Citation preview
Deadline Constrained Scientific Workflow Scheduling on
Dynamically Provisioned Cloud Resources
Vahid Arabnejad, Kris Bubendorfer, Bryan Ng
School of Engineering and Computer Science, Victoria University of Wellington, NewZealand
Abstract
Commercial cloud computing resources are rapidly becoming the target plat-form on which to perform scientific computation, due to the massive lever-age possible and elastic pay-as-you-go pricing model. The cloud allows re-searchers and institutions to only provision compute when required, and toscale seamlessly as needed. The cloud computing paradigm therefore presentsa low capital, low barrier to operating dedicated HPC eScience infrastruc-ture. However, there are still significant technical hurdles associated withobtaining sufficient execution performance while limiting the financial cost,in particular, a naive scheduling algorithm may increase the cost of compu-tation to the point that using cloud resources is no longer a viable option.
The work in this article concentrates on the problem of scheduling dead-line constrained scientific workloads on dynamically provisioned cloud re-sources, while reducing the cost of computation. Specifically we present twoalgorithms, Proportional Deadline Constrained (PDC) and Deadline Con-strained Critical Path (DCCP) that address the workflow scheduling prob-lem on such dynamically provisioned cloud resources. These algorithms areadditionally extended to refine their operation in task prioritization and back-filling respectively. The results in this article indicate that both PDC andDCCP algorithms achieve higher cost efficiences and success rates when com-pared to existing algorithms.
Keywords: Scheduling, Cloud Computing, Resource Allocation, ScientificWorkflows, Resource Provisioning.
Email addresses: [email protected] (Vahid Arabnejad),[email protected] (Kris Bubendorfer), [email protected] (Bryan Ng)
Preprint submitted to Future Generation of Computer Science January 27, 2017
1. Introduction
Elastic, on demand cloud computing enables significant computationalleverage to be applied to real world problems, be they medical, commercial,industrial or scientific. From a scheduling point of view, the most interestingsubset of these problems are those that can be represented as workflows.Workflows are being extensively used by scientists to model and to managecomplex, compute and data intensive experiments [1], and more of theseworkflows are being progressively moved onto commercial clouds [2, 3, 4, 5, 6].
Typical commercial cloud services charge on the basis of the number ofhours the resources (such as CPU, network bandwidth and amount of stor-age) are used. This charging model is referred to as pay-per-use. Otheradvantages of using commercial clouds for scientific computation include re-liability and fault tolerance, and access to specialized resources such as GPUs.The flexibility inherent in the elastic cloud model, while powerful, may alsoresult in inefficient usage and high costs when inadequate scheduling andprovisioning decisions are made [7].
This article focuses on extended versions of two related algorithms fromour recent work, Proportional Deadline Constrained (PDC) [8] and DeadlineConstrained Critical Path (DCCP) [9]. Both algorithms belong to the class oflist-based scheduling algorithms [10] consisting of a task prioritization phaseand a task assignment phase. The PDC algorithm maximizes parallelism ina workflow by separating it into logical levels and then proportionally sub-dividing the overall workflow deadline over different levels. In this work wefurther improve the performance of the PDC algorithm by refining the taskranking carried out during the task prioritization step. The DCCP algo-rithm uses the concept of Constrained Critical Paths (CCP) to execute a setof tasks on the same instance with the goal of reducing communication costbetween instances. In this work we extend the DCCP algorithm to strategi-cally backfill left over capacity (residuals) in provisioned instances and applyseveral different backfilling strategies. This approach is most effective in dataintensive workflows due to the reduction in data movement. In addition wealso conduct a considerably more comprehensive and detailed evaluation ofboth algorithms to better understand the limitations on their performance.
We evaluate the algorithms using a CloudSim [11] simulation that fea-tures dynamically provisioned cloud resources and a pay-per-use model de-
2
rived from Amazon’s EC2 pricing model. The simulations were conductedusing five scientific workflows generated by the pegasus workflow generator,Montage, SIPHT, LIGO, Cybershake and Epigenomics and each of thesegenerated workflows consisted of 1000 tasks.
We compare the performance of the extended PDC and DCCP againsttwo previously published algorithms (IC-PCP [12] and JIT [13]) by measur-ing success rate, normalized cost, and throughput. In terms of cost, PDCand DCCP algorithms generally returned the lowest compute costs over allworkflows and instance configurations. Of particular note, for the Cyber-shake workflow, PDC returned costs approximately 10% of those incurred byJIT.
The remainder of this article is organized as follows: Section 2 gives anoverview of existing approaches to scheduling workflows. In Section 3, wedefine the workflow scheduling problem and describe the basis of our algo-rithms with a graph model. Section 4 describes the two extended DCCP andthe PDC algorithms. The cost and success rate of the different scheduling al-gorithms are evaluated and the results are extensively discussed in Section 5.Finally, we summarize our work in Section 6.
2. Related Work
Various algorithms based on heuristic, search-based, and meta-heuristicstrategies, have been proposed for efficient resource scheduling – where tasksare either considered independent (bag of tasks) or dependent (workflows).As a well studied problem there are several comprehensive reviews of work-flow scheduling methods in distributed environments [14, 15, 16]. Schedulingworkflow tasks to resources while meeting workflow dependencies and con-straints is known as the Workflow Scheduling Problem (WSP) – a class ofproblem that is known to be NP-complete [17]. When multiple Quality ofService (QoS) parameters are used as constraints, the problem of workflowscheduling becomes even more challenging. We will focus on heuristic ap-proaches to solving the WSP. The reason is, that while search-based andmeta-heuristic strategies produce acceptable answers, in a dynamic schedul-ing environment the need for a training, or initialisation phase, limits theiruse in practice.
Allocating workflow tasks to resources can be separated into two stages,the first being scheduling and the second is provisioning [14]. Given a set
3
of resources, the workflow task scheduling phase aims to determine the op-timal execution order and task placement with respect to user and workflowconstraints [18, 19]. The resource provisioning phase aims to determine thenumber and type of resources required and then to reserve these resourcesfor workflow execution [20, 7].
While the majority of cloud scheduling systems necessarily include bothscheduling and provisioning stages, prior research tends to focus on thescheduling phase, under the assumption that a pre-identified pool of (oftenhomogenous) resources are used for execution, and with the goal of optimiz-ing workflow execution time (makespan) without considering resource cost.In a commercial cloud environment this set of assumptions no longer holds.
In this section, we classify recent related research into three categories:Budget Constrained, Deadline Constrained and Deadline and Budget Con-strained.
2.1. Budget Constrained Scheduling
A budget is the maximum amount of financial resource that users wish topay to run their workflows. Algorithms in the budget constrained categoryattempt to minimize workflow completion time for a given budget.
In [19] Sakellariou et al. presented two budget constrained algorithms,LOSS and GAIN. The algorithms start with one of two different initial as-signments. The first assignment is the Best assignment: a time optimizedassignment in which the execution time is the minimum possible. For ex-ample, the HEFT algorithm [21] is used as an initial assignment for theLOSS algorithm. HEFT is one of the most common scheduling algorithmsand attempts to reduce the workflow makespan. The second assignment isthe Cheapest assignment: a cost-optimized assignment wherein all tasks areassigned to resources having the least execution cost. For example, GAINuses the cheapest assignment as the initial assignment. Tasks are repeatedlyselected for reassignment until the user constrained budget is reached. Thesealgorithms however were designed for non elastic, grid environments.
An extension of the HEFT [21] algorithm called the Cost ConsciousScheduling Heuristic (CCSH) was presented by Li et al. in [22]. The CCSHfirst constructs a priority list of tasks and then assigns the task with thehighest priority value to the most cost-efficient virtual machine (VM). How-ever, only one VM type and one pricing model is considered. The authorslater introduced the Pareto dominance cost efficient heuristic to the CCSHto consider different cost models [23].
4
In [24], Zeng et al. presented a budget-aware backtracking algorithm forexecuting large scale many task workflows, referred to as ScaleStar. Theiralgorithm uses a new metric termed the Comparative Advantage (CA) toselect resources in a way to minimize cost. The CA metric attempts tobalance cost and execution time. This work while developed for cloud, useda grid cost model.
2.2. Deadline Constrained Scheduling
The deadline is the time by which a workflow must complete its execution.In most clouds, heterogeneous resources are available that offer different levelsof performance at different price offering. Generally, faster resources aremore expensive in comparison with slower ones. Therefore, there is often anexploitable trade off between execution time and the cost of resources. Therelated work in this section is the closest in prinicples to our work on PDCand DCCP.
In [25], Yuan et al. presented the Deadline Early Tree (DET) algorithm.In DET, tasks are partitioned into two types: critical and non-critical activi-ties. All tasks on the critical path are scheduled using dynamic programmingunder a given deadline. Non-critical tasks are backfilled between criticaltasks. However, the communication time between tasks in a workflow is nottaken into account by the DET scheduler.
The Hybrid Cloud Optimized Cost (HCOC) scheduling algorithm by Bit-tencourt and Madeira presented in [26] focuses on optimizing cloud-burstingfrom private to public clouds. The initial schedule starts to execute taskson private cloud resources; if the initial schedule cannot meet the deadline,additional resources are leased from a public cloud on demand. The combi-nation of private and public cloud models means that this work cannot beapplied to a purely commercial cloud context.
In [12], Abrishami et al. presented the Infrastructure as a service (IaaS)Cloud Partial Critical Paths (IC-PCP). All tasks in a partial critical path(PCP) are scheduled to the same cheapest instance that can complete themby the given deadline. This avoids incurring communication costs for eachPCP. However, the IC-PCP algorithm does not consider the boot and deploy-ment time of VMs, even though these are created on demand. One extensionof IC-PCP that attempts to further reduce cost is the Enhanced IC-PCPwith Replication (EIPR) algorithm [18] in which Calheiros and Buyya useidle instances and budget surpluses to replicate tasks. Their experimental re-sults show that the likelihood of meeting deadlines is increased by using task
5
replication. However, task replication in EIPR comes at an opportunity costas the resources could be used for new rather than replicated computation.
In [27], Byun et al. presented the Partitioned Balanced Time Scheduling(PBTS) algorithm that estimates the minimum number of instances requiredto meet the deadline in order to minimize execution cost. The PBTS algo-rithm has three phases which are task selection, resource capacity estimationand the task scheduling phase. However, only one VM type is considered forprovisioning and scheduling in order to simplify the estimation of resourcecapacity.
The Just in Time (JIT) algorithm proposed by Sahni and Vidyarthi in[13] is a dynamic cost minimization deadline constrained algorithm. TheJIT algorithm attempts to combine pipeline tasks into a single task that canabrogate the data transfer time between co-located tasks. The majority ofalgorithms prioritize tasks to find the best candidate for execution however,no such policy is used in JIT.
2.3. Deadline and Budget Constrained Scheduling
The following three algorithms consider more than one constraint inscheduling workflows. In [28], Zheng and Sakellariou, introduce an extendedversion of HEFT algorithm with both budget and deadline constraints (BHEFT).BHEFT checks if a workflow can be scheduled based on the available budgetand deadlines. To select the best possible instance in BHEFT, two variablesnamed Spare Application Budget and Current Task Budget are used. Thiswork while developed for cloud, used a grid cost model.
In [29], Poola et al. present a robust scheduling algorithm for hetero-geneous cloud resources with time and cost constraints. Three resource se-lection policies are used: deadline, cost and robustness, and each user canprioritize the policies independently.
The scheduling algorithms proposed by Malawski et al. in [30] aim tomaximize the number of serviced workflows while meeting given budget anddeadline constraints. These scheduling algorithms are designed for workflowsin an Infrastructure as a Service (IaaS) cloud. However, the authors consideronly one instance type rather than the wide variety of types that are currentlysupported by commercial providers.
6
3. Problem Definition
3.1. Application Model
Workflows are the most widely used models for representing and man-aging complex distributed scientific computations [1]. A Directed AcyclicGraph (DAG) is the most common abstraction of a workflow. Using aDAG abstraction, a workflow is defined as a graph G = (T,E) where T ={t0, t1, ..., tn} is a set of tasks represented by vertices and E = {ei,j | ti, tj ∈T} is a set of directed edges denoting data or control dependencies betweentasks. An edge ei,j ∈ E represents the precedence constraint as a directedarc between two tasks ti and tj where ti, tj ∈ T . The edge indicates thattask tj can start only after completing the execution of task ti with all datareceived from ti and this implies that task ti is the parent of task tj, and tasktj is the successor or child of task ti. Each task can have one or more parentsor children. Task ti cannot start until all parents have completed.
3.2. System Model
In this article, we adopt the IaaS service model. The IaaS paradigmprovides a service by offering instance types containing various amounts ofCPU, memory, storage and network bandwidth at different prices. Workflowsare executed on different instance types, and each instance type is associatedwith a set of resources.
We use a resource model based on the Amazon Elastic Compute cloud,where instances are provisioned on demand. The pricing model is a pay asyou go with minimum hourly billing. Under this pricing model, if an instanceis used for one minute, a user has to pay for the whole hour. We assumethat cloud vendors provide access to unlimited number of instances and theinstances are heterogeneous (denoted by P = {p0, p1 . . . ph}, where h is theindex of the instance type). We also assume that all instances and storageservices are located in the same region and also assume that the averagebandwidth between the instances is essentially identical.
3.3. Definitions
Most studies on workflow scheduling assume that estimated executiontime for workflow tasks are known. Task runtimes can be estimated usinganalytical modeling, empirical modeling and historical data. In this article,
7
Figure 1: A sample DAG with ten tasks
A
B C
D E FG
H I
J
we use scientific workflows generated using trace data from real applica-tions [31]. Execution time (computation cost) for task ti on instance pj isdenoted by w
pjti . All immediate predecessors of task ti are defined as:
pred(ti) = {tj | (tj, ti) ∈ T}. (1)
Also, all immediate successors of task ti are defined as:
succ(ti) = {tj | (ti, tj) ∈ T}. (2)
For example, predecessors of task H are D,E and G, successors of C are Gand F .
A task without any parent is an entry task and a task without any childrenis called an exit task. In Figure 1, task A is an entry task and task J is anexit task. Thus, by definition we have:
pred(tentry) = {∅} , (3)
succ(texit) = {∅} . (4)
8
The completion time of a workflow is called the schedule length or makespan(denoted by Lms). Because texit is the last task that can be executed, thetime until completing the exit task is defined as the makespan of a workflow.
Lms = FT (texit) , (5)
where FT (texit) is the finish time of the last task in a workflow.The amount of data transferred from task ti to task tj is called commu-
nication time (denoted by Ci,j), and this time is calculated as:
Ci,j =
{dataβ
, pi 6= pj0 , pi = pj.
(6)
If task ti and task tj are executed on the same instance (denoted by pi 6= pj),data transfer between them is local and the communication cost is definedas zero. Otherwise, the communication cost is the ratio between the size ofdata (data) to be transferred from task ti to tj to the average bandwidth (β)in the data-center.
The Earliest Start Time (EST) of a task ti is calculated on the instancewith the shortest execution time and defined as:
EST (i) =
{0 , ti = tentry
maxtj∈pred(ti)
{EST (tj) + wtj + Ci,j
},Otherwise, (7)
where wtj is the execution time of task tj on the fastest instance type.The cost of executing task ti on instance pj is calculated as:
TaskCostpjti =
⌈wpjti
Nt
⌉∗ cj, (8)
where cj is the cost of instance pj for one time interval and Nt is the numberof intervals. Finally, the overall cost of executing all tasks in a workflow isdefined as:
Costo =∑ti∈G
TaskCostpjti . (9)
4. PDC and DCCP Algorithms
In this section, we present the extended deadline constrained algorithms,Proportional Deadline Constrained (PDC) and Deadline Constrained Critical
9
Path (DCCP). As a guide to this this section Figure 2 shows the sequence ofsteps for scheduling a workflow in both PDC and DCCP and indicates whichsub-section details those parts of the algorithms.
Figure 2: Scheduling workflow with PDC and DCCP
4.1. Preprocessing Step
Both DCCP and PDC use a preprocessing step for partitioning tasks. Inthe preprocessing step, the tasks are partitioned into different levels based ontheir respective dependencies. Subsequently, the user-defined deadline TD isdistributed over the levels established in the preprocessing step. Each levelgets its own level deadline and all tasks in the same level have the samelevel-deadline.
4.1.1. Workflow Leveling
We aim to maximize task parallelism by partitioning tasks such that thereare no dependencies between tasks in each level. Each level can therefore bethought of as a bag of tasks (BoT) containing a set of independent tasks.
There are two main algorithms for allocating tasks into different levels,Deadline Bottom Level (DBL) [32] and Deadline Top Level (DTL) [33]. BothDBL and DBT algorithms categorize tasks in bottom-top direction and top-bottom direction, respectively. In this article, we use the DBL algorithm topartition tasks over the different levels.
10
We describe the level of task ti as an integer representing the maximumnumber of edges in the paths from task ti to the exit task (see Fig. 1). Thelevel number (denoted by NL) associates a task to a BoT. For the exit task,the level number is always 1, and for the other tasks, it is determined by:
NL (ti) = maxtj∈succ(ti)
{NL (tj) + 1}, (10)
where succ(ti) denotes the set of immediate successors of task ti. All tasksare then grouped into Task Level Sets (TLS) based on their levels:
TLS(`) = {ti|NL (ti) = `}, (11)
where ` is an integer denoting the level in [1 . . . NL (tentry)].
4.1.2. Proportional Deadline Distribution
Once all tasks are assigned to their respective levels, the tasks are propor-tionally distributed across each level based on the user deadline (TD). Eachsub-deadline assigned to a level is termed the level deadline (Tsd(`)). Tomeet the overall deadline, we attempt to ensure that every task in a level cancomplete its execution before the assigned sub-deadline. Firstly, the initialestimated deadline for each level (`) is calculated as:
InitialTsd(`) = maxti∈TLS(`)
{ECT(ti)}, (12)
where ECT(ti) denotes the Earliest Completion Time (ECT) of task ti overall instances and the ECT is defined as
ECT(ti) = max`∈pred(ti)
{InitialTsd(`), EST (ti)}+ wti , (13)
where EST (ti) is defined in equation. 7, pred (ti) denotes the set of prede-cessors of task ti; wti denotes the minimum execution duration for task ti and` indicates the parent level ti. The task, tentry has no predecessors, its ECTis equal to zero. In equation 12, the maximum ECT of all tasks in a levelis used as the overall estimate for that level. This duration is effectively theabsolute minimum time that is required for all tasks in a level to completeexecution in parallel.
11
After calculating the estimated deadline value for all levels, we distributethe user deadline among all tasks non-uniformly based on a deadline propor-tion denoted by ∝deadline in equation 14:
∝deadline=TD − InitialTsd(1)
InitialTsd(1), (14)
where InitialTsd(1) is the level that contains the exit task.We then compute length of each level deadline as a function of this dead-
line proportion to each level as follows:
Tsd(`) = InitialTsd(`) + (∝deadline ×|InitialTsd(`) |) . (15)
Intuitively, the levels with longer executing tasks gain a larger share of theuser deadline.
4.2. Task Prioritization
4.2.1. PDC Algorithm (A Single task)
In each step of the PDC algorithm, tasks that are ready to execute areplaced into the task ready list. A task is ready when all of its parents havebeen executed and all its required data is readily accessible. To select a task,at first all tasks in the ready list should first be prioritized. We used eightdifferent policies in order to show how the order of execution can influence thescheduling results, particularly the cost. These are summarized in table 1.We will discuss these result later (see section 5.2).
1. Upward Rank (ranku): This ranking is presented in [21]. The upwardrank is the length of critical path from task ti to task texit and iscalculated by equation (16), where wi and ci,j are the average executiontime and average communication time of task ti, respectively. The rankis called an upward rank because the ranking process starts from theexit node and ranks are calculated recursively by traversing the DAGto the entry node.
2. Downward Rank (rankd): The downward rank [21] starts from the en-try node and is computed recursively by traversing the DAG to the exitnode. rankd(ti) is the longest distance from tentry to task ti, excludingthe computation cost of task itself, where ranku(ti) is the length of thecritical path from task ti to texit, including the computation cost of thetask itself [21].
12
Tab
le1:
Ran
ks
valu
es
Pol
icy
Des
crip
tion
For
mu
laP
olic
yT
yp
e
Upw
ard
Rank
(ranku)
Th
ele
ngth
of
crit
ical
pat
hfr
omta
skt i
tota
skt exit
ranku(ti)
=wi+
max
t j∈succ(t
i)( c
i,j
+ra
nku(tj))
(16)
stat
ic
Dow
nw
ard
Ran
k(rankd)
Sta
rts
from
the
t entry
and
isco
mp
ute
dre
curs
ivel
yby
trav
ersi
ng
the
DA
Gto
t exit
rankd(ti)
=m
axt j∈pred
(ti)(w
j+c j,i
+ra
nkd(tj))
(17)
stat
ic
Su
mR
ank
(ranks)
Su
mof
the
upw
ard
and
dow
n-
ward
ran
k
ranks(ti)
=ra
nku(ti)
+ra
nkd(ti)
stat
ic
Min
imu
mE
x-
ecu
tion
Tim
e(M
inE
xe)
Low
est
exec
uti
onti
me
isgi
ven
firs
tp
riori
tym
in(w
i)st
atic
Max
imu
mE
x-
ecu
tion
Tim
e(M
axE
xe)
Task
sw
ith
alo
nge
rex
ecuti
onti
me
has
hig
her
pri
orit
ym
ax(w
i)st
atic
Ran
dom
Task
sar
ep
icked
from
the
read
yli
stat
ran
dom
—st
atic
Earl
iest
Com
-p
leti
on
Tim
e(E
CT
)
Th
eta
skth
at
fin
ish
esfi
rst
wil
lb
eth
eb
est
cand
idat
efo
rex
e-cu
tion
min
(FTwi)
dyn
amic
Earl
iest
Dea
d-
lin
eF
irst
(ED
F)
Task
sw
ith
min
imu
mE
DF
hav
eh
igh
est
pri
orit
y
EDF
(ti)
=Level` t
ideadline−EST
(ti)
dyn
amic
13
3. Sum Rank(ranks): This rank gives equal importance to both the up-rank and downrank and is calculateds as the arithmetic sum of ranksand rankd.
4. Minimum Execution Time (MinExe): For each task in the ready list,the minimum execution time on all VMs types is calculated and thetask with the lowest execution time is given first priority.
5. Maximum Execution Time (MaxExe): Similar in principle to the min-imum execution time, with the only difference being that the task witha longer execution time has higher priority.
6. Random: In this policy, tasks are picked from the ready list at random.
7. Earliest Completion Time (ECT): For each task the earliest completiontime on all VMs launched is calculated. The task that finishes first willbe the best candidate for execution.
8. Earliest Deadline First (EDF): Tasks with a minimum EDF have high-est priority among all ready tasks.
4.2.2. DCCP Algorithm (Multiple tasks: CCP definition)
A Critical Path (CP) is the longest path from the entry to exit node of atask graph [34]. The length of critical path (|CP |) is calculated as the sumof computation costs and communication costs, and can be considered as thelower bound for scheduling a workflow.
Several heuristics that utilize critical paths have been proposed for ad-dressing the workflow scheduling problem [21, 34, 35]. The set of tasks con-taining only the tasks ready for scheduling constitutes a constrained criticalpath (CCP) [36]. In the DCCP algorithm, the CCP in a workflow is deter-mined based on HEFT upward rank and downward rank [21], then we applya set of new ranking methods defined as follows:modified upward rank :
Mranku(ti) = wi +∑
tj∈succ(ti)
(ci,j) + maxtj∈succ(ti)
(ranku(tj)) (18)
modified downward rank :
Mrankd(ti) =∑
tk∈pred(ti)
(ck,i) + maxtk∈pred(ti)
(wk + rankd(tk)) (19)
The difference between our modified rank from standard rank is that themodified rank aggregates a task’s predecessors’ or successors’ communication
14
time instead of selecting the maximum. With the modified rank, tasks withhigher out-degree or in-degree have higher priorities. As a result, these tasksare executed first with higher probability and more tasks on the next CCPcan be considered as ready tasks.
In this article, we use the sum rank to find all CPs [21]:
ranks = ranku + rankd (20)
In DCCP, all tasks are first sorted based on their ranksum values and thosetasks with the highest values are selected as the first CP. All tasks in thefirst CP are labeled as visited tasks. Proceeding in the same way, all CPs ina workflow can be found.
4.2.3. An illustrative example
We consider a sample DAG that contains 11 tasks as shown in Figure 3.The numbers associated with each edge shows the data transfer time betweentasks. The data could either be direct or indirect via shared storage. For thisarticle the only difference is in the absolute time required, and for simulationwe only consider direct transfers. The average execution time (wi) of eachtask is displayed in Table 3.
1. A single task: Upon executing task 0, all its children are ready forexecution. The different start time, end time and data transfer time(blue intervals) are shown in Figure 4. Different policies select differenttask (Table 2).
Table 2: Selected Task by different policies
ranku rankd ranks MinExe MaxExe ECTSelected Task t2 t1 t1 t3 t1 t2
2. Multiple tasks: The first CP is obtained based on the highest sumrank which is the aggregation of ranku and rankd and this yields(0→ 1→ 4→ 9→ 11). Regardless of any previously selected tasks,proceeding in the same way, other CPs are found as displayed in Ta-ble 4. The next step is traversal of CPs to find CCPs in a round-robinorder. The first CCP consists of (0→ 1) as other tasks in the first CPare not yet ready. For example, consider t4 which is in the first CP,this task cannot be added to the CCP as one of its parents, t2, has
15
Figure 3: A sample DAG with 11 tasks
0
1
2
3
4
5
6
7 8
9
10
11
2616
17
29
918
14
11
16 1614
12
15
19
19
20
13
Figure 4: Ready tasks and rank values (shown within each bar) after execution of task 0
t0 22
t1 29
t2 22
t3 20
16
not yet been added to any CCPs. As no ready tasks can be found inthe first CP, a second (new) CP is constructed. In the new CP wehave t2 which is a ready task, as its only parent has already been in-cluded in a previous CCP. Thus, the second CCP consists of three tasks(2→ 5→ 8) having excluded t10 from the second CP. Similarly, otherCCPs are generated by using the remaining CPs. The different CCPscalculated by our modified rank approach are presented in Table 5.
Table 3: Ranks values
Standard Rank Modified RankTask wi ranku rankd ranks ranku rankd ranks
0 22 190 0 190 284 0 2841 29 142 48 190 142 48 1902 22 150 38 188 203 38 2413 20 96 39 135 96 39 1354 27 84 106 190 84 115 1995 21 110 78 188 140 78 2186 9 65 74 139 65 85 1507 14 70 115 185 89 115 2048 12 75 113 188 75 113 1889 11 41 149 190 41 173 21410 21 44 144 188 44 156 20011 10 10 180 190 10 236 246
4.3. Instance Selection in PDC
At the point the algorithms perform instance selection: (i) each task isalready assigned a level, (ii) the deadline for each level is already determined,and (iii) the priority of each ready task is already assigned. During instanceselection, a trade off must be made between execution time and cost. Todemonstrate this trade off, we show the expressions for both the time and thecost of executing each task on each instance type in equations (21) and (22),forming two sets of expressions for Time and Cost.
Firstly, the time needed for the current task, ti, on the instance pj iscalculated by ECT (ti, pj). The ECT is the earliest time that a task canfinish on an instance which is defined earlier in equation (13). Using thisobservation, we can then compute how much the estimated level deadline
17
Table 4: CPs and CCPs based on standard ranks
Critical Path Constrained CriticalPath
0→1→4→9→11 0→12→5→8→10 2→5→83→6 3→67 7
4→91011
Table 5: CPs and CCPs based on modified ranks
Critical Path Constrained Critical Path
0→2→5→7→11 0→2→5→71→4→9 1→4→9
3→6→10 3→68 8
1011
of the current task differs from the earliest completion time of task on theinstance pj:
Timepjti =
Tsd (NL (ti))− ECT (ti, pj)
Tsd (NL (ti))− ECT (ti)(21)
In equation (21), Tsd(·) is the deadline that is assigned to the level whichcontains the current task. Also, ECT (ti) is the minimum execution timeamong all instances that keeps our current task on schedule.
The values of Time for task ti are related to instance types, whereinthe lower value of Time means running on a cheaper instance. The reason isthat the values of ECT (ti, pj) is bigger on an instance with a lower processingcapacity. Also, if the value of Time is negative, it means that the currenttask on the selected instance will exceed the level deadline i.e. ECT (ti, pj) >
18
Tsd (NL (ti)).In the expression for Cost, given earlier in 8, TaskCosti refers to the cost
of scheduling the current task ti on instance pj. In equation, (22), the worstcost (maximum cost) and best cost (minimum cost) of executing the task tiamong all instances are TaskCostworst and TaskCostbest, respectively.
Costpjti =
TaskCostworst − TaskCostiTaskCostworst − TaskCostbest
(22)
To find the best instance, we use a Cost Time Trade-off Factor (CTTF)in equation ((23)) that considers a trade-off between cost and time.
CTTFpjti =
Costpjti
Timepjti
. (23)
When an instance is first provisioned, the instance is billed on an hourlyinterval until it is terminated. Therefore the first task assigned to an instancein a particular billing interval incurs the entire cost of that interval. As aconsequence, if other tasks can be executed during that paid interval, thenthere is no additional execution cost for executing them. Therefore, duringinstance selection, we first prioritize the reuse of such instances (i.e. whenCost in equation (22) is 1), provided that the level deadline is not exceeded(i.e. when Time in equation (21) is positive).
If there are more than one paid instances, the PDC selects the instancewith the minimum execution time (faster instances). If no such instances areavailable, it will attempt to use a provisioned but as yet unused instance, oras a last resort it creates a new instance.
4.4. Instance Selection in DCCP
In the Instance Selection phase, the DCCP algorithm identifies the mostappropriate instance to execute CCPs. All tasks in a CCP are executed onthe same instance to minimize communication cost between them. The timeneeded for the current CCP (denoted by (CCPi)), to execute on the instancepj is calculated by ECT (CCPi, pj). Work in scheduling generally assumessuch an estimate can be calculated. In practice this is difficult, however workis underway to profile workflow tools and underlying cloud systems to provideusable estimates for use in production systems [37].
The ECT is the earliest time that a CCP can complete execution onan instance (as defined in equation (13) for a single task). The differences
19
between the estimated level deadline and earliest completion time of thecurrent CCP on the instance pj is determined by:
TimepjCCPi
= Tsd (NL (ti))− ECT (CCPi, pj) (24)
where Tsd is the deadline that is assigned to the level (given by NL(·)) whichcontains the last task ti on the current CCP. There is a possibility thatthis value may be negative which means the current CCP exceeds the leveldeadline (ECT (CCPi, pj) > Tsd (NL (ti))). The cost of executing all taskson current CCP on instance pj is denoted by CostCCPi,pj .
CostCCPi,pj =∑
ti∈CCPi
TaskCostpjti (25)
Three different scenarios to find the most appropriate instance can beconsidered:
1. Most cloud providers, such as Amazon, charge based on 60 minutesinterval. When a task is scheduled on an instance, the whole billinginterval is charged no matter how much the instance is used. Therefore,if other tasks can be executed on the same VM during that paid inter-val, their execution cost is zero. To find the best instance in DCCP,the priority is to select an instance with residuals to execute a CCP.This is subject to its earliest completion time does not exceed the leveldeadline. The instance with minimum ECT is selected (the fastestone).
2. A new instance is provisioned if no instances could be found in theprevious step. For example, at the beginning of the scheduling to assignthe first CCP, an instance should be provisioned as there is no paidinstances. For this purpose, DCCP searches among instances that canmeet the level deadline and select the cheapest one.
3. In tight deadlines, there is a possibility that none of the instances canmeet the task level’s sub-deadline (i.e., when Time
pjCCPi
is negative).If this condition for a CCP is met, it does not mean that its impossibleto meet the overall user defined deadline. Rather, it means that thesub-deadline will be violated. In this case we select the best availableinstance - as overall the schedule may still be met.
20
4.4.1. Backfilling in DCCP
Scheduling of a workflow that consists of dependent tasks creates resourceutilization gaps between the execution of tasks. The principal reason is thattasks must wait for data from its parents. Therefore, there are idle time slotsformed between scheduled tasks on each resource. Moreover, the utilization ofcloud resources depends on how tasks are placed together. Instance fragmen-tation and resource wasting occurs if tasks are not packed firmly. Schedulingalgorithms can consider these time slots for executing ready tasks on differentresources. Consequently, filling up the idle slots decreases the makespan andmaximizes the overall instance utilization.
While backfilling policies are widely used to reduce fragmentation, thishas not been done in previously in workflow scheduling. To our knowledge theuse of back filling strategies in DCCP is unique. In this section, we show thatbackfilling increases instance utilization and this improved utilization leadsto cost saving. In PDC, we use a cost-time trade off for instance selection,this approach narrows our choices of instances for backfilling. Therefore, thebackfill algorithm is only used in conjunction with DCCP. Three differentpolicies are considered which exploit such idle slots to efficiently scheduletasks which are First Fit (FF), Best Fit (BF) and Worst Fit (WF). EachCCP can be placed in a residual according to one of the following policies:
1. First Fit (FF): a CCP can be inserted into the first gap where it fits.
2. Best Fit (BF): a CCP is placed into the schedule gap where it leavesthe minimum sized residuals.
3. Worst Fit (WF): a CCP is inserted into the schedule gap where it leavesthe maximum sized residuals.
In Section 5.3 we discuss how using of backfill policies significantly improvesoverall utilization.
4.5. Time Complexity
The time complexity of the two proposed algorithms is an important met-ric for benchmarking different scheduling algorithms. Consider a workflowrepresented by a DAG G = (T,E) with n tasks. If we assume that a DAGis fully connected, the maximum number of dependencies between tasks is(n)(n − 1)/2. Processing of all tasks and its dependencies requires a timecomplexity of O (n2). Besides, processing task dependencies, other compu-tations in PDC that must be taken into account are the task selection phaseand the instance selection phase, which are distinct from each other.
21
To compute the time complexity of task selection phase, all ready tasks(n) should be examined on all available processors (p) that need computationof O(n.p). Similarly, to select all workflow tasks, the time complexity oftask selection is O (n2.p). In the resource selection phase, selected tasksare evaluated on all available instances with complexity of O(p). Thus, thetotal time complexity of resource selection is O(n.p). The total time forPDC is O (n2 + n2.p+ n.p), where the algorithm complexity is of the orderO (n2.p). The only difference between DCCP with PDC is in calculatingconstrained critical paths. For this purpose, the calculation of upward anddownward rank occurs with time complexity of O (n2.p). Therefore, theDCCP algorithm is also of the order of O (n2.p).
5. Evaluation
In this section we compare the performance of the PDC and DCCP al-gorithms, with the well known IC-PCP [12] algorithm and the recently pub-lished JIT [13] algorithm. We use a simulator to compare the performanceof all four algorithms. Simulations are well accepted as the first approachto evaluate new techniques for workflow scheduling problem. It allows re-searchers to test the performance of newly developed algorithms under acontroller setting. For this purpose, all four algorithms are implemented andevaluated in CloudSim [11].
Our simulation scenario is configured as a single data-center and six dif-ferent instance types. The characteristics of the instances are based on theAmazon EC2 instance configurations presented in Table 6.
The average bandwidth between instances is fixed to 20 MBps, based onthe average bandwidth published by AWS [38]. The processing capacity of anEC2 unit is estimated at one Million Floating Point Operations Per Second(MFLOPS) [39]. The estimated execution times are scaled by instance typeCPU performance. In an ideal cloud environment, there is no provisioningdelay in resource allocation. However, some factors such as the time of day,operating system, instance type, location of the data center, and number ofrequested resources at the same time, can cause delays in startup time [40].Therefore, in our simulation, we adopted a 97-second boot time based onprevious measurements of EC2 [40].
In order to evaluate the performance of our algorithms with a realisticload, we use five common scientific workflows: Cybershake, Epigenomics,Montage, LIGO and SIPHT. The characteristics and task composition of
22
Table 6: Instance Types based on Amazon EC2
Type ECU Memory(GB) Cost($)
m3.medium 3 3.75 0.067
m4.large 6.5 8 0.126
m3.xlarge 13 15 0.266
m4.2xlarge 26 32 0.504
m4.4xlarge 53.5 64 1.008
m4.10xlarge 124.5 160 2.520
these workflows have been analyzed in published works cited in the relatedwork section [31, 41]. To evaluate the performance of these algorithms, wechoose different deadlines chosen from tight to relaxed. Additionally, wecalculate the fastest schedule (denoted by FS) as a baseline schedule. Effec-tively, this baseline is the fastest possible execution - ignoring costs.
FS =∑ti∈CP
(wji ) (26)
where wji is the computation cost of task ti on the fastest instance pj.We define the deadline as a function of the fastest schedule and this
deadline is expressed in equation 27 in which the deadline varies from tightto moderate to relaxed:
deadline = α ∗ FS, 0 < α < 20. (27)
The deadline factor α starts from 1 to consider very tight deadlines (typicallyapproaches the fastest schedule) and is increased by one up to a value of 20,which results in a very relaxed deadline.
The Amazon EC2 instances charge on an hourly interval from the timeof provisioning. We configure our simulator to reflect this charging modeland we use a time interval of 60 minutes in our simulations. To compareperformance with respect to different workflow sizes we evaluated workflowswith 50, 100, 200, 500 and 1000 tasks. However, as these results did not varysignificantly we present here only workflows with 1000 tasks.
23
We used the Pegasus workflow generator [31] to create representativeworkflows with the same structure as five real world scientific workflows(Cybershake, Epigenomics, Montage, LIGO and SIPHT). For each workflowstructure, and each deadline factor, 100 distinct Pegasus generated work-flows were scheduled in CloudSIM and the performance of the schedulingalgorithms are detailed in the following section.
5.1. Performance MetricsTo evaluate the algorithms under test, we elected to use the following
performance metrics: Success Rate (SR), Normalized Schedule Cost (NSC)and Throughput.
• Success Rate (SR): success rate of each algorithm (SR), calculated asthe ratio between the number of simulation runs that successfully metthe scheduling deadline and the total number of simulation runs (de-noted by nTot), defined as:
SR =n (k)
nTot, (28)
where n(k) is the cardinality of the set k and nTot = 100.
• Normalized Schedule Cost (NSC): To compare the monetary cost be-tween the algorithms, we consider the cost of failure in meeting a dead-line. For this purpose, a weight is assigned to average cost returned byeach algorithm. Let k denote the set of a simulation runs that success-fully meets the scheduling deadline, thus the weighted cost is calculatedas:
Costw =
∑k Costo (k)
SR, (29)
where Costo (k) was defined earlier in equation 9 and it is the cost forthe experiments that meet the deadline. Thus, the NSC is defined as:
Costns =CostwminC
, (30)
We consider the cheapest schedule (denoted by minC) as scheduling ofall tasks on the cheapest instance according to their precedence con-straints.
• Throughput: is the amount of work that can be done in a given dead-line interval by each algorithm. Million Floating Point Operations PerSecond (MFLOPS) is used as a measure of the throughput.
24
5.2. Task Selection in PDC
The task selection step is a characteristic of all list based schedulingalgorithms. In this section we evaluate task selection in PDC using a set ofeight different ranking policies in order to evaluate the importance of andsensitivity to ranking. We previously evaluated the task selection step forDCCP in [9] where two algorithms for constructing the constrained criticalpath were evaluated. Figures 5 to 9 show the results of different task selectionpolicies in PDC as defined in Section 4.2.1
2 3 4 5 6
7 8 9 10 11
12 13 14 15 16
17 18 19
40
50
60
70
80
20
25
30
35
40
45
12.5
15.0
17.5
20.0
8
10
12
14
6
7
8
9
10
6
8
10
4
5
6
7
8
4
5
6
3
4
5
6
3
4
5
3
4
5
2.5
3.0
3.5
4.0
4.5
2.0
2.5
3.0
3.5
4.0
2.0
2.5
3.0
3.5
4.0
2.0
2.5
3.0
3.5
2.0
2.5
3.0
3.5
1.5
2.0
2.5
3.0
1.5
2.0
2.5
3.0
1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4
6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8 11.0 11.2 11.4
11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4 15.6 15.8 16.0 16.2 16.4
16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4DeadlineFactor
Cos
t
Algorithm1.UpRank2.DownRank3.SumRank4.MinExe5.MaxExe6.ECT7.Random8.EDF
MONTAGE
Figure 5: Task selection results for Montage
Workflows differ remarkably in their characteristics including structure,size, computation and communication requirements. Each workflow is con-structed of various components including process, pipeline, data distribution,data aggregation and data redistribution [31]. The size of scientific work-flows varies from small number of tasks taking a few minutes to execute, tomillions of tasks that require days to execute. Moreover, workflows also differin terms of data transfer operations, example of such transfers are fetchinginput data, moving intermediate data generated within a workflow and out-put data. For each workflow structure, and each deadline factor, 100 distinctPegasus generated workflows are simulated using CloudSim.
25
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19
80
90
20
25
30
12
16
20
10
11
12
13
14
6
8
10
12
5
6
7
8
5
6
7
5
6
7
4
5
6
3
4
5
2.5
3.0
3.5
4.0
4.5
5.0
3
4
2.5
3.0
3.5
4.0
4.5
2.5
3.0
3.5
2.0
2.5
3.0
3.5
2.0
2.5
3.0
3.5
2.0
2.4
2.8
1.5
2.0
2.5
3.0
1.5
2.0
2.5
0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4
5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4
10.6 10.8 11.0 11.2 11.4 11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4
15.6 15.8 16.0 16.2 16.4 16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4DeadlineFactor
Cos
t
Algorithm1.UpRank2.DownRank3.SumRank4.MinExe5.MaxExe6.ECT7.Random8.EDF
SIPHT
Figure 6: Task selection results for SIPHT
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19
560
580
600
620
57.5
60.0
62.5
65.0
67.5
70.0
28
29
30
31
32
33
28
29
30
31
14
16
18
20
15
16
17
15.0
15.5
16.0
16.5
15.0
15.5
16.0
16.5
9
11
13
7.25
7.50
7.75
8.00
8.25
8.50
7.5
7.8
8.1
8.4
7.5
7.8
8.1
8.4
5
6
7
5.0
5.2
5.4
5.6
5.0
5.2
5.4
5.6
5.0
5.2
5.4
5.6
4.0
4.5
5.0
5.5
3.7
3.8
3.9
4.0
4.1
4.2
3.7
3.8
3.9
4.0
4.1
4.2
0.6 0.8 1.0 1.2 1.4 1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4
5.6 5.8 6.0 6.2 6.4 6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4
10.6 10.8 11.0 11.2 11.4 11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4
15.6 15.8 16.0 16.2 16.4 16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4DeadlineFactor
Cos
t
Algorithm1.UpRank2.DownRank3.SumRank4.MinExe5.MaxExe6.ECT7.Random8.EDF
LIGO
Figure 7: Task selection results for LIGO
26
2 3 4 5 6
7 8 9 10 11
12 13 14 15 16
17 18 19
8
10
12
3
4
5
6
2
3
4
1.5
2.0
2.5
3.0
3.5
1.5
2.0
2.5
3.0
1.0
1.5
2.0
2.5
1.0
1.5
2.0
1.0
1.5
2.0
0.75
1.00
1.25
1.50
0.50
0.75
1.00
1.25
0.50
0.75
1.00
1.25
0.50
0.75
1.00
1.25
0.6
0.8
1.0
1.2
0.4
0.6
0.8
1.0
1.2
0.4
0.6
0.8
1.0
0.4
0.6
0.8
1.0
0.4
0.6
0.8
0.4
0.5
0.6
0.7
0.8
1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4
6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8 11.0 11.2 11.4
11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4 15.6 15.8 16.0 16.2 16.4
16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4DeadlineFactor
Cos
t
Algorithm1.UpRank2.DownRank3.SumRank4.MinExe5.MaxExe6.ECT7.Random8.EDF
CYBERSHAKE
Figure 8: Task selection results for Cybershake
2 3 4 5 6
7 8 9 10 11
12 13 14 15 16
17 18 19
120
140
160
180
200
80
100
120
140
60
80
100
40
50
60
70
80
40
50
60
70
30
40
50
20
30
40
50
20
25
30
35
40
45
15
20
25
30
35
40
15
20
25
30
35
15
20
25
30
35
20
30
20
30
10
15
20
25
30
35
10
15
20
25
30
35
10
15
20
25
30
35
10
15
20
25
30
35
10
15
20
25
30
35
1.6 1.8 2.0 2.2 2.4 2.6 2.8 3.0 3.2 3.4 3.6 3.8 4.0 4.2 4.4 4.6 4.8 5.0 5.2 5.4 5.6 5.8 6.0 6.2 6.4
6.6 6.8 7.0 7.2 7.4 7.6 7.8 8.0 8.2 8.4 8.6 8.8 9.0 9.2 9.4 9.6 9.8 10.0 10.2 10.4 10.6 10.8 11.0 11.2 11.4
11.6 11.8 12.0 12.2 12.4 12.6 12.8 13.0 13.2 13.4 13.6 13.8 14.0 14.2 14.4 14.6 14.8 15.0 15.2 15.4 15.6 15.8 16.0 16.2 16.4
16.6 16.8 17.0 17.2 17.4 17.6 17.8 18.0 18.2 18.4 18.6 18.8 19.0 19.2 19.4DeadlineFactor
Cos
t
Algorithm1.UpRank2.DownRank3.SumRank4.MinExe5.MaxExe6.ECT7.Random8.EDF
EPIGENOMICS
Figure 9: Task selection results for Epigenomics
27
The average execution time and average communication time are usedfor task ranking by Upward, Downward and Sum rank. Accordingly, theimpact of instances that are launched during scheduling are not accountedfor by these policies. We call these static policies, as they do not changewhen additional instances are launched by the scheduler.
The Earliest Completion Time (ECT) and Earliest Deadline First (EDF)polices require continual recomputation as execution of a task on a VM leadsto changes in ECT and EST for all other tasks on that VM. We call thesedynamic policies, as they change when additional instances are launched bythe scheduler.
The dynamic ranking policies performed best on the Montage (Figure 5)and Cybershake (Figure 8) workflows, whereas the results were largely rank-ing agnostic in LIGO (Figure 7) and Epigenomics (Figure 9). The most in-teresting result was the SIPHT workflow (Figure 6), where the results werelargely unpredictable. None-the-less, overall the dynamic EDF policy pro-duced the lowest costs over all workflows tested.
The results of this set of experiments suggest that the structure of work-flows can significantly impact the ranking and scheduling cost. We note thatthe workflows where the policies performed most consistently had a highdegree of structural and runtime symmetry – where each task in such a se-quence usually has the same amount of data as input, and in turn generatesand distributes equal information as output to their children. Indeed, theworkflows for which the policy was agnostic suggest support for this conjec-ture. The most unpredictable workflow, SIPHT was strongly asymmetric inboth structure and runtime. The structures of these workflows can be foundin [31, 41].
In future work, we will investigate the impact of workflow structure –where we will look for a measure of symmetry and consider how this can beincorporated into scheduling decisions. Findings in this area may also leadto better practice in the design of workflows themselves.
Although cost differences may seem negligible between some of the poli-cies, in multiple of datasets the variance could be significant. This showsthat the task selection order could play a key role in minimizing the cost.
5.3. Backfilling in DCCP
In this section, we evaluate backfilling strategies in terms of the numberof provisioned instances. A more effective backfilling strategy will ultimately
28
provision fewer instances and thereby reduce cost. The three different strate-gies (FF, BF and WF) outlined in Section 4.4.1 are used. We also limitthe simulation to a single instance type (type 2 in Table 6) to ensure thatthe comparison across the algorithms is fair. Two data intensive workflows,LIGO and Montage, are chosen for evaluation with three different deadlineintervals. In Figures 10 and 11, graphs in left columns show the instanceutilization based on the VM creation order the right column shows the samesorted by utilization. The X-axis is the total number of instances and theY-axis is the utilization rate.
The worst fit policy has the best performance because it launches fewerinstances, and the number of launched instances has a direct effect on cost.Worst fit reduces further fragmentation of the residuals leaving larger allo-catable blocks. A small set of high utilized VMs leads to lower overall costin worst fit policy compared to further low utilized VMs in other policies.Overall, the number of instances decreases as the deadline is relaxed. In acase of the moderate and relaxed deadlines in LIGO, interval 12 and 19, itis observed that worst fit needs almost the half number of VMs. The sameobservation is true for MONTAGE with interval 19, worst fit approximatelyrequires one-third of VM numbers compared to others. Considering benefitsof cost saving in worst fit, we used this policy in DCCP algorithms for costcomparison analysis in Section 5.4.
5.4. Cost Comparison Analysis
Ultimately the cost vs. deadline performance of each algorithm is themost significant basis for evaluating their performance. In this section weevaluate each of the algorithms using six different instance types with dif-ferent characteristics as described in Table 6. As expected, experimentalresults in Figure 12 show that the cost of the workflow scheduling generallydecreases as the deadline factor increases.
In most cases, the PDC and DCCP algorithms outperform both IC-PCPand JIT, achieving the lowest overall cost over all workflows and deadlines.Like all heuristics, there are points at which their performance is not asgood, but these are in the minority, and for small values. For example, inCybershake, the cost of PDC at most deadlines is approximately 10% thatof the cost incurred by the worst performer, JIT.
Another interesting result is from the Epigenomics workflow, where whileIC-PCP achieves the lowest costs with relaxed deadlines (12 → 20), this
29
0.00
0.03
0.06
0.09
0.12
0 20 40 60VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(a) LIGO (interval 5)
0.00
0.03
0.06
0.09
0.12
0 20 40 60Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(b) LIGO (interval 5) sorted
0.0
0.1
0.2
0.3
0 10 20 30 40 50VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(c) LIGO(interval 12)
0.0
0.1
0.2
0.3
0 10 20 30 40 50Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(d) LIGO (interval 12) sorted
0.0
0.2
0.4
0 10 20VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(e) LIGO (interval 19)
0.0
0.2
0.4
0 10 20Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(f) LIGO (interval 19) sorted
Figure 10: VM utilization for three different deadline intervals with Backfilling policies forLIGO.
30
0.00
0.01
0.02
0.03
0.04
0.05
0 30 60 90 120VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(a) MONTAGE (interval 5)
0.00
0.01
0.02
0.03
0.04
0.05
0 30 60 90 120Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(b) MONTAGE (interval 5) sorted
0.00
0.02
0.04
0.06
0 20 40VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(c) MONTAGE (interval 12)
0.00
0.02
0.04
0.06
0 20 40Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(d) MONTAGE (interval 12) sorted
0.00
0.02
0.04
0.06
0.08
0 10 20 30 40VM number
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(e) MONTAGE (interval 19)
0.00
0.02
0.04
0.06
0.08
0 10 20 30 40Number of VMs
Util
izat
ion
DCCP_FFDCCP_BFDCCP_WF
(f) MONTAGE (interval 19) sorted
Figure 11: VM utilization for three different deadline intervals with backfilling policies forMONTAGE.
31
0
200
400
5 10 15Deadline Intervals
Nor
mal
ized
Cos
t
JITPCPPDCDCCP
(a) Montage
0
10
20
30
5 10 15Deadline Intervals
Nor
mal
ized
Cos
t
JITPCPPDCDCCP
(b) SIPHT
0
10
20
30
40
5 10 15Deadline Intervals
Nor
mal
ized
Cos
t
JITPCPPDCDCCP
(c) LIGO
0
50
100
150
5 10 15Deadline Intervals
Nor
mal
ized
Cos
t
JITPCPPDCDCCP
(d) Cybershake
0
5
10
5 10 15Deadline Intervals
Nor
mal
ized
Cos
t
JITPCPPDCDCCP
(e) Epigenomics
Figure 12: Normalized Cost vs. deadline for five different datasets.
32
algorithm also is unable to generate any viable schedule, at any cost, for themajority of the tighter deadlines.
As a final observation, from Figure 12, the cost of finding a schedule whendeadline is tight for Montage and Cybershake is extremely high. It can beexplained by considering the structure of these workflows. For example inMontage, more than 800 parallel tasks out of 1000 in first two levels need tobe scheduled. Therefore, when deadline is tight, all algorithms need to leasemany instances in parallel to finish elementary tasks that makes the schedulecost very expensive
5.5. Success Rate Analysis
Figure 13 shows the relative Success Rate (SR) of each algorithm as thedeadline factor, α, is increased from 1 to 20.
A low success rate indicates that the algorithm cannot find a makespanthat meets the deadline (in most datasets). The best overall performers arethe PDC and JIT algorithms, which exhibit a success rate of 100% for mostdeadlines. We observe that the relaxing of the deadline causes the successrate of each algorithm to increase except for IC-PCP. Although we expectedto have fewer failures when the deadline is relaxed, the behavior of IC-PCPin different intervals is contrary to these expectations. The highest failureoccurs when deadline factor, α < 4,with 100% failure, except in Cybershake.IC-PCP also has the worst performance in Epigenomics below a deadlinefactor of 16, IC-PCP can find a schedule for less than 2% of the datasetsbefore its deadline is reached.
The best performance of IC-PCP belongs to Cybershake, which has a suc-cess rate of above 60% in all intervals. The DCCP algorithm in all scientificworkflows has 100% success when deadline is more relaxed. The maximumfailure in DCCP happens in Epigenomics when deadline is tight. No sig-nificant differences were found between the PDC and JIT whereas both areable to finish workflows for more than 95% of the deadlines. Although inmost of the tested deadlines JIT can find a solution, JIT generates expensiveschedules as it is discussed in 5.4.
5.6. Throughput Analysis
The throughput of each algorithm is displayed in Figure 14. The X-axisin Figure 14 is the cost based on the deadline intervals. The Y-axis is thenumber of MFLOPS in billion.
33
0
25
50
75
100
5 10 15 20Deadline Intervals
Suc
cess
Rat
e
JITPCPPDCDCCP
(a) Montage
0
25
50
75
100
5 10 15 20Deadline Intervals
Suc
cess
Rat
e
JITPCPPDCDCCP
(b) SIPHT
0
25
50
75
100
5 10 15 20Deadline Intervals
Suc
cess
Rat
e
JITPCPPDCDCCP
(c) LIGO
0
25
50
75
100
5 10 15 20Deadline Intervals
Suc
cess
Rat
e
JITPCPPDCDCCP
(d) Cybershake
0
25
50
75
100
5 10 15 20Deadline Intervals
Suc
cess
Rat
e
JITPCPPDCDCCP
(e) Epigenomics
Figure 13: Success Rate for five different datasets.
34
1
2
3
4
5
0 200 400Cost($)
Thr
ough
put
JITPCPPDCDCCP
(a) Montage
0
20
40
60
80
0 10 20 30Cost($)
Thr
ough
put
JITPCPPDCDCCP
(b) SIPHT
0
25
50
75
100
0 10 20 30 40Cost($)
Thr
ough
put
JITPCPPDCDCCP
(c) LIGO
5
7
9
0 50 100 150Cost($)
Thr
ough
put
JITPCPPDCDCCP
(d) Cybershake
0
250
500
750
1000
0 5 10Cost($)
Thr
ough
put
JITPCPPDCDCCP
(e) Epigenomics
Figure 14: Throughput for five different datasets.
35
In Figure 14, the top left corner indicates better performance at a lowercost. Clearly the throughput is dependent on the success rate – and thereforeover all intervals and workflows the throughput of the best algorithms, PDCand JIT are essentially equal. However, the cost difference between PDC andJIT is significant as shown in the graph. The best performance of DCCP is inMONTAGE and LIGO (both are data intensive workflows), in which DCCPis close to PDC with a similar cost. IC-PCP has the worst performance inalmost all workflows, which is directly related to the low success rate of thisalgorithm.
6. Conclusion
In this article we address the problem of scientific workflow schedulingin dynamically provisioned commercial cloud environments. The ever in-creasing use of cloud computing by scientists has highlighted opportunitiesfor improving utilization of cloud infrastructure by improving response timewhile decreasing the net cost of computation. In this article we employworkflow scheduling to achieve lower cost with good response times in cloudenvironment. Workflow scheduling in cloud environment differs from gridand cluster computing environments primarily in the elastic resource pro-visioning and pay-per-use charging model. Therefore, workflow schedulingin clouds requires a different approach in mapping tasks to resources whilelimiting the cost.
To address this we introduced new algorithms, Proportional DeadlineConstrained (PDC) and Deadline Constrained Critical Path (DCCP). PDCoperates by maximising the parallelism in a workflow by separating it intological levels and then proportionally sub dividing the overall workflow dead-line over them. The DCCP algorithm is similar to PDC, the main differenceis that it also determines the constrained critical path through the work-flow in order to co-locate tasks that communicate on the same instance. Wethen evaluated these algorithms (via CloudSim simulation) against two pre-viously published algorithms (IC-PCP and JIT), using a variety of metrics– including success rate, normalised cost, and throughput. We also inves-tigated the influence of the task selection step in the PDC algorithm andbackfilling strategies in DCCP. The simulations were conducted using fivescientific workflows, Montage, SIPHT, LIGO, Cybershake and Epigenomics,and these were generated by the Pegasus workflow generator.
36
In terms of cost performance, overall the PDC and DCCP algorithmsreturned the lowest compute cost, over all workflows and instance configura-tions. Of particular note, for the Cybershake workflow, PDC returned costsapproximately 10% of those incurred by JIT. In terms of success rate andthroughput, the best overall performers were the PDC and JIT algorithms,although the JIT algorithm was many times more expensive in terms of cost.We also investigated the effect of eight different policies to evaluate task se-lection order on scheduling performance in PDC. In doing so, we appear tohave found an interesting connection between the symmetry of a workflowand the performance of the scheduling algorithms, which is worthy of furtherinvestigation.
For DCCP we strategically backfill residuals in provisioned instances.This approach is most effective in data intensive workflows due to the re-duction in data movement. Worst fit resulted in higher utilisation on fewerinstances than either first fit or best fit. Worst fit reduces further fragmen-tation of the residuals by leaving larger allocatable blocks.
In this article we present two algorithms for scheduling workflows ondynamically provisioned elastic cloud resources. Overall, both our algorithmsare able to achieve a consistently high success rates and throughput, while inmost cases presenting the lowest overall pay-per-use cost. In general DCCPslightly outperforms PDC as most workflows we tested are dataintensive.
[1] I. J. Taylor, E. Deelman, D. B. Gannon, M. Shields, Workflows for e-Science: Scientific Workflows for Grids, Springer Publishing Company,Incorporated, 2014.
[2] G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B. Berman,P. Maechling, Scientific workflow applications on amazon ec2, in: 5thIEEE International Conference on E-Science, 2009, pp. 59–66. doi:
10.1109/ESCIW.2009.5408002.
[3] E. Deelman, G. Singh, M. Livny, B. Berriman, J. Good, The cost ofdoing science on the cloud: The montage example, in: Proceedings ofthe 2008 ACM/IEEE Conference on Supercomputing, SC ’08, IEEEPress, Piscataway, NJ, USA, 2008, pp. 50:1–50:12.URL http://dl.acm.org/citation.cfm?id=1413370.1413421
[4] C. Hoffa, G. Mehta, T. Freeman, E. Deelman, K. Keahey, B. Berriman,J. Good, On the use of cloud computing for scientific workflows, in:
37
eScience, 2008. eScience ’08. IEEE Fourth International Conference on,2008, pp. 640–645. doi:10.1109/eScience.2008.167.
[5] J.-S. Vockler, G. Juve, E. Deelman, M. Rynge, B. Berriman, Experiencesusing cloud computing for a scientific workflow application, in: Pro-ceedings of the 2Nd International Workshop on Scientific Cloud Com-puting, ScienceCloud ’11, ACM, New York, NY, USA, 2011, pp. 15–24.doi:10.1145/1996109.1996114.URL http://doi.acm.org/10.1145/1996109.1996114
[6] R. Madduri, K. Chard, R. Chard, L. Lacinski, A. Rodriguez, D. Sulakhe,D. Kelly, U. Dave, I. Foster, The Globus Galaxies platform: deliveringscience gateways as a service, Concurrency and Computation: Practiceand Experience (2015) n/a–n/adoi:10.1002/cpe.3486.URL http://dx.doi.org/10.1002/cpe.3486
[7] R. Chard, K. Chard, K. Bubendorfer, L. Lacinski, R. Madduri, I. Foster,Cost-aware cloud provisioning, in: the IEEE 11th International Confer-ence on E-Science, 2015.
[8] V. Arabnejad, K. Bubendorfer, Cost effective and deadline constrainedscientific workflow scheduling for commercial clouds, in: proceedings ofthe 14th IEEE International Symposium on Network Computing andApplications (NCA), IEEE, Cambridge, MA USA, 2015.
[9] V. Arabnejad, K. Bubendorfer, B. Ng, K. Chard, A deadline constrainedcritical path heuristic for cost-effectively scheduling workflows, in: toappear in the proceedings of the 8th IEEE International Conference onUtility and Cloud Computing (UCC)), IEEE, Limassol, Cyprus, 2015.
[10] Y.-K. Kwok, I. Ahmad, Static scheduling algorithms for allocatingdirected task graphs to multiprocessors, ACM Computing Surveys(CSUR) 31 (4) (1999) 406–471.
[11] R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. F. De Rose, R. Buyya,CloudSim: a toolkit for modeling and simulation of cloud computing en-vironments and evaluation of resource provisioning algorithms, Software:Practice and Experience 41 (1) (2011) 2350. doi:10.1002/spe.995.
[12] S. Abrishami, M. Naghibzadeh, D. H. Epema, Deadline-constrainedworkflow scheduling algorithms for infrastructure as a service clouds,
38
Future Generation Computer Systems 29 (1) (2013) 158 – 169, includ-ing Special section: AIRCC-NetCoM 2009 and Special section: Cloudsand Service-Oriented Architectures.
[13] J. Sahni, D. Vidyarthi, A cost-effective deadline-constrained dynamicscheduling algorithm for scientific workflows in a cloud environment,Cloud Computing, IEEE Transactions on PP (99) (2015) 1–1. doi:
10.1109/TCC.2015.2451649.
[14] F. Wu, Q. Wu, Y. Tan, Workflow scheduling in cloud: a survey, TheJournal of Supercomputing (2015) 1–46.
[15] S. Smanchat, K. Viriyapant, Taxonomies of workflow scheduling prob-lem and techniques in the cloud, Future Generation Computer Systems52 (2015) 1–12.
[16] E. N. Alkhanak, S. P. Lee, S. U. R. Khan, Cost-aware challenges forworkflow scheduling approaches in cloud computing environments: Tax-onomy and opportunities, Future Generation Computer Systems.
[17] J. Ullman, Np-complete scheduling problems, Journal of Computer andSystem Sciences 10 (3) (1975) 384 – 393.
[18] R. Calheiros, R. Buyya, Meeting deadlines of scientific workflows inpublic clouds with tasks replication, Parallel and Distributed Systems,IEEE Transactions on 25 (7) (2014) 1787–1796.
[19] R. Sakellariou, H. Zhao, E. Tsiakkouri, M. D. Dikaiakos, Schedul-ing workflows with budget constraints, in: in Integrated Research inGrid Computing, S. Gorlatch and M. Danelutto, Eds.: CoreGrid series,Springer-Verlag, 2007.
[20] K. Chard, K. Bubendorfer, P. Komisarczuk, High occupancy resourceallocation for grid and cloud systems, a study with drive, in: proceedingsof the ACM International Symposium on High Performance DistributedComputing (HPDC), Chicago, Illinois, 2010.URL publications/HPDC2010.pdf
[21] H. Topcuoglu, S. Hariri, M.-Y. Wu, Performance-effective and low-complexity task scheduling for heterogeneous computing, Parallel andDistributed Systems, IEEE Transactions on 13 (3) (2002) 260–274.
39
[22] J. Li, S. Su, X. Cheng, Q. Huang, Z. Zhang, Cost-conscious schedulingfor large graph processing in the cloud, in: High Performance Com-puting and Communications (HPCC), 2011 IEEE 13th InternationalConference on, 2011, pp. 808–813. doi:10.1109/HPCC.2011.147.
[23] S. Su, J. Li, Q. Huang, X. Huang, K. Shuang, J. Wang, Cost-efficienttask scheduling for executing large programs in the cloud, Parallel Com-puting 39 (45) (2013) 177 – 188.
[24] L. Zeng, B. Veeravalli, X. Li, Scalestar: Budget conscious schedul-ing precedence-constrained many-task workflow applications in cloud,in: Advanced Information Networking and Applications (AINA), 2012IEEE 26th International Conference on, 2012, pp. 534–541. doi:
10.1109/AINA.2012.12.
[25] Y. Yuan, X. Li, Q. Wang, X. Zhu, Deadline division-based heuristic forcost optimization in workflow scheduling, Information Sciences 179 (15)(2009) 2562–2575.
[26] L. Bittencourt, E. Madeira, HCOC: a cost optimization algorithm forworkflow scheduling in hybrid clouds, Journal of Internet Services andApplications 2 (3) (2011) 207–227. doi:10.1007/s13174-011-0032-0.
[27] E.-K. Byun, Y.-S. Kee, J.-S. Kim, S. Maeng, Cost optimized pro-visioning of elastic resources for application workflows, Future Gen-eration Computer Systems 27 (8) (2011) 1011 – 1026. doi:http:
//dx.doi.org/10.1016/j.future.2011.05.001.
[28] W. Zheng, R. Sakellariou, Budget-deadline constrained workflow plan-ning for admission control, Journal of Grid Computing 11 (4) (2013)633–651. doi:10.1007/s10723-013-9257-4.URL http://dx.doi.org/10.1007/s10723-013-9257-4
[29] D. Poola, S. Garg, R. Buyya, Y. Yang, K. Ramamohanarao, Robustscheduling of scientific workflows with deadline and budget constraints inclouds, in: Advanced Information Networking and Applications (AINA),2014 IEEE 28th International Conference on, 2014, pp. 858–865. doi:
10.1109/AINA.2014.105.
[30] M. Malawski, G. Juve, E. Deelman, J. Nabrzyski, Algorithms forcost- and deadline-constrained provisioning for scientific workflow
40
ensembles in iaas clouds, Future Generation Computer Systems 48(2015) 1 – 18, special Section: Business and Industry Specific Cloud.doi:http://dx.doi.org/10.1016/j.future.2015.01.004.URL http://www.sciencedirect.com/science/article/pii/
S0167739X15000059
[31] S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M.-H. Su, K. Vahi,Characterization of scientific workflows, in: Workflows in Support ofLarge-Scale Science, 2008. WORKS 2008. Third Workshop on, 2008,pp. 1–10.
[32] Y. Yuan, X. Li, Q. Wang, Y. Zhang, Bottom level based heuristic forworkflow scheduling in grids, Chinese Journal of Computers-ChineseEdition- 31 (2) (2008) 282.
[33] J. Yu, R. Buyya, C. K. Tham, Cost-based scheduling of scientificworkflow applications on utility grids, in: e-Science and Grid Com-puting, 2005. First International Conference on, 2005, pp. 8 pp.–147.doi:10.1109/E-SCIENCE.2005.26.
[34] Y.-K. Kwok, L. Ahmad, Dynamic critical-path scheduling: An effectivetechnique for allocating task graphs to multiprocessors, Parallel andDistributed Systems, IEEE Transactions on 7 (5) (1996) 506–521.
[35] G. Sih, E. Lee, A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures, Parallel and Dis-tributed Systems, IEEE Transactions on 4 (2) (1993) 175–187.
[36] M. A. Khan, Scheduling for heterogeneous systems using constrainedcritical paths, Parallel Computing 38 (4) (2012) 175–193.
[37] R. Chard, K. Bubendorfer, B. Ng, Network health and e-science in com-mercial clouds, Accepted: Future Generation Computer Systemsdoi:10.1016/j.future.2015.06.001.
[38] M. R. Palankar, A. Iamnitchi, M. Ripeanu, S. Garfinkel, Amazon S3 forscience grids: a viable solution?, in: Proceedings of the 2008 interna-tional workshop on Data-aware distributed computing, ACM, 2008, pp.55–64.
41
[39] S. Ostermann, A. Iosup, N. Yigitbasi, R. Prodan, T. Fahringer,D. Epema, A performance analysis of ec2 cloud computing services forscientific computing, in: D. Avresky, M. Diaz, A. Bode, B. Ciciani,E. Dekel (Eds.), Cloud Computing, Vol. 34 of Lecture Notes of theInstitute for Computer Sciences, Social-Informatics and Telecommu-nications Engineering, Springer Berlin Heidelberg, 2010, pp. 115–131.doi:10.1007/978-3-642-12636-9_9.
[40] M. Mao, M. Humphrey, A performance study on the vm startup time inthe cloud, in: Proceedings of the 2012 IEEE Fifth International Confer-ence on Cloud Computing, CLOUD ’12, IEEE Computer Society, Wash-ington, DC, USA, 2012, pp. 423–430. doi:10.1109/CLOUD.2012.103.
[41] G. Juve, A. Chervenak, E. Deelman, S. Bharathi, G. Mehta, K. Vahi,Characterizing and profiling scientific workflows, Future GenerationComputer Systems 29 (3) (2013) 682 – 692, special Section: Recent De-velopments in High Performance Computing and Security. doi:http:
//dx.doi.org/10.1016/j.future.2012.08.015.
42
![Resource-sharing Policy in Multi-tenant Scienti c Work ow ... · geographical distribution, and multi-tenancy [3]. These problems have attracted many computer scien-tists into cloud](https://img.pdfslide.net/doc/110x75/5fbfcb13009f1f007216684f/resource-sharing-policy-in-multi-tenant-scienti-c-work-ow-geographical-distribution.jpg)
