A Budget Constrained Scheduling of Workflow Applications on Utility Grids using Genetic Algorithms

Jia Yu and Rajkumar Buyya

Grid Computing and Distributed Systems LaboratoryDept. of Computer Science and Software EngineeringThe University of Melbourne, Australia

Content

Introduction Utility Grids Problem overview Genetic Algorithms

Proposed Work Experiment Results Related work Conclusion and future work

Utility Computing and Utility Grids Utility Computing

New service provisioning model. Providing computing services such as servers,

storage and applications. Pay-per-use.

Utility Grids Grid computing provides a global infrastructure for

resource sharing and integration. Enabling users to consume utility services

transparently over a secure, shared, scalable and standard world-wide network environment.

Community Grids vs. Utility Grids

Community Grids

Utility Grids

Availability Best effort Advanced Reservation

QoS Best effort Contract/SLA

Pricing Not considered /

free access

Usage, QoS level, Market supply and demand

Workflow Scheduling

Scheduling on Community Grids Minimize the execution time ignoring other

factors such as monetary cost of resource access and various users’ QoS satisfaction levels.

Scheduling on Utility Grids Optimize performance under most important QoS

constraints imposed by users. Minimize execution cost while meeting a specified

deadline. Minimize execution time while meeting a specified

budget.

Genetic Algorithms

Random search method based on the principle of evolution.

Exploitation of best solutions from past searches.

Exploration of new regions of the solution space.

A high-quality solution to be derived from a large search space.

Genetic AlgorithmsStart

Initialize the population of possible solutions

Generate offspring solutions by genetic operators

Evaluate the fitness of each individual in the population

Select the fittest solutions in the population

Terminated?

Stop

Yes

No

Each individual in the search space of the problem represents a solution.

A GA maintains a population of individuals that evolves over generations.

The quality of an individual is determined by a fitness function.

Proposed Work

Existing GAs Schedule dependent tasks in homogeneous

multiprocessor systems. Minimize execution time or maximize system

throughput. Our work

Schedule dependent tasks in heterogeneous environments.

Minimize execution time while meeting users’ budget.

Application Model

A

B C

D

Directed Acyclic Graph (DAG)

There is no cycle in the graph. A task cannot be executed

until all of its parent tasks are completed.

Construction of a Genetic Algorithm

Representation of individual in the population.

Determination of the fitness function. Design of genetic operators.

Problem encoding

T0 T1 T2

T3 T4

T5 T6

T7

T0 T1 T2

T3 T4

T5 T6

T7

Workflow

S1

S2

S3

S4

time

Schedule

T0 T2 T7

T1

T3 T5

T4 T6

T0(1)-T2(1)-T7(1)-T1(2)-T3(3)-T5(3)-T4(4)-T6(4)

S1:T0-T2-T7

S2:T1

S3:T3-T5

S4:T4-T6

Two-dimensional strings

One-dimensional string

Cost-fitness: encourages the formation of the solutions that achieve the budget constraint.

c(I) is the sum of the task execution cost and data transmission cost of I ,

and B is the budget of the workflow.

Time-fitness: encourages the GA to choose individuals with earliest completion time in the current population.

where t(I) is the completion time of I and maxTime is the largest

completion time of the current population.

Fitness function

Fitness function

B

IcIF t

)()(cos

maxTime

ItIFtime

)()(

otherwise

1)(

),(

),()( coscos

IFif

IF

IFIF t

time

t

Genetic operators

Selection Retain fittest individuals in the population as

successive generations evolve. Crossover

Produce new individuals by combining the two existing individuals.

Mutation

Crossover

Before crossover

Crossover

After crossover

S1:T0-T2-T7S2:T1S3:T3-T5S4:T4-T6

parent1

S1: T0-T1S7: T2-T7S8: T3S9: T4-T6S10:T5

parent2

T0(1)-T2(1)-T7(1)-T1(2)-T3(3)-T5(3)-T4(4)-T6(4)

T0(1)-T1(1)-T2(7)-T7(7)-T3(8)-T4(9)-T6(9)-T5(10)

Randomly select crossover window

S1: T0-T2-T1S4: T4-T6S7: T7S8: T3S10:T5

S1: T0-T7S2: T1S3: T3-T5S7: T2S9:T4-T6

offspring1 offspring2

Mutation Operations Mutation operations:

Allow a certain offspring to obtain features that are not possessed by either parent.

Swapping mutation Swapping mutation aims to change the execution

order of tasks in an individual that compete for a same time slot.

Replacing mutation Replacing mutation aims to re-allocate an

alternative service to a task in an individual.

Schedule refinement

T0 T1 T2

T3 T4

T5

T6

0-1878

1878-2050

2050-2650

5166-5666

4450-5166

0-2450 0-4450

T0 T1 T2

T3 T4

T5

T6

0-1878

1878-3050

3050-5000

5166-5666

4450-5166

0-4440 0-4450

Rescheduled tasks

(a) Before refinement (b) After refinement

(G$300) (G$200)

(G$150) (G$100)

(G$180) (G$100)

Experiments

GridSim experiment environment

WorkflowSystem

GISGrid

Service

1.register(service type)

1. register4. AvailableSlotQuery(duration)

Grid Service

2. query(type A)

3.service list

5. slots

GIS: Grid Index System

6. makeReservation(task )

Experiments

Applications

1 3 5 7

2 4 6 8

10 11 12

13 14 15

Align_wap

reslice

softmean

slicer

convert

(300000)

9

(600000)

(300000)

(600000)

(300000)

Align_wap

reslice

Align_wap Align_wap

reslice reslice

slicer slicer

convert convert

(300000) (300000) (300000)

(600000) (600000) (600000)

(300000) (300000)

(600000) (600000)

1 3 5 7

2 4 6 8

10 11 12

13 14 15

Align_wap

reslice

softmean

slicer

convert

(300000)

9

(600000)

(300000)

(600000)

(300000)

Align_wap

reslice

Align_wap Align_wap

reslice reslice

slicer slicer

convert convert

(300000) (300000) (300000)

(600000) (600000) (600000)

(300000) (300000)

(600000) (600000)

(900000)1

5

6

2 3 4

109

11

12 13

15

7

14

SignalP COILS2 SEG PROSITE

TMHMM

Prospero HMMer

PSI-BLAST BLAST IMPALA

Summary

PSI-PRED

3D-PSSM

Genome

Summary

SCOP

(300000) (600000) (600000)

(300000)

(150000)

8

(150000)

(300000) (300000) (300000)

(600000)

(600000)

(300000)

(150000)

(300000)

(900000)1

5

6

2 3 4

109

11

12 13

15

7

14

SignalP COILS2 SEG PROSITE

TMHMM

Prospero HMMer

PSI-BLAST BLAST IMPALA

Summary

PSI-PRED

3D-PSSM

Genome

Summary

SCOP

(300000) (600000) (600000)

(300000)

(150000)

8

(150000)

(300000) (300000) (300000)

(600000)

(600000)

(300000)

(150000)

(300000)

1

5

6

2 3 4

109

11

12 13

15

7

14

SignalP COILS2 SEG PROSITE

TMHMM

Prospero HMMer

PSI-BLAST BLAST IMPALA

Summary

PSI-PRED

3D-PSSM

Genome

Summary

SCOP

(300000) (600000) (600000)

(300000)

(150000)

8

(150000)

(300000) (300000) (300000)

(600000)

(600000)

(300000)

(150000)

(300000)

Balanced structure Unbalanced structure

Experiments

Service type represents different types of services. 15 types of services, each supported by 10 different

service providers with different processing capability.

ServiceID

Processing Time(sec)

Cost (G$)

1 1200 300

2 600 600

3 400 900

4 300 1200

Bandwidth(Mbps)

Cost/sec (G$/sec)

100 1

200 2

512 5.12

1024 10.24

Table I. Service speed andcorresponding price for executing a task.

Table II. Transmission bandwidth and corresponding price.

Evolution of execution time and cost during 100 generations.

Evolution of execution time and cost in response to different refinement rate when budget is G$3000.

Heuristics compared

Greedy time Assigns a planed budget to each task in the workflow

based on the average estimated execution costs of tasks and the total budget of the workflow.

Assigns each task to a service which can complete at earliest time within its assigned sub-budget.

Related Work

Time optimization algorithms Min-Min: vGrADS, Pegasus HEFT: ASKLON GRASP: Pegasus Simulated Annealing: ICENI Genetic Algorithms: ASKALON

Genetic algorithms in multiprocessors systems Heuristics

E. Tsiakkouri et al., “Scheduling Workflows with Budget Constraints”, the CoreGRID Workshop on Integrated Research in Grid Computing, Nov. 28-30, 2005.

Conclusion and Future Work

Budget constrained workflow scheduling Minimize execution time while meeting user’s budget Genetic algorithms

Fitness function Crossover and Mutation

Future work Different negotiation models Run time rescheduling Other QoS constraints

Thank You… Any ??

Thank You… Any ??