|epcc| NeSC Workshop Open Issues in Grid Scheduling Ali Anjomshoaa EPCC, University of Edinburgh Tuesday, 21 October 2003 Overview of a Grid Scheduling

|epcc||epcc|

NeSC Workshop

Open Issues in Grid Scheduling

Ali Anjomshoaa

EPCC, University of Edinburgh

Tuesday, 21 October 2003

Overview of a Grid Scheduling Paradigm

|epcc|

Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling

22

Grid Scheduling

Why is scheduling so important for Grids?

In order to make coherent and coordinated use of ubiquitous and heterogeneous Grids, Grid management is essential.

Grid management is not an option.

“Scheduling is all about management.”

|epcc|


33

Scheduling is about Management

Grids are complex multivariate environments, which are made up of numerous Grid entities that need to be managed.

These Grid entities fulfill various roles, which places them within the remit of different management environments.

Grid entities and their roles are interdependent on each other and their environments.

The management of these Grid entities is, for the most part, the undertaking of Grid scheduling.

|epcc|


44

Have introduced the concepts of:

Grid entities Grid entity roles Grid management environments

There is a mapping:

entity roles ↔ management environments

In addition, there are interdependencies:

Between the Grid entities Between the Grid entity roles Between the Grid management environments

Scheduling is about Management

|epcc|


55

Grid Entities

So what are these Grid entities?

A Grid entity is defined by its roles.

Grid entity roles can include:

A Grid entity can have one or more roles.

Resource Account Contract User

Job Policy Workflow …

|epcc|


66

Grid Entity Roles

A Grid entity can fulfill many roles. For example:

it can have both a Job role and a Resource role, it can have both a Job role and a Workflow role, or indeed any other combination of roles.

A Grid entity role is defined by a set of role specific attributes.

A Grid entity should be managed within different management environments, depending on the various roles that it fulfills.

Could say that the interest of Grid scheduling is in the role management of the various roles of Grid entities.

|epcc|


77

Grid Entity Definition

A Grid entity is defined by its roles.

A Grid entity role is defined by a set of role specific attributes.

“A Grid entity is defined by the sets of role specific attributes of its roles.”

The definition of a Grid entity could be captured in a description document as sets of role attributes.

The description document will then allow the Grid entity to be managed according to its various roles, within a Grid.

|epcc|


88


GridEntity

|epcc|


99


ResourceRole

JobRole

GridEntity

…

…

|epcc|


1010


GridEntityJob

AttributeSet

ResourceAttribute

Set

…

…

|epcc|


1111


GridEntityJob

AttributeSet

ResourceAttribute

Set

DescriptionDocument …

…

|epcc|


1212

Summary of Concepts

Previously introduced the concepts of:

Grid entities Grid entity roles Grid management environments

Have now introduced two new concepts of:

Grid entity role specific attributes Grid entity description documents

|epcc|


1313

Grid Entity Roles and Management

Different roles require different management environments.

“The management environment within which a Grid entity is being managed, will depend on the particular role of that Grid entity which is being managed.”

A role type can not be managed in more than one management environment.

A management environment can manage more than one type of role.

|epcc|


1414


Role interdependencies need to be managed also.

This will mean that the boundaries of management environments will need to be crossed.

Common factorisations of methods and protocols, from various management environments, need to be made, in order to allow the management of role interdependencies.

|epcc|


1515

Grid Management Environments

Resource Management Environment Computing Resource Management Environment Database Resource Management Environment Network Resource Management Environment

Job Management Environment

User Management Environment

Policy Management Environment

Account/Finance Management Environment

Security Management Environment

|epcc|


1616


A Grid management environment sets the rules for the management of a set of Grid entity role types.

The types of Grid entity roles managed in a particular management environment, can not be managed by any other.

|epcc|


1717


GridEntity

|epcc|


1818


JobResourceResource

RoleJobRole

GridEntity


Resource Management Environment

|epcc|


1919


GridEntity


Resource Management Environment

JobAttribute

Set

ResourceAttribute

Set

|epcc|


2020


Different management environments should allow for custom management models, methods, protocols, languages, and semantics.

However, due to the interdependencies between these management environments, they must also share a common set of languages, semantics, and ontologies in an interoperability layer to allow them to interoperate.

Without this common set of interoperability features, it would not be possible to manage role and environment interdependencies.

|epcc|


2121


Account

Security

Policy

UserJob

Resource

|epcc|


2222

InteroperabilityLayer


Account

Security

Policy

UserJob

Resource

|epcc|


2323

The Result?

What is the advantage of these concepts and abstractions?

Discrete roles and management environments allows them to be developed in a highly independent way.

This framework is easily extendable to include new roles and management environments.

This framework allows role definition and attribute sets to be extended or updated, without breaking the framework.

|epcc|


2424

Meta-Scheduling

Scheduling should allow the management of roles within their respective management environments.

Meta-scheduling should handle the interdependencies between roles and their environments, both within Grid entities, and between Grid entities.

|epcc|


2525

Summary

Have introduced the concepts of:

Grid entities Grid entity roles

Grid entity role specific attributes Grid entity description documents Grid management environments

Roles are managed within their respective management environments.

Role interdependencies must be managed.

|epcc|


2626

Conclusions

Managing discrete Grid entity roles in respective management environments allows for the development of highly independent pieces of the Grid management puzzle.

This framework enables manageable extensions and upgrades of the various pieces of the Grid management puzzle.

This framework gives flexibility in defining Grid entities through the attribution of an extensible set of roles.

This framework allows independent management of discrete roles, while enabling management of role interdependencies through the interoperability layer.

This framework should allow the consolidation of disparate scheduling efforts in various collaborations.

|epcc|


2727

The Holy Grail!

Interoperable Grid architectures based on heterogeneous infrastructure layers, managing heterogeneous computing environments.

Need common interoperability layers: protocols, languages, semantics, ontologies

Documents

|epcc| NeSC Workshop Open Issues in Grid Scheduling Ali Anjomshoaa EPCC, University of Edinburgh Tuesday, 21 October 2003 Overview of a Grid Scheduling