CSE 8383 - Advanced Computer Architecture

CSE 8383 - Advanced Computer Architecture

Week-13April 15, 2004

engr.smu.edu/~rewini/8383

Contents

Warm up Big Picture Clusters Scheduling Mobile IP Reliability issues in Mobile IP

Let’s test your network background

Group Work

What?

Application

Presentation

Session

Transport

Network

Data Link

Physical

Application

Presentation

Session

Transport

Network

Data Link

Physical

LAN LANInternet

Internet

Explain

Application

Presentation

Session

Transport

Network

Data Link

Physical

Mail ftp Telnet

Transmission Control Protocol

(TCP)

Internet Protocol(IP)

Ethernet Token ring

Big Picture

Leopold’s View of the FieldNumerous Application Programs

Concrete Architectures

Pthreads Java ThreadsOpenMP

Skeletons

MPI PVMThreads

Shared Memory Message Passing

Distributed SMCluster

SMP CC-NUMA ATMMyrinet

Hiding Details

High

Low

Parallel and Distributed Architecture (Leopold, 2001)

Degree of Coupling

SIMD MIMD

Shared Memory

Distributed Memory

Supported Grain Sizes

Communication Speedslowfast

fine coarse

loosetight

SIMD SMP CC-NUMA DMPC Cluster Grid

Clusters(Commodity Off The Shelf)

What is a Cluster? A Collection of interconnected

stand-alone computers working working together as a single integrated computing resource

Cluster nod3es may exist in a single cabinet or be physically separated and connected via a LAN

Clusters

M

C

P

I/O

OS

M

C

P

I/O

OS

M

C

P

I/O

OS

Middleware

Programming Environment

Interconnection Network

Clusters offer these features High Performance Expandability and Scalability High Throughput High Availability

High Performance Clusters

Tuned to derive maximum

performance

Share processing load

Needs software customization

E.g. Beowulf

Cluster Components

Homogeneous Clusters All nodes have same configuration

Heterogeneous Clusters Nodes with different configurations e.g.

different OSs

Cluster Architecture and OS Cluster of PCs Cluster of Workstations Cluster of SMPs (Symmetric

Multiprocessors) Linux clusters (Beowulf) Solaris clusters (NOW) NT clusters

Cluster Size Group clusters – 2-99 nodes Departmental clusters – 99-999

nodes Organizational clusters – many

100’s Global clusters – 1000’s +

Internet wide

Typical Cluster EnvironmentUser Application

PVM/MPI

OS/Hardware

Middleware

PVM & MPI PVM (Parallel Virtual Machine)

Allows heterogeneous collection of computers linked by a network to be used as a single large parallel computer

MPI (Message Passing Interface) Library specification for message passing Free and vendor supplied

implementations available

PVM Introduction http://www.netlib.org/pvm3/ http://www.epm.ornl.gov/pvm/ Started as a research project in 1989 Developed at Oak Ridge National Lab &

University of Tennessee It makes it possible to develop

applications on a set of heterogeneous computers connected by a network that appears logically to user as a single parallel computer

PVM Environment Virtual machine Dynamic set of heterogeneous

computer systems connected via a network and managed as a single parallel computer

Computer nodes hosts Hosts are uniprocessors or

multiprocessors running PVM software

PVM Software Two Components:

Library of PVM routines Daemon

Should reside on all hosts in the virtual machine

Before running an application, the user must start up PVM and configure a virtual machine

PVM Application A number of sequential programs,

each of which will correspond to one or more processes in a parallel program

These programs are compiled individually for each host in the virtual machine

Object files are placed in locations accessible from other hosts

PVM Application (Cont.) One of these sequential programs,

which is called the initiation task, has to be started manually on one of the hosts

Tasks on other hosts are started automatically by the initiation task

Tasks comprising a PVM application can be identical (SPMD) [common in most applications] or can be different (pipeline: input processing, output)

Application Structure Start graph

Middle node is call supervisor or master

Supervisor-workers or Master-slaves Tree

Root is the top supervisor Hierarchy

Task Creation A task in PVM can be started manually

or can be spawned from another task The function pvm_spawn() is used for

dynamic task creation. The task that calls the function

pvm_spawn() is referred to as the parent

The newly created tasks are called children.

To Create a child, you must specify: 1. The machine on which the child will be

started2. A path to the executable file on the specified

machine3. The number of copies of the child to be

created4. An array of arguments to the child tasks

Task ID All PVM tasks are identified by an integer

task identifier When a task is created it is assigned a

unique identifier (TID) Task identifiers can be used to identify

senders and receivers during communication. It can also be used to assign functions to different tasks based on their TIDs

Task ID Retrieval Task’s TID pvm_mytid()

Mytid = pvm_mytid;

Child’s TID pvm_spawn()pvm_spawn(…,…,…,…,…, &tid);

Parent’s TID pvm_parent()my_parent_tid = pvm_parent();

Daemon’s TID pvm_tidtohost()daemon_tid = pvm_tidtohost(id);

Scheduling

Partitioner

Grains of Sequential Code

Parallel/Distributed System

Parallel Program Tasks

Scheduler

Schedule

Processors

Time

Program Tasks

Sequential Program

Explicit ApproachImplicit Approach

Dependence Analyzer

Ideal Parallelism

Scheduling

Scheduling Introduction Model

Program tasks Machine Schedule Execution and communication time

Problem Complexity

Introduction to Scheduling This problem has been described in

a number of different ways in different fields

Classical problem of job sequencing in production management has influenced most of the solutions

Set of resources and set of consumers

Scheduling System

ConsumersConsumers ResourcesResources

SchedulerScheduler

PolicyPolicy

Program Tasks

(T, <, D, A)

T set of tasks < partial order on T D Communication Data A amount of computation

Task GraphA

10

D

15

E

10

F

20

B

15

C

10

G

15

H

15

I

30

558 7

5

5 5

10

5

4 5 4

20

Machine m heterogeneous processors Connected via an arbitrary

interconnection network (network graph)

Associated with each processor Pi is its speed Si

Associated with each edge (i,j) is the transfer rate Rij

Examples of Network Graphs

Linear Array

Ring

Mesh

6 3

1 2

5 4 Fully Connected

Task Schedule Gantt Chart Mapping (f) of tasks to a processing

element and a starting time Formally: f: T {1,2,3, …, m} x [0infinity f(v) = (i,t) task v is scheduled to

be processed by processor i starting at time t

Gantt Chart

Processor

Time

1 2 3 4 5 0

1

2

3

4

Stop

Start

W-1 W-2 W-3 W-4 W-5

Execution and Communication Times If task ti is executed on pj

Execution time = Ai/Sj

The communication delay between ti and tj, when executed on adjacent processing elements pk and pl is

Dij/Rkl

Complexity Computationally intractable in

general Small number of polynomial optimal

algorithms in restricted cases A large number of heuristics in more

general cases Quality of the scheduleschedule vs. Quality of

the schedulerscheduler

Mobile Computing

What is Driving Mobile Computing Advances in Wireless

Communication Technology Advances in Portable

Computing Technology Reliance on Network

Computing Mobile Workforce

Mobile Computing Using small size portable

computers, hand-helds, and other small wearable devices,

To run applications and access information resources via wireless connections

By mobile and nomadic users

Mobility versus Nomadicity Mobile Node

The node is able to change its point of attachment from one subnet to another while maintaining all existing communication.

Nomadic NodeThe node must terminate all existing communication before changing its point of attachment.

Main Components Mobile Hosts

Backbone Network Wired Wireless Multi-hop Hybrid

Wired Backbone

Fixed Communication Network

Fixed Host

Fixed host

Fixed HostFixed Host

Base Station

Base Station

Base Station

Base Station

Wired Backbone

Mobile Host

Mobile Host

Mobile Host

Mobile Host

Wireless Multi-hop Backbone


Mobile Host

Mobile Host

Mobile Host

Mobile Host Mobile Host

Mobile HostMobile HostMobile Host

Mobile HostMobile Host


Hybrid backbone

Fixed Communication

Network

Fixed Host

Fixed host

Fixed HostFixed Host

Base StationBase Station

Base StationBase Station

Wired Backbone


Mobile Host

Mobile Host


Mobile Host

Mobile Host

Mobile Host Mobile Host




Mobile Host

Hybrid Backbone

Mobile Host

IETF Mobile IP System Components

Mobile host Home address Home agent Foreign agent (NOT in V6) Care of address

Mobile IP (Cont.) Mobile IP Functionality

Agent Discovery Registration Tunneling

Optimization Binding Caches Foreign Agent Smooth Handoff

Mobile IP (Cont.)

Arbitrary Topology of Routers and

Links

Home Agent

Mobile Host at Home

Foreign Agent

Foreign Agent

Mobile Host visitingA foreign subnet

Home subnet

Foreign subnet

Foreign subnet

Update in Mobile IP Mobile Host at Home

Two-Tier Update Can we reduce the cost of update?

The use of Proxy Agents

Neighborhood (Location Area)

Tradeoff Between Update and Search

Mobile Host at Home

Location area

Documents

CSE 8383 - Advanced Computer Architecture