KUAS.EE

Parallel Computing at a Glance

KUAS.EE

History Parallel Computing

KUAS.EE

What is Parallel Processing ?

Processing of multiple tasks simultaneously on multipleProcessors is called parallel processing.

D1 D2 D3

P1 P2 P3

R

Pm

KUAS.EE

Why Parallel Processing ?

. Computational requirements are ever increasing, both in the area of scientific and business. grand challenge problems. Sequential architecture reaches physical limitation.

. Hardware improvements in pipelining, superscalar are non-scalable and requires sophisticated complier technology.

. Vector processing works well for certain of problems.

. The technology of parallel processing is mature.

. Significant development in networking technology is paving a way for heterogeneous computing.

KUAS.EE

Hardware Architecture for Parallel Processing ?

1. Single instruction single data (SISD)2. Single instruction multiple data (SIMD)3. Multiple instruction and single data (MISD)4. Multiple instruction and multiple data (MIMD)

KUAS.EE

Single instruction single data (SISD)

Sequential computer : PC, Macintosh, Workstation

KUAS.EE

Single instruction multiple data (SIMD)

Vector machines CRAY, Thinking Machines

KUAS.EE

Multiple instruction and single data (MISD)

KUAS.EE

Multiple instruction and multiple data (MIMD)

work asynchronously

KUAS.EE

Shared Memory MIMD Machine

Tightly-couple multiprocessor

Silicon Graphics machinesSun’s SMP

addressshared memorysingle address spacereal address vs. virtual addressthreadNUMA v.s UMAMessage passing v.s shared memory

KUAS.EE

Distributed Shared Memory MIMD Machine

Loosely-coupled multiprocessor

C-DAC’s PARAMIBM’s SP/2Intel’s Paragaon

KUAS.EE

Comparison between Shared Memory MIMDand Distributed Shared MIMD

Shared MemoryMIMD

Distributed Memory

MIMD (MPP)

Manufacturability Easy Easy

Programmability Easy Slightly difficult

Reliability Poor Good

ExtensibilityScalability

Difficult Easy

KUAS.EE

Approaches to Parallel Programming

.Data Parallelism (SIMD)

.Process Parallelism

.Farmer and Worker Models (Master and Slaves)

KUAS.EE

PARAM Supercomputers

KUAS.EE

PARAS Operating Environment

It is a complete parallel programming environment.

1. OS kernel2. Host servers3. Compliers4. Run-time environment5. Parallel file system6. On-line debugger and profiling tool7. Graphics and visualization support8. Networking interface9. Off-line parallel processing tools10. Program restructures11. libraries

Program development environmentProgram run-time environmentUtilities

KUAS.EE

PARAS Programming Model

. PARAS Microkernel

. CONcurrent threads Environment(CORE)

. POSIX threads interface

. Popular Message Passing interface such as . MPI . PVM. Parallelizing Compliers. Tools and debuggers for parallel programming . Load balancing and distribution tools

KUAS.EE

Levels of Parallelism

KUAS.EE

Levels of Parallelism