Multi-core. What is parallel programming ? Classification of parallel architectures Dimension of...

Multi-core

What is parallel programming ? Classification of parallel architectures

Dimension of instruction Dimension of data

Memory models for parallel programming Distributed memory model Share memory model

Multi-core architectures Steps in parallelization

Jaruloj Chongstitvatana 2Parallel Programming: Introduction

CONCURRENT

Two or more actions progress at the same time.

PARALLEL

Two or more actions execute simultaneously.

Jaruloj Chongstitvatana Parallel Programming: Introduction 3

How to write a program that can execute correctly on two proccesors or more.

Design considerations Correctness Efficiency Simplicity Scalability

Jaruloj Chongstitvatana Parallel Programming: Introduction 4

Single Instruction Single Data: SISD

Single Instruction Multiple Data: SIMD

Multiple Instruction Single Data: MISD

Multiple Instruction Multiple Data: MIMD

Jaruloj Chongstitvatana Parallel Programming: Introduction 5

No parallelism

Jaruloj Chongstitvatana Parallel Programming: Introduction 6

Vector machine Graphic processors

Jaruloj Chongstitvatana Parallel Programming: Introduction 7

No application (at this time)

Jaruloj Chongstitvatana Parallel Programming: Introduction 8

Multi-core architectures

Jaruloj Chongstitvatana Parallel Programming: Introduction 9

SHARED MEMORY MODEL

Multiple processors connect to one shared memory.

All processors can access same memory locations.

DISTRIBUTED MEMORY MODEL

Each processor connects to its private memory.

Jaruloj Chongstitvatana Parallel Programming: Introduction 10

SHARED MEMORY

Bottleneck for memory access

No data transfer

Easy for share data

DISTRIBUTED MEMORY

Better memory access

Data transfer from non-local memory

Easy for private data

Jaruloj Chongstitvatana Parallel Programming: Introduction 11

AMD Multicore Opteron Sun UltraSparc T1 IBM Cell Broadband Engine (CBE) Intel Core2 Duo

Jaruloj Chongstitvatana Parallel Programming: Introduction 12

Jaruloj Chongstitvatana Parallel Programming: Introduction 13

Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

Jaruloj Chongstitvatana Parallel Programming: Introduction 14

Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

Jaruloj Chongstitvatana Parallel Programming: Introduction 15

Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

Jaruloj Chongstitvatana Parallel Programming: Introduction 16

Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

Jaruloj Chongstitvatana Parallel Programming: Introduction 17

Source: C. Hughes &T. Hughes, Professional Multicore Programming: Design and Implementation for C++ Developers, Wiley, 2008.

Starting from sequential code

Identify possible concurrency Design and implement Test for correctness Tuning for performance

May effect correctness If tuning is impossible, consider redesign

Jaruloj Chongstitvatana Parallel Programming: Introduction 18

Data race Deadlock

Notes: Results might be slightly different for each

run due to round-off error for different sequence of operations.

Jaruloj Chongstitvatana Parallel Programming: Introduction 19