1

Parallel processing speedup Parallel processing speedup performance laws and their performance laws and their

characteristicscharacteristics

Sukhnandan Kaur Sukhnandan Kaur M-tech(CSE)M-tech(CSE)

1717

2

AbstractAbstract Use Amdahl's Law and Gustafson's law

to measure the speedup factor Characteristics

3

What is Amdahl's law What is Amdahl's law Amdahl’s law states that the speedup

achieved through parallelization of a program is limited by the percentage of its workload that is inherently serial

We can get no more than a maximum speedup equal to 1 / (s + p / N )

4

What is Gustafson’s law Gustafson’s law states that, with

increasing data size, the speedup obtained through parallelization increases, because the parallel work increases with data size

The speedup factor is S + N ( 1 – S )

5

In detailsIn details Amdahl's Law Gustafson's law

6

Amdahl’ law: quantify parallelizability

Amdahl's law is named after computer achitect Gene Amdahl, and was made in 1967 when Amdahl was working in IBM

Amdahl's Law quantifies the theoretical speedup that can be obtained by parallelizing a computational load among a set number of processors

7

Classic Model of Parallel Classic Model of Parallel ProcessingProcessing

Multiple Processors available (4)

A Process can be divided into serial and parallel portions

The parallel parts are executed concurrently

Serial Time: 10 time units

Parallel Time: 4 time units

S - Serial or non-parallel portion

A - All A parts can be executed concurrently

B - All B parts can be executed concurrently

All A parts must be completed prior to executing the B parts

An example parallel process of time 10:

Executed on a single processor:

Executed in parallel on 4 processors:

S A A A A B B B B S

SA

A

A

A

B

B

B

B

S

8

Amdahl’s Law (Analytical Amdahl’s Law (Analytical Model)Model)

Analytical model of parallel speedup from 1960s

Parallel fraction () is run over n processors taking /n time

The part that must be executed in serial (1- ) gets no speedup

Overall performance is limited by the fraction of the work that cannot be done in parallel (1- )

diminishing returns with increasing processors (n)

processors ofnumber parallelin done

becan work thatoffraction

,)1(

1

n

where

n

meParallelTiSerialTimeSpeedup

9

Shortcomings of Amdahl’s lawShortcomings of Amdahl’s law Using Amdahl's Law as an argument

against massively parallel processing is not valid

The serial percentage is not practically obtainable

10

We have Gustafson’s lawWe have Gustafson’s law In 1988, John Gustafson refined Amdahl's

model Adding due consideration for large-scale

resources and tasks View an example to prove that why we

need Gustafson’s law

11

Equations of Gustafson’s lawEquations of Gustafson’s law

12

Compare the two lawsCompare the two laws Amdahl’s law: Suppose two cities are 60 km apart, a car

has spent one hour travelling the first 30 km. No matter how fast it drives the last 30 km, it is impossible to achieve 90 km/h before arriving the destination

13

Compare the two laws (Cont’d)Compare the two laws (Cont’d) Gustafson’s law: Suppose a car has already been travelling for

some time at speed of less than 90km/h, and when given enough time and distance to travel, the car’s average speed can reach 90km/h as long as it drives faster than 90 km/h for some time. And also the average speed can reach 120km/h and even 150km/h as long as it drives fast enough in the following part

14

CharacteristicsCharacteristicsHigh PerformanceExpandability and ScalabilityHigh ThroughputHigh Availability

15

ConclusionConclusion Amdahl’s presumption of fixed data size

is obviously a restriction which does not map into reality for many problems

Both laws are in fact different perspective over the same truth – one sees data size as fixed and the other sees the relation as a function of data size

16

Thank you