High Peformance Computer

8/10/2019 High Peformance Computer

1/43

High Perfo rm an c e ParallelSupercompute r

Dien Taufan LessyMCSCE

Dien Taufan Lessy (3011464)High Performace Parallel Supercomputer

Spring Semester 2014


2/43

g p p


Contents

IntroductionParallelismFuture ResearchConclusion

ReferencesLiteratures


3/43


I ntr oduction

The first SupercomputerIBM Naval Ordnance Research Calculator

15000 operations/s ADD(15 s), MUL(31 s), DIV(227 s)

[1]


4/43


I ntr oduction

The first SupercomputerControl Data Corporation 66001 MFLPOS

[2]


5/43


I ntr oduction

Today (November 2013)Tianhe-2 (MilkyWay-2)

[3]

Cores: 3.120.000

Rmax: 33.862,7 (TFLOPs)Power: 17.808 kW


6/43


I ntr oduction

Todays Ranking

[3]


7/43


I ntr oduction

HPC Vendor

[3]


8/43


I ntr oduction

Processor Generation

[3]


9/43


I ntr oduction

Segment user

[3]


10/43


I ntr oduction

OS

[3]


11/43


I ntr oduction

[3]


12/43


Parallelism

History


13/43


I ntr oduction

[3]

Concept and Terminology


14/43


Parallelism

The von Neumman Computer

Walk-Through: c=a+b

1. Get next instruction2. Decode: Fetch a3. Fetch a to internal register4. Get next instruction

5. Decode: fetch b6. Fetch b to internal register7. Get next instruction8. Decode: add a and b (c in register)9. Do the addition in ALU10. Get next instruction11. Decode: store c in main memory12. Move c from internal register to main memory

Note: Some units are idle while others are workingwaste of cycles. Pipelining (modularization) & Cashing (advance decoding)parallelism


15/43


16/43


Parallelism

Increasing Cycle TimeMoores Law


17/43


Parallelism

Increasing Cycle TimeCore Voltage increase with frequency

[6]


18/43


Parallelism

~

~

~

Power cost of Frequency


19/43


Parallelism

High Performance Serial Processor needs

high power


20/43


Parallelism

Processor Memory GAP (Bootleneck)


21/43


Parallelism

Definition

Concurrent vs Parallel

a parallel computer is collection of processingelements that communicate and cooperate to solvelarge problems quickly - Almasi and Gottlieb 1989


22/43


Parallelism

Speedup vs Efficiency

For given problem:

speedup(using P Processors) =

10 Processor with 2 times Speedup?

exec. time (P Processor)

exec. time (1 Processor)


23/43


Parallelism

Serial vs Parallel

[3]


24/43


Parallelism

Serial vs Parallel

[3]


25/43


Parallelism

Serial vs Parallel

[3]


26/43


Parallelism

Serial vs Parallel

[3]


27/43


Parallelism

Processor Type

Scalar processorCISC: Complex Instruction Set Computer

Intel 80x86 (IA32)

RISC: Reduced Instruction Set ComputerSun SPARC, IBM Power #, SGI MIPSVLIW: Very Long Instruction Word; Explicitly parallelinstruction computing (EPIC); Probably dying

Intel IA64 (Itanium)

Vector processor;Cray X1/T90; NEC SX#; Japan Earth Simulator; EarlyCray machines; Japan Life Simulator (hybrid)


28/43


Parallelism

CISC vs RISC vs VLWI

[5]


29/43


Parallelism

[3]

Flynns Classical Taxonomy


30/43


Parallelism

[3]

SISD


31/43


Parallelism

[3]

SIMD


32/43


Parallelism

[3]

MISD


33/43


Parallelism

[3]

MIMD


34/43


Parallelism

Memory ArchitectureShared Memory

Superscalar processors with L2cache connected to memorymodules through a bus or crossbar

All processors have access to allmachine resources including

memory and I/O devicesSMP (symmetric multiprocessor): ifprocessors are all the same andhave equal access to machineresources, i.e. it is symmetric.SMP are UMA (Uniform Memory

Access) machinese.g., A node of IBM SP machine;SUN Ultraenterprise 10000


35/43


Parallelism

Memory ArchitectureShared Memory

If bus,Only one processor can access thememory at a time.Processors contend for bus toaccess memory

If crossbar,Multiple processors can accessmemory through independent pathsContention when differentprocessors access same memorymoduleCrossbar can be very expensive.

Processor count limited by memorycontention and bandwidth

Max usually 64 or 128


36/43


Parallelism

Memory ArchitectureDistributed Memory

Superscalar processors withlocal memory connectedthrough communicationnetwork.Each processor can only workon data in local memory

Access to remote memoryrequires explicitcommunication.Present-day largesupercomputers are all somesort of distributed-memorymachines


37/43


Parallelism

Memory ArchitectureHybrid Distributed-Shared Memory

Overall distributedmemory, SMP nodesMost modern

supercomputers andworkstation clusters areof this typeMessage passing; orhybrid messagepassing/threading.


38/43


Parallelism

[3]

Amdahls Law

Suppose only part of an application seems parallel Amdahl

s law

let s be the fraction of work done sequentially, so(1-s) is fraction parallelizable

P = number of processors

Speedup(P) = Time(1)/Time(P)


39/43


Parallelism

[3]

Amdahls Law


40/43


41/43


Conclusion

Improvement of Single-instruction-streamrequires a lot of effort for little gainParallel computing the only way to achievehigher performance in the foreseeablefutureSupercomputer combines all of parallelcomputing technology such as Paralle CPU,Multicore, Scalar, Vector, etc


42/43


References

[1]: http://www.columbia.edu/cu/computinghistory/norc.html, May 2014[2]: http://en.wikipedia.org/wiki/File:CDC_6600.jc.jpg[3]: http://www.top500.org/, May 2014[4]: https://computing.llnl.gov/tutorials/parallel_comp/[5]: http://15418.courses.cs.cmu.edu/spring2014/lecture/whyparallelism[6]: http://www.intel.com

[7]: http://discovermagazine.com/galleries/zen-photo/m/moores-law[8]: http://people.cs.clemson.edu/~mark/464/acmse_epic.pdf


43/43

Dien Taufan Lessy (3011464)

Literature

Parallel Computer Architecture: A Hardware / Software

Approach, D.E. Culler, J.P. Singh Computer Architecture: A Quantitative Approach, J.L.

Hennessy, D.A. Patterson https://computing.llnl.gov/tutorials/parallel_comp/

https://computing.llnl.gov/tutorials/parallel_comp/OverviewRecentSupercomputers.2008.pdf http://www-users.cs.umn.edu/~karypis/parbook/ http://www.top500.org/ http://15418.courses.cs.cmu.edu

Documents

High Peformance Computer