CSC 7600 Lecture 28 : Final Exam Review Spring 2010 HIGH PERFORMANCE COMPUTING: MODELS, METHODS, & MEANS FINAL EXAM REVIEW Daniel Kogler, Chirag Dekate

CSC 7600 Lecture 28 : Final Exam Review Spring 2010

HIGH PERFORMANCE COMPUTING: MODELS, METHODS, & MEANS

FINAL EXAM REVIEW

Daniel Kogler, Chirag Dekate & Timur GilmanovDepartment of Computer ScienceLouisiana State UniversityMay 5th, 2011


Lecture 2 : Parallel Computer Architecture

• HPC System Stack (5)• Performance Factors (7, 8, 9, 10)• Scalability (21, 22)• MIMD, SIMD, Vector Processing (Pipelineing –

Important), Shared Memory

2


Lecture 3: Commodity Clusters

• Commodity clusters vs Constellations (8)• Key parameters for cluster computing (24)• Where is the parallelism (25)• Constituent hardware elements (27)• Decoupled Work Queue Model (46)

3


Lecture 4 : Benchmarking

• Basic Performance metrics (4)• Benchmarking Definition (5)• Purpose of Benchmarking (6)• Linpack and HPL

4


Lecture 5: Capacity Computing

• Speedup & Efficiency (5)• Capacity, Capability, Cooperative – Important (7,8,11)• Ideal Speedup (18,19)• Granularities in parallelism (20)• Overhead (21, 22, 23, 24)• Condor Class Ads (30, 31)• Condor MatchMaker(32)• Condor commands (37) • Capacity Computing Performance issues (53)

5


Lecture 6: Communicating Sequential Processes (CSP)

• Scalability, Strong Scaling, Weak Scaling (7, 8, 9, 10)• Cooperative Computing (12)• Cooperative Computing Highlights :

– Data decomposition Goals in CSP (13) – Distributed Concurrent Processes (14)– Data Exchange (15)– Synchronization(16)

• Performance issues in CSP (64, 65)

6


Lecture 7: MPI

• Point to point communication in-depth• Deadlock & how to resolve deadlocks• Be able to Understand MPI programs and detect

conceptual flaws in the programs and correct the errors etc.

7


Lecture 8: MPI

• Collective Calls • Be able to understand MPI programs and detect

conceptual flaws in the program and correct the errors

8


Lecture 9: SMP

• IMPORTANT - AMDAHLS LAW(9,10,11,12, 13) • Levels of Memory Hierarchy (30) • Cache Measures (31, 32)• IMPORTANT - CACHE PERFORMANCE (33) refer to

lecture 10 that has more complete and comprehensive coverage of this topic.

9


Lecture 10: Enabling Technologies

• Logic technology metrics (12)• What is bisection bandwidth (67)

10


Lecture11: Pthreads

• IMPORTANT – CPI – Cycles Per Instructions (7, 8, 9, 10)

• Race Conditions, Critical Sections (15, 16)• Thread Synchronization Mechanisms(17 – 22)• Important - Deadlock, Livelock, Starvation(29)• Priority inversion (30)

11


Lecture 12: OpenMP

• Where are we? (4) – use as a guide for material to study• HPC Modelities (5) • OpenMP data environment (28)• OpenMP work-sharing directives (29)• OpenMP thread synchronizations (40, 41, 42, 43)• OpenMP reduction• Be able to read and understand OpenMP C source code,

and detect anomalies, and correct the errors.

12


Lecture 13: Performance

• Performance Counters (11, 12, 13, 14)• Performance Analysis Tools (15) • Gprof (20)• PerfSuite (24,25)• PAPI (28, 29, 30, 31, 32)• TAU (49, 50, 51, 78, 79)• SMP to MPP (60, 62, 63)

13


Lecture 14: Visualization

• Basic gnuplot commands (48 – 56)

14


Lecture 15: Parallel Algorithms 1

• OpenMP and MPI Matrix Multiplication (54 – 70)• Review source code

15



• Parallel Matrix Processing and Locality (5, 6, 7, 8, 9)• Matrix Transpose (18-19, 21-23)

16



• Review Parallel Sorting Algorithms

17



• Not Testable

18


Lecture 19: Parallel File I/O 1

• RAID (9, 10,11, 12, 13, 14)• Distributed File Systems: NFS (16, 17, 18)• Parallel File Systems (20, 21, 22, 23)

19


Lecture 20: Parallel File I/O 2

• Not Testable

20


Lecture 21: Operating System 1

• Operating System (7, 12, 13, 14, 15, 16)• Process Management (18, 19, 20, 23, 24, 25)• Threads (27, 28)• Memory Management (31, 32)• Storage Management (35, 36)• OS Kernel (39) • Modern Operating Systems (41, 42, 44, 45)• Unix and Linux (57, 67, 68)

21


Lecture 22 Libraries 1

• Lecture 19:– Static & Dynamic libraries (27, 28)

22


Lecture 23 PFIO3 + Libraries 2

• Not testable

23


Lecture 24: Operating Systems 2

• Linux and Unix Concurrency mechanisms (24, 25)• Linux and Unix Scheduling (26, 27,28)• Unix & Linux I/O (32, 33)• Lightweight Kernels (49, 50)• Compute Node Kernels (55, 56, 57)

24


Lecture 25: Scheduling

• Job Scheduling (4, 5)• CPU Scheduling algorithms (9 – 15)• Workload Management Systems (20 – 27) • Scheduling Algorithms for WMS FIFO, FIFO with Backfill,

EASY, Conservative (29 – 46)

25


Lecture 26 : Checkpointing and System Administration

• Not Testable

26


Lecture 27: Beyond and Beyond

• Not Testable

27

CSC 7600 Lecture 28 : Final Exam Review Spring 2010 28

Documents

CSC 7600 Lecture 28 : Final Exam Review Spring 2010 HIGH PERFORMANCE COMPUTING: MODELS, METHODS, & MEANS FINAL EXAM REVIEW Daniel Kogler, Chirag Dekate