Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
CISC 879 : Advanced Parallel Programming
John Cavazos!Dept of Computer & Information Sciences!
University of Delaware!
www.cis.udel.edu/~cavazos/cisc879!
Lecture 1!The Multicore !
Revolution!
CISC 879 : Advanced Parallel Programming
Bio
• John Cavazos <cavazos@cis> • Assistant Professor, CIS • Research
• Applying machine learning to hard systems problems • Compiler Optimizations • Interplay between compiler and architecture • Rethinking fundamental structure of optimizing compilers • GPGPU
CISC 879 : Advanced Parallel Programming
Lecture 1: Overview
• Structure of Course • Motivation • A Brief Overview of the Landscape • The Rest of the Course • Administrivia
CISC 879 : Advanced Parallel Programming
Structure of the Course • First N (N>5) lectures *not* by you!
• General parallel programming • GPUs
• mostly OpenCL, some CUDA and HMPP • Student presentations (20 mins.) • Two projects (next slide)
CISC 879 : Advanced Parallel Programming
Project 1: Intro to GPUs • Choose two programs
• 1 Kernel and 1 Application • Kernel: parallelize sequential code
• We provide this code • Use OpenCL, CUDA, or HMPP
• Application: Compile, Run, Profile • You choose • Can any of “hot” portions be parallelized?
• Goal : Familiarization with environments
CISC 879 : Advanced Parallel Programming
Project 1: Intro to GPUs • Deliverable: Project Report
• 2-3 pages • Template available online • Conference paper format
• Handout available next lecture
CISC 879 : Advanced Parallel Programming
Project 2:Parallelization • Choose two real-work applications
• Bioinformatics, Data mining, Financial, Standard Benchmarks (e.g., SPEC)
• Does not have to be same application from previous project
• Parallelize some important loops
CISC 879 : Advanced Parallel Programming
Project 2:Parallelization • Deliverable: Report (~5 pgs)
• Conference paper format • Project presentation (~15 mins)
• Handout available in a couple weeks
CISC 879 : Advanced Parallel Programming
Machines Available • Several GPU (and other) systems available • EECIS server (primary)
• cuda.acad.ece.udel.edu • Machine has four nVidia GT200 (Tesla C1060)
GPU cards • Cavazos Lab (secondary; only a select few)
• Several NVIDIA and AMD GPUs • Thanks to kind donations from Nvidia and AMD! • Also available: Playstation 3s (Cell processor)
• Your own: many of you have laptops with GPUs
CISC 879 : Advanced Parallel Programming
Basis for Grading
• Paper presentation (20%) • Projects (80%)
• Project 1 (20%) • Project report/checkpoint
• Project 2 (60%) • Project report/presentation/checkpoints
No Midterm or Final!
CISC 879 : Advanced Parallel Programming
Lecture 1: Overview
• Structure of Course • Motivation • A Brief Overview of the Landscape • The Rest of the Course • Administrivia
CISC 879 : Advanced Parallel Programming
Technology Trends
Moore’s Law Gordon Moore (Intel co-founder) predicted in 1965 that transistor density of semiconductor chips would double every 18 months.
Slide Source: Jack Dongarra
CISC 879 : Advanced Parallel Programming
Transistors and Clock Rate
So, processors got faster every 18 months (roughly 50% to 100%!) Why bother with parallel programming? Just wait a year or two…
Slide Source: http://audilab.bmed.mcgill.ca/~funnell/Bacon/HW/trends.html
CISC 879 : Advanced Parallel Programming
Transistors and Clock Rate
So, processors got faster every 18 months (roughly 50% to 100%!) Why bother with parallel programming? Just wait a year or two…
Slide Source: http://audilab.bmed.mcgill.ca/~funnell/Bacon/HW/trends.html
CISC 879 : Advanced Parallel Programming
The “Power Wall”
• High Power Consumption and Heat Dissipation
CISC 879 : Advanced Parallel Programming
Multicores Save Power • Multicores with simple cores decreases
frequency and power • Example : Uni-processor with power
budget N • Increase frequency by 20%
• Increases power, by more than 50% • Only increase performance by 13%
CISC 879 : Advanced Parallel Programming
Multicores Save Power • Multicores with simple cores decreases
frequency and power • Decrease frequency by 20% (e.g.,
simplifying core) • Decreases power by 50% • Can now add another simple core • Power budget stays at N with increased
performance!
CISC 879 : Advanced Parallel Programming
Architects Passed the Ball
• “We are dedicating all of our future product development to multicore designs. … This is a sea change in computing”
- Paul Otellini, President, Intel (2005)
• “Multicore: This is the one which will have the biggest impact on us. We have never had a problem to solve like this. A breakthrough is needed in how applications are done on multicore devices.”
- Bill Gates, Microsoft
CISC 879 : Advanced Parallel Programming
Architects Passed the Ball
• “When we start talking about parallelism and ease of use of truly parallel computers, we're talking about a problem that's as hard as any that computer science has faced. … I would be panicked if I were in industry.”
- John Hennessy, President of Stanford ACM Queue Magazine, 1/07
CISC 879 : Advanced Parallel Programming
Looming “Multicore Crisis”
Slide Source: Berkeley View of Landscape
CISC 879 : Advanced Parallel Programming
Lecture 1: Overview
• Structure of Course • Motivation • A Brief Overview of the Landscape • The Rest of the Course • Administrivia
CISC 879 : Advanced Parallel Programming
The Rise of Multicores
Slide Source: Amarasinghe, 6189 IAP 2007
CISC 879 : Advanced Parallel Programming
Classes of Multicores • Shared-Memory
• Intel “Clovertown” • AMD “Barcelona” • Sun Niagara
• Distributed-Memory • MIT Raw • Cell
• Many-cores • Intel Teraflops • GPUs
CISC 879 : Advanced Parallel Programming
Shared-Memory Multicores
Slide Source: http://www.cosc.brocku.ca/Offerings/3P93/KimFrith/3P99.html
CISC 879 : Advanced Parallel Programming
Distributed-Memory Multicores
Slide Source: http://www.csee.umbc.edu/~plusquel/611/index.html
CISC 879 : Advanced Parallel Programming
Many-cores (GPUs)
Slide Source: http://www.csee.umbc.edu/~plusquel/611/index.html
CISC 879 : Advanced Parallel Programming
Many-cores (GPUs) cont’d
CISC 879 : Advanced Parallel Programming
Lecture 1: Overview
• Structure of Course • Motivation • A Brief Overview of the Landscape • The Rest of the Course • Administrivia
CISC 879 : Advanced Parallel Programming
Background/References
• Background • Parallel programming not required • Computer architecture not required • Should be familiar with the C programming
language
Textbook: (at the campus bookstore)
Heterogeneous Computing with OpenCL
Benedict Gaster, Lee Howes, David Kaeli, Perhaad Mistry, and Dana Schaa
CISC 879 : Advanced Parallel Programming
Project Guidelines
• Papers should be • Well-written and formatted correctly • Properly referenced • Results should be presented with graphs • Intellectual merit most important factor
• Negative result is fine • However, must demonstrate something
interesting
Think of this as a conference submission!
CISC 879 : Advanced Parallel Programming
Expectations
• Class participation • Ask questions • Challenge all speakers. • NOT a lecture class or a passive experience.
ACTIVE learning. • Most common project problem: Not getting started • Ask for help if you need it!
• I will hold office hours by appointment • Require checkpoints to show me status!
CISC 879 : Advanced Parallel Programming
For Next Time
• Read the following IEEE Micro article • SINGLE-THREADED VS. MULTITHREADED: WHERE SHOULD WE
FOCUS? • http://www.cs.wisc.edu/multifacet/papers/ieeemicro07_debate.pdf
• AND watch the video • Single-Threaded vs. Multi-Threaded
• http://www.ele.uri.edu/CARD/summary.html