Introduction To Parallel ProcessorsComputer Science and Engineering
Copyright by Hesham El-Rewini
Grosch’s Law (1960s)
“To sell a computer for twice as much, it must be four times as fast”
Vendors skip small speed improvements in favor of waiting for large ones
Buyers of expensive machines would wait for a twofold improvement in performance for the same price.
Computer Science and Engineering
Copyright by Hesham El-Rewini
Processor performance would double every 18 months
This prediction has held for several decades
Unlikely that single-processor performance continues to increase indefinitely
Computer Science and Engineering
Copyright by Hesham El-Rewini
Von Neumann’s bottleneck
Single control unit connecting a memory to a processing unit
Instructions and data are fetched one at a time from memory and fed to processing unit
Speed is limited by the rate at which instructions and data are transferred from memory to the processing unit.
Computer Science and Engineering
Copyright by Hesham El-Rewini
Problem
Assume that a switching component such as a transistor can switch in zero time. We propose to construct a disk-shaped computer chip with such a component. The only limitation is the time it takes to send electronic signals from one edge of the chip to the other. Make the simplifying assumption that electronic signals travel 300,000 kilometers per second. What must be the diameter of a round chip so that it can switch 109 times per second? What would the diameter be if the switching requirements were 1012 time per second?
Computer Science and Engineering
Copyright by Hesham El-Rewini
Longer R&D time (2-3 years)
Expensive systems
Advances in commodity processors and network technology
Network of PCs and workstations connected via LAN or WAN forms a Parallel System
Network Computing
Computer Science and Engineering
Copyright by Hesham El-Rewini
Amdahl’s Law
The performance improvement to be gained from using some faster mode of execution is limited by the fraction of the time the faster mode can be used
Computer Science and Engineering
Copyright by Hesham El-Rewini
Walk 4 miles /hour 50 + 20 = 70 hours S = 1
Bike 10 miles / hour 20 + 20 = 40 hours S = 1.8
Car-1 50 miles / hour 4 + 20 = 24 hours S = 2.9
Car-2 120 miles / hour 1.67 + 20 = 21.67 hours S = 3.2
Car-3 600 miles /hour 0.33 + 20 = 20.33 hours S = 3.4
must walk
(1- ): The fraction of the program that is naturally parallel
Computer Science and Engineering
Copyright by Hesham El-Rewini
T(N) = 1
Computer Science and Engineering
Copyright by Hesham El-Rewini