CS252Graduate Computer Architecture
Lecture 21
Multiprocessor Networks (con’t)
John Kubiatowicz
Electrical Engineering and Computer Sciences
University of California, Berkeley
http://www.eecs.berkeley.edu/~kubitron/cs252
4/15/2009 cs252-S09, Lecture 21 2
Review: On Chip: Embeddings in two dimensions
• Embed multiple logical dimension in one physical dimension using long wires
• When embedding higher-dimension in lower one, either some wires longer than others, or all wires long
6 x 3 x 2
4/15/2009 cs252-S09, Lecture 21 3
Review:Store&Forward vs Cut-Through Routing
Time: h(n/b + ) vs n/b + h OR(cycles): h(n/w + ) vs n/w + h
• what if message is fragmented?• wormhole vs virtual cut-through
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1 0
23 1
023
3 1 0
2 1 0
23 1 0
0
1
2
3
23 1 0Time
Store & Forward Routing Cut-Through Routing
Source Dest Dest
4/15/2009 cs252-S09, Lecture 21 4
Contention
• Two packets trying to use the same link at same time– limited buffering
– drop?
• Most parallel mach. networks block in place– link-level flow control
– tree saturation
• Closed system - offered load depends on delivered– Source Squelching
4/15/2009 cs252-S09, Lecture 21 5
Bandwidth• What affects local bandwidth?
– packet density b x ndata/n
– routing delay b x ndata /(n + w)
– contention» endpoints
» within the network
• Aggregate bandwidth– bisection bandwidth
» sum of bandwidth of smallest set of links that partition the network
– total bandwidth of all the channels: Cb
– suppose N hosts issue packet every M cycles with ave dist » each msg occupies h channels for l = n/w cycles each
» C/N channels available per node
» link utilization for store-and-forward: = (hl/M channel cycles/node)/(C/N) = Nhl/MC < 1!
» link utilization for wormhole routing?
4/15/2009 cs252-S09, Lecture 21 6
Saturation
0
10
20
30
40
50
60
70
80
0 0.2 0.4 0.6 0.8 1
Delivered Bandwidth
Lat
ency
Saturation
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0 0.2 0.4 0.6 0.8 1 1.2
Offered BandwidthD
eliv
ered
Ban
dw
idth
Saturation
4/15/2009 cs252-S09, Lecture 21 7
How Many Dimensions?• n = 2 or n = 3
– Short wires, easy to build
– Many hops, low bisection bandwidth
– Requires traffic locality
• n >= 4– Harder to build, more wires, longer average length
– Fewer hops, better bisection bandwidth
– Can handle non-local traffic
• k-ary d-cubes provide a consistent framework for comparison
– N = kd
– scale dimension (d) or nodes per dimension (k)
– assume cut-through
4/15/2009 cs252-S09, Lecture 21 8
Traditional Scaling: Latency scaling with N
• Assumes equal channel width– independent of node count or dimension
– dominated by average distance
0
20
40
60
80
100
120
140
0 5000 10000
Machine Size (N)
Ave L
ate
ncy T
(n=40)
d=2
d=3
d=4
k=2
n/w
0
50
100
150
200
250
0 2000 4000 6000 8000 10000
Machine Size (N)
Ave L
ate
ncy T
(n=140)
4/15/2009 cs252-S09, Lecture 21 9
Average Distance
• but, equal channel width is not equal cost!
• Higher dimension => more channels
0
10
20
30
40
50
60
70
80
90
100
0 5 10 15 20 25
Dimension
Ave D
ista
nce
256
1024
16384
1048576
ave dist = d (k-1)/2
4/15/2009 cs252-S09, Lecture 21 10
In the 3D world• For n nodes, bisection area is O(n2/3 )
• For large n, bisection bandwidth is limited to O(n2/3 )– Bill Dally, IEEE TPDS, [Dal90a]
– For fixed bisection bandwidth, low-dimensional k-ary n-cubes are better (otherwise higher is better)
– i.e., a few short fat wires are better than many long thin wires
– What about many long fat wires?
4/15/2009 cs252-S09, Lecture 21 11
Dally paper (con’t)• Equal Bisection,W=1 for hypercube W= ½k
• Three wire models:– Constant delay, independent of length
– Logarithmic delay with length (exponential driver tree)
– Linear delay (speed of light/optimal repeaters)
Logarithmic Delay Linear Delay
4/15/2009 cs252-S09, Lecture 21 12
Equal cost in k-ary n-cubes• Equal number of nodes?• Equal number of pins/wires?• Equal bisection bandwidth?• Equal area?• Equal wire length?
What do we know?• switch degree: d diameter = d(k-1)• total links = Nd• pins per node = 2wd• bisection = kd-1 = N/k links in each directions• 2Nw/k wires cross the middle
4/15/2009 cs252-S09, Lecture 21 13
Latency for Equal Width Channels
• total links(N) = Nd
0
50
100
150
200
250
0 5 10 15 20 25
Dimension
Average L
ate
ncy (n =
40,
= 2
)256
1024
16384
1048576
4/15/2009 cs252-S09, Lecture 21 14
Latency with Equal Pin Count
• Baseline d=2, has w = 32 (128 wires per node)
• fix 2dw pins => w(d) = 64/d
• distance up with d, but channel time down
0
50
100
150
200
250
300
0 5 10 15 20 25
Dimension (d)
Ave
Lat
ency
T(n
=40B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
0
50
100
150
200
250
300
0 5 10 15 20 25
Dimension (d)
Ave
Lat
ency
T(n
= 14
0 B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
4/15/2009 cs252-S09, Lecture 21 15
Latency with Equal Bisection Width
• N-node hypercube has N bisection links
• 2d torus has 2N 1/2
• Fixed bisection => w(d) = N 1/d / 2 = k/2
• 1 M nodes, d=2 has w=512!0
100
200
300
400
500
600
700
800
900
1000
0 5 10 15 20 25
Dimension (d)
Ave L
ate
ncy T
(n=40)
256 nodes
1024 nodes
16 k nodes
1M nodes
4/15/2009 cs252-S09, Lecture 21 16
Larger Routing Delay (w/ equal pin)
• Dally’s conclusions strongly influenced by assumption of small routing delay
– Here, Routing delay =20
0
100
200
300
400
500
600
700
800
900
1000
0 5 10 15 20 25
Dimension (d)
Ave L
ate
ncy T
(n= 1
40 B
)
256 nodes
1024 nodes
16 k nodes
1M nodes
4/15/2009 cs252-S09, Lecture 21 17
Latency under Contention
• Optimal packet size? Channel utilization?
0
50
100
150
200
250
300
0 0.2 0.4 0.6 0.8 1
Channel Utilization
Late
ncy
n40,d2,k32
n40,d3,k10
n16,d2,k32
n16,d3,k10
n8,d2,k32
n8,d3,k10
n4,d2,k32
n4,d3,k10
4/15/2009 cs252-S09, Lecture 21 18
Saturation
• Fatter links shorten queuing delays
0
50
100
150
200
250
0 0.2 0.4 0.6 0.8 1
Ave Channel Utilization
Late
ncy
n/w=40
n/w=16
n/w=8
n/w = 4
4/15/2009 cs252-S09, Lecture 21 19
Phits per cycle
• higher degree network has larger available bandwidth– cost?
0
50
100
150
200
250
300
350
0 0.05 0.1 0.15 0.2 0.25
Flits per cycle per processor
Late
ncy
n8, d3, k10
n8, d2, k32
4/15/2009 cs252-S09, Lecture 21 20
Discussion• Rich set of topological alternatives with deep
relationships
• Design point depends heavily on cost model– nodes, pins, area, ...
– Wire length or wire delay metrics favor small dimension
– Long (pipelined) links increase optimal dimension
• Need a consistent framework and analysis to separate opinion from design
• Optimal point changes with technology
4/15/2009 cs252-S09, Lecture 21 21
• Problem: Low-dimensional networks have high k– Consequence: may have to travel many hops in single dimension– Routing latency can dominate long-distance traffic patterns
• Solution: Provide one or more “express” links
– Like express trains, express elevators, etc» Delay linear with distance, lower constant» Closer to “speed of light” in medium» Lower power, since no router cost
– “Express Cubes: Improving performance of k-ary n-cube interconnection networks,” Bill Dally 1991
• Another Idea: route with pass transistors through links
Another Idea: Express Cubes
4/15/2009 cs252-S09, Lecture 21 22
The Routing problem: Local decisions
• Routing at each hop: Pick next output port!
Cross-bar
InputBuffer
Control
OutputPorts
Input Receiver Transmiter
Ports
Routing, Scheduling
OutputBuffer
4/15/2009 cs252-S09, Lecture 21 23
How do you build a crossbar?
Io
I1
I2
I3
Io I1 I2 I3
O0
Oi
O2
O3
RAMphase
O0
Oi
O2
O3
DoutDin
Io
I1
I2
I3
addr
4/15/2009 cs252-S09, Lecture 21 24
Input buffered swtich
• Independent routing logic per input– FSM
• Scheduler logic arbitrates each output– priority, FIFO, random
• Head-of-line blocking problem
Cross-bar
OutputPorts
Input Ports
Scheduling
R0
R1
R2
R3
4/15/2009 cs252-S09, Lecture 21 25
Output Buffered Switch
• How would you build a shared pool?
Control
OutputPorts
Input Ports
OutputPorts
OutputPorts
OutputPorts
R0
R1
R2
R3
4/15/2009 cs252-S09, Lecture 21 26
Output scheduling
• n independent arbitration problems?– static priority, random, round-robin
• simplifications due to routing algorithm?
• general case is max bipartite matching
Cross-bar
OutputPorts
R0
R1
R2
R3
O0
O1
O2
InputBuffers
4/15/2009 cs252-S09, Lecture 21 27
Switch Components
• Output ports– transmitter (typically drives clock and data)
• Input ports– synchronizer aligns data signal with local clock domain– essentially FIFO buffer
• Crossbar– connects each input to any output– degree limited by area or pinout
• Buffering• Control logic
– complexity depends on routing logic and scheduling algorithm
– determine output port for each incoming packet– arbitrate among inputs directed at same output
4/15/2009 cs252-S09, Lecture 21 28
Properties of Routing Algorithms• Routing algorithm:
– R: N x N -> C, which at each switch maps the destination node nd to the next channel on the route
– which of the possible paths are used as routes?– how is the next hop determined?
» arithmetic» source-based port select» table driven» general computation
• Deterministic– route determined by (source, dest), not intermediate state (i.e. traffic)
• Adaptive– route influenced by traffic along the way
• Minimal– only selects shortest paths
• Deadlock free– no traffic pattern can lead to a situation where packets are deadlocked
and never move forward
4/15/2009 cs252-S09, Lecture 21 29
Routing Mechanism• need to select output port for each input packet
– in a few cycles
• Simple arithmetic in regular topologies– ex: x, y routing in a grid
» west (-x) x < 0
» east (+x) x > 0
» south (-y) x = 0, y < 0
» north (+y) x = 0, y > 0
» processor x = 0, y = 0
• Reduce relative address of each dimension in order– Dimension-order routing in k-ary d-cubes
– e-cube routing in n-cube
4/15/2009 cs252-S09, Lecture 21 30
Deadlock Freedom• How can deadlock arise?
– necessary conditions:» shared resource» incrementally allocated» non-preemptible
– think of a channel as a shared resource that is acquired incrementally
» source buffer then dest. buffer» channels along a route
• How do you avoid it?– constrain how channel resources are allocated– ex: dimension order
• How do you prove that a routing algorithm is deadlock free?
– Show that channel dependency graph has no cycles!
4/15/2009 cs252-S09, Lecture 21 31
Consider Trees
• Why is the obvious routing on X deadlock free?– butterfly?
– tree?
– fat tree?
• Any assumptions about routing mechanism? amount of buffering?
4/15/2009 cs252-S09, Lecture 21 32
Up*-Down* routing for general topology• Given any bidirectional network
• Construct a spanning tree
• Number of the nodes increasing from leaves to roots
• UP increase node numbers
• Any Source -> Dest by UP*-DOWN* route– up edges, single turn, down edges
– Proof of deadlock freedom?
• Performance?– Some numberings and routes much better than others
– interacts with topology in strange ways
4/15/2009 cs252-S09, Lecture 21 33
Turn Restrictions in X,Y
• XY routing forbids 4 of 8 turns and leaves no room for adaptive routing
• Can you allow more turns and still be deadlock free?
+Y
-Y
+X-X
4/15/2009 cs252-S09, Lecture 21 34
Minimal turn restrictions in 2D
West-first
north-last negative first
-x +x
+y
-y
4/15/2009 cs252-S09, Lecture 21 35
Example legal west-first routes
• Can route around failures or congestion
• Can combine turn restrictions with virtual channels
4/15/2009 cs252-S09, Lecture 21 36
General Proof Technique• resources are logically associated with channels
• messages introduce dependences between resources as they move forward
• need to articulate the possible dependences that can arise between channels
• show that there are no cycles in Channel Dependence Graph
– find a numbering of channel resources such that every legal route follows a monotonic sequence no traffic pattern can lead to deadlock
• network need not be acyclic, just channel dependence graph
4/15/2009 cs252-S09, Lecture 21 37
Example: k-ary 2D array• Thm: Dimension-ordered (x,y) routing
is deadlock free
• Numbering– +x channel (i,y) -> (i+1,y) gets i
– similarly for -x with 0 as most positive edge
– +y channel (x,j) -> (x,j+1) gets N+j
– similary for -y channels
• any routing sequence: x direction, turn, y direction is increasing
• Generalization: – “e-cube routing” on 3-D: X then Y then Z
1 2 3
01200 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
17
18
1916
17
18
4/15/2009 cs252-S09, Lecture 21 38
Channel Dependence Graph
1 2 3
01200 01 02 03
10 11 12 13
20 21 22 23
30 31 32 33
17
18
1916
17
18
1 2 3
012
1718 1718 1718 1718
1 2 3
012
1817 1817 1817 1817
1 2 3
012
1916 1916 1916 1916
1 2 3
012
4/15/2009 cs252-S09, Lecture 21 39
More examples:• What about wormhole routing on a ring?
• Or: Unidirectional Torus of higher dimension?
012
3
45
6
7
4/15/2009 cs252-S09, Lecture 21 40
Deadlock free wormhole networks?• Basic dimension order routing techniques don’t work for
unidirectional k-ary d-cubes– only for k-ary d-arrays (bi-directional)– And – dimension-ordered routing not adaptive!
• Idea: add channels!– provide multiple “virtual channels” to break the dependence cycle
– good for BW too!
– Do not need to add links, or xbar, only buffer resources
• This adds nodes to the CDG, remove edges?
OutputPorts
Input Ports
Cross-Bar
4/15/2009 cs252-S09, Lecture 21 41
When are virtual channels allocated?
• Two separate processes:– Virtual channel allocation– Switch/connection allocation
• Virtual Channel Allocation– Choose route and free output virtual channel
• Switch Allocation– For each incoming virtual channel, must negotiate switch on
outgoing pin
• In ideal case (not highly loaded), would like to optimistically allocate a virtual channel
OutputPorts
Input Ports
Cross-Bar
Hardware efficient designFor crossbar
4/15/2009 cs252-S09, Lecture 21 42
Breaking deadlock with virtual channels
Packet switchesfrom lo to hi channel
4/15/2009 cs252-S09, Lecture 21 43
Paper Discusion: Linder and Harden
• Paper: “An Adaptive and Fault Tolerant Wormhole Routing Stategy for k-ary n-cubes”
– Daniel Linder and Jim Harden
• General virtual-channel scheme for k-ary n-cubes– With wrap-around paths
• Properties of result for uni-directional k-ary n-cube:– 1 virtual interconnection network
– n+1 levels
• Properties of result for bi-directional k-ary n-cube:– 2n-1 virtual interconnection networks
– n+1 levels per network
4/15/2009 cs252-S09, Lecture 21 44
Example: Unidirectional 4-ary 2-cube
Physical Network• Wrap-around channels
necessary but cancause deadlock
Virtual Network• Use VCs to avoid deadlock• 1 level for each wrap-around
4/15/2009 cs252-S09, Lecture 21 45
Bi-directional 4-ary 2-cube: 2 virtual networks
Virtual Network 1 Virtual Network 2
Recommended
