Embed Size (px)
Citation preview
CSE 222A: Computer Communication Networks Alex C. Snoeren
Lecture 9:Hybrid Networks"
Thanks: Nathan Farrington
Lecture 9 Overview" Project discussion Overview optical technology
Helios paper discussion
2 CSE 222A – Lecture 9: Hybrid Networks"
The Case for Optics"
3
Electrical Packet Switch
$500/port 10 Gb/s fixed rate 12 W/port Requires transceivers Per-packet switching For bursty, uniform traffic
Optical Circuit Switch
$500/port Rate free 240 mW/port No transceivers 12 ms switching time For stable, pair-wise traffic
3 CSE 222A – Lecture 9: Hybrid Networks"
Optical Circuit Switch"
Lenses Fixed Mirror
Mirrors on Motors
Glass Fiber Bundle
Input 1 Output 2 Output 1
Rotate Mirror 1. Full crossbar switch 2. Does not decode packets 3. Needs external scheduler
4 CSE 222A – Lecture 9: Hybrid Networks"
Electrical Packet Switch 1 2 3 4 5 6 7 8
WDM MUX WDM DEMUX
Optical Circuit Switch
Superlink
10G WDM Optical Transceivers
No Transceivers Required 80G
Wavelength Division Mux"
5 CSE 222A – Lecture 9: Hybrid Networks"
Bisection Bandwidth
10% Electrical (10:1
Oversubscribed)
100% Electrical
Helios Example 10% Electrical + 90%
Optical
Cost $6.3 M
Power 96.5 kW
Cables 6,656
Example: N=64 pods * k=1024 hosts/pod = 64K hosts total; 8 wavelengths
N pods, k-ports each
k switches, N-ports each
Bisection Bandwidth
10% Electrical (10:1
Oversubscribed)
100% Electrical
Helios Example 10% Electrical + 90%
Optical
Cost $6.3 M $62.3 M
Power 96.5 kW 950.3 kW
Cables 6,656 65,536
Example: N=64 pods * k=1024 hosts/pod = 64K hosts total; 8 wavelengths
N pods, k-ports each
k switches, N-ports each
Bisection Bandwidth
10% Electrical (10:1
Oversubscribed)
100% Electrical
Helios Example 10% Electrical + 90%
Optical
Cost $6.3 M $62.2 M $22.1 M 2.8x Less
Power 96.5 kW 950.3 kW 157.2 kW 6.0x Less
Cables 6,656 65,536 14,016 4.7x Less
Example: N=64 pods * k=1024 hosts/pod = 64K hosts total; 8 wavelengths
Fewer Core Switches
N pods, k-ports each
Less than k switches, N-ports each
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 10G • Demand = 10G • Throughput = 10G Pod 1 -> 3: • Capacity = 80G • Demand = 80G • Throughput = 80G
OCS EPS
Setup a Circuit
Pod 1 Pod 2 Pod 3
9 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 10G • Demand = 10G • Throughput = 10G Pod 1 -> 3: • Capacity = 80G • Demand = 80G • Throughput = 80G
OCS EPS
Traffic Patterns Change
Pod 1 Pod 2 Pod 3
10 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 10G • Demand = 10G 80G • Throughput = 10G Pod 1 -> 3: • Capacity = 80G • Demand = 80G 10G • Throughput = 10G
OCS EPS
Traffic Patterns Change
Pod 1 Pod 2 Pod 3
11 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 10G • Demand = 10G 80G • Throughput = 10G Pod 1 -> 3: • Capacity = 80G • Demand = 80G 10G • Throughput = 10G
OCS EPS
Pod 1 Pod 2 Pod 3
Break a Circuit
12 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 10G • Demand = 10G 80G • Throughput = 10G Pod 1 -> 3: • Capacity = 80G • Demand = 80G 10G • Throughput = 10G
OCS EPS
Pod 1 Pod 2 Pod 3
Setup a Circuit
13 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 80G • Demand = 80G • Throughput = 80G Pod 1 -> 3: • Capacity = 80G • Demand = 80G 10G • Throughput = 10G
OCS EPS
Pod 1 Pod 2 Pod 3
14 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
Pod 1 -> 2: • Capacity = 80G • Demand = 80G • Throughput = 80G Pod 1 -> 3: • Capacity = 10G • Demand = 10G • Throughput = 10G
OCS EPS
Pod 1 Pod 2 Pod 3
15 CSE 222A – Lecture 9: Hybrid Networks"
10G 10G 10G 80G 80G 80G
OCS EPS
Pod 1 Pod 2 Pod 3
Pod Switch Manager
Pod Switch Manager
Pod Switch Manager
Circuit Switch Manager
Topology Manager
16 CSE 222A – Lecture 9: Hybrid Networks"
Outline of Control Loop"1. Estimate traffic demand 2. Compute optimal topology for maximum throughput 3. Program the pod switches and circuit switches
17 CSE 222A – Lecture 9: Hybrid Networks"
1. Estimate Traffic Demand"Question: Will this flow use more bandwidth if we give it
more capacity?
1. Identify elephant flows (mice don’t grow) Problem: Measurements are biased by current topology
2. Pretend all hosts are connected to an ideal crossbar switch
3. Compute the max-min fair bandwidth fixpoint
Mohammad Al-Fares, Sivasankar Radhakrishnan, Barath Raghavan, Nelson Huang, and Amin Vahdat. Hedera: Dynamic Flow Scheduling for Data Center Networks. In NSDI’10.
18 CSE 222A – Lecture 9: Hybrid Networks"
2. Compute Optimal Topology"1. Formulate as instance of max-weight perfect matching
problem on bipartite graph 2. Solve with Edmonds algorithm
1
2
3
4
1
2
3
4
Source Pods Destination Pods
a) Pods do not send traffic to themselves b) Edge weights represent interpod
demand c) Algorithm is run iteratively for each
circuit switch, making use of the previous results
19 CSE 222A – Lecture 9: Hybrid Networks"
Example: Compute Optimal Topology"
20 CSE 222A – Lecture 9: Hybrid Networks"
Example: Compute Optimal Topology"
21 CSE 222A – Lecture 9: Hybrid Networks"
Example: Compute Optimal Topology"
22 CSE 222A – Lecture 9: Hybrid Networks"
Traditional Network Helios Network
100% bisection bandwidth (240 Gb/s)
23 CSE 222A – Lecture 9: Hybrid Networks"
Hardware" 24 servers
◆ HP DL380 ◆ 2 socket (E5520)
Nehalem ◆ Dual Myricom 10G NICs
7 switches ◆ One Dell 1G 48-port ◆ Three Fulcrum 10G 24-
port ◆ One Glimmerglass 64-
port optical circuit switch ◆ Two Cisco Nexus 5020
10G 52-port
24 CSE 222A – Lecture 9: Hybrid Networks"
25 CSE 222A – Lecture 9: Hybrid Networks"
Hash Collisions TCP/IP Overhead
190 Gb/s Peak 171 Gb/s Avg
Traditional Network"
26 CSE 222A – Lecture 9: Hybrid Networks"
160 Gb/s Peak 43 Gb/s Avg
Helios Network (Baseline)"
27 CSE 222A – Lecture 9: Hybrid Networks"
Port Debouncing"
0.0 0.25 0.5 0.75 1.0 1.25 1.5 1.75 2.0
Time (s)
1. Layer 1 PHY signal locked (bits are detected) 2. Switch thread wakes up and polls for PHY status
• Makes note to enable link after 2 seconds 3. Switch thread enables Layer 2 link
28 CSE 222A – Lecture 9: Hybrid Networks"
Without Debouncing"
160 Gb/s Peak 87 Gb/s Avg
29 CSE 222A – Lecture 9: Hybrid Networks"
Without EDC"Th
roug
hput
(Gb/
s)
Time (s)
160 Gb/s Peak 142 Gb/s Avg
Software Limitation
Thro
ughp
ut (G
b/s)
Time (s)
27 ms Gaps
30 CSE 222A – Lecture 9: Hybrid Networks"
Traffic Stability & Throughput"
31 CSE 222A – Lecture 9: Hybrid Networks"
For Next Class…"
Read and review FairCloud paper
Keep moving on term projects!
32 CSE 222A – Lecture 9: Hybrid Networks"
