1

High PerformanceSwitching and RoutingTelecom Center Workshop: Sept 4, 1997.

Performance Guarantees for Internet Routers

ISL Affiliates MeetingApril 4th 2002

Nick McKeownProfessor of Electrical Engineering and Computer Science, Stanford University

nickm@stanford.eduwww.stanford.edu/~nickm

2

What a Router Looks LikeCisco GSR 12416 Juniper M160

6ft

19”

2ft

Capacity: 160Gb/sPower: 4.2kW

3ft

2.5ft

19”

Capacity: 80Gb/sPower: 2.6kW

3

Basic Architectural Components

of an IP Router

Control Plane

Datapathper-packet processing

SwitchingForwarding

Table

Routing Table

Routing Protocols

4

Generic Router Architecture

LookupIP Address

UpdateHeader

Header ProcessingData Hdr Data Hdr

~1M prefixesOff-chip DRAM

AddressTable

AddressTable

IP Address Next Hop

QueuePacket

BufferMemoryBuffer

Memory~1M packetsOff-chip DRAM

5

Generic Router Architecture

LookupIP Address

UpdateHeader

Header Processing

AddressTable

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

AddressTable

LookupIP Address

UpdateHeader

Header Processing

AddressTable

AddressTable

BufferManager

BufferMemory

BufferMemory

BufferManager

BufferMemory

BufferMemory

BufferManager

BufferMemory

BufferMemory

6

High Performance Networking Research Group

1. Adisak Mekkitikul: Crossbar scheduling algorithms that provide 100% throughput.

2. Pankaj Gupta: IP address lookup and classification algorithms.

3. Sundar Iyer: Parallel packet switches; High performance packet buffers; Distributed shared memory routers.

4. Isaac Keslassy, Shang-tse Chuang: Incorporating optics into routers.

5. Pablo Molinero Fernandez: The use of circuit switching in the Internet.

6. Nandita Dukkipati, Rui Zhang: Congestion control for short-lived flows.

7. Yashar Ganjali: Multipath routing.

Graduated

7

Performance metrics of routers

1. Capacity “maximize C, s.t. volume < 2m3 and power < 5kW”

2. Throughput Operators like to maximize usage of expensive long-

haul links. This would be trivial with work-conserving output-

queued routers

3. Controllable Delay Some users would like predictable delay. This is feasible with output-queueing plus weighted

fair queueing (WFQ).

WFQ( , ) ( , )

8

The Problem

Output queued switches are impractical

R

R

RR

DRAMDRAM

NR NR

data

9

Memory BandwidthCommercial DRAM

1. It’s hard to keep up with Moore’s Law: The bottleneck is memory speed. Memory speed is not keeping up with Moore’s Law.

0.0001

0.001

0.01

0.1

1

10

100

1000

1980 1983 1986 1989 1992 1995 1998 2001

Acc

ess

Tim

e (n

s) DRAM1.1x / 18months

Moore’s Law2x / 18 months

Router Capacity2.2x / 18months

Line Capacity2x / 7 months

10

Potted history1. [Karol et al. 1987] Throughput limited to by

head-of-line blocking for Bernoulli IID uniform traffic.

2. [Tamir 1989] Observed that with “Virtual Output Queues” (VOQs) Head-of-Line blocking is reduced and throughput goes up.

%5822

11

Potted history3. [Anderson et al. 1993] Observed analogy to maximum size

matching in a bipartite graph.

4. [M et al. 1995] (a) Maximum size match can not guarantee 100% throughput.(b) But maximum weight match can – O(N3).

5. [Mekkittikul and M 1998] A carefully picked maximum size match can give 100% throughput.

Matching

O(N2.5)

12

Potted history Speedup

5. [Chuang, Goel et al. 1997] Precise emulation of an output queued switch is possible with a speedup of two and a “stable marriage” scheduling algorithm.

6. [Prabhakar and Dai 2000] 100% throughput possible for maximal matching with a speedup of two.

13

Potted historyNewer approaches

7. [Tassiulas 1998] 100% throughput possible for simple randomized algorithm with memory.

8. [Giaccone et al. 2001] “Apsara” algorithms.

9. [Iyer and M 2000] Parallel switches can achieve 100% throughput and emulate an output queued switch.

10. [Chang et al. 2000] A 2-stage switch with a TDM scheduler can give 100% throughput.

11. [Iyer, Zhang and M 2002] Distributed shared memory switches can emulate an output queued switch.

14

Basic Switch Model

A1(n)

S(n)

N NLNN(n)

A1N(n)

A11(n)L11(n)

1 1

AN(n)

ANN(n)

AN1(n)

D1(n)

DN(n)

15

Some definitions

matrix. npermutatio a is and :where

:matrix Service 2.

".admissible" is traffic the say we If

where

:matrix Traffic 1.

SssS

nAE

ijij

jij

iij

ijijij

1,0],[

1,1

)]([:,

3. Queue occupancies:

Occupancy

L11(n) LNN(n)

16

Some definitions of throughput

( ) ,

. [ ( )] ,

[ ( )] ,

( )( )lim lim

1. Work conservation

2. "100% throughput"

3.

4

5.

6.

ij

ij

ij

ijijij

n n

L n C n

E L n C

E L n

A nD n

n n

When traffic is

admissible

17

Scheduling algorithms to achieve 100% throughput

1. When traffic is uniform (Many algorithms…)2. When traffic is non-uniform, but traffic matrix is known

• Technique: Birkhoff-von Neumann decomposition. [Chang ‘99]

3. When matrix is not known.• Technique: Lyapunov function. [M et al. ‘96]

4. When algorithm is pipelined, or information is incomplete.• Technique: Lyapunov function. [Keslassy & M ’01]

5. When algorithm does not complete.• Technique: Randomized algorithm. [Tassiulas ’00]

6. When there is speedup.• Technique: Fluid model. [Dai & Prabhakar ’00]

7. When there is no algorithm.• Technique: 2-stage load-balancing switch. [Chang ’01]• Technique: Parallel Packet Switch. [Iyer & M ’01]

18

When the traffic matrix is not known

( 1) ( ) ( ) 0

( ) ( ) ( ) ( ) | ( ) 0.

1. We would like to show that:

i.e.

2. I nstead we fi nd that if the scheduling algorithm is a maximum

weight bipartite

ij

ij ijij ij

E V L n V L n | L n ,ij

E V L n S n A n V L n L n

( 1) ( 1) ( ) ( ) ( ) ( ) .

( ) ( ) ( ),

( )

matching, then

where: is our Lyapunov f unction.

3. Hence, if is large enough, there is an expected

single-step downward drif t in

T T

T

E L n L n L n L n | L n c L n

V L n L n L n

L n

[ ( )] .

occupancy, and so

E L n

19

Throughput resultsTheory:

Practice:

InputQueueing

(IQ)

InputQueueing

(IQ)

InputQueueing

(IQ)

InputQueueing

(IQ)

58% [Karol, 1987]

IQ + VOQ,Maximum weight matching

IQ + VOQ,Maximum weight matching

IQ + VOQ,Sub-maximal size matching

e.g. PIM, iSLIP.

IQ + VOQ,Sub-maximal size matching

e.g. PIM, iSLIP.

100% [M et al., 1996]

Different weight functions,incomplete information, pipelining.

Different weight functions,incomplete information, pipelining.

Randomized algorithmsRandomized algorithms

100% [Tassiulas, 1998]

100% [Various]

Various heuristics, distributed algorithms,

and amounts of speedup

Various heuristics, distributed algorithms,

and amounts of speedup

IQ + VOQ,Maximal size matching,

Speedup of two.

IQ + VOQ,Maximal size matching,

Speedup of two.

100% [Dai & Prabhakar, 2000]