Ion Stoica,Scott Shenker, and Hui Zhang SIGCOMM’99, Vancouver, August 1999

1

Core-Stateless Fair Queueing:Core-Stateless Fair Queueing: Achieving Approximately Fair Achieving Approximately Fair

Bandwidth Allocations in High Speed Bandwidth Allocations in High Speed NetworksNetworks

Ion Stoica,Scott Shenker, and Hui Zhang

SIGCOMM’99, Vancouver, August 1999

Presented by Bob Kinicki

Advanced Computer Networks – CSFQ

Paper2

OutlineOutline Introduction Core-Stateless Fair Queueing (CSFQ)

– Fluid Model Algorithm– Packet Algorithm– Flow Arrival Rate– Link Fair Share Rate Estimation

NS Simulations Conclusions


Paper3

IntroductionIntroduction This paper brings forward the concept of “fair”

allocation. The claim is that fair allocation inherently

requires routers to maintain statestate and perform operations on a per flow basis.

The authors present an architecture and a set of algorithms that is “approximately” fair while using FIFO queueing at internal routers.


Paper4

An “Island” of RoutersAn “Island” of Routers

EdgeRouter

CoreRouter

SourceDestination

Destination


Paper5

Core-Stateless Fair Core-Stateless Fair QueueingQueueing

Ingress edge routers compute per-flow rate estimates and insert these estimates as labelslabels into each packet header.

Only edge routers maintain per flow state.Labels are updated at each router based

only on aggregate information.FIFO queueing with probabilistic dropping

of packets on input is employed at the core routers.


Paper6

Edge – Core Router Edge – Core Router ArchitectureArchitecture


Paper7

Fluid Model AlgorithmFluid Model Algorithm Assume the bottleneck router has an output

link with capacity CC. Assume each flow’s arrival rate, rrii (t)(t) , is

known precisely. The main idea is that max-min fair

bandwidth allocations are characterized such that all flows that are bottlenecked by a router have the same output rate.

This rate is called the fair share rate of the link. Let α(t)α(t) be the fair share rate at time t.


Paper8

Fluid Model AlgorithmFluid Model Algorithm

If max-min bandwidth allocations are achieved, each flow receives service at a rate given by

min min (r(ri (t), i (t), α(α(tt))))

Let A(t)A(t) denote the total arrival rate:

If A(t) > CA(t) > C then the fair share is the unique solution to


Paper9

Fluid Model AlgorithmFluid Model AlgorithmThus, the probabilistic fluid forwarding

algorithm that achieves fair bandwidth allocation is:

Each incoming bit of flowEach incoming bit of flow ii is is dropped with probabilitydropped with probability

max (0,1-max (0,1-α(t)/rα(t)/rii(t)(t)) (2)) (2)

These dropping probabilities yield fair share arrival rates at the next hop.


Paper10

Packet AlgorithmPacket Algorithm

Moving from a bit-level, buffer-less fluid model to a packet-based, buffer model leaves two challenges:– Estimate the flow arrival rates rrii(t)(t)

– Estimate the fair share α(t)α(t)This is possible because the rate

estimator incorporates the packet size.


Paper11

Flow Arrival RateFlow Arrival RateAt each edge router, use exponential averaging to estimate the

rate of a flow. For flow ii, let

lliikk be the length of the kkthth packet.

ttiikk be the arrival time of the kkthth packet.

Then the estimated rate of flow i, ri, rii is updated every time a new packet is received:

rrii

new new = (1-e= (1-e-T/K-T/K)*L / T + e)*L / T + e-T/K-T/K* r* riioldold (3)

where

TT == TTiikk = t= tii

kk – – ttiik-1k-1

L = lL = liikk and KK is a constant


Paper12

Link Fair Rate EstimationLink Fair Rate Estimation

If we denote the estimate of the fair share by and the acceptance rate by , we have

Note – if we know rrii(t)(t) then can be determined by finding the unique solution to F(x) = CF(x) = C.

However, this requires per-flow state !


Paper13

Heuristic AlgorithmHeuristic Algorithm

The heuristic algorithm needs three aggregate state variables:

, , where is the estimated aggregate arrival rate.

When a packet arrives, the router computes:

(5)

and similarly computes .


Paper14

CSFQ Algorithm CSFQ Algorithm

When a packet arrives, is updated using exponential averaging (equation 5).

If the packet is dropped, remains the same.

If the packet is not dropped, is updated using exponential averaging.

At the end of an epoch (defined by KKcc ), if the link is congested, update :


Paper15

CSFQ Algorithm (cont.)CSFQ Algorithm (cont.)

If the link is not congested, is set to the largest rate of any active flow.

feeds into the calculation of drop probability, pp, for the next arriving packet as αα in

p = max (0 , 1 – p = max (0 , 1 – α α / label)/ label)


Paper16

CSFQ Pseudo CodeCSFQ Pseudo Code

Figure 3Figure 3


Paper17

CSFQ CSFQ PseudoPseudo Code Code


Paper18

Label RewritingLabel Rewriting

At core routers, outgoing rate is merely the minimum between the incoming rate and the fair rate, αα .

Hence, the packet label L can be rewritten by

L L newnew == minmin (L (L oldold , , α )α )


Paper19

SimulationsSimulations

A major effort of the paper is to compare CSFQ to four algorithms via ns-2 simulations.

FIFOREDFRED (Flow Random Early Drop)DRR (Deficit Round Robin)


Paper20

FRED (Flow Random Early FRED (Flow Random Early Drop)Drop)

Maintains per flow state in router. FRED preferentially drops a packet that has

either:– Had many packets dropped in the past– A queue larger than the average queue size

Main goal : Fairness FRED-2 guarantees to each flow a minimum

number of buffers.


Paper21

DRR (Deficit Round Robin)DRR (Deficit Round Robin)Represents an efficient implementation of

WFQ.A sophisticated per-flow queueing

algorithm.Scheme assumes that when router buffer is

full the packet from the longest queue is dropped.

Can be viewed as “best case” algorithm with respect to fairness.


Paper22

ns-2 Simulation Detailsns-2 Simulation DetailsUse TCP, UDP, RLM and On-Off traffic

sources in separate simulations.Bottleneck link: 10 Mbps, 1ms latency,

64KB bufferRED, FRED (min, max) thresholds:

(16KB, 32KB)KK and KKcc = 100 ms. = 200ms.KKαα


Paper23

A Single Congested LinkA Single Congested Link

First Experiment : 32 UDP flows– Each UDP flow is indexed from 0 to 31 with

flow 0 sending at 0.3125 Mbps and each of the i subsequent flows sending (i+ 1) times its fair share of 0.3125 Mbps.

Second Experiment : 1 UDP flow, 31 TCP flows– UDP flow sends at 10 Mbps– 31 TCP flows share a single 10 Mbps link.


Paper24

Figure 3a: 32 UDP Flows Figure 3a: 32 UDP Flows

Only CSFQ andDRR can containUDP flows!!


Paper25

Figure 3b : One UDP Flow, 31 TCP Figure 3b : One UDP Flow, 31 TCP FlowsFlows

Only CSFQ andDRR can containFlow 0 – the onlyUDP flow!


Paper26

A Single Congested LinkA Single Congested Link

Third Experiment Set : 31 simulations– Each simulation has a different N,

N = 2 … 32.– One TCP and N-1 UDP flows with each

UDP flow sending at twice fair share rate of 10/N Mbps.


Paper27

Figure 4 : One TCP Flow, N-1 UDP Figure 4 : One TCP Flow, N-1 UDP FlowsFlows

Normalized fair sharethroughput for one TCP source

DRR good for lessthan 22 flows.

CSFQ better thanDRR when a largenumber of flows.

CSFQ beats FRED.


Paper28

Multiple Congested LinksMultiple Congested Links

Router RouterRouter Router

UDPSinks

TCP/UDP-0Sink

TCP/UDP-0Source

UDPSources

1

1-10 K1-K10

10 11 20 K10K1


Paper29

Multiple Congested LinksMultiple Congested Links

First experiment : UDP flow 0 sends at its fair share rate, 0.909 Mbps while the other ten “crossing” UDP flows send at 2 Mbps.

Second experiment: Replace the UDP flow with one TCP flow and leave the ten crossing UDP flows.


Paper30

Figure 6a : UDP sourceFigure 6a : UDP source

Fraction of UDP-0 traffic forwardedversus the number of congested links


Paper31

Figure 6b : TCP SourceFigure 6b : TCP Source

Fraction of TCP-0 traffic forwardedversus the number of congested links


Paper32

Receiver-driven Layered Receiver-driven Layered MulticastMulticast

RLM is an adaptive scheme in which the source sends the information encoded in a number of layers.

Each layer represents a different multicast group.

Receivers join and leave multicast groups based on packet drop rates experienced.


Paper33

Receiver-driven Layered Receiver-driven Layered MulticastMulticast

Simulation of three RLM flows and one TCP flow with a 4 Mbps link.

Fair share for each is 1 Mbps.Since router buffer set to 64 KB, KK, KKcc,

and are set to 250 ms.Each RLM layer II sends 2i+4 Kbps with

each receiver subscribing to the first five layers.

KKαα


Paper34

Figure 7c : FREDFigure 7c : FRED


Paper35

Figure 7d : REDFigure 7d : RED


Paper36

Figure 7e : FIFOFigure 7e : FIFO


Paper37

Figure 7a : DRRFigure 7a : DRR


Paper38

Figure 7b : CSFQFigure 7b : CSFQ

KK,== Kc Kc = = Kα Kα = = 250 ms.250 ms.


Paper39

CSFQ [journal article]CSFQ [journal article]


Paper40

CSFQ [journal article]CSFQ [journal article]


Paper41

On-Off Flow ModelOn-Off Flow Model

One approach to modeling interactive, Web traffic :: OFF represents “think time”

ON and OFF drawn from exponential distribution with means of 100 ms and 1900 ms respectively.

During ON period source sends at 10 Mbps.


Paper42

Table 1 : One On-Off Flow, 19 TCP Table 1 : One On-Off Flow, 19 TCP FlowsFlows

Algorithm Delivered Dropped

DRR 601 6157

CSFQ 1680 5078

FRED 1714 5044

RED 5322 1436

FIFO 5452 1306


Paper43

Web TrafficWeb Traffic

A second approach to modeling Web traffic that uses Pareto Distribution to model the length of a TCP connection.

In this simulation 60 TCP flows whose interarrivals are exponentially distributed with mean 0.05 ms and Pareto distribution that yields a mean connection length of 20,1 KB packets.


Paper44

Table 2 : 60 Short TCP Flows, One UDP Table 2 : 60 Short TCP Flows, One UDP FlowFlow

Algorithm Mean Transfer Time for TCP

Standard Deviation

DRR 25 99

CSFQ 62 142

FRED 40 174

RED 592 1274

FIFO 840 1695


Paper45

Table 3 :Table 3 : 19 TCP Flows, One UDP 19 TCP Flows, One UDP Flow with propagation delay of 100 Flow with propagation delay of 100

ms.ms.Algorithm Mean

ThroughputStandard Deviation

DRR 6080 64

CSFQ 5761 220

FRED 4974 190

RED 628 80

FIFO 378 69


Paper46

Packet RelabelingPacket Relabeling

RouterFlow 2

Router

Sink

Flow 1

Sources

Flow 3

Link 210 Mbps

Link 110 Mbps

10 Mbps

10 Mbps

10 Mbps


Paper47

Table 4 : UDP and TCP Table 4 : UDP and TCP with Packet Relabeling with Packet Relabeling

Traffic Flow 1 Flow 2 Flow 3

UDP 3.36 3.32 3.28

TCP 3.43 3.13 3.43


Paper48

Unfriendly FlowsUnfriendly FlowsUsing TCP congestion control requires

cooperation from other flows.Three types cooperation violators:

– Unresponsive flows (e.g., Real Audio)– Not TCP-friendly flows– Flows that lie to cheat.

This paper deals with unfriendly flows!!


Paper49

ConclusionsConclusionsThis paper presents Core Stateless Fair

Queueing and offers many simulations to show how CSFQ provides better fairness than RED or FIFO.

They mention issue of “large latencies”. This is the robust versus fragile flow issue from FRED paper.

CSFQ ‘clobbers’ UDP flows!


Paper50

SignificanceSignificance

First paper to use hints from the edge of the subnet.

Deals with UDP. Many AQM algorithms ignore UDP.

Makes a reasonable attempt to look at a variety of traffic types.


Paper51

Problems/ WeaknessesProblems/ Weaknesses

“Epoch” is related to three constants in a way that can produce different results.

How does one set K constants for a variety of situations.

No discussion of algorithm “stability”


Paper52

AcknowledgmentsAcknowledgments

Figures extracted from presentation by Nagaraj Shirali and Choong-Soo Lee in Spring 2002 and modified for annotations.

Documents

Ion Stoica,Scott Shenker, and Hui Zhang SIGCOMM’99, Vancouver, August 1999