Network Elements based on Partial State A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University [email protected] reddy

Network Elements based on Partial State

A. L. Narasimha Reddy

Dept. of Electrical Engineering

Texas A & M University

[email protected]

http://ee.tamu.edu/~reddy/

Narasimha Reddy


2

Acknowledgements

• Deying Tong (Cisco)

• Sai Gopalakrishnan (Cisco)

• Smitha (Intel)

• Phani Achanta (Graduating in Aug. 2002)

Narasimha Reddy


3

Introduction

• Proposals for new network architectures– Full State (IntServ)

• Difficult to scale per-flow state with # of flows

– No State (DiffServ)• Flow isolation difficult

Narasimha Reddy


4

Introduction

• What if we can build network elements with some fixed amount of state?– State is not enough for all the flows– What kind of services can we provide?

• Hypothesis: Only few flows need state, most flows can be aggregated.

Narasimha Reddy


5

Motivation

• Typical Internet traffic consists of – Many short-lived flows (“mice”)

• pump below 20 packets (approximately 20KB)

– Few large flows (“elephants”)

• Current resource management techniques do not distinguish the flows

• Dropping packets from short-lived flows may do little to ease congestion– Also, mice flows are latency sensitive

Narasimha Reddy


6

Motivation (contd..)

• Congestion management “should” rely on controlling high bandwidth flows– Offer more control on traffic– Likely to be consuming disproportionate bandwidth– Likely to be “robust” (ftp for e.g..)

• May need mechanisms to control unresponsive applications– To improve fairness and to prevent congestion collapse

Narasimha Reddy


7

Flow Classification

• Long-lived flows• TCP flows (FTP Applications)

• UDP flows (Video Applications)

• Short-lived flows• Telnet, HTTP transfers

• Responsive vs. Nonresponsive flows– ftp vs. some video transfers

Narasimha Reddy


8

Basis for Partial State

• A Small fraction of flows contribute large fraction of bytes.

• If state can be allocated to these flows, resource management can be done efficiently without requiring full state.

Narasimha Reddy


9


Narasimha Reddy


10


Narasimha Reddy


11


Narasimha Reddy


12

Partial State Approach

• Maintain Fixed amount of Partial State– State is not dependent on number of flows– State depends on engineering concerns

• Manage the state information to retain history of high-BW flows -- How?

• Adopt appropriate resource management based on the goals

Narasimha Reddy


13

Partial State Approach

• Similar to how caches are employed in computer memory systems– Exploit locality

• Employ an engineering solution in an architecture-transparent fashion

Narasimha Reddy


14

State Management

• Sampling is employed as a basic tool– High-BW flows more likely to be selected

• State organized as a Cache– Caches allow quick identification if flow is

allocated state

• State Allocation can be– Policy Driven– Traffic Driven

Narasimha Reddy


15

Policy Driven State Management

• An ISP could decide to monitor flows above 1Mbps– Will need state >= link capacity/1 Mbps

• Could monitor flows consuming more than 1% of link capacity– For security reasons– At most 100 flows with 1% BW consumption

Narasimha Reddy


16

Traffic Driven State Management

• Monitor top 100 flows at any time– Don’t know the identity of these flows– Don’t know how much BW these may consume

• Employ LRU Cache management– Flows have to arrive at cache frequently to stay

in cache– Maintains High-BW flows in a self-organizing

way

Narasimha Reddy


17

Traffic Driven State Management (contd…)

• Flows probabilistically admitted into cache, ‘p’.– Reduces the chance of short-term flows disturbing the

cache state.

• Keep count of packet arrivals of cached flows– Declare a “high-BW” flow if count > Threshold

Narasimha Reddy


18

The Algorithm

New Packet

In Cache?

Cache size < ‘S’

Admit the flow into the cache with a probability ‘p’, count = 1

Update position and count

Make a new entry, count=1

Yes

No

Yes

No

Narasimha Reddy


19

Why an LRU Cache?• High bandwidth flows arrive often

– Stay in the cache for longer periods

• Smooth flows stay in the cache longer compared to bursty flows– UDP flows (smooth)– TCP flows (bursty)

• Responsive flows reduce rate and get replaced – Nonresponsive flows remain in cache

Narasimha Reddy


20

UDP Cache Occupancy

0100200300400500600

0.1

0.4

0.6 1

1.25 2.

12.

7 33.

5 4

Rate in Mb

Tim

e in

se

co

nd

s

Narasimha Reddy


21

TCP Cache Occupancy

0.70.720.740.760.78

0.80.820.840.86

1 3 5 7 9 11 13 15 17 19

Flow Number

Tim

e in

se

co

nd

s

Narasimha Reddy


22

Resource Management

• Cached flows can be treated individually

• Noncached flows treated in an aggregate manner

• With larger state, finer control on traffic

Narasimha Reddy


23

Resource Management

• Preferential Dropping (RED based)– Drop cached flows more often– Use Packet count as a scaling function

• Fair queuing – Cached flows, noncached flows in separate

queues, employ WFQ – Possible to protect noncached flows from

cached flows

Narasimha Reddy


24

Resource Management

Narasimha Reddy


25

Preferential Dropping

drop prob

Queue lengthdrop prob for high bandwidth flows

minth maxth

maxp

1

drop prob for other flows

Narasimha Reddy


26

Preferential Dropping (contd..)

• As congestion builds up, above min_th, – if (flow->count >=‘threshold’)

• Pdrop = pred * flow->count / ‘threshold’

– else• Pdrop = pred

• High-BW nonresponsive flows get higher drops

• Low-BW and responsive flows see lower drops

Narasimha Reddy


27

Two Studies

• LRU-RED: Simulation based study– Provide lower drop rates for responsive and

short-term flows– Approximately fair BW distribution

• LRU-FQ: Linux-based partial state router prototype– Contain DOS attacks– Provide shorter delays for short-term flows

Narasimha Reddy


28

LRU-RED Simulations

R1 R2

40Mb

20Mb 20Mb

Narasimha Reddy


29

Topology 2

R1 R2 R3

40Mb 30Mb

20Mb20Mb

Narasimha Reddy


30

LRU-RED Results

0

10

20

30

40

50

50 67 75 80

% UDP flows

% T

CP

Th

rou

gh

pu

t

Droptail

LQD

CHOKe

LRU

RED

Narasimha Reddy


31

LRU-RED Results

0

5

10

15

20

25

30

50 67 75 80

% UDP Flows

% T

CP

Dro

pra

te

Droptail

LQD

CHOKe

LRU

RED

Narasimha Reddy


32

LRU-RED Results

0

10

20

30

40

50 67 75 80

% UDP Flows

% U

DP

Dro

pra

te

Droptail

LQD

CHOKe

LRU

RED

Narasimha Reddy


33

LRU-RED Results

05

101520253035

50 67 75 80

% UDP Flows

% H

TT

P D

rop

rate

Droptail

LQD

CHOKe

LRU

RED

Narasimha Reddy


34

Varying Load

0

10

20

30

40

50

25 50 100 150

% load on bottleneck link

% T

CP

Dro

pra

te

CHOKe

RED

DropTail

LQD

LRU

Narasimha Reddy


35

RTT Bias -TCP flows

0

1

2

3

4

5

6

7

8

8 8 44 84 84 124

204

204

404

RTT in ms

% D

rop

rate

CHOKe

RED

DropTail

LQD

LRU

Narasimha Reddy


36

Summary of LRU-RED

• LRU cache is effective in identifying high bandwidth, nonresponsive flows

• Combined the above with RED to propose a novel active queue management scheme

• Simulation results show the effectiveness of the scheme

• Sampling can further reduce the per-packet cost

Narasimha Reddy


37

LRU-FQ Resource Management

Narasimha Reddy


38

LRU-FQ Flow Chart – Enque Packet Arrival

Is Flow in Cache?

Yes

No Does Cache Have

space?

Yes

Admit flow with Probability ‘p’

No

Is Flow Admitted?

Record flow detailsInitialize ‘count’ to 0

Yes

Increment ‘count’Move flow to top of cache No

Is‘count’ >= ‘threshold’

No

Yes

Enqueue in Non-responsiveQueue

Enqueue in ResponsiveQueue

Narasimha Reddy


39

LRU-FQ – Dequeue event Dequeue event results in selection of a packet from

either queues based on the Fair Queue algorithm used.

The weight assigned to the individual queues determine the proportion of bandwidth they are assigned.

Implementation Issueson

Linux

Narasimha Reddy


41

Linux IP Packet Forwarding

Packet Arrival Check & StorePacket

Enqueue pkt

Request SchedulerTo invoke bottom half

Device Prepares

packet Packet Departure

Error checkingVerify

Destination

Route to destinationUpdate Packet

Packet Enqueued

Scheduler invokesBottom half Scheduler runs

Device driver

Local packetDeliver to upper layers UPPER LAYERS

IP LAYER

LINK LAYER

Design space

Narasimha Reddy


42

Linux Kernel traffic control

• Filters are used to distinguish between different classes of flows.

• Each class of flows can be further categorized into sub-classes using filters.

• Queuing disciplines control how the packets are enqueued and dequeued

Narasimha Reddy


43

LRU-FQ Implementation

• LRU component of the scheme is implemented as a filter. – All parameters: threshold, probability and

cache size are passed as parameters to the filter

• Fair Queuing employed as a queuing discipline. – Scheduling based on queue’s weight.– Start-time Fair Queuing

LRU-FQ - Results

Narasimha Reddy


45

Timing ResultsTiming Analysis

95.6

95.62

95.64

95.66

95.68

95.7

95.72

0 5 10 15 20 25 30 35 40 45

Time Delay (usec)

Rec

eive

d Tp

ut (M

bps)

Normal Routing

Diffserv Routing

Start Time FQ & LRU

Narasimha Reddy


46

Control of Non-responsive Proportion

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

9 8 7 6 5 4 3 2 1

LRU Weight (x/10)

TC

P T

hro

ug

hp

ut

Fra

ctio

n (

20 T

CP

Flo

ws)

Ideal

UDP Flows = 2

UDP Flows = 3

UDP Flows = 4

UDP Flows = 5

Normal Router

Long-Term flow differentiation

Probability = 1/25 Cache size= 11 threshold= 125

Normal TCP fraction = 0.07

Narasimha Reddy


47

Long-term flow differentiationUDP Rate Based Experiments

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

0.95

1 2 3 4

LRU Weight Proportion (x/10)

TC

P T

hro

ug

hp

ut

frac

tio

n

Ideal

UDP Rate = 100%

UDP Rate = 80%

UDP Rate = 60%

UDP Rate = 40%

Probability = 1/25 Cache size= 11 threshold= 125

Narasimha Reddy


48

Histogram of Web File Distribution

0

100

200

300

400

500

600

File Size

Fre

qu

ency

Histogram of Web File Distribution 350 500 140 9 1

500 5k 50k 500k 5m

Protecting Web Mice

Narasimha Reddy


49

Protecting Web mice

1:1LRU : Normal Queue

11LRU Cache Size

125Threshold

1/50Probability

20Web Clients

2 – 4LongTerm UDP Flows

20Long Term TCP Flows

Experimental Setup

Narasimha Reddy


50

Protecting Web MiceBandwidth Results

0.06566.268100789.2654

0.06786.478109889.0003

0.06586.28596089.1872

TCP Fraction

TCP Tput

# Web Requests

UDP Tput

UDP Flows

0.495346.677296547.5474

0.510647.863295645.8713

0.497346.884313147.3922

TCP Fraction

TCP Tput

# Web Requests

UDP Tput

UDP Flows

Normal Router

LRU-FQ Router

Narasimha Reddy


51

Protecting Web MiceTiming Results

UDP AvgRsp DevRsp MinRsp MaxRsp AvgConn DevConn MinConn MaxConn2 1.117 2.62 0.01 45.062 0.704 1.543 0.0027 21.0263 1.111 2.624 0.004 45.067 0.703 1.657 0.0007 21.034 1.193 2.484 0.029 43.13 0.839 1.798 0.0033 21.031

Normal Router

LRU-FQ Router

UDP AvgRsp DevRsp MinRsp MaxRsp AvgConn DevConn MinConn MaxConn2 3.558 5.919 0.03 93.125 1.842 2.913 0.0136 45.0133 3.178 5.4 0.03 90.067 1.857 2.761 0.0136 21.0154 3.472 6.369 0.029 93.024 1.821 3.149 0.0132 45.007

Narasimha Reddy


52

Summary of LRU-FQ

• Provides a good control of DOS attacks with limited number of flows

• Provides better delays for short-term flows

• Allows DDOS attack detection through wavelet signatures on miss traffic

• Automatically identifies resource hogs

• Partial state packet handling cost -not an issue at 100Mbps.

Narasimha Reddy


53

References• SACRED (Tong, Reddy ‘99) IWQOS 1999.• SACRIO (Gopalakrishnan, Reddy ‘01): Partial state

in Diff-serv Networks, NOSSDAV 2001.• LRU-RED (Smitha, Reddy ‘01): Globecom 2001• LRU-FQ (Achanta, Reddy ‘02): In preparation• WADeS (Ramanathan, Reddy ‘02): DDOS detection

• Please visit the following URLs for references – http://ee.tamu.edu/~reddy/papers/– http://www.cs.tamu.edu/people/phani/research/index.htm

Narasimha Reddy


54

Applications of Partial State• More intelligent control of network traffic

• Accounting and measurement of high bandwidth flows

• Denial of Service (DOS) attack prevention

• DDOS attack detection– Wavelet signatures of miss traffic give

indications of attacks

• Tracing of high bandwidth flows

• QOS routing

Narasimha Reddy


55

Related Work• Route caching in LANs

• RED-PD[Mahajan,Floyd ’01]: RED drop history used to guide decisions

• Approximate Fairness through Differential Dropping [Pan, Breslau, Prabhakar, Shenker ’01]: Similar to RED-PD

• Traffic Measurement [Estan, Verghese ‘01]: Employ a lot more state, limited to measurement.

Narasimha Reddy


56

Future Work

• Analyze impact of cache size and traffic behavior on QOS

• Provide mechanisms for “protection” of cached flows

• Implement on network processors to adapt to changing traffic conditions

• QOS routing of cached flows

Narasimha Reddy


57

Thank you !!

For more information, send e-mail to

A.L. Narasimha Reddy [email protected]

Documents

Network Elements based on Partial State A. L. Narasimha Reddy Dept. of Electrical Engineering Texas A & M University [email protected] reddy