Opportunistic Traffic Scheduling Over Multiple Network Path

Coskun Cetinkaya and Edward Knightly

Edward Knightly

Multi-Path Routing

Establishes and simultaneously uses multiple parallel paths– Key advantage is efficiency

Routing protocol assigns weights to paths– OSPF, QoS routing, traffic engineering

Edward Knightly

Existing Splitting Techniques

Per packet round robin forwarding– Simplest and most frequently used– Degrades TCP throughput due to re-ordering

Per flow hashing – Fine splitting granularity and no TCP re-ordering– Per-TCP-flow lookup limits implementation feasibility

Destination prefix based forwarding– Coarse-granularity splitting and no TCP re-ordering– Unpredictable load splitting that may not match desired

weights

All ignore path quality in splitting decision

Edward Knightly

Our Thesis

Observe– Routing weights change slowly (from traffic engineering) – Quality of paths changes continuously

Opportunistic Multipath Scheduling– Exploits short-term capacity variations on different paths via

scheduling packets to opportunistically favor low-delay paths– Obey weights at long time scales to ensure “global”

objectives

Hypothesis– Improve throughput/delay, no per-flow lookup, satisfy

weights– TCP throughput improvements due to RTT reduction will

overwhelm re-ordering effects

Edward Knightly

System Model

Design: scheduling/traffic splitting policy

Objective: minimize mean delay of multipath traffic– Decrease RTT and loss rate increase TCP throughput

Subject to: mean traffic on path i = i (path weight)

Multipath traffic

…

Cross traffic

Splitter

Edward Knightly

Xk = size of packet k I(sk,i) = 1 of packet k is scheduled on path i, 0

otherwise For equal capacity paths minimizing delay is

equivalent to minimizing the expected queue length

Mathematical Formulation

)),((

}{..

}{min

1ik

kki

ki

ki

ik

k

CisIXAQQ

isPts

ksQE

Edward Knightly

Optimal Scheduler

Assumptions:– Cross-traffic and multi-path traffic are stationary

processes queue length is stationary – Multi-path traffic does not change path conditions

Using a wireless scheduling analogy [LCS02], we can show that the optimal scheduler is threshold based:

iiijjji

i

ii

vQvQPtsv

vQQS

}min{..

)min(arg)(***

**

Contrast to “join the shortest queue” policy which ignores weights

Edward Knightly

Evaluate the performance under self-similar cross traffic

Queue size distribution is Weibull:

Expected queue size (and delay):

Round Robin Optimal Scheduler

Performance of the Optimal Policy

iix

i exQP }{

)1(2 ii H

iH

i

ii

iiii H

Hm

Hma

2

2

)1)(1(

)1(2

1

i

i

i

iRRQE

/1

1 )1(}{

*

* )(}{

v

vx

opt dxeQEi

ii

)max( **ivv

Edward Knightly

On-Line Computation of v*

In practice, we do not know the queue length or its distribution

Threshold update:– stochastic approximation technique [KC78,LCS02]

Scheduling decision:– Qi

k estimated via probes

)),((11 isIvv ki

ki

ki k

}{minarg ki

ki

ik vQs

Edward Knightly

Evaluation Scenario

Two paths with capacity 10 Mb/sec Cross-traffic: self-similar with mean rate m[0.3,

0.9], variance coefficient a[0.5,4], and Hurst Parameter H[0.5,0.9]

Multi-path traffic is constant-rate or TCP Gain defined as

}{

}{1

RR

OMS

QE

QEG

…

Edward Knightly

Model: gain depends only on H and # paths and is 50%

Higher N more path diversity higher gain

Large H long-time scale path correlation higher gain

Homogeneous Paths: Model

)1(2/1

11

HNG

Edward Knightly

Simulated gains higher than predicted by model – Model serves as lower bound– Queue distribution is asymptotic lower bound, tighter for larger queues

Delay increases with increasing mean (m) and variance coefficient (a) Gain (relative) is highest under higher H, lower m, lower a

Homogeneous Paths: Simulation

Edward Knightly

Gain increases with path diversity (increasing ratio of variance coefficient) – OMS exploits different path properties subject to

weights

Heterogeneous Paths: Impact of Variance Coefficient Ratio

H=0.6

m=0.7

Edward Knightly

So far, assumed path information is immediately available at scheduler/splitter

RTT-scale delay to obtain buffer state (via probes or ECN) Gain decreases as information delay increases High gain for measured values of traffic (0.7 < H < 0.85)

and delay (1 < RTT < 100 msec)

Effect of Information Delay

m=0.9

a=0.5

Edward Knightly

When can OMS do worse than RR? Three combined factors:– iid traffic having no long-time-scale bursts– High information delay– High ratio of multi-path traffic to cross-traffic (scheduled

traffic itself determines conditions)

Limits of OMS

Edward Knightly

TCP Multi-Path Traffic

With RR, multipath traffic achieves only 20% to 38% of fair share– High cost of mis-ordering and delay

TCP/OMS significantly outperforms TCP/RR– TCP/OMS requires an aggregate level of only 10 cross-traffic

flows to achieve maximum performance– OMS impact overwhelms effect of TCP variants

10 msec probing interval

32 kb/s probing overhead

(0.32% of capacity)

Edward Knightly

Probing Interval and TCP Traffic

Base case probing interval: 10 msec interval and 32 kb/sec Faster 1 msec probing yields higher-than-fair share for

multi-path flows Slower probing (e.g., 3.2 kb/sec) reduces performance

Edward Knightly

Summary

Multipath routing promises increased efficiency and performance

Today’s traffic splitting ignores path dynamics and– inhibits TCP throughput via reordering,– requires expensive per-TCP flow lookups, or– cannot achieve weights via prefix splitting

Opportunistic Multipath Scheduling– Improves throughput/delay via a measurement

based opportunistic policy that satisfies routing weights

– Gains overwhelm occasional misordering

http://www.ece.rice.edu/networks