Uni Innsbruck Informatik - Uni Innsbruck Informatik - 11

We Don‘t Need No Control We Don‘t Need No Control PlanePlane

Michael Welzl, Kashif MunirMichael Welzl, Kashif Munir

Michael WelzlMichael WelzlDPS NSG Team DPS NSG Team http://dps.uibk.ac.at/nsgInstitute of Computer ScienceInstitute of Computer ScienceUniversity of InnsbruckUniversity of Innsbruck

AGNM 2006AGNM 2006Dublin, IrelandDublin, Ireland

26-27 October 200626-27 October 2006

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 22

Proposed architectureProposed architecture

• Goal: efficient per-flow QoS without signaling to routers– ultimate dream (very long-term goal): without any router involvement!

(99% instead of 100% reliable guarantees)

• Idea: use traditional coarse-grain QoS (DiffServ) to differentiate between– long-lived bulk data transfer with advance reservation (EF) and– everything else (= SOAP etc. over TCP) (best effort)

• Allows us to assume isolated traffic; planned to drop this requirement later

• Because data transfers are long lived, apply admission control– Flows signal to resource broker (RB) when joining or leaving the network

• Mandate usage of one particular congestion control mechanism for all flows in the EF aggregate– Enables efficient resource usage because flows are elastic

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 33

Key ingredients of our QoS soupKey ingredients of our QoS soup

• Link capacities must be known, paths should be stable(capacity information should be updated upon routing change)

• Shared bottlenecks must be known

• Bottlenecks must be fairly shared by congestion control mechanism irrespective of RTT (max-main fairness required, i.e. all flows must increase their rates until they reach their limit)

• No signaling to routers = no way to enforce proper behavior there must be no cheaters– User incentive: fair behavior among cooperating nodes among

which Grid application is distributed– Unfair behavior between Grid apps 1 and 2 in same Grid neglected

(usually acceptable, as used by same Virtual Organization)

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 44

Link capacities must be knownLink capacities must be known

• Can be attained with measurements• Working on permanently active, (mostly) passive measurement

system for the Grid that detects capacity with packet pair– send two packets p1 and p2 in a row; high probability that p2 is

enqueued exactly behind p1 at bottleneck– at receiver: calculate bottleneck bandwidth via time between p1

and p2– e.g. TCP: “Delayed ACK“

receiver automatically sendspacket pairs passive TCP receivermonitoring is quite good!

– exploit longevity - minimizeerror by listening for along time

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 55

Shared bottlenecks must be knownShared bottlenecks must be known

• Simple basis: distributed traceroute tool– enhancement: traceroute terminates early upon detection of known hop

• Handle “black holes“ in traceroute– generate test messages from A, B to C - identify signature from B in A‘s traffic– method has worked in the past: “controlled flooding“ for DDoS detection

A

B

C

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 66

Congestion Control mechanism Congestion Control mechanism must be max-min fairmust be max-min fair• Was once said to be impossible without per-flow state in routers

– not true; XCP and some others– but these explicit require router support...

• Main problem: dependence on RTT– three good indications that this can be removed without router support

1. CADPC/PTP (my Ph.D. thesis)...• max-min fairness based on router feedback, but only capacity and

available bandwidth (could also be obtain by measuring)

2. Result in old paper on phase effects by Sally Floyd3. TCP Libra

• Problem: efficiency - no max-min fair “high-speed“ CC mechanism without router support– now: plan to change existing one based on knowledge from above

examples

increase/decreasefactors are f(RTT)

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 77

Per-flow QoS without signaling to Per-flow QoS without signaling to routersrouters

continuous measurements;update to BB upon path change

Synchronization ofdistributed (P2P based)database; link capacities

known to all brokers1. may I join?2. yes

Synchronization ofdistributed (P2P based)database; all flows known

to all brokers3. I quit4. ok

Traditional method: signaling

to edge routers (e.g. with COPS) at

this point!

Synchronization ofdistributed (P2P based)database; all flows known

to all brokers

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 88

Efficiency via elasticityEfficiency via elasticity

• QoS guarantees in Grid: „File will be transferred within X seconds“ enables flexible resource usage

Time (seconds)

Flow 1

Bottleneck Bandwidth (kbs)

4 kbs

End 1 End 2 End 3 End 4

Flow 2

Flow 3

Flow 4

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 99

Efficiency via elasticity /2Efficiency via elasticity /2

• Flow 1 stopped, flows 2-4 automatically increase their rates– leading to earlier termination times E2‘-E4‘; known to (calculated by) BB

Time (seconds)

Flow 4

Bottleneck Bandwidth (kbs)

4 kbs

t1 - E1 E2 E3 E4 E2’ E3’ E4’

Flow 3

Flow 2

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1010

Efficiency via elasticity /3Efficiency via elasticity /3

• Flow 5 asks BB for admission– BB knows about current rates and promised E2-E4, grants access

Time (seconds)

Bottleneck Bandwidth (kbs)

Flow 4

4 kbs

t2 – Flow 5 wants to get admission here

E2 E3 E4 E2’ E3’ E4’

Flow 3

Flow 2

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1111

Efficiency via elasticity /4Efficiency via elasticity /4

• Flow 2 terminates in time– Flows 3-5 will also terminate in time

Time (seconds)

Flow 4

Bottleneck Bandwidth (kbs)

4 kbs

t2 – Flow 5 gets admission here

E2 E3 E4 E2’ E3’ E4’

Flow 3

Flow 2

E5

Flow 5

E4” E3” E2”

Additional flow admitted and earlier termination times than promised!

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1212

Elasticity without Congestion Elasticity without Congestion Control?Control?

• Significant amount of additional signaling necessary

Time

Flow-1

Flow-2

Flow-3

Flow-4

Bottleneck Bandwidth

4 kbs

As Flow-1 stops, Flows 2-4 could increase their

rates

Without congestion control, signal “increase your rates“ to flows 2-4

required!

As flow 5 is admitted, signal “reduce your

rates“ toflows 2-4 required!

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1313

Additional considerationsAdditional considerations

• How to assign different rates to different flows?– max-min fairness: if a sender “acts“ like two, it obtains twice the

rate– consider rate consisting of slots (e.g. 1 kbit/s = 1 slot)– flows can consist of several slots– let congestion control mechanism operate on slots

• Possibility: admit new flows even in scenario below

Time (seconds)

Flow 4

Bottleneck Bandwidth (kbs)

4 kbs

t2 – Flow 5 gets admission here

E2 E3 E4 E2’ E3’ E4’

Flow 3

Flow 2

E5

Flow 5

E4” E3” E2”

Must introduce unfairness: only flow 2

can reduce rate

Disadvantage: more signaling again!

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1414

Difficult & distant future workDifficult & distant future work

• Drop requirement of traffic isolation via DiffServ– constantly obtain and update conservative estimate of available

bandwidth using packet pair (works without saturating link)– ensure that limit is never exceeded; “condition red“ otherwise!– Some open questions...

• does this require the CC mechanism to be TCP-friendly?• condition red: reduce slots, or let flows be aggressive for a short

time?

• How to handle routing changes– will be noticed, but can reduce capacity break QoS guarantee– condition red; can happen in worst case, but to be avoided at all cost– mitigation methods

• very conservative estimate of available bandwidth; leave headroom• tell senders to reroute via intermediate end systems

• Bottom line: lots of complicated issues, but possible to solve them

Uni Innsbruck Informatik - Uni Innsbruck Informatik - 1515

Thank you!Thank you!

Questions?