Potential Applications ofShared Bottleneck Detection (SBD)

Michael Welzl

NNUW-1Simula, Oslo18. 09. 2013

2

Context

• SBD has been around for a while– Papers dating back more than a decade– Approach: correlate per-path measurements (e.g. OWD, ..)– Never been a “hot topic”; said to be too unreliable, e.g. in IETF– Note: applications have different reliability requirements

• Major problem: real-life validation– Most papers are poor in this aspect

• E.g. real-life test without really validating– Hard to do in most testbeds

• We tried and failed in PlanetLab: no shared bottlenecks

• NorNet could make a real difference here

3

A gallery of potential applications of SBD

4

Multi-path transfer

• Better congestion control for MPTCP, CMT-SCTP– cf.: Sofiane Hassayoun, Janardhan Iyengar and David Ros,

“Dynamic Window Coupling for Multipath Congestion Control”. In Proceedings of IEEE ICNP 2011, Vancouver, October 2011.

• Basic issue:On N paths, act like 1 flow or like N?– N might be unfair if the flows share the same bottleneck– 1 might be too conservative if they don’t

5

Multi-path overlay:EC-GIN LFT scenario

Multipath file transfer

(AB + ACB) beneficial

Multipath file transfer not beneficial

due to shared bottleneck

6

File transfer delay prediction

• Consider telling A and B that it takes 5 minutes to send a 10 GB file to C– But then they send at the same time...

• This is needed for scheduling in distributed computing

A

B

C

7

Distributed computing: abstract-concrete WF mapping Tasks = {T1, T2, T3, T4}

Resources = {R1, R2, R3, R4}Data transfers = {D1, D2, D3, D4}

Unnoticed by traditionalscheduling algorithms!

8

Incorporating the net

1. Theory (analysis, “abstract” simulations)– Network-aware variant of Heterogeneous Earliest Finish Time

(HEFT) scheduling algorithm developed in Ph.D. thesisMurtaza Yousaf, "Accurate Shared Bottleneck Detection and File Transfer Delay Prediction for Grid Scheduling", December 2008, University of Innsbruck, Austria.

2. Next step: more realistic simulations, using ns-2– Simulator available, developed in master thesis– Never been used! Get it from:

http://heim.ifi.uio.no/michawe/research/projects/ec-gin-uibk/

3. Next step: real-life tests using SBD– Nothing done yet!

9

Per-flow QoS without signaling to routers

continuous measurements;update to BB upon path change

Synchronization ofdistributed (P2P based)database; link capacitiesknown to all brokers1. may I join?2. yes

Synchronization ofdistributed (P2P based)database; all flows knownto all brokers3. I quit4. ok

Traditional method: signaling to edge routers (e.g. with COPS) at this point!

Synchronization ofdistributed (P2P based)database; all flows knownto all brokers

10

Realistic?

• Yes, when you’re in control of all traffic, know shared bottlenecks, and know their capacity– Several Ph.D. theses, some in “EC-GIN” EU project

One of them:Kashif Munir, "Admission and Congestion Control for Deadline Constrained Data Transfers", May 2009, University of Innsbruck, Austria.

– BDTS system at INRIA Lyon, successfully applied in French Grid’5000 environment

– Incorporated in INRIA spin-off Lyatiss

11

INRIA’s

JBDTS GUI

12

Network monitoring

• “With CloudWeaver, users are able to gain full insight into the underlying topology, capacity, usage and vulnerability of a cloud deployment; anticipate congestion and fix bottlenecks, maximize resource utilization according to flow patterns and minimize infrastructure costs.”– Wouldn’t it be nice to do something along these lines

for networks where you cannot control everything?

13

How we use the Internet today: 2 stories

1. I clean our flat while listening to Spotify via my wife’s laptop– in parallel, downloading files via my own– Suddenly I begin to think:

“please, dear downloads, don’t make the music stop!”

2. I am in a hotel room, using Skype with video to see my daughter– Quality barely good enough– I avoid clicking on anything– Note: that’s different when I talk to my mother...

14

A major problem

• We may have become used to this, but that doesn’t mean it’s good?!– Would like to specify: do not interrupt Spotify / Skype

(or know: do downloads disturb Spotify / Skype or not?)

• These were just two examples– Downloads can also have different priorities– When I download two files, I try to guess whether the

downloads slow each other down

15

Opinions: 139 of my work colleagues, students, and Facebook “friends”

16

17

18

Ingredients of this controlled-fairness soup

• Shared bottleneck detection– for the user: know about mutual influence of transfers– for upload and download: control fairness only among flows that

share a bottleneck

• Coupled congestion control for separate flows– Solutions exist (CM, TCB interdependence (RFC 2140))

• Alternative to coupled cc.: tunable-aggression-TCP– Solutions exist

• E2E-signaling of fairness requirements– Doesn’t exist?!

• ... and a GUI that shows transfers by application;existing tools can do that

19

Coupling cc. of separate flows

• Does something very wrong if flows do NOT share the same bottleneck!

• But if they do, a ton of potential very positive side-effects– Less delay– Less packet loss– Less signalling

(N flows don’t need N*feedback about the same path)– More controllable behavior

(sender-side scheduling vs. “fighting it out” on the bottleneck)– Better performance for short or application-limited flows

(TCP does use it or lose it; with shared congestion control, ifflow 1 doesn’t use it, maybe flow 2 does.Skip slow start: again less queuing delay from slow start overshoot)

20

IETF RMCAT proposal:“Flow State Exchange” (FSE)

• The result of searching for minimum-necessary-standardization: only define what goes in / out, how data are maintained– Could reside in a single app (e.g. browser) and/or in the OS

CM

Stream 1

Stream 2

CM

Stream 1

Stream 2

FSEStream 1

Stream 2

FSE

Traditional CM FSE-based CM Another possibleimplementationof flow coordination

21

Coupled cc: thinking ahead

• If we can relax update frequency, can we coordinaterates across hosts?

• May need to detect shared bottlenecks on both sides

H1 H3

H2

22

Conclusion

• Some SBD mechanisms exist– real-life validation is urgently needed!

• A long list of potential applications– Better cc for multi-path transfers

• Or decisions to apply multi-path, e.g. in an overlay– Predicting file transfer delay

• Important input for scheduling in distributed processing– Deeper knowledge about a net; good for monitoring or even

QoS (-ish) mechanisms• At the “least QoS” end of the spectrum, just tell users which flows

influence each other, let them apply priorities– Coupled congestion control

• Many possibilities there, e.g. cross-host-control

23

Thank you!

Questions?