Using&Mul*path&TCP&to&enable& …ew2017.european-wireless.org/wp-uploads/2017/05/... · Mul/path!Networks! Mobile devices have mulplewireless& connecons&& Mul/path!Networks! ... MPTCP

Using Mul*path TCP to enable smooth connec*vity in 5G networks

University Politehnica of Bucharest

Cos$n Raiciu [email protected]

THANKS TO SUPERFLUIDITY

Joint work with Dragos Niculescu, Andrei Croitoru, Catalin Nicutar (UPB), Mark Handley, Damon Wichik (UCL), Olivier Bonaventure, Christoph Paasch, Sebastien Barre, Gregory Dettal (U. Catholique de Louvain)

Mul/path Networks

Mobile devices have mul*ple wireless connec*ons

Mul/path Networks

Servers are multi-homed

Client

How do we use these networks? TCP.

Used by most applica/ons, offers byte-‐oriented reliable delivery, adjusts load to network condi/ons

[Labovits et al – Internet Interdomain traffic – Sigcomm 2010]

TCP is single path A TCP connec/on Uses a single-‐path in the network regardless of network topology

Is *ed to the source and des*na*on addresses of the endpoints

Is allergic to packet loss, forcing wireless networks to hide loss at lower layers despite latency increases.

Mismatch between network and transport

creates problems

Poor Performance for Mobile Users

celltower


celltower


celltower


Offload to WiFi

celltower


All ongoing TCP connections die

celltower

Mul*path TCP

Multipath TCP RFC 6824 (protocol) and RFC 6356 (conges4on control) Evolves TCP to use mul$ple paths in the network.

Supports unmodified applica4ons

Works well over deployed cellular and Wifi networks, exis*ng datacenters and the Internet at large.

Linux kernel and IOS implementa*ons used in produc*on

Deployed by Apple, Samsung and LG on mobile phones, and used by network operators to boost throughput

Mul/path TCP components

Connec4on setup Sending data over mul4ple paths Encoding control informa4on Dealing with (many) middleboxes Conges4on control

[Raiciu et. al – NSDI 2012]

[Wischik et. al – NSDI 2011]

MPTCP Connec/on Management


SYN

MP_CAPABLE X


SYN/ACK MP_CAPABLE Y


ACK,

DataACK


Subflow 1

MPTCP conn. X

Subflow 1

MPTCP conn. Y


SYN JOIN Y

Subflow 1

MPTCP conn. X

Subflow 1

MPTCP conn. Y


SYN/ACK

JOIN X

Subflow 1

MPTCP conn. X

Subflow 1

MPTCP conn. Y


ACK, DataACK

Subflow 1

MPTCP conn. X

Subflow 1

MPTCP conn. Y


Subflow 2 Subflow 2

Subflow 1

MPTCP conn. X

Subflow 1

MPTCP conn. Y

Sending Data

TCP Opera/on

1

TCP Opera/on

1 2

TCP Opera/on

1 2 3

TCP Opera/on

2 3 4 1

TCP Opera/on

3 4 2

ACK 1

TCP Opera/on

4 3

ACK 2 ACK 1

TCP Opera/on

4

ACK 3 ACK 2 ACK 1

TCP Opera/on

ACK 4 ACK 3 ACK 2

ACK 1

Strawman Design

1

Strawman Design

1

2

Strawman Design

3

2

1

Strawman Design

2

3

4

1

Strawman Design

2 4

3

ACK 1

Strawman Design

4

3

ACK 1

ACK 2

Strawman Design

4

ACK 3

ACK 2

ACK 1

A third of access networks will

“correct” or drop ACKs of unseen data [Honda, IMC 2011]

Strawman Design

ACK 3

ACK 4

ACK 1

Strawman Design

Ok, so what does work?

•  We need a sequence space for each subflow – This will drive loss detec/on and retransmissions

•  We need a data sequence number – This will put segments in order at the receiver

•  We need a data ACK for flow control – Receive window is rela/ve to Data ACK

MPTCP Data Transmission SUBFLOW: 100

DATA:1


DATA:1

SUBFLOW: 200 DATA:2


DATA:3

SUBFLOW: 200 DATA:2

SUBFLOW: 100 DATA:1


DATA:3

SUBFLOW: 200 DATA:2

SUBFLOW: 100 DATA:1


DATA:3

SUBFLOW: 200 DATA:2

SUBFLOW: 100 DATA:1

SUBFLOW: 102 DATA:2

TCP Packet Header

Source Port Destination Port

Sequence Number

Acknowledgment Number

Receive Window Header Length Reserved Code bits

Checksum

Options

Urgent Pointer

Data

Bit 0 Bit 15 Bit 16 Bit 31

20 Bytes

0 - 40 Bytes

MPTCP Packet Header


Sequence Number



Checksum

Options

Urgent Pointer

Data


20 Bytes

0 - 40 Bytes

Subflow

Subflow

Subflow Subflow

MPTCP Packet Header


Sequence Number



Checksum

Options

Urgent Pointer

Data


20 Bytes

0 - 40 Bytes

Subflow

Subflow

Subflow Subflow

Connection relative to Data ACK

Connection Sequence Mapping Data ACK

Implementa/on Issues

Receive Buffer Sizing

MPTCP over WiFi/3G

8Mbps, 20ms

2Mbps, 150ms

TCP over WiFi/3G

MPTCP over WiFi/3G

MPTCP over WiFi/3G

Multipath TCP increases

throughput

MPTCP over WiFi/3G

What happened

here?

Needed to keep the pipe full during fast retransmit

Allow all paths to send at full rate

While the worst path is sending a packet

Needed to keep the pipe full

• TCP 2 * BW * RTT

• MPTCP 2 * sum(BWi) * max (RTTi)

Receive Buffer Sizing

MPTCP needs large receive buffer to cope with paths with significantly different delay

MPTCP over WiFi/3G

1 Receive Window

MPTCP over WiFi/3G

1 Receive Window

4 3 2

MPTCP over WiFi/3G

1

3 2 4

Receive Window

MPTCP over WiFi/3G

1

ACK

3 2 4

Receive Window

MPTCP over WiFi/3G

1

Wifi path blocked by the Receive Window

3 2 4

Receive Window

MPTCP over WiFi/3G

1

1

3 2 4

Receive Window

REINJECT SEGMENT ON WIFI

MPTCP over WiFi/3G

1 REINJECT SEGMENT ON WIFI

HALVE CONGESTION WINDOW

OF 3G SUBFLOW 3 2 4

Receive Window 1

MPTCP over WiFi/3G

1 3 2 4

Receive Window 1

MPTCP over WiFi/3G

Receive Window 1

MPTCP over WiFi/3G

MPTCP deployments

MPTCP stacks: Linux, IOS, Solaris

•  Apple’s MPTCP stack runs on all IOS devices since IOS7 (2013) –  it is used by default by Siri – AF_MULTIPATH socket and special API for MPTCP

•  MPTCP Linux kernel: hhp://mul/path-‐tcp.org/

•  Samsung, LG offer Android based MPTCP-‐enabled phones (Galaxy S6, S6 edge, S7)

Commercial deployments

•  Giga LTE – LTE + 802.11ac with Mul/path TCP – Deployed by Korea Telekom and Turkish Telekom

•  LTE + DSL – Large market: Intel, Zyxel Comm., Tessares, … – Deployment/trials: DT, Vodafone, BT, Orange.

•  Link bonding: low rate links for flights (NASA), cellular (mari/me).

Exis/ng deployments use operator proxy

3G celltower

MPTCP Proxy

Proxy Func/onality: Server Does Not Speak MPTCP

3G celltower

MPTCP Proxy

Proxy Func/onality: Server Speaks MPTCP

3G celltower

MPTCP Proxy

Proxy Func/onality: Server Speaks MPTCP

3G celltower

MPTCP Proxy

Mobility in the Bucharest Underground

Time (min) 0 10 5 15 25 20 3

0

Spe

ed (M

bps)

0

10

20

30

40

Wifi Mobility with Mul/path TCP [Croitoru et al., NSDI’15]

WiFi mobility back in fashion

Cellular data growing at a rate that is not sustainable in the long run Offloading to WiFi touted as a solu/on Ubiquitous Access Point deployments in urban areas

AP deployment in Bucharest Center

WiFi is mostly a sta$c connec$vity solu$on

L2 Mobility mantra One AP at a *me When link fades, scan for new APs and connect to one of them based on

– Signal strength – User preference – Es/mated capacity

Client doesn’t know best AP un*l it tries it Best AP changes when client is mobile or sta*c

FAST HANDOVER lots of research

Why bother selec*ng best AP?

Connect to all APs Spread traffic with Mul*path TCP while having a single NIC in the client Implementa/on is straighsorward •  On a single channel, use virtual interfaces •  Channel switch between different channels

Connect to all APs

So how should traffic be allocated? Most traffic should come via the best AP

while probing all other APs

Mul/path TCP + Mul/ple APs

Experiments •  Single channel

– Hidden terminal – Carrier-‐sense

•  Mul/ple channels – Use channel switching [see paper]

AP1 AP2

Hidden terminal Experiment

Client near AP1

Client near AP2

In-between

APs

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8

Thro

ughp

ut [%

]

Position

UDP1


Client near AP1

Client near AP2

In-between

APs

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8

Thro

ughp

ut [%

]

Position

UDP1UDP2


Client near AP1

Client near AP2

In-between

APs

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8

Thro

ughp

ut [%

]

Position

UDP1UDP2

UDP1+2

AP1 transmissions capture transmissions

from AP2

UDP throughput drops significantly

0

0.2

0.4

0.6

0.8

1

1 2 3 4 5 6 7 8

Thro

ughp

ut [%

]

Position

UDP1UDP2

UDP1+2MPTCP1+2


Client near AP1

Client near AP2

In-between

APs

Near optimal throughput with MPTCP One subflow always dominates airtime.

Carrier Sense experiment APs hear each other

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35

Thro

ughp

ut [%

]

Position

TCP via AP1TCP via AP2

MPTCP over AP1+2


0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35

Thro

ughp

ut [%

]

Position


MPTCP over AP1+2

MAC with two APs sending AP1 Client

X

AP2

X

0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35

Thro

ughp

ut [%

]

Position


MPTCP over AP1+2

The closest AP wins contention most times,

optimizing airtime usage


0

0.2

0.4

0.6

0.8

1

0 5 10 15 20 25 30 35

Thro

ughp

ut [%

]

Position


MPTCP over AP1+2

MPTCP allows Wifi to exploit channel diversity


With 802.11n, MPTCP client gets half the throughput it should receive

MAC with two APs sending AP1 Client AP2

lower rate

The Wifi MAC gives packet level fairness, reducing total throughput

Carrier-‐Sense performance depends on rate control algorithm •  Near-‐op/mal behaviour when:

•  All APs used the same fixed rate •  Using minstrel (the default algorithm in Linux)

•  Bad behaviour when: •  Using less aggressive rate control algorithms •  Using minstrel-‐ht (802.11n)

Rate control algorithms are specific to each AP vendor. Change is very difficult.

MPTCP conges/on control primer

Move traffic away from congested links •  Put most data on the subflow with the

smallest loss rate •  Subflows with higher loss rate only receive

probe traffic

MPTCP conges/on control in WiFi

AP1 AP2

p1 p2

When the MAC gives packet level fairness, p1 = p2

Client knows which AP is beher Client es*mates the average *me Ti it takes APi to send one packet

•  Assuming AP is alone accessing the medium •  Passively es/mate PHY loss rates [see paper]

Client orders its APs according to Ti •  Informs server that APj is bad when

Tj > 1.3 mini=1,N(Ti)

Client ECN marking to steer traffic

AP1 AP2

p1 p2

Mark 10% of incoming packets with ECN CE Codepoint

Client ECN marking to steer traffic

AP1 AP2

p1

Mark 10% of incoming packets with ECN CE Codepoint

MPTCP conges/on control primer

Move traffic away from congested links •  Put most data on the subflow with the

smallest loss rate •  Subflows with higher loss rate only receive

probe traffic

Do at least as good as TCP on the best path •  Use RTT and loss es/mates on each subflow

to es/mate what TCP would get

Safe ECN Marking

Mark δ packets as congested using ECN

Safe ECN Marking

The client can compute the safe marking threshold delta: Implemented in device-‐independent part of the WiFi driver of the Linux kernel.

How well does ECN marking work for 802.11n?

0 5

10 15 20 25 30 35 40

0 1 2 3 4 5 6 7 8

Thro

ughp

ut (M

bps)

Position

AP1AP2

MPTCPMPTCP+ECN

Indoor client, walking speed A mobility experiment

0 5

10 15 20 25 30

0 50 100 150 200 250

Thro

ughp

ut (M

bps)

Position

AP1 AP2 AP3


0 5

10 15 20 25 30

0 50 100 150 200 250

Thro

ughp

ut (M

bps)

Position

AP1AP2

AP3Handover


0 5

10 15 20 25 30

0 50 100 150 200 250

Thro

ughp

ut (M

bps)

Position

AP1AP2

AP3Handover

MPTCP+ECN

MPTCP + WiFi = Match made in heaven:

•  Hidden terminals •  CS, all senders use the same rate •  CS, minstrel

Human interven*on needed via ECN: •  Robust across all cases we tested

A research agenda for 5G

Fast handover is the wrong goal: it is be_er to use all connec*ons at once Mul*path TCP allows mixing and matching wireless connec*ons dynamically, moving traffic to the least congested one. • With exis/ng infrastructure (S-‐GW) •  Across legacy APs and cellular networks and can provide •  Robust connec/vity when mobile •  Improved throughput •  Smaller energy consump/on

Q1. When should data be sent over cellular interface?

• Always

• Only when app saturates Wifi

Q2. Improving latency for short flows

• Duplicate data on all subflows

But when should we enable duplica*on? • Doesn’t make sense for network-‐limited flows. • Don’t want to change apps. • Server knows if app is backlogged, client controls interfaces.

Q3. Saving mobile energy

• Es/mate energy efficiency of each interface, use the best one

Issues • Trade-‐off between probing and using • How deal with tail energy

Q4. Mul*Cell: Mul* Wifi for cellular networks

Does it make sense to connect to mul/ple cells simultaneously?

• How to do it with single NIC? • Interac/ons with the cellular rate alloca/on and handover algorithms?

Q5. 5G radios op*mized for MPTCP

What can we do at lower levels to leverage MPTCP deployment at users?

• Low loss rate not crucial anymore => lower latency, higher capacity.

• Less Distributed MIMO => independent channels exploited by MPTCP?

Conclusions Multipath TCP offers mobility and congestion-aware bonding support at layer 4 and is gaining a lot of traction MPTCP can improve 5G networks’ robustness and throughput• while integrating existing Wifi APs and

• keeping 5G networks simple

Documents

Using&Mul*path&TCP&to&enable& …ew2017.european-wireless.org/wp-uploads/2017/05/... · Mul/path!Networks! Mobile devices have mulplewireless& connecons&& Mul/path!Networks! ... MPTCP