Practicalizing Delay-Tolerant Mobile Apps with Cedos

YoungGyoun Moon, Donghwi Kim, Younghwan Go, Yeongjin Kim,Yung Yi, Song Chong, and KyoungSoo Park

Dept. of Electrical Engineering, KAIST

2

Bandwidth-hungry Mobile Apps

Global Mobile Data Traffic Forecast

EB/month

Source: Cisco VNI Mobile, 2015

10x increase

3

Mobile Network Capacity Overload

It is unclear if the physical capacity can meet the future demand

4

Cellular Data Offloading to Wi-Fi

Wi-Fi as a secondary network interface in smartphones

Cellular Network (3G/LTE) Wi-Fi

Wide coverage Pros Higher bandwidth

Expensive link cost Cons Limited coverage

5

Delayed Wi-Fi Offloading

Delay-tolerance provides more Wi-Fi offloading opportunities• Wiffler [MobiSys’10], DTap [CoNEXT’10]

On-the-spot of-floading

Delayed of-floading

3G/4G 3G/4G 3G/4G

340MB

PM 3:00

PM 9:00

Download Completed

Download Completed

Time until Deadline ≤

LTE Transfer Time

340MB 280MB

Deadline

How many mobile apps supportdelayed offloading?

6

1. TCP connection failure at every disruption

Behavior of mobile apps at network disruption

Challenges in Opportunistic Wi-Fi Usage

143.90.7.20110.89.47.114

Reconnect-ing…

Load static webpage Failed

Stream video Failed

Stream video Resume

Post comment Failed

Little system support for disruption handling

7

Challenges in Opportunistic Wi-Fi Usage

2. No complete support for delayed offloading

When there is no Wi-Fi connectivity,

Burden of handling network disruptions/delayis placed solely on app developers.

8

Practical delay-tolerant mobile network access architecture The first comprehensive solution to development and deploy-

ment of delay-tolerant mobile apps

Handle disruptions and long delays

Delayed Wi-Fi offloadingDelayed offloading schedulersfor static and dynamic files

Cedos: Cellular Data Offloading System

Challenges Our contributions

Transport protocol which handles disruptions and delays

Real-world mobile apps

9

Overall Architecture

D2TP

Client application TCP server

D2BufMgrD2SchedD2TP TCPTCP

D2Prox

D2TP

D2Prox

D2SchedD2BufMgr

D2TP↔TCP translation proxy

Delay-tolerant offloading scheduler

Disruption- and Delay-tolerant transport protocol

Backward compatibility

Handling disruption

Delayed offloading

10

D2TP

D2TP allows disruption- and delay-tolerant transmission• By handling the network disruptions/delays at mobility transparently

D2TP

D2TP end host

Handle disruptions/delays

BSD socket-like API

D2TP

D2TP end host

BSD socket-like API

D2TP

TCP-like, reliable data transfer in stationary environments Hides the mobility events from the application layer

• Key enabler: “flow identifier (flow id)”

11

DATA (flow id = X, IP = A)

Authentication Process

Detect address change

C S

Continue data transfer

DATA (flow id = X, IP = B)

Validation

11

12

D2Sched and D2BufMgr

D2TP

D2TP client apps

D2BufMgr(dynamic contents)

D2Sched(static contents)

Opportunistic Wi-Fi offloading

D2TP server

D2TP

D2Sched and D2BufMgr enable opportunistic Wi-Fi offloading• Exploit users’ delay tolerance

13

D2Sched

Cedos network API explicitly exposes how to handle delays• “I want to complete a 1 GB file download at least within 6 hours.”

D2Sched • Delay-tolerant offloading scheduler for static files• Goal: To meet QoE requirement with maximum Wi-Fi usage

14

D2BufMgr

Can we find any delay tolerance in real-time streaming apps?

15

D2BufMgr

Usage scenario

LTE Wi-Fi LTE Wi-Fi LTE Wi-Fi

1st st. 2nd st. 3rd st.

Wi-Fi data transfer is possible for small portion of time only

D2BufMgr aims to(1) Match minimal QoE to avoid buffer underrun

• “I want to watch a 320 Kbps video without any buffer underrun.”

(2) Maximize Wi-Fi usage

16

D2BufMgr

Delay transfer in 3G/LTE until buffer is going to be drained out

LTE LTE

Th_high

Sufficient buffered data Stop using LTE

Buffer draining out Switch back to LTE

Sufficient buffered data Stop using LTE

Time

Amt. datain buffer

Th_low

Guarantee QoE

17

D2Prox

D2Prox works as a protocol translation proxy• For incremental real-world deployment

D2TP

Client apps TCP server

D2BufMgr(dynamic contents)

D2Sched(static contents)

D2TP TCPTCP

D2Prox

18

Cedos

Motivation Design

• D2TP• D2Sched and D2BufMgr• D2Prox

Implementation• Cedos socket API• Portability

Evaluation • Real-world Applications

19

Cedos Socket API

/* BSD socket-like APIs */int d2tp_socket();int d2tp_bind(fd, &addr, addr_len)int d2tp_connect(fd, &addr, addr_len);int d2tp_accept(fd, &addr, &addr_len);int d2tp_close(fd);int d2tp_read(fd, buf, count);int d2tp_write(fd, buf, count);int d2tp_select(nfds, readfds, writefds, exceptfds, timeout);

/* Cedos-specific APIs (for delayed offloading) */d2tp_setsockopt(fd, D2TP_SO_RCVBUFTH, &th, sizeof(th));d2tp_setsockopt(fd, D2TP_SO_DEADLINE, &th, sizeof(th));d2tp_setsockopt(fd, D2TP_SO_ESIZE, &th, sizeof(th));

20

D2Prox and Real-world Applications

D2Prox implementation based on nginx (v1.2.6)• By porting its web proxy module to use D2TP socket functions

Real-world applications

Implementation Total (LoC) Modified (LoC)

D2Prox 129,340 55

Cedos-Firefox 8,851,611 123

Cedos-VLC 3,733,651 61

ReadyCast 9,868 -

a delay-tolerant podcast downloader built on Ce-dos

Migration effort ↓

21

Evaluation with Real-world Apps

Opportunistic data offloading (Cedos-VLC with D2BufMgr)• Video streaming in a subway train

Delayed data offloading (ReadyCast with D2Sched)• User study with delay-tolerant podcast app

22

Opportunistic Data Offloading with VLC

Test environment• Client: Nexus 5

– Watch a 640 Kbps streaming video– Take a round trip for 22 stations in Daejeon subway Line 1

LTE Wi-Fi LTE Wi-Fi LTE Wi-Fi

1st st. 2nd st. 3rd st.

Wi-Fi data transfer is possible only at stations

23

D2BufMgr’s low and high threshold = 200 KB and 3 MB• Enough to cover 0.3 second of interface switching time

Results• Not a single video pause during the playback

Opportunistic Data Offloading with VLC

Wi-Fi temporal coverage

11%

Wi-Fi offloading ratio

48%

24

Delayed Data Offloading with ReadyCast

25

Delayed Data Offloading with ReadyCast

26

Delayed Data Offloading with ReadyCast User study with 50 podcast users for 8 weeks

Internet

ServerServer Server

Wi-Fi AP

Client

D2Prox

KAIST campus

Monitoring server

27

Delayed Data Offloading with ReadyCast

Offloading ratios by user-configured delays

60

70

80

90

100

79.5%

93.4%96.2%

99.5%

92.4%

Wi-F

i Off

load

ing

Rat

io (

%)

On-the-spot Delayed(short)

Delayed(long)

Delayed(long) Total

Allowed de-lay 0 0 – 1 hr 1 – 3 hr 3 – 6 hr -

# Downloads 974 362 330 1,168 2,834

Users allowed delay for65% of their downloads

28

Delayed Data Offloading with ReadyCast

Delay selection by users

Week 1 Week 2 Week 3 - 80%

20%

40%

60%

80%

100%

53% 51%

71%

8%21%

13%39%28%

16%

On-the-spotDelayed (Short)Delayed (Long)

Fra

cti

on

of

Da

ta T

ran

sm

itte

d (

%)

People allow more delays as theybecome accustomed to delayed transmission

29

Related Works

TCP migrate DTN Protocols(e.g., Bundle) Wiffler

Handling disruption V V (done by apps)

Handling long delay ‒ V (done by apps)

API BSD custom custom

Delayed offloading N/A N/A (static files only)

Cedos provides a comprehensive solution for delayed offloading applications

30

Conclusion

Delay-tolerant Wi-Fi offloading is known to be a great idea,but is not widely adopted in real-world mobile apps

Cedos bridges the gap between theory and practice in Wi-Fi of-floading by providing a platform for delay-tolerant apps

We hope Cedos can further reduce the cost in providingdata-intensive mobile services to users.

Thank you!

http://cedos.kaist.edu

Backup Slides

33

D2BufMgr

D2BufMgr algorithm• If buffered data is smaller than the low threshold Thlow,

start filling up the buffer using LTE up to the high threshold Thhigh

• If buffered data becomes larger than the high threshold,switch to Wi-Fi interface and try downloading

• Calculating low and high threshold– Thlow = (bitrate) * (time to switch Wi-Fi-to-LTE & start download)

– Thhigh = Thlow + (bitrate) * (time to switch LTE-to-Wi-Fi & start download)

Guarantees no buffer underrun with minimal LTE usage

34

Wi-Fi is Nuisance

Frequent connection closure due to network disruption

Smartphone user survey result• Half (48%) of smartphone users usually keep Wi-Fi interface off

Increased battery consump-tion,

enough cellular data cap, etc..

72%

Reasons for turning off Wi-Fi interface

35

Delayed Data Offloading with ReadyCast

Delay selection by monthly data cap

5GB1GB0

Delayed download:

48%Delayed download:

52%Delayed download:

63%

36

Cedos: Cellular Data Offloading System

Practical delay-tolerant mobile network access architecture

Contributions• The first comprehensive solution to the development and deploy-

ment of delay-tolerant mobile apps– Hide network disruptions and maximize Wi-Fi usage opportunities

• Easy to port existing mobile apps to support cellular data offloading– Both for streaming and delay-tolerant apps

• Backward compatible to existing TCP-based servers– For incremental real-world deployment

37

Initialization of D2TP Connection TCP-like 3-way handshake + shared key exchange

SYN (flow id = X) + Public-Key

ACK

SYN+ACK + ENC(key)

Create/encrypt secret key

Extract secret key via decryption

C S

Both hosts now share the same secret key and flow id

38

D2TP Flow Handling at Mobility Events Authentication process to prevent connection hijacking

DATA (flow id = X, IP = A)

RSP (HMAC-SHA1_key{nonce})

CHG (nonce)Detect address change

C S

Continue data transfer

DATA (flow id = X, IP = B)

AUTH

Validation

39

D2Sched

D2TP flow scheduler for deadline-based flow management• Delay data transfer in cellular network until deadline approaches

D2Sched aims to(1) Complete all flows within user-specified deadline

(2) Maximize Wi-Fi usage

D2Sched should support multiple concurrent D2TP flows• Sequential scheduling in an earliest deadline first (EDF) manner

– An urgent flow has less data transfer opportunities than others

40

D2Sched

Scheduling algorithm• Flows are indexed in an increasing order of time until deadline

– e.g., flow 1 (earliest deadline), …, flow N (latest deadline)

• At each scheduling epoch, 0. If the data transfer was allowed, update the estimated throughput

For k = 1 to N,

1. Pick a flow whose time until deadline (D) is the k-th earliest one

2. Calculate the potential completion time (PCT) of the flow

3-1. If PCT <= D, start transferring flow 1 only if connected to Wi-Fi

3-2. If PCT > D, start transferring flow 1 using faster interface• Either cellular or Wi-Fi network

41

Cedos Components

D2TP socket API

D2Prox

Real-world Cedos applications

D-Lock

42

BSD socket-like API for developers’ minimal migration effort

Example• Streaming app (opportunistic Wi-Fi offloading)

D2TP Socket API

fd = socket();th.low = 20*1024; // low threshold = 2KBth.high = 1000*1024; // high threshold = 1MBd2tp_setsockopt(fd, D2TP_SO_RCVBUFTH, &th, sizeof(th)); connect(fd, &serv_addr, sizeof(serv_addr); write(fd, request_buf, BUFSIZE); read(fd, response_buf, BUFSIZE); close(fd);

d2tp_d2tp_d2tp_d2tp_

fd = d2tp_

43

D2TP Socket API

fd = socket();d2tp_setsockopt(fd, D2TP_SO_DEADLINE, &t, sizeof(t));d2tp_setsockopt(fd, D2TP_SO_ESIZE, &size, sizeof(size)); connect(fd, &serv_addr, sizeof(serv_addr); write(fd, request_buf, BUFSIZE); read(fd, response_buf, BUFSIZE); close(fd);

d2tp_d2tp_d2tp_d2tp_

fd = d2tp_

BSD socket-like API for developers’ minimal migration effort

Example• Non-interactive app (delayed data offloading)

44

D-Lock

CPU sleep or Wi-Fi “no-attach” mode on idle state• To minimize the battery consumption• D2TP flows would stop even if Wi-Fi exists

Android WakeLock and WifiLock• Battery would be drained out quickly

D-Lock is a power efficient CPU/Wi-Fi wakeup module• Acquires CPU/Wi-Fi lock only when requested by D2TP

– Periodically wakes up CPU to probe nearby Wi-Fi APs– D2Sched allows data transfer when a preferred network is available

45

Microbenchmark

Test environment• Client

– Galaxy S3 (1.4 GHz quadcore CPU, 2 GB RAM, and Android 4.1.2.)– Nexus 5 (2.3 GHz quadcore CPU, 2 GB RAM, and Android 4.4.2.)

• Server machine (TCP/D2TP server or D2Prox)– Intel Xeon E5-2650v2 octacore CPU, 32 GB RAM, and Ubuntu 12.04

• Wi-Fi AP– ipTIME N8004R 802.11n (using 5GHz channel)

46

Throughput and Fairness

TCP/D2TP client ↔ TCP/D2TP server• Client: Nexus 5• Comparable per-flow bandwidth fairness to that of TCP

D2TP client ↔ D2Prox ↔ TCP server• Client and D2Prox: E5-2650v2 machines

– 64K concurrent flows (each downloads a 1GB file)– Change idle flow ratio from 0% to 100%, and back to 0%

• D2Prox keeps 10 Gbps line rate (with JFI >= 0.9) throughout the test

Protocol (# conns) Aggregate Throughput JFI**

TCP (100) 95.1 Mbps 0.99

D2TP (100) 96.3 Mbps 0.99

TCP (50) + D2TP (50) 95.0 Mbps 0.98

** JFI = Jain’s Fairness Index

47

Power Usage at Idle States

D2TP client ↔ D2Prox ↔ TCP server• Client: Galaxy S3

– No active connection (connections are waiting for Wi-Fi opportunity)– Turn on Wi-Fi module for periodic scanning

Sleep WakeLock(2min)

D-Lock (1min)

D-Lock (2min)

D-Lock (10min)

0

50

100

150

200

250

43.0

231.9

67.5 55.1 45.2

Po

wer

(m

W)

Battery consumption reduced by 76%

48

Video Streaming in a Subway Train

LTE Wi-Fi LTE Wi-Fi LTE Wi-Fi

Wolpyeong (KAIST) Government Complex Daejon City Hall

Limitations of current streaming applications• Unable to fully utilize Wi-Fi due to short periods of connectivity• Wi-Fi signal strength can be unstable for continuous streaming

Current Streaming Apps

49

Consume the prefetched chunks

Video Streaming in a Subway Train

LTE Wi-Fi LTE Wi-Fi LTE Wi-Fi

Wolpyeong (KAIST) Government Complex Daejon City Hall

Benefits of Cedos application• Real-time buffer management exploiting opportunistic Wi-Fi connections• Prevent buffer underrun by switching back to LTE if Wi-Fi is too slow

Prefetch chunks via Wi-Fi

Cedos-based Streaming App (VLC using D2BufMgr)

50

Related Works

Mobility for IP networks• Mobile IP [JACS’95], i3 [SIGCOMM’02], HIP [RFC4423]

– Separate host identity with location by relaying to ID– Require additional infrastructure support

• Migrate TCP option [MobiCom’00]– Reuse connection after IP address change– Cannot handle large disruption delays

Existing DTN protocols• Bundle Protocol [RFC5050], LTP [RFC5325-5327]

– Designed as transport protocols in multi-hop opportunistic networks– Unsuitable for sending dynamically generated data– Programming API is largely different from BSD-socket API

e.g., require endpoint IP starting with “dtn://”

51

Related Works

Wi-Fi offloading with delay• TUBE [SIGCOMM’12], DTap [CoNEXT’10]

– Show the potential of Wi-Fi offloading through mobility studies– No practical system design

• Wiffler [MobiSys’10]– Wi-Fi offloading system leveraging the delay tolerance– Focus only on fixed-sized files, no multi-flow consideration

52

Cellular Network vs. Wi-Fi

• Cost savings by Wi-Fi deployment• Simulation analysis result in New York City [1]

• By deploying 33,000 APs (density of 42 APs/km2),44.7% of network facility cost can be saved.

[1] White Paper: Building a Business Case for Carrier WiFi Offload – March 2012, Wireless 20/20 (http://www.wireless2020.com/docs/CarrierWiFiOffloadWhitePaper03202012.pdf)

LTE Only LTE + Wi-FiLTE spots 1,879 432Wi-Fi APs 0 33,000

Total CapEx $514M $284M