6
Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan Braud * , Teemu K¨ am¨ ar¨ ainen , Matti Siekkinen †∧ , Pan Hui *† * The Hong Kong University of Science and Technology Aalto University University of Helsinki Abstract—Real time interactive cloud-based mobile applica- tions such as augmented reality and cloud gaming require low and stable latency, especially in urban areas. These conditions are difficult to meet with the traditional single carrier LTE network access and consolidated server deployment in a cloud. Yet, with multiple SIM/multiple radio devices, latency can be kept under a given threshold through dynamic selection among multiple carriers and server deployment at network edge. To this end, it is necessary to understand how mobile network latency changes over time during a session with different carriers and how the server placement affects the latencies. In this paper, we present results from a measurement study of mobile network latency and jitter in 4G networks of Hong Kong and Helsinki, two very different cities in terms of population density and mobile infrastructure. Based on the results, we introduce a lightweight carrier selection algorithm that displays latencies 10 to 20% lower than single carrier operation. I. I NTRODUCTION There is a growing number of interactive mobile multime- dia applications that have strict requirements on end-to-end latency in addition to bandwidth, such as Mobile Augmented Reality (MAR) and Mobile Cloud Gaming (MCG). MAR consists in superimposing a virtual layer over the physical world and relies on computation-hungry vision algorithms which are usually offloaded to a cloud. However, offloading puts additional stress on already constrained network latency and upstream bandwidth [1]. In MCG, a thin mobile client sends user control events to a remote server which runs the game and transmits a video stream back to the client. MCG requires both very short latency and sufficient downstream bandwidth to provide an enjoyable experience [2]. With such applications, wireless mobile access networks are responsible for a substantial part of the end-to-end latency. In particular, they can cause significant jitter, which is especially detrimental for interactive applications. In controlled environ- ments, LTE can provide latency low enough for real-time operation, below 20ms. However, commercial networks pro- vide very different results depending on network deployment, provisioning, and usage patterns. In November 2016, a report from Opensignal displayed an average downlink throughput of 17.78 Mb/s with 84.52% availability 1 for Hong Kong and 1 Amount of time the user has access to 4G on his device 23.34 Mb/s with 76.36% availability for Finland [3]. Using a single carrier and fixed cloud-based servers often fails to provide sufficiently low and stable latency for these latency- critical applications. However, more and more mobiles are providing multiple SIM capabilities, so that a single device may be connected to several networks at all times. Such devices might enable dynamically selecting the carrier with the lowest or least variable latency in conjunction with edge deployment and possible live migration. Alternatively, the device can use a single universal SIM card to switch between carriers as is done in Google’s Project Fi [4], Apple SIM [5] and Samsung e-SIM [6]. However, efficient carrier selection requires models of mobile network latency in operational environments. In other words, a fine-grained understanding of mobile network latency “in the wild” is necessary. In this paper, we study the latency of LTE network access in two distinct cities: Hong Kong and Helsinki. Hong Kong is characterized by a very high population density in certain areas, while Helsinki has very high mobile network penetration but far fewer users. Many measurement studies on mobile network latency have already been conducted by others before us (see Section V). The key novelty of our study is that we perform continuous latency measurements to several different locations, simultaneously with several carriers. This enables us to investigate the latency-wise benefits of edge server deployment compared to consolidated data centre deployments and to evaluate the benefits of dynamic carrier selection. Besides displaying apparent differences between the two cities, we make several interesting findings, such as discrepancies in intra-session latency variation between carriers and consistent latency benefits with close-to-mobile edge deployments. We then exploit these findings to develop a simple model for dynamic carrier selection and show its benefits over the single carrier paradigm. Our contribution is two-fold: A comparative measurement study of commercial LTE network latency for three main operators in Hong Kong and four main operators in Helsinki. A geography-based predictive model of latency for the city of Hong Kong. Through this model, it is possible to reduce the network latency up to 20%.

Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

Multi-Carrier Measurement Study of MobileNetwork Latency: The Tale of Hong Kong and

HelsinkiTristan Braud∗, Teemu Kamarainen∧, Matti Siekkinen†∧, Pan Hui∗†

∗ The Hong Kong University of Science and Technology∧ Aalto University

† University of Helsinki

Abstract—Real time interactive cloud-based mobile applica-tions such as augmented reality and cloud gaming require lowand stable latency, especially in urban areas. These conditionsare difficult to meet with the traditional single carrier LTEnetwork access and consolidated server deployment in a cloud.Yet, with multiple SIM/multiple radio devices, latency can bekept under a given threshold through dynamic selection amongmultiple carriers and server deployment at network edge. To thisend, it is necessary to understand how mobile network latencychanges over time during a session with different carriers andhow the server placement affects the latencies. In this paper, wepresent results from a measurement study of mobile networklatency and jitter in 4G networks of Hong Kong and Helsinki,two very different cities in terms of population density and mobileinfrastructure. Based on the results, we introduce a lightweightcarrier selection algorithm that displays latencies 10 to 20%lower than single carrier operation.

I. INTRODUCTION

There is a growing number of interactive mobile multime-dia applications that have strict requirements on end-to-endlatency in addition to bandwidth, such as Mobile AugmentedReality (MAR) and Mobile Cloud Gaming (MCG). MARconsists in superimposing a virtual layer over the physicalworld and relies on computation-hungry vision algorithmswhich are usually offloaded to a cloud. However, offloadingputs additional stress on already constrained network latencyand upstream bandwidth [1]. In MCG, a thin mobile clientsends user control events to a remote server which runs thegame and transmits a video stream back to the client. MCGrequires both very short latency and sufficient downstreambandwidth to provide an enjoyable experience [2].

With such applications, wireless mobile access networks areresponsible for a substantial part of the end-to-end latency. Inparticular, they can cause significant jitter, which is especiallydetrimental for interactive applications. In controlled environ-ments, LTE can provide latency low enough for real-timeoperation, below 20ms. However, commercial networks pro-vide very different results depending on network deployment,provisioning, and usage patterns. In November 2016, a reportfrom Opensignal displayed an average downlink throughputof 17.78 Mb/s with 84.52% availability1 for Hong Kong and

1Amount of time the user has access to 4G on his device

23.34 Mb/s with 76.36% availability for Finland [3]. Usinga single carrier and fixed cloud-based servers often fails toprovide sufficiently low and stable latency for these latency-critical applications. However, more and more mobiles areproviding multiple SIM capabilities, so that a single devicemay be connected to several networks at all times. Suchdevices might enable dynamically selecting the carrier withthe lowest or least variable latency in conjunction with edgedeployment and possible live migration. Alternatively, thedevice can use a single universal SIM card to switch betweencarriers as is done in Google’s Project Fi [4], Apple SIM [5]and Samsung e-SIM [6]. However, efficient carrier selectionrequires models of mobile network latency in operationalenvironments. In other words, a fine-grained understanding ofmobile network latency “in the wild” is necessary.

In this paper, we study the latency of LTE network accessin two distinct cities: Hong Kong and Helsinki. Hong Kongis characterized by a very high population density in certainareas, while Helsinki has very high mobile network penetrationbut far fewer users. Many measurement studies on mobilenetwork latency have already been conducted by others beforeus (see Section V). The key novelty of our study is that weperform continuous latency measurements to several differentlocations, simultaneously with several carriers. This enablesus to investigate the latency-wise benefits of edge serverdeployment compared to consolidated data centre deploymentsand to evaluate the benefits of dynamic carrier selection.Besides displaying apparent differences between the two cities,we make several interesting findings, such as discrepancies inintra-session latency variation between carriers and consistentlatency benefits with close-to-mobile edge deployments. Wethen exploit these findings to develop a simple model fordynamic carrier selection and show its benefits over the singlecarrier paradigm.

Our contribution is two-fold:• A comparative measurement study of commercial LTE

network latency for three main operators in Hong Kongand four main operators in Helsinki.

• A geography-based predictive model of latency for thecity of Hong Kong. Through this model, it is possible toreduce the network latency up to 20%.

Page 2: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

The rest of this paper is organized as follows: After de-scribing our experimental protocol and the collected dataset inSection II, we present the results of the comparative analysisin Section III, and introduce our predictive latency model inSection IV. We finally discuss related works in Section V.

II. EXPERIMENTAL PROTOCOL

A. Measurement Setup

We created an Android application to collect network mea-surement data. The application records time, user location,network quality parameters (RSRP, RSRQ), cell ID, localarea code, carrier and the type of connection (e.g. 3G, 4G).Besides, the application measures the network latency to 7distinct servers using the ping tool. The latency is measuredto the five geographically closest Speedtest.net servers, thenearest Amazon EC2 location and to the closest pingable IPaddress (found using the traceroute tool). Each sample isthe average of 5 consecutive pings for which we also log thestandard deviation for each location. On supported phones, theapplication also logs the neighbour cell IDs.

The application is configured to collect one sample every 5seconds in 4G networks and every 20 seconds in other typesof networks. We gathered the data in two very differently pop-ulated cities, Hong Kong and Helsinki, over five days, visitingthe same places both at peak and quiet hour. The populationdensity in Hong Kong is roughly 6644 pop./km2 while thepopulation density of Helsinki is only 2967 pop./km2. Theexperiments are done using SIM cards from several majornetwork operators at the same time. This allows us to comparethe differences in latency with different operators.

B. Measurement Data

We collected 183 different traces with a total of 33164data points. In Hong Kong, the mobile network operatorsincluded in the traces are CSL Mobile Limited (csl.), ChinaMobile Hong Kong (CMHK) and SmarTone HK. In Helsinki,the operators are Elisa, Telia, Saunalahti and DNA. Theseproviders have been anonymized as HK1, HK2 and HK3 inHong Kong, and HEL1, HEL2, HEL3 and HEL4 in Helsinki.The measurements are performed using a variety of LTEcapable mobile phones. This dataset includes results from bothwalking and vehicular speeds (20 to 70 km/h).

III. EXPERIMENTAL RESULTS

A. Overall Latency in Different Networks

Table I presents the 4G coverage of Hong Kong andHelsinki, by provider. As expected, all operators cover the vastmajority of both cities, as during our experiments, the phoneswere connected to 4G 95-99.8% of the time. Hong Kongpresents a slightly better coverage than Helsinki, probably dueto the fact that we rarely crossed rural areas. It is worth notingthat for both Hong Kong and Helsinki, one provider performsslightly less well than the other, with only 95% 4G coverage.

Figure 1 shows the average and median latencies for alloperators to the geographically closest Speedtest.net server. InHong Kong, HK2 and HK3 perform similarly with median

TABLE I4G COVERAGE (IN PERCENTAGE OF TIME) AND PERCENTAGE OF

HANDOVER BACK TO 3G, RELATIVELY TO THE TOTAL NUMBER OFHANDOVER IN HONG KONG (TOP) HELSINKI (BOTTOM).

HK1 HK2 HK3

Time in 4G 95.5% 99.8% 99.5%Handover to 3G 1.79% 0.30% 0.59%

HEL1 HEL2 HEL3 HEL4

Time in 4G 95.0% 98.6% 97.7% 99.4%Handover to 3G 1.9% 1.0% 1.4% 0.5%

HK1 HK2 HK30

20

40

60

80

100

RTT

to th

e clo

sest

serv

er (m

s) Average LatencyMedian latency

HEL1 HEL2 HEL3 HEL40

20

40

60

80

100

RTT

to th

e clo

sest

serv

er (m

s) Average LatencyMedian latency

Fig. 1. Average and median latencies to the closest Speedtest.net server forall operators both in Hong Kong (left) and Helsinki (right). HK1, HEL1 andHEL2 display a large difference between average and median latencies dueto a few very high values.

latencies ranging from 40 to 50 ms and the average latenciesaround 55 ms. HK1 outperforms the two with a median latencyof around 20 ms. However, its average latency is similar tothe others due to a few extremely high values. In Helsinki,the average latencies between operators are very close to eachother between 30 and 40 ms. However, the lower medianlatencies for HEL1 and HEL2 (practically the same operatorwith a different brand) show a higher deviation compared tothe other network operators.

The cumulative distribution functions of the latency datato the closest Speedtest.net server shown in Figure 2 confirmthat while HK1 operator’s median latency is around 20 mslower than the others, the 90th percentile is similarly around60 ms as is the case for HK3. HK2’s 90th percentile is 10ms higher than the other two. In Helsinki HEL1, HEL2 andHEL4 share very similar CDFs. HEL2, however, has both thelowest median latency and highest 90th percentile.

B. Intra-Session Latency Variation

As the traces are continuous within a single measurement,we can identify handovers and show their impact on thelatency measurements. Figure 3 shows that HK2 and HK3in Hong Kong and HEL1, HEL2 and HEL4 seem to handlehandovers from one cell to another without additional latencyincrease. HK1 and HEL3, however, show that in their net-works, the handovers increases the overall latency. However,the 95% confidence interval is high in the data, and furthermeasurements are needed to confirm this finding.

Latency stability during a session is crucial for interactivemobile multimedia applications to enable a consistent quality

Page 3: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

0 20 40 60 80 100 120Latency (ms)

0.0

0.2

0.4

0.6

0.8

1.0CD

F of

late

ncy

dist

ribut

ion HK1

HK2HK3

0 20 40 60 80 100 120Latency (ms)

0.0

0.2

0.4

0.6

0.8

1.0

CDF

of la

tenc

y di

strib

utio

n

HEL1HEL2HEL3HEL4

Fig. 2. The CDF of latency distribution both for Hong Kong (left) andHelsinki (right). HK1 and HEL2 have the lowest median latency while havingdistinct bimodal CDFs.

HK1 HK2 HK30

20

40

60

80

100

120

RTT

to th

e clo

sest

serv

er (m

s) Latency handoverLatency no handover

HEL1 HEL2 HEL3 HEL40

20

40

60

80

100

120

RTT

to th

e clo

sest

serv

er (m

s) Latency handoverLatency no handover

Fig. 3. Average latencies to the closest Speedtest.net server with and withouta handover (cell change) between the previous measurement. Handoversignificantly increase the latency for HK1 and HEL3.

of experience. Humans can adapt to latency, e.g. when gamingbut jitter can be particularly detrimental [7]. Figure 4 shows thestandard deviation in latency for different session distances.We notice that for the Helsinki data, the latency variationincreases rapidly once session distance grows. This variationtends to stabilize to a certain level when session distance isbeyond 500m. In Hong Kong, the results look different: onlywith HK1 does the latency variation show a clear increasingtrend with session distance, while the variation does notchange much with the two other operators.

We also notice quite different amounts of latency variationbetween operators. In Helsinki, HEL4, the standard deviationstays below 20 ms, while with the other carriers it stabilizes toa value between 30 and 45 ms. In Hong Kong, HK2, latencyvaries slightly, with a standard deviation being below 10 ms.HK1 has a very high standard deviation ranging from 80 to120 ms, while HK2 is between the two with 40 ms.

The multi-carrier results on latency stability are plotted inFigure 10. The large confidence intervals reflect the relativelysmall number of samples. The trend is similar to the singlecarrier case, but especially for the Helsinki dataset, the la-tency variation increases less rapidly as a function of sessiondistance (note that the x-axis scale is different from Figure 4).Hence, the latency remains stable for longer distances whencombining multiple carriers.

The traces also include signal strength values for each datapoint. Our findings confirm our previous work in which weshowed that outside extreme values, the signal strength doesnot have a significant effect on the latency [2].

0 500 1000 1500 2000Session distance (m)

20

40

60

80

100

120

Late

ncy

Stan

dard

Dev

iatio

n (m

s)

HK1HK2HK3

0 500 1000 1500 2000Session distance (m)

1015202530354045

Late

ncy

Stan

dard

Dev

iatio

n (m

s)

HEL1HEL2HEL3HEL4

Fig. 4. Standard deviation of latency with different session distances (HongKong left, Helsinki right). Latency increases quickly with distance beforestabilizing at a given value.

0 20 40 60 80 100 120Latency (ms)

0.0

0.2

0.4

0.6

0.8

1.0

CDF

of la

tenc

y di

strib

utio

n di

ffere

nce

HK1HK2HK3

0 20 40 60 80 100 120Latency (ms)

0.0

0.2

0.4

0.6

0.8

1.0

CDF

of la

tenc

y di

strib

utio

n di

ffere

nce

HEL1HEL2HEL3HEL4

Fig. 5. The CDF of latency differences between the closest Speedtest serverand a distant Amazon server both for Hong Kong (left) and Helsinki (right).The median difference is around 20 to 30 ms in both cities.

C. Server Placement Effect

We next study the impact of server placement, namelyedge vs consolidated data centre, by comparing latency to thenearest Speedtest server, which is sometimes located insidethe carrier’s network, to a closest Amazon EC2 server (inFrankfurt for Helsinki and Singapore for Hong Kong).

Figure 5 displays the CDF of the absolute difference be-tween the round trip time to the closest Speedtest server andthe Amazon server. The median difference is between 20 and30ms both for measurements performed in Helsinki and HongKong, which seems reasonable as both Amazon servers areapproximately 2500km away from the measurement point,which already adds 8ms latency. Interestingly, among HongKong carriers, HK1 delivered shortest latencies in Figure 2 butexhibits largest overall latency difference between the closestvs Amazon server, which indicates that the short-latency tothe closest server is at least partially caused by the serverproximity to this carrier’s network and not just a characteristicof the mobile network itself.

When initiating a measurement session, the app rankscandidate Speedtest servers based on ping results. Histogramsof these rankings in Figure 6 reveal that in all the measurementsessions in Helsinki, top 5 ranking was the same. In contrast,the top 3 positions were split between two servers with allthree carriers in Hong Kong. Hence, server placement forlow latency does not appear to be carrier dependent butnot entirely static either. Carrier dependency would changewith the deployment of Multi-Access Edge Computing whereservers are placed in the RAN [8].

We also wanted to understand whether the server that

Page 4: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

HK1 HK2 HK3

HEL1 HEL2 HEL3

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

0255075

100

0255075

100

rank

coun

t (%

)server

123456789

101112131415161718

Fig. 6. Histograms of ranking of servers based on latency measurement atthe start of a session. The carrier choice does not have a strong influence onserver placement.

0.00

0.25

0.50

0.75

1.00

−50 −25 0 25 50

latency difference (ms)

frac

tion

of s

ampl

es

carrierHEL1HEL2HEL3HK1HK2HK3

Fig. 7. CDFs of latency differences between the Speedtest server initiallychosen as closest one and the closest one among the other four servers. In25% of the case, the closest server initially chosen was not the closest one.

provides the shortest latency upon the start of a measurementsession also remains that way during the session. To this end,we calculated for each set of samples the difference in latencybetween the first ranked server and the server providing thelowest latency among the other four ones. Figure 7 showsthe results. Negative values indicate that the server that wasinitially chosen as the one to provide the shortest latencydid not, in fact, provide the shortest latency among the fivemeasured, whereas positive values indicate that it did. Inroughly 25% of the samples, the initially chosen closest serverwas not the closest one latency wise. HEL1 is an exception inthat the initial choice of server is almost always the best one.With the other carriers, sticking to the initially chosen serverwould increase the latency by 5 ms or more in 5-10% of thetime, while ”penalties” of 20 ms or more occur only at mostwith a few percents of the samples.

D. Location Dependency

In Figure 8, we show the average latency measured inseveral activity centres in Hong Kong. HKUST campus islocated in a sparsely populated area. Choi Hung representsthe typical residential area. We also included three majorneighbourhoods: Sham Shui Po, characterized by medium-risebuildings, Mong Kok, one of the most densely populated areas

TABLE IITHE IMPACT OF MULTI-CARRIER ACCESS WITH OPTIMAL CARRIER

SELECTION ON LATENCY (MEDIAN / STD DEV).

c1 c2 c3 c1+c2 c1+c3 c2+c3∑3

i=1 ci

HK 24/155 47/47 56/14 23/13 23/15 46/9 23/12HEL 23/11 16/49 30/18 16/9 23/5 16/10 16/5

in the world, and Tsim Sha Tsui, a tourist hub. On the Islandside, we visited Central, the business centre of Hong Kongand Wan Chai, a more diverse international hub with offices,museums and shopping areas. All areas were visited both atpeak and off-hours.

In order to offer offloaded MAR, MCG, and other latencyconstrained applications while keeping mobility, the latencyprofile of the city should not only present small delays withlow variance, but also remain consistent within different areas.The figure shows that the latency conditions for offloadedMAR or MCG are not fulfilled by any operator. HK2 and HK3display average latencies that are two to three times higherthan the recommended values all over the map. On the otherhand, HK1 displays a strong spatial dependence, with averagelatencies low enough to consider offloading applications inthe north of the city. This sharp north/south dichotomy partlyexplains the bimodal CDF displayed Section III-A. Eachcarrier performs better in specific areas, whether in terms oflatency or jitter. A simple predictive model could exploit thisspatial dependency for latency estimation.

IV. PREDICTIVE LATENCY MODELING

A. Multi-Carrier Access

There is much variance in latency between operators as wellas over space and time. Multi-carrier access can potentiallyyield a lower and less varying latency. Since most of our datawere collected by simultaneously measuring three differentcarriers, it allowed us to do some analysis on the achievablelatency using a multi-carrier mobile access. We identifiedthose portions of data where all three operators were active,compared the latency samples between the carriers in a time-synchronized manner, and always selected the smallest sampleamong the carriers. In this way, we obtain results for one, two,and three simultaneous carriers shown in Table II.

We observe that a combination of carriers yields a signif-icantly lower and less variable latency compared to a singleone. The latency samples in Figure 9 exemplify the advantageof using multiple carriers (selected latency samples are markedwith a black cross). Our data suggest that some combinationsof two carriers can achieve as low and stable latency as acombination of three carriers. Combining HK2 with HK3yields smaller std deviation of latency than a combination ofall three carriers, but the median latency is twice as high.

In Figure 10, we plot the mean of the standard deviationof latency within multi-carrier sessions as a function of thesession distance. Interestingly, we do not reach the lowest

Page 5: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

28.35

HKUST

34.17

Choi Hung

35.04

Sham Shui Po

72.17

Tsim Sha Tsui

127.7

Central

125.7Wan Chai HK1

41.28Mong Kok

46.73

HKUST

58.75

Choi Hung

55.31

57.58Mong Kok

Sham Shui Po

60.34

Tsim Sha Tsui

56.46

Central

58.77Wan Chai HK2

71.69

HKUST

50.09

Choi Hung

47.37

52.32Mong Kok

Sham Shui Po

51.73

Tsim Sha Tsui

44.74

Central

61.45Wan Chai HK3

Fig. 8. Geographical representation of latencies in various neighborhoods of Hong Kong. HK1 displays a strong geographic dependency, while HK2 andHK3 remain stable over the territory

0

25

50

75

100

0 5000 10000 20000

Timestamp

Late

ncy

to c

lose

st

Speedte

st s

erv

er

(ms)

Carrier 1Carrier 2Carrier 3Multicarrier

0

25

50

75

100

0 2000 4000 6000

Timestamp

Late

ncy

to c

lose

st

Speedte

st s

erv

er

(ms)

Carrier 1Carrier 2Carrier 3Multicarrier

Fig. 9. Hong Kong (left) and Helsinki (right) 4G latency samples from multi-carrier analysis. The average latency significantly decreases when using multi-carrier access.

3

6

9

12

0 1000 2000 3000 4000session distance (m)

late

ncy

std

devi

atio

n (m

s)

HK1+HK2HK1+HK2+HK3HK1+HK3HK2+HK3

2

4

6

0 1000 2000 3000 4000session distance (m)

late

ncy

std

devi

atio

n (m

s)

HEL1+HEL2HEL1+HEL2+HEL3HEL1+HEL3HEL2+HEL3

Fig. 10. Standard deviation with 95% CI of latency with multi-carrier sessionsof different distances (Hong Kong left, Helsinki right). Using multiple carriersdivides the standard deviation by 10 compared to a single carrier.

standard deviation for the combination of three operators, yetusing multiple carriers divides the standard deviation by tencompared to a single carrier.

The above-presented results assume the ability to perfectlypredict the lowest latency carrier at all times, which may not bepossible in practice. Next, we present a simple carrier selectionalgorithm that can directly be applied to our measurementS.

B. Simple Carrier Selection Algorithm

In Section III-D, we noticed that at a given time, the latencyis strongly dependent on the operator as well as the geographiclocation of the user. This observation, combined with theCDF presented Section III-A allows us to state the followingpostulate: average latency tends to vary more when movingbetween neighbourhoods than at a smaller time scale. Basedon this postulate, we designed a simple algorithm for carrierselection: 1. every 5 seconds, the latency of each operatoris measured. 2. when data has to be transmitted, it is sent

TABLE IIIAVERAGE LATENCY MEASURED IN HONG KONG WITH BOTH SINGLE AND

MULTIPLE CARRIER MODELS. IN EDGE COMPUTING SCENARIOS, THEAVERAGE LATENCY DECREASES BY 10 TO 20% WITH OUR MODEL.

HK1 HK2 HK3∑

HKn∑

HKn,opt

Edge 57.5 ms 56.9 ms 53.0 ms 48.1 ms 28.8 msCloud 80.4 ms 83.5 ms 76.0 ms 68.9 ms 60.3 ms

through the carrier presenting the lowest latency at the lastmeasurement episode.

In Table III, we represent the average latency measuredfor each carrier, the optimal achievable latency in a multi-carrier context, and the latency achieved by our multi-carriermodel for both dynamic edge (closest pingable IP) and cloud(Amazon server) situations. Assuming that the edge serversare dynamically migrated with no delays or that the serviceis stateless, our simple model allows decreasing the latencyby 10 to 20% compared to single carrier operation. Curiously,the absolute improvement is higher in the case of the cloudserver with a latency difference of 7 to 14 ms compared to5 to 9 ms for an edge server. This phenomenon is probablycaused by the wired wide area network for which latencyvariance tends to be smaller. We also computed the lowestaverage latency reachable if the carrier with the lowest latencyis always selected. With better models, we can envision areduction of 30 to 40% of the latency.

V. RELATED WORK

Many studies of mobile network latency have been con-ducted. Several papers on 3G networks exist [9], [10] but wefocus on those that studied latency of LTE. In 2012, Laner etal. [11] did comparative measurements of one-way delays incellular networks. Their setup was highly controlled, and nomobility was involved. Sommers et al. [12] studied a largeSpeedTest dataset for throughput and latency with a focuson Wi-Fi vs cellular network. Their data consists of one-shot measurements, while we study continuous measurements.Becker et al. [13] studied application-level performance ofLTE and the effect of discontinuous reception (DRX) onlatency. No mobility was considered. Rosen et al. [14] in-vestigated the radio resource control (RRC) protocol and itsimpact on latency. For continuous measurements, such effectsshould be limited with LTE unless timer values are set tounusually low values. Chetty et al. [15] measured mobile

Page 6: Multi-Carrier Measurement Study of Mobile Network …braudt/papers/Multi_carrier...Multi-Carrier Measurement Study of Mobile Network Latency: The Tale of Hong Kong and Helsinki Tristan

network performance in South Africa. LTE penetration atthat time was modest and the measurements non-continuous.Albaladejo et al. [16] examined LTE latency in Dublin andderived a model for RTT as a function of link quality. Wedid not find such a clear relationship in our measurementdata. Larson et al. [17] studied causes, such as inter-cellhandovers, behind extreme latencies in Norwegian mobilenetworks. Nikravesh et al. [18] studied throughput and latencybetween mobile devices and Google servers. Their dataset ismuch larger and longitudinal but coarser-grained. We strive tounderstand relatively fine-grained latency variations during anapplication session and, therefore, measured simultaneouslymultiple carriers and server locations using a short interval.

Li et al. [19] used low-level cellular information to improvesingle SIM multi-carrier access. They showed improvementcompared to existing multi-carrier services such as ProjectFi and highlighted the cost of switching between carriers.

VI. CONCLUSION AND FUTURE WORK

In this article, we reported on a measurement study of la-tency in 4G networks of Hong Kong and Helsinki. Our resultssuggest that current 4G network deployments may not yetprovide low and stable enough latency for highly interactiveapplications, such as MAR or MCG. However, the study alsoindicates that some carriers partially fulfil the requirements,either in terms of average latency or jitter. We also examinedlatency-wise benefits of using nearby servers compared tofurther away consolidated data centres with mobile networks.Finally, we showed the benefits of using multiple carriers. Witha simple algorithm, we managed to reduce the average latencyby 10 to 20%. We also exposed that the latency could decreaseby up to 40% with more precise models.

This work is our first stab at the problem and further effortswith more extensive measurements are necessary to developprecise predictive models of mobile network latency and jitter.We also wish to extend our study to include throughput andexamine the impact of both cross-traffic and parallel traffic onlatency. Preliminary measurements showed a dramatic increaseof the latency in the presence of both downlink (170 to 250%)and uplink (270 to 380%) traffic, indicating that the presenceof uplink cross-traffic can be even more detrimental thandownlink cross-traffic. Other extensions of this work includebeing able to characterize the mobile phone’s impact on thelatency measurements to crowdsource our measurements tousers in different cities. This would enable us to collect enoughsamples to explore the temporal dimension in latency, as wellas its stability. A larger dataset is also required to createpredictive models not tied to constant active measurements.

Finally, we expect the development of 5G to challenge themodels presented in this paper. Due to the denser deploymentof macro, micro and pico base stations, we expect to encountermuch less traffic at a given base station, thus more stablelatency and throughput. However, we still expect high variancein very densely populated areas and increased inequality ofresources between city centres and more rural areas. As such,5G will aggravate the need for latency prediction and increase

the utility of multi-carrier communication for stable low-latency service.

ACKNOWLEDGEMENTS

This research has been supported in part by project16214817 from the Research Grants Council of Hong Kongand the 5GEAR project from the Academy of Finland.

REFERENCES

[1] T. Braud, F. H. Bijarbooneh, D. Chatzopoulos, and P. Hui, “Futurenetworking challenges: The case of mobile augmented reality.”

[2] T. Kamarainen, M. Siekkinen, A. Yla-Jaaski, W. Zhang, and P. Hui, “Ameasurement study on achieving imperceptible latency in mobile cloudgaming,” in 8th ACM Multimedia Systems Conference - MMSys ’17,2017.

[3] OpenSignal. (2016) The state of lte. Accessed 12-05-2017. [Online].Available: https://opensignal.com/reports/2016/11/state-of-lte

[4] Google. Project Fi. https://fi.google.com/. Accessed 19-10-2017.[Online]. Available: https://fi.google.com/

[5] Apple. Apple SIM. https://www.apple.com/ipad/apple-sim/. Accessed19-10-2017. [Online]. Available: https://www.apple.com/ipad/apple-sim/

[6] Samsung. Samsung eSIM. http://developer.samsung.com/tech-insights/esim/samsungs-esim-solution. Accessed 19-10-2017. [On-line]. Available: http://developer.samsung.com/tech-insights/eSIM/samsungs-esim-solution

[7] B. Lee and A. Oulasvirta, “Modelling error rates in temporal pointing,”in 2016 CHI Conference on Human Factors in Computing Systems CHI’16, 2016.

[8] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, “Onmulti-access edge computing: A survey of the emerging 5g network edgecloud architecture and orchestration,” IEEE Communications SurveysTutorials, vol. 19, no. 3, pp. 1657–1681, thirdquarter 2017.

[9] C. Serrano, B. Garriga, J. Velasco, J. Urbano, S. Tenorio, and M. Sierra,“Latency in broad-band mobile networks,” in VTC Spring 2009 - IEEE69th Vehicular Technology Conference, 2009.

[10] M. Laner, P. Svoboda, E. Hasenleithner, and M. Rupp, “Dissecting 3guplink delay by measuring in an operational hspa network,” in 12thInternational Conference on Passive and Active Measurement PAM’11,2011.

[11] M. Laner, P. Svoboda, P. Romirer-Maierhofer, N. Nikaein, F. Ricciato,and M. Rupp, “A comparison between one-way delays in operating hspaand lte networks,” in 2012 10th International Symposium on Modelingand Optimization in Mobile, Ad Hoc and Wireless Networks (WiOpt),2012.

[12] J. Sommers and P. Barford, “Cell vs. wifi: On the performance of metroarea mobile connections,” in 2012 Internet Measurement ConferenceIMC ’12, 2012.

[13] N. Becker, A. Rizk, and M. Fidler, “A measurement study on theapplication-level performance of lte,” in 2014 IFIP Networking Con-ference, 2014.

[14] S. Rosen, H. Luo, Q. A. Chen, Z. M. Mao, J. Hui, A. Drake, and K. Lau,“Discovering fine-grained rrc state dynamics and performance impacts incellular networks,” in 20th Annual International Conference on MobileComputing and Networking Mobicom ’14, 2014.

[15] M. Chetty, S. Sundaresan, S. Muckaden, N. Feamster, and E. Calandro,“Measuring broadband performance in south africa,” in 4th AnnualSymposium on Computing for Development ACM DEV-4 ’13, 2013.

[16] M. B. Albaladejo, D. J. Leith, and P. Manzoni, “Measurement-basedmodelling of lte performance in dublin city,” in 2016 IEEE 27th AnnualInternational Symposium on Personal, Indoor, and Mobile Radio Com-munications (PIMRC), 2016.

[17] N. Larson, D. Baltrunas, A. Kvalbein, A. Dhamdhere, k. claffy, andA. Elmokashfi, “Investigating excessive delays in mobile broadbandnetworks,” in 5th Workshop on All Things Cellular: Operations, Ap-plications and Challenges AllThingsCellular ’15, 2015.

[18] A. Nikravesh, D. R. Choffnes, E. Katz-Bassett, Z. M. Mao, andM. Welsh, Mobile Network Performance from User Devices: A Lon-gitudinal, Multidimensional Analysis, 2014.

[19] Y. Li, H. Deng, C. Peng, Z. Yuan, G.-H. Tu, J. Li, and S. Lu, “icellular:Device-customized cellular network access on commodity smartphones.”

in NSDI, 2016, pp. 643–656.