5G coverage and performance assessment in UK top cities

Public

5G coverage and performance assessment in UK top cities and Combined Authorities

Customer: West Midlands 5G 11th December 2020

Public

Picture

Contents.

5G Coverage

5G Throughput

5G Latency/Data Availability

Appendix: Methodology

Public

Data Collection.Data analysis of samples collected over a 4-month period.

311-Dec-20

Data collection period: July 20th (wk 30) – November 29th (wk 48)

KPI Definition

Coverage – 5G%% of tiles covered with 5G. The tile size is ≈ 100m x 100m. For a tile to be considered as covered at least 1 5G (NR)

sample is required.

Throughput Avg & P95 of throughputs from 5G capable users. A user is considered 5G capable when he has reported more

than 5 5G samples in a week

Data Availability % of successful connection tests when device is on 5G

Latency Avg latency when device is on 5G

Data collection methodology details per KPI are shown in slides 21-32.

Devices having contributed 5G data in the time period and polygons of interest

Device Name % of total samplesSG Note 10 24.5%SG S20 5G 22.7%

SG S20+ 5G 17.9%SG S20 Ultra (5G) 14.0%SG Note20 Ultra 6.1%

SG S10 5G 5.1%SG A90 5G 2.8%

SG Fold (5G) 1.9%Sony Xperia 1 II 0.8%

SG Note20 0.8%OPPO Find X2 Pro 0.6%

SG A51 5G 0.5%Huawei Mate 20 X (5G) 0.5%

SG S20 FE 5G 0.4%Motorola moto g 5G plus 0.2%

Pixel 5 0.2%Sony Xperia 5 II 0.2%

Oppo Find X2 Lite 0.2%SG Z Fold2 5G 0.1%

Others 0.5%

5G coverage and performance assessment

Public

5G Coverage.

Public

Combined Authorities: 5G%.West Midlands ranks 1st in 5G geographical coverage.

511-Dec-205G coverage and performance assessment

Public

Cities: 5G%.Birmingham ranks 4th in 5G geographical coverage.


Public

City maps: 5G%.Tiles with 5G coverage are shown in blue.

711-Dec-20

Manchester Liverpool


Public


811-Dec-20

Glasgow Birmingham


Public


911-Dec-20

Bristol London


Public

Combined Authority maps: 5G%.Tiles with 5G coverage are shown in blue.

1011-Dec-20

West of England Cambridgeshire and Peterborough


Public


1111-Dec-20

West Midlands Greater Manchester


Public


1211-Dec-20

Liverpool City Region North East


Public


1311-Dec-20

North of Tyne Sheffield City Region


Public


1411-Dec-20

Tees Valley West Yorkshire


Public

5G Throughput.

Public

Cities: DL/UL Throughput.Birmingham ranks 2nd in 5G DL/UL throughput behind Liverpool.


Public

Combined Authorities: 5G Throughput.Users in West Midlands experience 60% DL thr. improvement in their daily activities when connected on 5G.


Public

5G Latency/Data Availability.

Public

Cities: 5G Data Availability and Latency.


Public

Combined Authorities: 5G Data Availability and Latency.


Public

Methodology.

Public

Methodology: Combined Authorities polygons.Polygon set for the Combined Authorities.

225G coverage and performance assessment 11-Dec-20

Combined Authority polygons were extracted from:

https://data.gov.uk/dataset/d2094bbf-8c63-4218-95df-92e870b1eecc/combined-authorities-march-2017-full-clipped-boundaries-in-england

The latest available government polygon set is from 2017 and thus includes outdated boundaries for the North East combined authority. The newly established combined authorities include the following councils:

• North of Tyne: Newcastle, North Tyneside and Northumberland

• North East: Durham, Gateshead, South Tyneside and Sunderland

Council polygons were extracted from:

https://data.gov.uk/dataset/11302ddc-65bc-4a8f-96a9-af5c456e442c/counties-and-unitary-authorities-december-2016-full-clipped-boundaries-in-england-and-wales

https://data.gov.uk/dataset/d2094bbf-8c63-4218-95df-92e870b1eecc/combined-authorities-march-2017-full-clipped-boundaries-in-england

https://data.gov.uk/dataset/11302ddc-65bc-4a8f-96a9-af5c456e442c/counties-and-unitary-authorities-december-2016-full-clipped-boundaries-in-england-and-wales

Public

Methodology: Cities polyogns.Polygon set for the top 6 cities.

2311-Dec-20

City polygons for England:

Birmingham, London, Manchester, Liverpool and Bristol

https://data.gov.uk/dataset/7879ab82-2863-401e-8a29-a56e264d2182/major-towns-and-cities-december-2015-boundaries

Polygon for Glasgow:

https://data.gov.uk/dataset/1a9d2abf-9050-42ac-b508-02cf2f4532b2/localities-urban-areas


https://data.gov.uk/dataset/7879ab82-2863-401e-8a29-a56e264d2182/major-towns-and-cities-december-2015-boundaries

https://data.gov.uk/dataset/1a9d2abf-9050-42ac-b508-02cf2f4532b2/localities-urban-areas

Public

Coverage

Public

Coverage.GPS location information

Network Information use case

⚫ Network Information result use case is triggered after every GPS update from the operating system. For example, during a Google Maps navigation.

⚫ Our apps only trigger GPS during an active voice call to get exact location in case the call drops.

⚫ It is the use case with highest density of samples due to the fact that each sample is triggered every time there is a GPS update on the phone

⚫ Samples collected from both the Network Information and Connection Test use cases are included in the coverage analysis.

25

Network Information samples are produced every second, when GPS is enabled and updated.

The app with the umlaut SDK does not have to run actively. Installation is sufficient.

t

Trigger events:

1 GPS: Location update


Public

Throughput

Public

Network Traffic/Throughput Result (NTR).Passive Data Collection

⚫ Calculate and store the “fastest representative value” throughput rate within the 15min interval⚫ per traffic direction (UL/DL) and⚫ per technology (2G/3G/4G/5G/WiFi)

⚫ The used aggregation of the per-second throughput rate depends on the statistics (number of seconds) accumulated within the 15 minutes for each technology

⚫ The “fastest representative value“ is represented by a measurement point from the top data rates to avoid limited throughputs and outliers but always trying to keep the statistical re-semblance, calculated as follows:

Description

27

Filtering

Number of samples Percentile

< 10 No Throughput considered

[10 – 20] P80

[21 – 50] P95

> 51 P99

⚫ Passive collection of user experienced throughput is monitoring user activity

⚫ A big proportion of throughput samples are not reflecting the network capabilities as they are derived from less demanding data sessions (i.e. WhattsApp messaging)

⚫ As such, an analysis taking into account all throughput samples will result in artificially low values and mask any network performance differences

⚫ Filtering to filter out data sessions without sufficient data traffic:

Radio Access Technology Data Traffic Filter

2G Traffic bytes >= 500,000

3G Traffic bytes >= 2,000,000

4G & 5G Traffic bytes >= 10,000,000


Public

Data Availability

Public

Connection Test.Internet availability and accesibility

⚫ Download a small icon (596 bytes) via

TCP/SSL.

⚫ Test is executed every 15 min in the

background.

⚫ The test result is either successful,

representing mobile state internet state

(cellular or WIFI), or failed.

⚫ Connection test 4 stages:

⚫ DNS lookup

⚫ TCP Connect

⚫ HTTP received

⚫ Overall success

Connection Test description: Connection Test Sequence:Client

DurationDNS

DurationTCP connect

DurationSSL handshake

HTTPs GET

DurationHTTPs receive

DNS Server

Target:Multi-CDN

Server

Prox

yPr

oxy

DNS Lookup

DNS Response

SYNSYN /ACK

ACK

SSL Client Hello

SSL Initialization

HTTPs GET (HTTP 1.1)

HTTPs Receive

11-Dec-20 295G coverage and performance assessment

Public

Connection Test.Internet availability and accesibility

Restrictions:

⚫ Battery level must be higher than 15%.

⚫ Device / host app must not be in battery saving or

doze modes.

Metrics The test returns several metrics, e.g. :

⚫ DurationDNS: DNS lookup time in milliseconds

⚫ DurationTcpConnect: TCP connect time in milliseconds

⚫ DurationHttpReceive: Duration of file content transfer in

milliseconds

⚫ Success Per test stage; TRUE if test stage was successful


Public

Latency

Public

Latency Test Result.

11-Dec-20 32

Latency measures

⚫ A latency test is executed after every successful connection test (CT). This is perform by doing 10 pings to the Amazon or Akamai CDN addresses.

⚫ It is collecting the round trip time (RRT) for each of the pings and the results are the min, max, avg, med and jitter values of the pings

⚫ Latency Results with minValue < = 0 or maxvalue > 60000 are filtered out and also those where the number of successful pings does not match the number of executed pings. As such, only samples where each ping was successful and gives a reasonable latency value are used for subsequent analysis

⚫ Latency tests that started more than 3 seconds after Connection Test are also discarded

Description:

Considerations:

Sequence Schematic :

Delta

Connection Test (CT)

DNS

Latency Test (LTR)

LTR Timestamp onStart

Run 10 pings with 200 ms pause, 30 s timeout

(ping -i 0.2 -c 10 –W 30 …)

… … …i

RTT 1 RTT10

Close/ Write Log

LTR Timestamp onEnd

…


Public

Key contacts for further information.

Hakan Ekmen

+49 151 [email protected]

Velissarios Lataris


Efstathios Tsardinakis


11-Dec-205G coverage and performance assessment 33

Public

www.umlaut.com


Public 35

Disclaimer

This document and all information contained herein is the sole property of umlaut.No intellectual property rights are granted by the delivery of this document or the disclosure of its content. This document shall not be reproduced or disclosed to a third party without the express written consent of umlaut. This document and its content shall not be used for any purpose other than that for which it is supplied.


Documents

5G coverage and performance assessment in UK top cities