Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
@THEUAETRA
www.tra.gov.ae
MEDIA Report 2015
Mobile Network QoSBenchmarking Campaign
02MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
The TRA is committed to ensure both quality and variety in telecommunication services throughout the UAE. This is an essential part of the long term plan to help the UAE become a recognized regional ICT hub.
The TRA is also keen to ensure that the terms of licensees are well adhered to. The Telecommunications
Regulatory Authority (TRA) of the United Arab Emirates (UAE) has carried out an extensive benchmark-ing survey of the Mobile networks and services provided by mobile licensees (Etisalat and du).The survey has covered more than 17,000 Km of the UAE’s unique roads between November – December, 2015 at peak hours during working days.
All of the emirates including: Abu Dhabi, Dubai, Sharjah, Ajman, Um Al Quwain, Ras Al Khaimah, Fujairah, the Western Region and major highways were covered. The TRA has performed thousands of test calls during the survey in addition to similar tests were conducted indoor for 34 static venues that are covered in this Indoor Report.
Background
03MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
The survey addressed 2nd, 3rd and 4th generation mobile networks for both licensees (du and Etisalat). Key performance indicators used in the survey include: 1. Call Completion Success Rate 2. Call Setup Success Rate 3. Call Drop Rate 4. Voice Quality 5. Service Coverage 6. Data Transfer Rate
Definition of User Simulation QoS
1. Call Completion Success Rate: Is the measure of calls that were successfully set up and normally terminated, note that a high Call Completion Success Rate is desirable.
2. Call Setup Success Rate: Percentage of attempts that resulted in successful, note that a high Call Setup Success Rate is desirable.
3. Call Drop Rate: This refers to the disconnection of mobile calls by the network during a 120-second call-holding period for each call, note that a low Call Drop Rate is desirable. The target is to achieve a ratio of less than 2% in the index of the dropped calls.
% Dropped calls =Number of Dropped Calls / Number of Good
Initiations
4. Voice Quality: The overall voice quality rate is equal to the average voice quality on the downlink and uplink which refers to the network›s ability in achieving an acceptable level of voice quality using the Mean Opinion Score (MOS) measure and 2.8 score has been set as the MOS threshold, note that a high Voice Quality Rate is desirable.
5. Service Coverage: This is based on signal strength and refers to the network›s ability in achieving a signal strength of -100 dBm or higher.
6. Data Transfer Rate at which data is transmitted over the application protocol levels. Provided in megabit-per-second Mbps. Both FTP & HTTP have been tested:
- FTP (File Transfer Protocol): used to upload files from a workstation to an FTP server or download files from an FTP server to a workstation. - HTTP (Hyper Text Transfer Protocol): used to transfer files from a Web server onto a browser in order to view a Web page that is on the Internet.
What was measured?
04
To ensure that the testing provides a fair competitor comparison of the service provided by the mobile networks and that it provides a reliable basis for the en-couragement of ever improving quality of service throughout the UAE, the TRA has employed “state of the art” test equipment and services. By using latest equipment; the TRA ensured the measurements where done in a manner, that was statistically valid, repeatable, technically consistent, providing absolute comparability of KPIs for all networks, technology independent, using ITU & ETSI standards as a minimum and are robust enough to stand legal scrutiny of claims.
The equipment measures “key performance indicators” that directly relate to the public’s experience through Outdoor (in-car user experience) and Indoor user experi-ence. These include network coverage, dropped calls and the clarity of calls.The ratio of the voice calls attempts in regard to Mobile Originated Calls /Mobile Terminated Calls (MOC/MTC) was 1/1. The call test scenario consisted of a 120 sec-onds holding period followed by a 30 seconds idle time. The measurement systems included both the older “featured phones” as well as the latest “smartphones”.
Results InterpretationIn interpreting the results, it should be noted that:• The drive test results represent a snapshot of the mobile service provider›s
network in-car user experience based on the specified routes during the time of day when the measurements were carried out and using a particular type of handset. The reported level of service quality may therefore not be exactly comparable with the consumer›s own experience;
• Consumers should not treat the drive test results as recommended best buys. Factors such as price plans, value added services, customer care and support are not captured in this report.
Methodology
05MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Survey Results 1- Outdoor
Call Completion Success RateThis parameter measures the capability of the network to successful setting up a call and normally terminating it. The call completion success rate shown below is based on more than 10,000 voice call attempts per operator.
Call Setup Success Rate This parameter measures the capability of the network to suc-cessful setting up a call normally. The call completion success rate below is based on more than 10,000 voice call attempts for each operator.
Figure 3: Call Setup Success Rate in Outdoor Coverage
95.0% 95.0%
97.5% 97.5%
100.0% 100.0%
98.96% 99.56%99.76% 99.85%
Etisalat Etisalatdu du
06MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Call Drop RateThis parameter measures the capability of the network to successful setting up a call and normally terminating it. The call completion success rate shown below is based on more than 10,000 voice call attempts per operator.
0.0%
90.0%
0.5%
95.5%
1.0%
100.0%
0.60%
96.09%
0.09%
95.52%
Etisalat
Etisalat
du
du
Voice QualityThe MOS method is a means to provide an objective result of voice quality of calls in the operator’s network. In each call, speech samples are sent to the uplink and downlink direction. These samples are then analyzed in both directions to measure the speech quality using the ITU-T Rec. P.863 (POLQA) algo-rithm. This is a recognized method widely used to evaluate voice quality showing the following categories:
Conversation description
Understandable without interference noise
Understandable with minor interference noise
Fairly understandable
Requires a lot of effort to understand
Not understandable
Quality
Excellent
Good
Acceptable
Not good
Poor
Result (MOS)
5
4
3
2
1
Table 1: MOS score values.Based on the definition that an acceptable level of voice quality is one with an opinion score of at least “2.8”, the percentage of samples with opinion scores greater than “2.8” for each mobile network is as shown:
Figure 5: Voice quality (MOS) in outdoor coverage
07MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Service CoverageThis parameter measures the availability of service coverage in the regions. Therefore, it measures the network’s ability to achieve the minimum signal strength of -100dBm.
95%
98%
100%
99.10%98.84%
99.87% 99.67%
Etisalat Etisalatdu
du
Figure 6: Coverage level in the *Single Mode (2G) and **Dual Mode (2G/ 3G)* Single Mode: Here the mobile device is locked to the 2G Network (GSM) and not allowed to handover to the 3G Network
(UMTS).** Dual Mode: Here the mobile device is allowed to handover between both 2G and 3G networks based on the received signal
from the network. Data Transfer RatePacket Data Performance (Throughputs): Rate at which data is transmitted over the application protocol levels. Provided in kilo-bit-per-second (kbps), where 1 kilobit = 1000 bits. Both FTP & HTTP have been tested.• FTP (File Transfer Protocol): used to upload files from a Mobile Device to a FTP server or download files from a FTP server to a
Mobile Device. File size of (20 MB) where used for the test.• HTTP (Hyper Text Transfer Protocol): used to transfer files from a Web server onto the Mobile Device. File size (10 MB) where
used for the test.
Single Mode (2G) Dual Mode
08MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
a-Data transfer rate in Dual Mode(2G / 3G)
b-Data transferrates in LTE 4G
0
0
4
20
2
10
6
30
8
40
1
5
5
25
3
15
7
35
910
du
du
Etisalat
Etisalat
7.3
32.8
0.6
1.9
19.6
0.3
8.7
34.7
1.9
3.8
24.6
0.7Figure 7: Data transfer rate in Dual Mode (2G / 3G) in outdoor coverage
Figure 8: Data transfer rates in LTE (4G) in outdoor coverage
FTP DL
FTP DL
FTP UL
FTP UL
HTTP DL
FTP DL
HTTP UL
FTP UL
Figure 8: Data transfer rates in LTE (4G) in outdoor coverage
09MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Call Completion Success Rate
Overall
Call SetupSuccess Rate
Overall
95%
95%
98%
98%
100%
100%
du
du
Etisalat
Etisalat
2012
2012
2012
2012
2013
2013
2013
2013
2014
2014
2014
2014
2015
2015
2015
2015
98.62%
99.00%
98.27%
98.81%
98.82%
99.09%
98.56%
99.12%
98.55%
98.84%
98.59%
98.83%
98.96%
99.56%
99.76%
99.85%
Mobile Network Performance Trends
(between 2012, 2013, 2014, & 2015) in the Outdoor Survey The below graph (Figure 9) compares the performance of both Etisalat’s and Du’s mobile networks in terms of the major Quality of Service Parameters (between 2012, 2013, 2014, & 2015)
10MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Call Drop Rate
Overall
Voice Quality
Overall
0.0%
90%
0.5%
95%
1.0%
100%
du
du
Etisalat
Etisalat
2012
2012
2012
2012
2013
2013
2013
2013
2014
2014
2014
2014
2015
2015
2015
2015
0.35%
97.48%
0.49%
94.96%
0.27%
97.50%
0.56%
95.13%
0.30%
97.02%
0.25%
95.33%
0.60%
96.09%
0.09%
95.52%
11MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Per-task Harmonic MeanThroughput-Etisalat (Dual Mode)
Per-task Harmonic MeanThroughput-Du (Dual Mode)
0.00
0.00
4.00
4.00
8.00
8.00
2.00
2.00
6.00
6.00
10.00
1.00
1.00
5.00
5.00
9.00
3.00
3.00
7.00
7.00
0.73
0.25
3.80
1.90
1.90
0.60
8.70
7.30
0.72
0.85
5.63
5.57
0.68
0.79
5.72
4.58
0.29
0.16
0.70
0.58
0.35
0.36
1.93
2.27
0.23
0.20
1.00
0.69
0.19
0.24
1.79
2.97
2012
2012
2013
2013
2014
2014
2015
2015
FTP DL
FTP DL
FTP UL
FTP UL
HTTP DL
HTTP DL
HTTP UL
HTTP UL
Figure 9: Mobile Network Performance Trends (between 2012, 2013, 2014, & 2015) in outdoor coverage
12MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Call Completion Rate %
Call Drop Rate %
Call Setup Rate %
Voice Quality MOS
95.0%
0.0%
95.0%
90.0%
97.5%
0.5%
97.5%
95.0%
100%
1.0%
100%
100%
du
du
du
du
Etisalat
Etisalat
Etisalat
Etisalat
99.25%
0.70%
99.65%
98.31%
1- Indoor The TRA has conducted an Indoor Benchmarking Survey similar to the Outdoor Benchmarking Survey to a number of 34 Static Venues. The results have shown that the licensees (Etisalat and Du) achieved very good results in the KPI’s of voice and data services in 2G, 3G, and 4G.
99.74%
0.13%
99.87%
97.52%
Figure 10: Voice performance (overall) in Indoor Locations
13MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
0.0 0.00
4.0
20.00
35.00
2.010.00
6.0
30.00
45.0050.00
1.05.00
5.0
25.00
40.00
3.0
15.00
du duEtisalat Etisalat
4.9
Data Transfer Rate a- Data transfer rate overall (2G / 3G/ 4G)
0.7
2.1
0.4
5.7
1.1
3.0
0.3
39.46
10.51
46.12
10.45
Figure 14: Data transfer rates overall (2G / 3G / 4G) in indoor locations
Figure 15: Data transfer rates overall (4G) in indoor locations
FTP DL FTP DLFTP UL FTP ULHTTP DL HTTP UL
14MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Mobile Network Performance Trends(between 2012, 2013, 2104, & 2015) in the Indoor Locations
Call Completion Success Rate
Overall
Call SetupSuccess Rate
Overall
95%
95%
98%
98%
100%
100%
du
du
Etisalat
Etisalat
2012
2012
2012
2012
2013
2013
2013
2013
2014
2014
2014
2014
2015
2015
2015
2015
98.42%
99.02%
99.07%
98.29%
99.07%
99.63%
99.06%
99.57%
98.80%
98.42%
98.89%
99.24%
98.95%
99.65%
99.74%
99.87%
15MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Call DropRate
Overall
Voice QualityMOS
Overall
0.0%
90%
0.5%
95%
1.0%
100%
du
du
Etisalat
Etisalat
2012
2012
2012
2012
2013
2013
2013
2013
2014
2014
2014
2014
2015
2015
2015
2015
0.61%
96.89%
0.44%
97.81%
0.56%
96.27%
0.52%
98.41%
0.63%
96.31%
0.36%
97.51%
0.70%
98.31%
0.13%
97.50%
16MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
Per-task Harmonic MeanThroughput-Etisalat (Dual Mode)
Per-task Harmonic MeanThroughput-Du (Dual Mode)
0.00
0.00
4.00
4.00
8.00
8.00
2.00
2.00
6.00
6.00
10.00
1.00
1.00
5.00
5.00
9.00
3.00
3.00
7.00
7.00
0.73
0.25
3.80
1.90
1.90
0.60
8.70
7.30
0.72
0.85
5.63
5.57
0.68
0.79
5.72
4.58
0.29
0.16
0.70
0.58
0.35
0.36
1.93
2.27
0.23
0.20
1.00
0.69
0.19
0.24
1.79
2.97
2012
2012
2013
2013
2014
2014
2015
2015
FTP DL
FTP DL
FTP UL
FTP UL
HTTP DL
HTTP DL
HTTP UL
HTTP UL
Figure 16: Mobile Network Performance Trends(between 2012, 2013, 2104 & 2015) in the Indoor Locations
17MEDIA Report 2015 – Mobile Network QoS Benchmarking Campaign
ConclusionIt should be noted that the results were benchmarked against other similar Benchmarking Surveys con-ducted by other countries worldwide. The telecom sector in UAE represented by the Licensees (Etisalat and Du) has achieved a competitive performance which has exceeded other countries in some KPI’s such as Call Completion Success Rate, Call Drop Rate, and Data Transfer Rate. Outlook; Both Licensees (Etisalat and Du) have challenging times ahead; with the demands on networks changing due to the continued explosion of Mobile data traffic and the increasing complexity of the network. It is TRA aim to provide continued feedback on the mobile network quality of its Licensees in its aim to
provide a reliable basis for continual improving quality of service throughout the UAE. And help UAE become a recognized regional ICT hub.