Wireline Video QoE Using U-vMOS - Ovum Ltd./media/Informa-Shop... · QoE (quality of experience) is a key service differentiator as home-based viewers have choices regarding their

Wireline Video QoE Using U-vMOS

OvumOvumTMT intelligence |

Sponsored by

The role of U-vMOS in QoE standardization

2 TMT intelligence informa © 2016 Ovum. All rights reserved.

ContentsSummary.............................................................. 3

Importance of QoE for wireline video leads to U-vMOS ..............................................3

Recommendations for service providers .......3

Introduction ......................................................... 4

Market overview: video services .....................4

Ovum video research findings ........................4

Video QoE: The need for a standard metric ......... 5

U-vMOS ............................................................5

U-vMOS measurement results ............................ 7

Measurement methodology ............................7

Results .............................................................7

Video QoE: Wireline consumer survey insights ................................................... 10

Wireline video quality matters to consumers.................................................10

The consumer perspective on current wireline video QoE ...........................10

Exploring different aspects of

consumer wireline video QoE .......................11

U-vMOS and qualitative analysis are important ..................................................... 13

Together, the service provider has a comprehensive QoE view ...................13

Conclusions........................................................ 14

U-vMOS becomes an important metric for QoE ...............................................14

Appendix ............................................................ 15

Methodology ..................................................15

About the authors

© Copyright Ovum 2016. All rights reserved.

The contents of this product are protected by international copyright laws, database rights and other intellectual property rights. The owner of these rights is Informa Telecoms and Media Limited, our affiliates or other third party licensors. All product and company names and logos contained within or appearing on this product are the trademarks, service marks or trading names of their respective owners, including Informa Telecoms and Media Limited. This product may not be copied, reproduced, distributed or transmitted in any form or by any means without the prior permission of Informa Telecoms and Media Limited. Whilst reasonable efforts have been made to ensure that the information and content of this product was correct as at the date of first publication, neither Informa Telecoms and Media Limited nor any person engaged or employed by Informa Telecoms and Media Limited accepts any liability for any errors, omissions or other inaccuracies. Readers should independently verify any facts and figures as no liability can be accepted in this regard - readers assume full responsibility and risk accordingly for their use of such information and content. Any views and/or opinions expressed in this product by individual authors or contributors are their personal views and/or opinions and do not necessarily reflect the views and/or opinions of Informa Telecoms and Media Limited.

Angel Dobardziev

Angel Dobardziev is a Consulting Director with Ovum’s Consumer, Media, and Entertainment Practice, where he provides custom advice and consulting services to telecoms and media service providers and technology vendors, in addition to contributing thought-leading research on the consumer, media, and entertainment markets.

Julie Kunstler

Julie Kunstler has over 25 years of experience in the communications components, equipment, and software industry. She is responsible for Ovum’s coverage of the wireline broadband access component, equipment, and software markets, closely following technology developments and subsequent effects on the vendor ecosystem. Julie is paying close attention to SDN/NFV and its impact on wireline broadband access.

Julie has worked as an executive, investor, and board member for communications companies. Having served in these numerous roles, Julie understands what it takes for new communications technologies to be deployed in large numbers and across multiple geographies. Julie brings her knowledge of the pressure on telcos to offer bandwidth-hungry applications along with the margin requirements of component vendors. Julie is a regular participant at industry events around the globe.

3TMT intelligence informa© 2016 Ovum. All rights reserved.

Recommendations for service providersQoE is gaining importance as a key competitive differentiator among wireline communications service providers – both telco and cable. Based on the quantitative and qualitative analysis, we believe that service providers need to significantly step up their abilities concerning wireline video QoE. Our recommendations include:•AdoptastandardizedandrobustQoEmeasurementtool,suchasU-vMOS,whichwillprovideconsistent

measurement regardless of end-user (viewer) expectations.•CombinetheuseofaQoEmeasurementtoolwithnetworkmonitoringandperformancetoolstoimprove

overall customer experience management.•ImplementQoEmeasurementsbeforeandafternetworkupgradesinordertodeterminewhichnetwork

elements have the greatest impact on QoE. •IntegratetheuseofQoEintoKPIstoensurethatmanagersthroughouttheorganizationarefocusedon

QoE using the same data.•EvaluateevolvingsubscriberexpectationsaroundqualitativeQoEandhowtheseexpectationsinfluence

the overall customer experience, customer retention, and churn.

SummaryImportance of QoE for wireline video leads to U-vMOSThe importance of watching high-quality content on the home TV continues to grow. Linear TV viewing is not going away and OTT content is joining traditional pay-TV content services. While content is king, QoE (quality of experience) is a key service differentiator as home-based viewers have choices regarding their service providers.

While QoE is not a new concept, few service providers have tried to measure end-user QoE. Instead, operators have resorted to increasing downstream bandwidth. However, more bandwidth does not necessarily improve QoE and with increasing sophistication of video services, audiences are becoming more demanding regarding picture quality, instantaneous channel changing, and multiscreen viewing, for example. In addition, viewers are influenced by previous viewing experiences and expectations.

Service providers are seeking a standard approach to measuring QoE across their wireline networks. They want to be able to test QoE over time and without relying on qualitative feedback from customers. Consequently, Huawei, a major communications and IT equipment, software, and services vendor, has developed a QoE measurement solution called U-vMOS (User/Unified/Ubiquitous video Mean Opinion Score).

The following sections examine the use of U-vMOS in measuring QoE for wireline-based video in three European countries – Germany, Spain, and Italy. In addition, the results of an extensive customer QoE survey are presented along with a comparison of the quantitative U-vMOS results to the qualitative survey responses.


IntroductionMarket overview: video servicesOvum research indicates that video is becoming a major driver for service revenues, for both content-driven telco operators and pure-access providers. The emergence of video as an important service is fairly recent, and telcos have treated video requirements on the telecom network similar to other services: overprovisioning throughout the telecom network stack. However, given that video uses more network resources, such as bandwidth, compared to other applications (e.g. voice, SMS, and Web access) and that the telecoms network is merely the last leg of the content journey, many telcos have realized that sufficient network capacity does not necessarily guarantee a positive video QoE. For example, systemic issues with set-top-box (STB) software may affect a hosted IPTV service severely even though the network is healthy.

The onset of 4K adds to the QoE challenges facing operators. Many operators cited network constraints; they believe that their networks are not technically capable of providing a smooth linear 4K customer experience. Yet, the successful launch of 4K TV services would provide a strong competitive advantage to an operator. A successful launch requires sufficient 4K content, seamless end-to-end networking, and superb QoE in order to garner premium tariffs.

One also needs to keep in the mind the high content premiums that operators pay for major sports content. Operators need to make massive investments to acquire the rights, and the favored format is live, linear broadcast. Return on investment requires consistent, high-quality viewer experiences.

As such, telcos are now attempting to understand QoE from the end-user perspective and act upon it, rather than merely monitoring network resources and reactively repairing faults or problems. This is no easy task according to the operators interviewed. These operators stated that there is no industry benchmark and that QoE metrics differ according to each vendor and consultant. Equally important, there is no QoE industry benchmark across operators, and this would help operators with competitive positioning, internal resource allocations, and KPIs.

According to initial Ovum findings, which include internal secondary and primary research, operators are still in the very early stages of understanding video QoE and of course, mapping relevant parameters to network KPIs and eventually, improving them. In fact, the interviewed operators have different processes in place to measure QoE and these may also differ between different operator companies at the same multinational operator group (more detail on the operator interviews below). While we expect convergence around QoE within the same operator group over time, the lack of standardized QoE measurements highlights the early stage of QoE in terms of definition, standardization, implementation, and widespread adoption.

Ovum video research findingsThe section below illustrates a few strong market trends that are shaping the telco video service market.

Growing sports rights fees are driving cutting-edge user experience. The favored premium format is the live, linear broadcast. BT Sport's launch of its 4K channel in September 2015 was a by-product of its large investment into soccer rights, as it sought to differentiate itself from its rivals. In addition, it assisted the NBA (National Basketball Association) to become the first sports broadcaster of 4K in North America. BT has broadcast more 4K live events than any other TV provider in Europe. Maintaining its newly found reputation as a pioneer in user experience, BT also trialed virtual reality (VR) technology with a hundred users, who were able to "virtually" place themselves at the heart of the action from various camera angles in the stadium. Such initiatives are expected to form part of the next generation of user experience in the long term, as outlined by BT Sport panelists at TV Connect 2016. Soon, the industry will see premium, second-tier, and tertiary offerings differentiated not only by the portfolio of sports on offer, but also by the user experience promised within the packages. Ovum expects that there will be no shortage of consumer demand and therefore opportunities will be created in all parts of the value chain.


Bottom line:•Operatorscanrealizereturnsinproportiontothesizeoftheirinvestments,whichiswhereoperators,

such as Vodafone UK, are keen to make entry-level investments. •Theenhancementoftheuserexperiencewillbecoupledwithongoingadjustments,optimizations,and

restructuring of packages. Operators should constantly evaluate and update their premium propositions to identify where the value lies and pivot their businesses around those centers of value.

Linear TV is alive. Over the past few years, much has been made of the shifts in people's viewing habits, with the success of OTT services and the emergence of smaller, second screens pushing consumers away from watching live content on the main TV set. However, 86% of viewing in the UK is still of live TV, and the use of mobile devices is still generally supplementing viewing rather than replacing existing viewing habits. This more positive outlook for linear TV presents a number of opportunities for those looking to work with OTT content, rather than against it.

Bottom line:•Bothpay-TVoperatorsandfree-to-airprovidersshouldlooktoensurethattheyarekeepingupwiththe

expectations of their audience by offering broadcast TV integrated with on-demand and catch-up services. •Pay-TVoperatorsshouldnotbeafraidtoworkinconjunctionwithOTTservices.Oneoptionisfor

operators to offer services like Netflix through their own platforms – a strategy that has already been adopted by Virgin Media, BT, TalkTalk, and EE in the UK.

•Conversely,thefactthatlinearTVremainsstrongalsomeansthatstandaloneOTTservicesfrompay-TVoperators – as Sky has found with Now TV – pose little risk of cannibalizing core pay-TV offerings and can contribute to further growth, albeit by attracting lower-ARPU customers.

Video QoE: The need for a standard metricVideo QoE is a subjective metric and depends on many intrinsic and external factors. For example, subscribers in markets with very high-speed broadband may perceive 1080p video as basic and with a lower video QoE compared to 4K. However, in a developing market, 1080p video may appear as the highest QoE imagined. Consequently, there needs to be reference video QoE models that quantify the user experience regardless of environment conditions. The ITU (International Telecommunication Union) has defined several of these models that are actually in use today by many vendors and telcos. However, none of these models can translate technical network parameters to actual perceived video QoE and none can identify what needs to be done in order for video QoE to be improved. Moreover, ITU formulas do not account for the interactive, immersive, and high-resolution video services that are being rolled out at the moment, meaning that there is a need in the market for a unified, standardized, user-facing metric.

U-vMOSQuantifying a natively qualitative experience is not easy. It takes time and research. Driven by requests from service providers, Huawei has developed a methodology called the User/Unified/Ubiquitous video Mean Opinion Score (U-vMOS) to help telcos measure the quality of the streaming video their subscriber customers receive.

In concept, U-vMOS is similar to the ITU video quality mean opinion score (MOS) methodology for measuring voice quality. Specifically, U-vMOS gives telcos a methodology for measuring the end user's video experience at the viewing device level.

U-vMOS focuses on three areas that need to be measured to quantify the end-user experience: •videoquality•interactiveexperience•viewingexperience.


Huawei assigns a numeric value to the different performance levels within each category. Using a formula it has developed, it assigns an overall number that signifies the overall quality of a streaming video session. Figure 1 shows the U-vMOS formula and how the different elements are rated.

The U-vMOS equation produces a score from 1 to 5, which translates to the following classification.

Huawei did not develop this formula for measuring QoE entirely on its own. The approach and resulting formula are from extensive research collaboration between Huawei and several universities. For example, Huawei conducted joint research with the Oxford Internet Institute and the School of Journalism and Communication of Peking University. Thousands of end users (viewers) were studied to see how they reacted to different streaming video performances and to help verify the methodology along with the model. Huawei received input on how to best standardize U-vMOS as a methodology for measuring the video service experience.

In Ovum's opinion, Huawei's methodology for measuring streaming video QoE is promising. U-vMOS appears to be a well-defined methodology, and the vendor has tested the methodology extensively with residential viewers.

The in-depth research behind the U-vMOS approach was critical because end users will often rate QoE based on their expectations from prior viewing experiences. For example, viewers will rate differently the same viewing experience in a controlled lab setting on the same TV viewing device. For example, some viewers may consider a three-second buffering delay to the start of a video to represent a good experience, especially if they are used to longer wait times. Conversely, other viewers may rate a delay of more than one second as being poor quality, especially if they recently upgraded their service to higher-bandwidth offerings, along with higher tariffs, to diminish buffering.

Table 1: U-vMOS video service ratings

U-vMOS score Rating

5 Excellent

4 Good

3 Fair

2 Poor

1 Bad

Source: Huawei

U-vMOSmodelingmethods

Video quality Interactive experience View experience

sQuality* sInteraction sView

U-vMOS = f (sQuality, sInteraction, sView)

*s = score

Figure 1: U-vMOS equation parameters

Source: Huawei


While the U-vMOS scoring would remain the same in this example, lower scores may be more acceptable in some markets than they are in others. Consequently, telcos need to be vigilant about changing market expectations from customers and potential customers. QoE analysis and subsequent network improvements need to be an ongoing process rather than simply a one-time test.

Next, Ovum discusses both quantitative U-vMOS measurements and an extensive qualitative survey conducted in Germany, Italy, and Spain.

U-vMOS measurement resultsIn order to quantify and place U-vMOS in a real-world context, Ovum partnered with Huawei to perform measurements in the following three countries: Germany, Italy, and Spain. The choice of these three countries was based on Huawei's ability to provide sufficient test measurements. The following sections describe the technical details of these tests along with key results.

Measurement methodologyThe U-vMOS network measurements were performed in various locations throughout each country, with a focus on the residential environment. Table 2 summarizes the number of U-vMOS samples collected in each country.

The measurement tool used was Huawei's SpeedPro, and the measurements were conducted in the home environment across these countries in the evening hours. Most users were subscribers of wireline broadband services, including fiber, xDSL, and cable, so that they resemble a typical user in these countries. SpeedPro is a native application installed in desktop or notebook computers. These computers were connected to a fixed broadband modem via a cable in order to remove possible Wi-Fi performance degradations.

The video resolutions tested were 720p and 1080p and some tests were run using higher resolutions, at 1440p and 2160p. However, the focus of the analysis is at 720p and 1080p resolutions, since these were the most common video resolutions across the measurement environments in the three countries.

ResultsAll individual measurements were grouped by country, in order to understand the actual level of video QoE in these countries. As will be discussed, some results correlated with the qualitative survey results. Other results contradicted survey responses, highlighting end-user expectations. The following quantitative test results illustrate some interesting trends.

Table 2: U-vMOS measurement samples

Country Number of U-vMOS measurements

Germany 357

Italy 617

Spain 748

Source: Huawei


GermanyFigure 2 illustrates U-vMOS scores from measurements performed in Germany.

As expected, higher-resolution videos perform better in the U-vMOS scale. Measurements in 720p and 1080p were much higher in number compared with all other resolutions, meaning that the wider variance illustrated in Figure 2 is expected. These measurements indicate that Germany's wireline networks were very capable of delivering 1080p, suggesting that telcos can prepare their networks to offer 4K services.

Average U-vMOS scores for all measurements were 3.02 and 3.36 for 720p and 1080p, respectively, corresponding to a fair rating in both cases.

ItalyFigure 3 illustrates measurements performed in Italy.

Italy's network performance was similar to that of German networks, with networks being more than capable of delivering 1080p videos. There was a great variance in 720p and 1080p measurements, but the average U-vMOS scores for these were 2.82 (poor) and 3.28 (fair), respectively.

U-v

MO

S sc

ores

Video resolution (pixels)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

1080720480360

Figure 2: Germany – U-vMOS measurement scores

Note: U-vMOS score: 1 = Bad, 2 = Poor, 3 = Fair, 4 = Good, 5 = ExcellentSource: Huawei and Ovum

U-v

MO

S sc

ores


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

1080720480

Figure 3: Italy – U-vMOS measurement scores



SpainFigure 4 illustrates the measurement results from Spain's networks.

The results from Spain illustrate that network performance follows similar patterns to both Germany and Italy. The average U-vMOS score for Spain's measurements are 3.06 and 3.19 for 720p and 1080p, respectively, corresponding to a fair rating in both cases.

Country comparisonThe following figure illustrates the mean scores for all measurements performed in each country.

Wireline networks in all three countries show that both 720p and 1080p are adequately delivered in the measurement environments. The U-vMOS scores increased from 720p to 1080p, as expected, but there are variations in the different country measurements. For example, among the three countries, Spain performed best in 720p but worse in 1080p, in contrast to Germany and Italy, which performed better for 1080p. This could mean that Spain's wireline networks perform better between 720p and 1080p, but Germany's and Italy's networks can support higher resolutions without perceived quality impairments.

Next, we move to the consumer survey insights.

Mea

n U

-vM

OS

scor

e

Germany Spain Italy

0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

1080

3.20 3.29

720

3.02 3.072.83

3.36

Measured video resolution (pixels)

Figure 5: Summary for Germany, Spain, Italy – U-vMOS measurement scores


U-v

MO

S sc

ores


0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

1080720

Figure 4: Spain – U-vMOS measurement scores



Video QoE: Wireline consumer survey insightsWireline video quality matters to consumersWireline video quality is a major issue for consumers, and remains a key consideration for their pay-TV service loyalty – or churn decision – today. Of course, content remains king in the visual entertainment industry. Consumers' desire to get the best deal on the best content has been, is, and is likely to remain a major issue for consumers – but they also want a top video quality experience to fully enjoy that content.

The survey asked the roughly one-third (31%) of the consumers who had changed pay-TV providers in the last year to select the single most important reason for the change. Almost half of the consumers across the three countries (48%) stated that the primary driver was to get a better content offering, as shown in Figure 6. Other key reasons for churn included to get a better deal and better picture quality (21% and 19%, respectively).

It is interesting to note that picture quality was a bigger issue for Germany, where for about a third (31%) of the respondents it was the primary motive for churning.

The consumer perspective on current wireline video QoEVery few consumers find their video QoE poor today, regardless of the platform they use, as outlined in Figure 7. However, we note that only a minority (19–29%) are truly delighted either. Indeed, the vast majority of consumers (69–76%) find their video QoE good or adequate. Consumer experiences are very similar overall across platforms, although paid Internet services seem slightly ahead of live IPTV and on-demand videos. The results are roughly similar across countries, although German consumers are slightly happier than Spanish consumers who in turn are slightly happier than Italian consumers, if the proportion of those that rated their overall experience as excellent is used as a guide, with 22%, 20%, and 16%, respectively, of the total.

On their own, the findings outlined in Figure 7 could be seen as a "glass half full" – good, with room for steady improvement – by the industry. But when viewed alongside the findings outlined in Figure 6, where picture quality was a major driver for churn for about a fifth of the consumers (more in Germany) that changed pay-TV providers in the last 12 months, the case for bigger, stronger, and faster video QoE improvements becomes much more urgent.

Res

pond

ents

(%)

Germany Italy Spain

Question 12. What was the primary motive for changing your pay-TV service?

0

10

20

30

40

50

60

OtherTo get better customer service

To get a better / cheaper deal

To get a better / more reliable

broadband service

To get a better content offering

To get a better picture quality

31%

17% 18%

39%43%

53%

14%

6%11% 14%

29%

16%

0% 0% 0% 3% 5%2%

Figure 6: Drivers for wireline consumer pay-TV churn

N = 468 for all countries. N for Spain = 234; N for Italy = 175; N for Germany = 59 Source: Ovum survey, August 2016


Exploring different aspects of consumer wireline video QoEWhile all of the different aspects of the video QoE in general, and on-demand video in particular, matter to consumers, some are more important than others, as outlined in Figure 8. Consumers care most about the smooth playing of the video (i.e. without buffering) and the overall picture quality, both of which were rated on average at 4.2 on a scale of 1–5, where 5 is most important. These two core factors are closely followed by video-loading speed, which consumers rated on average at 3.8.

Factors such as the interactive experience (e.g. channel change, or zapping time) and the quality of multiple video streams clearly matter less, at an average score of 3.4, and are of secondary importance to consumers. Interestingly, there are negligible differences across the three countries on these ratings.

Buffering remains a major issueBuffering, when the video unexpectedly pauses, remains a major video QoE detractor; approximately a quarter of the consumers in our survey regularly experience it. It is also a nuisance for approximately half that reported experiencing it infrequently, as outlined in Figure 9. Looked at another way, only a quarter of the consumers reported to have never experienced buffering, suggesting that network improvements are needed in this area.

Res

pond

ents

(%)

Live programs on your IPTV service On-demand videos Videos on your paid Internet service

Questions 20, 25, & 30. How would you rate your overall experience of watching the following?

3% 4% 2% 1% 2% 0%0

10

20

30

40

50

60

Very poorPoorAdequateGoodExcellent

22% 19%

29%

53% 53% 54%

21% 23%

15%

Figure 7: The overall wireline video QoE

N for IPTV service = 270; N for on-demand videos = 1,502; N for paid Internet service = 501 Source: Ovum survey, August 2016

Scal

e

Question 31. Can you please rate the importance of each of the following while watching on-demand videos on your TV? On a scale of 1‒5, where 5 is most important, and 1 is least important

0

1

2

3

4

5

Multiple video stream quality (i.e. player cam)

Interactive experience (e.g. changing channels)

Smooth playing of video (i.e. no buffering)

Video-loading speed

Picture quality

4.163.83

4.23

3.36 3.41

Figure 8: Importance of different video QoE components

N = 1,502 Source: Ovum survey, August 2016


We note that OTT services have the weakest QoE perception among different pay-TV services. This is to be expected, given that these services are over-the-top – unmanaged video services in many cases.

Looking across the countries, we note that German consumers report the highest occurrence of regular buffering, with Italy and Spain alternating positions behind Germany in terms of buffering incidence across the different pay-TV service platforms.

Picture quality perceived strongOn the whole, consumers have few issues with the picture quality, as shown in Figure 10. Approximately 8 out of 10 respondents have rated it good or excellent across different pay-TV services, and a negligible fraction (2%) rate their picture quality as poor or very poor.

There are, overall, few differences across the consumer perceptions of picture quality for on-demand videos of cable and satellite services. However, when it comes to IPTV and paid Internet services, a significantly higher percentage of German consumers rated picture quality much more strongly compared to Spanish or Italian respondents.

Res

pond

ents

(%)

Questions 16, 22, & 27. How often do you suffer from buffering (i.e. the video unexpectedly pausing) when watching the following?

0

10

20

30

40

50

NeverYes, but very occasionallyYes, it buffers now and againYes, it regularly buffers (i.e. at least once per video)

7% 8% 9%15% 17%

21%

48% 48% 48%

30%27%

22%

A live program via your IPTV service On-demand video A video from your Internet service

Figure 9: Wireline consumers’ buffering experience

N for IPTV service = 270; N for on-demand videos = 1,502; N for paid Internet service = 501Source: Ovum survey, August 2016

Res

pond

ents

(%)

Live programs on your IPTV service An on-demand video A video from your Internet service

Questions 17, 23, & 28. In general, how would you describe the picture quality you receive while watching the following?

0

10

20

30

40

50

60

Very poorPoorAcceptableGoodExcellent

27%

55% 54% 53%

17% 19%14%

1% 1% 1% 1% 0%2%

24%

32%

Figure 10: Wireline consumer – general picture QoE



Video load timeVideo load time is an area where consumer perceptions of pay-TV provider performance looks less than stellar. Figure 11 illustrates that, on average, only about a fifth (19–24%) report instant loading of long videos, with the majority reporting up to 5-second video-loading time – and a significant minority much longer than that. Looking across pay-TV service categories, it is interesting to note that paid Internet services (we used the Netflix example) perform slightly better than IPTV services, which in turn perform better than on-demand videos from paid cable and satellite TV services. Interestingly, a significantly higher proportion of Spanish consumers report instant loading of videos across all three pay-TV service categories versus the other two countries.

Consumer survey conclusionsVideo quality is clearly very important to consumers. Getting the best content at the best package is always a top concern for consumers, but only when this is accompanied by good video QoE. This survey indicates that pay-TV service providers have significant room for improving their video QoE, given that only a minority of the consumers are truly delighted with it, and for a significant proportion, video QoE is the main driver for their decision to switch providers.

Next, this survey indicates that picture quality, no buffering, and fast video-loading times are the most important factors for consumers, while other factors, such as channel change time and multiple video stream quality, are less critical. Consumers still report considerable issues with buffering and slow video-loading times, which indicates that these issues should be a key focus for improvement for pay-TV and network service providers.

U-vMOS and qualitative analysis are importantTogether, the service provider has a comprehensive QoE view Both qualitative (such as a survey) and quantitative QoE (such as U-vMOS) analysis are important to service providers. As briefly discussed earlier, wireline consumers are influenced by their expectations when it comes to rating video QoE experiences. Sometimes their ratings correlate with actual network measurements, while at other times their ratings are not strongly correlated to network measurements. Qualitative measurements highlight perceived areas of satisfaction and dissatisfaction, while U-vMOS provides quantitative measurements. Together, they provide service providers a comprehensive view of both perceived and actual QoE.

Res

pond

ents

(%)

Your IPTV service An on-demand video An Internet service

Questions 15, 21, & 26. How long would you say it takes for a long video to load on your main TV when using/watching the following?

0

5

10

15

20

25

30

35

40

Longer than 10 secondsWithin 10 secondsWithin 5 secondsWithin 3 secondsInstantly

24%21%

19%

28%

21%

36%

21%24%

29%

14% 15%

10%13%

19%

6%

Figure 11: Wireline consumers’ video load-time QoE



When we dig deeper into the qualitative survey findings, we discover several correlations between wireline access bandwidth and video load time, buffering times, and overall viewing experience, as shown in Table 3. This table provides a summary of selected responses for those respondents on 100Mbps or higher networks versus those on 1–10Mbps networks. As expected, the answers to these questions reflect that video viewers on 100Mbps or more of broadband speed led to faster times or higher overall experiences than video viewers on the low-speed tier of 1–10Mbps.

For other questions, the highest speed tier did not lead to meaningful better experiences or faster times. When asked about the time to change programs on an IPTV service, 27% of respondents with broadband speed of 100Mbps or higher responded "instantly," compared to 26% for those in the 1–10Mbps tier.

The perception of time is highly subjective and is often based on expectations. Consequently, the use of quantitative measurements, such as U-vMOS, along with surveys provides service providers with both actual video QoE and perceived performance data.

Table 3: Comparison of selected answers based on download speeds

Question 100Mbps or more answers

1–10Mbps answers

How long does it take for a long video to load when using your IPTV service? (question 15) 43% = instantly 16% = instantly

How often do you suffer from buffering issues when watching a live program via your IPTV service? (question 16) 37% = never 24% = never

In general, how would describe the picture quality you receive while watching a live program via your IPTV service? (question 17) 43% = excellent 20% = excellent

When watching multiple video streams on your IPTV service, what is the picture quality you receive? (question 19) 29% = excellent 20% = excellent

What is your overall experience of watching live programs on IPTV service? (question 20) 37% = excellent 20% = excellent

How long would you say it takes to change programs when watching on-demand video? (question 24) 23% = instantly 16% = instantly

Source: Ovum

ConclusionsU-vMOS becomes an important metric for QoEAs wireline video dominates network consumption and as viewers rate content as king, service providers must incorporate robust measurement tools to quantify QoE. While qualitative surveys can provide guidelines concerning customer satisfaction, they are not an accurate, consistent, or universal tool for clear differentiation across bandwidth tiers or for identifying key access network performance metrics.

U-vMOS represents a robust tool that can be adopted by wireline service providers, enabling them to move past viewer perceptions to true network performance. The U-vMOS approach is comprehensive, because it covers the key areas of QoE for video viewers. In addition, U-vMOS can be used by service providers to highlight areas where access networks are not performing well, enabling the service providers to focus their network improvements on areas that directly impact QoE – a key element of customer satisfaction.

Furthermore, with U-vMOS, service providers have an objective, repeatable test measurement that can be incorporated into KPIs.


AppendixMethodologyThe network measurements took place between August and September 2016 and were conducted in metropolitan areas in the same countries in which the survey took place: Germany, Italy, and Spain. The measurements were performed during the evening, to mimic typical consumer behavior and were conducted through a variety of broadband connections, including DSL, cable, and in some cases, fiber. The measurements were conducted using Huawei's SpeedPro tool in personal computers, by friendly users. To avoid Wi-Fi interference and degradation issues, these computers were connected to the broadband modem via a cable.

The consumer survey included 1,502 consumers, with nationally representative samples (age and gender) of 500/501 respondents each in Germany, Italy, and Spain, who had a broadband connection and a pay-TV subscription at home. The online survey fieldwork was carried out during end of August and beginning of September 2016.

Figures 12, 13, and 14 provide additional detail on the survey respondent demographics.

Res

pond

ents

(%)

Question 3. Age of respondents

16-24 25-34 35-44 45-54 55+

0

20

40

60

80

100

SpainItalyGermany11%

19%

21%

28%

21%

12%

22%

23%

27%

16%

12%

22%

25%

24%

17%

Figure 12: Age of the survey respondents

N = 1,502 across three countries, 500/501 in each countrySource: Ovum survey, August 2016

Res

pond

ents

(%)

Question 4. Gender of respondents

Male Female

0

20

40

60

80

100

SpainItalyGermany

52%

48%

47%

53%

48%

52%

Figure 13: Gender of respondents


Res

pond

ents

(%)

Over 100Mbps Don’t know

Question 7. What is the maximum broadband speed you have at home?

Less than 1Mbps Between 1Mbps to 10Mbps Between 10Mbps to 50Mbps Between 50Mbps to 100Mbps

0

20

40

60

80

100

SpainItalyGermany

Figure 14: Broadband connection speed of respondents


ABOUT OVUMOvum is a leading global technology research and advisory firm. Through its 180 analysts worldwide it offers expert analysis and strategic insight across the IT, telecoms, and media industries. Founded in 1985, Ovum has one of the most experienced analyst teams in the industry and is a respected source of guidance for technology business leaders, CIOs, vendors, service providers, and regulators looking for comprehensive, accurate and insightful market data, research and consulting. With 23 offices across six continents, Ovum offers a truly global perspective on technology and media markets and provides thousands of clients with insight including workflow tools, forecasts, surveys, market assessments, technology audits and opinion. In 2012, Ovum was jointly named Global Analyst Firm of the Year by the IIAR.

For more details on Ovum and how we can help your company identify future trends and opportunities, please contact us at [email protected] or visit www.ovum.com. To hear more from our analyst team join our Analyst Community group on LinkedIn www.ovum.com/linkedin and follow us on Twitter www.twitter.com/OvumTelecoms.

OvumOvumTMT intelligence |

Documents

Wireline Video QoE Using U-vMOS - Ovum Ltd./media/Informa-Shop... · QoE (quality of experience) is a key service differentiator as home-based viewers have choices regarding their