Optimizing Quality-of-Experience for HTTP-based Adaptive ...ivms.stanford.edu/~zhuxq/papers/ijmdem2016.pdf · and different flavors of QoE optimization objectives. Extensive evaluations

DOI: 10.4018/IJMDEM.2016100102

Copyright © 2016, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

International Journal of Multimedia Data Engineering and ManagementVolume 7 • Issue 4 • October-December 2016

Optimizing Quality-of-Experience for HTTP-based Adaptive Video Streaming:An SDN-based ApproachSangeeta Ramakrishnan, Cisco Systems, San Jose, CA, USA

Xiaoqing Zhu, Cisco Systems, San Jose, CA, USA

Frank Chan, Cisco Systems, San Jose, CA, USA

Kashyap Kodanda Ram Kambhatla, Cisco Systems, San Jose, CA, USA

Zheng Lu, Cisco Systems, San Jose, CA, USA

Cindy Chan, Cisco Systems, San Jose, CA, USA

Bhanu Krishnamurthy, Cisco Systems, San Jose, CA, USA

ABSTRACT

In this work, the authors present a novel bandwidth management solution for optimizing overall quality of experience (QoE) of multiple video streaming sessions. Instead of allocating bandwidth equally among competing flows, they propose to tailor the bandwidth allocation to both content complexity of requested video and playout buffer status of individual clients. The authors formulate the multi-client bandwidth allocation problem within the convex optimization framework, which is flexible enough to accommodate a wide variety of video quality metrics. Further, the authors present a practical architecture based on software defined networking (SDN) with two components: video quality monitoring and video quality optimization. Testbed-based experiments confirm that with quality-optimized allocation the network can support up to 75% more users at the same level of quality-of-experience (QoE) than conventional equal-rate allocations.

KEywORDSConvex Optimization, HTTP-based Adaptive Streaming (HAS), Quality-of-Experience (QoE), Software Based Networks (SDN), Video Quality Analytics

1. INTRODUCTION

Over the past few years, video traffic has been growing rapidly and is anticipated to dominate next-generation networks. Globally, video traffic will comprise 80% of all consumers Internet traffic in 2019, up from 64% in 2014 (Cisco Systems Inc., 2015). Service providers and network operators alike are seeking novel solutions to fend off the impending bandwidth crunch introduced by video traffic on their existing network infrastructure.

Today, the technology for video streaming over the Internet is converging towards a paradigm named HTTP-based adaptive streaming of video. Figure 1 depicts the overall architecture of such a system. A HTTP-based adaptive streaming client can dynamically change video rate and quality on a per-segment basis. It typically switches to a low-quality version of the video to avoid buffer underflow during temporary network congestion. HTTP-based adaptive video streaming is widely deployed in commercial systems, including Netflix, Akamai, Microsoft Smooth Streaming (Zambelli, 2009), Apple HTTP Live Streaming (HLS) (Apple Inc., 2016), and Adobe HTTP Dynamic Streaming

22


23

(HDS). In 2012, the Motion Picture Experts Group (MPEG) joined forces with 3GPP (3rd Generation Partnership Project) in defining the recently standardized Dynamic Adaptive Streaming over HTTP (DASH) specifications (MPEG, 2012). MPEG-DASH has intentionally left out of its scope the definition of client behavior for content fetching, rate adaptation strategies, and video playout, thereby leaving room for innovation-based competition in industry.

In this work, we consider how to allocate bottleneck bandwidth across multiple competing HTTP-based adaptive video streams. We note that for streaming video, complex video contents (e.g., action movies) require a higher data rate to achieve the same perceptual quality than sequences with more static scenes (e.g., talking head in news). Therefore, it is preferable to equalize video quality amongst video streams to maximize the overall quality-of-experience (QoE) for all users. Furthermore, the bandwidth allocation decision should avoid playout buffer underflow at video clients. Effectively, such an approach achieves statistical multiplexing not only across different video streams, but also across temporal content variations within each stream.

We investigate two aspects of optimizing QoE for HTTP-based adaptive video streaming: system architecture and intelligent bandwidth management algorithms. We envision an architecture based on software-defined networking (SDN), where a video QoE optimization application (VQOA) collects information from various points in the network and analyzes it to provide more accurate estimates of end-user QoE. The VQOA is also in charge of coordinating rate adaptation decisions across competing HTTP-based adaptive video streaming clients while dynamically allocating bandwidth at the bottleneck link.

We formulate the joint video rate selection and bandwidth allocation problem within a convex optimization framework. It is flexible enough to accommodate a wide variety of video quality metrics and different flavors of QoE optimization objectives. Extensive evaluations from simulations and testbed-based experiments confirm that with quality-optimized allocation the network can support up to 75% more users at the same level of quality-of-experience (QoE) than conventional equal-rate allocations.

The rest of the paper is organized as follows. The next section discusses related work. Section 3 provides an overview of our proposed SDN-based architecture. Section 4 presents the quality optimization framework. Section 5 describes the practical implementation of the proposed bandwidth allocation scheme. Sections 6 and 7 evaluate the proposed solution via extensive simulations and testbed-based experiments.

Figure 1. Overall system architecture for HTTP-based adaptive streaming of video


24

2. RELATED wORK

In this section, we discuss related research works that optimize QoE of video streaming services. They roughly fall into three categories: a) network-centric approaches based on software defined networking (SDN); b) client-centric approaches which focus on the rate adaptation logic in HTTP-based adaptive video streaming clients; and c) joint consideration of network operations and client behaviors. The proposed solutions in this paper belong to the first and third categories. They stand apart from existing research by leveraging an SDN framework to enable QoE-driven dynamic bandwidth allocation amongst HTTP-based adaptive video streaming clients.

The concept of SDN can be traced back to the mention of OpenFlow in 2008 (McKeown et al., 2008), and has gained wide adoption in industry since then. The main advantage of SDN lies in the separation of control plane from the data plane, which allows for network visibility, dynamic resource provisioning, and flexible service deployments. For instance, Nam et al., (2014) propose an SDN platform for over-the-top video service providers, so that they can pinpoint and avoid congested bottlenecks by dynamically selecting paths for HTTP-based adaptive video streaming traffic. The GENI Cinema system presented in (Wang et al., 2014) provides a live video streaming service. It supports on-demand instantiation of video relay servers in a distributed cloud environment, and can dynamically steer traffic via SDN. These efforts focus on the dynamic path selection capability of SDN. In contrast, our paper leverages SDN for its ability to coordinate quality-driven dynamic bandwidth allocation across competing HTTP-based adaptive video streams.

A large body of work exists for improving the rate adaptation logic of HTTP-based adaptive video streaming clients. Earlier attempts (Liu, Bouazizi, & Gabbouj, 2011; De Cicco, Mascolo, & Palmisano, 2011) propose novel rate adaptation algorithms with several competing performance goals: higher average rate and video quality, reduced rate oscillations, and minimal client re-buffering. Hu et al. (2014) further propose to perform content-aware adaptation from the perspective of a single client. Akhshabi & Begen (2012) have observed that multiple competing clients tend to suffer from frequent quality oscillations. This has inspired new designs of client rate adaptation algorithms to also strive for fairness and stability across multiple clients (Jiang, Sekar, and Zhang, 2012; Zhu et al., 2013; Li et al., 2014). While these works share the same goal as our paper in aiming to improve QoE of HTTP-based adaptive video streaming, they focus only on client rate adaptation logics without modifying network operations.

The third category of related research share the same spirit as our work in jointly considering network and client operations. For instance, Ma & Bartos (2011) propose a congestion-aware rate adaptation scheme that augments existing HTTP-based segment-by-segment delivery protocols to provide network bandwidth management and to ensure fairness. Houdaille & Gouache (2012) use traffic shaping in the residential gateway as a mechanism to implement bandwidth arbitration amongst competing video clients sharing the same home network. This approach is similar to the equal-rate allocation scheme discussed in our paper. In (Essaili et al., 2013), the authors present a proactive QoE based approach for rewriting client HTTP requests at a proxy in the mobile network. It also mentions the idea about shaping TCP throughput for each client according to the QoE optimizer feedback. These two methods are similar to the approaches described in our paper in that they either explicitly modify client rate adaptation behavior, or dynamically perform quality-optimized bandwidth allocation. However, this paper does not present a framework to combine them together. Joseph & de Veciana (2014) present a scheme for joint optimization of network resource allocation and video quality adaptation for video delivery over HTTP-based adaptive streaming. The joint optimization approach is similar to the one proposed in our paper, except that our proposed solution leverages SDN to implement a centralized approach.

This paper extends an earlier version of the work (Ramakrishnan et al., 2015) by presenting one more variant of the optimization scheme that does not involve modifying client behaviors: quality-aware bandwidth allocation. Additional testbed-based evaluations are carried out to evaluate several variants of the optimization schemes, and to study the impact of different HTTP-based adaptive streaming clients and different bandwidth allocation update intervals.


25

3. ARCHITECTURE OVERVIEw

We envision an SDN-based architecture for supporting QoE-based analytics and optimization in HTTP-based adaptive video streaming. As illustrated in Figure 2, the VQOA interacts with the SDN controller and collects information regarding various entities in the network. The VQOA may obtain network topology and device types (e.g., smart phones vs. set-top-boxes) from the SDN controller, metadata of requested video contents from the streamer, and receiver playout buffer status from clients. It can then aggregate and publish video QoE statistics at a central location, dynamically adjust network bandwidth allocation at the bottleneck position, and coordinate video rate selections amongst competing clients.

The video QoE analytics can be used by operators for identifying most congested network segments within the video delivery network to be upgraded first, for debugging ongoing QoE-related issues in the field, and for proactively preventing QoE-related complaints from content subscribers. The proposed QoE-based optimization solutions enable operators to optimally allocate network resource to improve end-user QoE. They can dynamically tune bandwidth allocation according to user’s groups based on device type (e.g., STB vs. tablets vs. smart phones), on codec in use (e.g., HEVC vs. MPEG-4), or on video content complexity (e.g., sports vs. talk shows). Business policies such as premium subscriber or premium content may also influence the QoS configuration that each flow receives. As explained later in the next section, VQOA can use any other information — e.g., bitrate and video quality scores carried within the Media Presentation Description (MPD) or as companion meta data — to optimize QoE of individual video streams, so as to achieve certain overall performance objectives.

4. QUALITy OPTIMIZATION FRAMEwORK

In this section, we present a convex optimization framework for joint bandwidth allocation and video rate selection.

A. Network Bandwidth Sharing Model

Figure 2. SDN based architecture for quality-of-experience (QoE) optimization in HTTP-based adaptive video streaming


26

Consider N video streams sharing a bottleneck link of capacity C . We assume that the centralized controller is capable of updating the bandwidth allocation to each stream on a periodical basis, once per ∆t seconds. At time t , the stream i is assigned a bandwidth of c

i t,. Obviously, the total bandwidth

assigned to all streams should always be below the link capacity: ii tc C t∑ < ∀

,, . In our formulation,

we consider a time horizon of T seconds, corresponding to T t/∆ time slots.

B. Video Quality Model

In HTTP-based adaptive streaming, video contents are requested and downloaded by the client on a per-segment basis. For each segment, the client can choose from K different rate-quality levels. We denote the segment duration as τ , and consider a total of M T=� / τ video segments. The sets of available rates and corresponding quality levels for the mth segment in the ith client is designated as:

i m i m i m

k

i m

Kr r r

, , , ,, , , ,= … …{ }( ) ( ) ( )1 .

i m i m i m

k

i m

Kq q q

, , , ,, , , ,= … …{ }( ) ( ) ( )1 .

Note that our proposed optimization framework is general enough to accommodate many other quality metrics, including peak-signal-to-noise-ratio (PSNR), structure-similarity-index (SSIM) (Wang, 2004), and subjective mean-opinion-score (MOS) — if such information is available.

For notational convenience, we use qi m,.( ) to indicate the empirical rate-quality tradeoff function

for the mth segment in the ith stream. In other words, q r qi m i m

k

, ,( ) = ( ) for r ri m

k= ( )�,�

. The function qi m,.( )

can either be expressed as a parametric model via non-linear regression or simply follow the form of a piece-wise linear function.

C. Client Playout Buffer Evolution

Figure 3 shows the viewpoint of a single client, with time-varying allocated bandwidth ci t,

’s and video rate selections r

i m,’s. Accordingly, one can derive the cumulative data being downloaded

si t,( ) and consumed l

i t,( ) at the client playout buffer at time t, as follows:

s b c ti t

i

t

t

i t, ,= +

′′

=∑01

∆ (1)

�,� ,l ri t

m t m

t

i m

i

=′ ′

′

= +

∑0 τ

τ (2)

Here, the initial playout buffer size is denoted by bi0 and the playout latency of the client is

denoted by ti0 . In other words, we assume that the client may start off with a non-empty playout


27

buffer with data size bi0 , and will wait for t

i0 seconds before viewing the first video segment. In order

to avoid client playout buffer underflow, we need

s l i ti t i t, ,

, ,> ∀ ∀ (3)

Note that the above constraint is only tight when a client has just finished playing out a new segment, i.e., when t t m m Mi= + ∀ = …� ,� ,� ,�

01τ .

In live streaming scenarios, the clients are further constrained in the number of segments they can request in advance. This translates into another set of linear constraints, such that

s l i ti t i t

U, ,

, ,< ∀ ∀ (4)

Here, l li tU

i t m, ,= + 0τ

, where m0 is the number of video segments within the look-ahead horizon. Figure 4 illustrates the cumulative view of the client playout buffer status.

C. Optimization Objective

The goal of the joint bandwidth allocation and video rate selection problem is to find the optimal choices of c

i t,’s and r

i m,’s, so as to maximize the overall video quality across all streams, for all

segments with the time horizon. This can be expressed formally as:

max, , ,c r

i

N

m

M

i m i mq r

= =∑∑ ( )1 1

(5)

s.t. r r r i mi m i m i m

K

, , ,, ,

1( ) ( )≤ ≤ ∀ ∀ (6)

Figure 3. Illustration of a single client with time-varying bandwidth allocation and video rate selections


28

ii tc C t∑ ≤ ∀

,, (7)

l s l i ti t i t i t

U, , ,

, ,< < ∀ ∀ (8)

For streaming on-demand video contents, it is possible to obtain the rate-quality profiles in advance. Typically, the rate-quality tradeoff curves of encoded video segments are concave: improvement in quality diminishes with rate increments. Consequently, the optimization problem Equations 5-8 has a convex objective function with a set of linear constraints. When all information is available at a central entity, the above convex optimization problem could be solved using standard numerical techniques, such as the interior point method (Boyd & Vandenberghe, 2004). Complexity of the optimization algorithm for solving Equation 5-8 increases with N, M, and K. On the other hand, overall system complexity is independent of M: the effect of more frequent updates balances out the effect of shorter horizon within each update.

The proposed convex optimization framework is flexible enough to accommodate many other variations of quality-based bandwidth allocation via slight modification of the objective functions and linear constraints. For instance, it is also possible to equalize quality across all flows in a given user group, or to ensure quality differentiation for flows belonging to different user groups.

5. PRACTICAL IMPLEMENTATIONS OF VQOA

We now describe two variants of allocations schemes in a practical system, and discuss several design considerations.

Figure 4. A cumulative view of how client playout buffer level evolves over time. The client playout level corresponds to the cumulative difference between download and playout data size


29

A. Quality-Aware Bandwidth Allocation

In certain systems, it is infeasible to modify the behavior of an off-the-shelf HTTP-based adaptive streaming client. We therefore consider a quality-aware bandwidth allocation scheme, which involves only bandwidth allocation on the network via the SDN controller, without any modifications to client adaptation logic.

The combinatorial nature of this problem leads to a complexity of O(K^N) with a brute-force solution for N clients and K rate profile levels. Instead, we propose a greedy algorithm that works well for large values of K and N. The proposed scheme starts out by choosing the lowest available rate for all streams, and sequentially promotes the rate one stream at a time, selecting streams with the lowest quality at the current allocation, until the total allocated rate of all streams reaches bottleneck link capacity. The quality of each stream corresponds to minimum quality of all segments within the time window.

B. Joint Optimization

For scenarios where both network bandwidth allocation and video streaming clients are controlled by the VQOA, we propose a joint optimization scheme that strives to equalize the video quality across all clients and all segments sharing the bottleneck bandwidth of C during an update interval T.

The proposed algorithm works in a similar fashion as the previous quality-aware bandwidth allocation scheme: it sequentially promotes the profile assignment of individual video segments with the lowest quality until total data budget for the next update interval (B_total = T*C) runs out. The bandwidth allocation is then computed as the average profile rate assigned to all segments of each client. This can be considered as a discrete approximation to solving the continuous optimization problem in Equations 5-8.

It is assumed that at the start of each update interval, each video client has accumulated at least a buffer duration of T: B_start > T. Therefore, there is no danger of buffer under-flowing during the upcoming update interval. At the end of the interval, the algorithm ensures that the playout buffer of each client will either increase or remain the same with respect to its starting position B_end ≥ B_start.

C. Choice of Video Quality Metric

Throughout rest of this paper, we focus on two representative video quality metrics:

• Peak signal-to-noise ratio (PSNR): Since it can be easily calculated and has been widely used quality metric in the image/video coding community due to its mathematical simplicity;

• Stream Video Quality (SVQ): A Cisco proprietary non-reference video quality metric, which is lightweight to calculate and has demonstrated via internal investigations high correlations to subjective mean-opinion-scores. The SVQ score ranges from 1 to 10, with 10 being the highest perceived quality. Typically, a score below 6 corresponds to “bad” visual quality whereas a score above “9” is considered “very good”.

It needs to be noted, however, that the optimization framework itself is generic enough to accommodate other video quality metrics.

D. Aggregation of Video Quality Metric

Previous studies (Hu, Choudhury, & Gibson, 2007; Keimel, Oelbaum & Diepold, 2010; Chen et al., 2015) have shown that subjective ratings of a user in terms of overall quality-of-experience (QoE) for a short video sequence is mainly dominated by the period of “worst” instantaneous video


30

qualities. Based on such findings, we have chosen to represent overall QoE of the system in terms of 5-th percentile of per-segment and per-client video quality. However, our proposed approach is general enough to work well if a different aggregation of quality metric (such as average quality) is preferred.

6. SIMULATION RESULTS

We first evaluate potential benefit of the proposed optimization framework via numerical simulations. The simulation considers N video streams sharing a bottleneck link of C = 100� Mbps. Video sequence from a high-definition movie clip is encoded into segments with a duration of τ = 2 second each, at 12 different rate and quality levels. Each stream contains M = 40� segments with a random starting point in the movie.

We compare performance of the proposed solution against two reference schemes. In equal-rate allocation scheme, all clients are allocated the same bandwidth from the bottleneck link, and choose the video rate for all segments accordingly. Namely, c r C N t m i

i t i m, ,/ , , ,= = ∀ ∀ ∀ . In the client-only

optimization scheme, all clients are allocated equal bandwidth c C N t ii t,

/ , ,= ∀ ∀( ) ; each client optimizes its video rate selection over time so as to minimize the total encoded video distortion of all its segments without incurring playout buffer underflow. Following the same manner, we refer to our proposed solution as the joint optimization scheme.

Figure 5 shows rate and quality traces resulting from all three schemes when N=30 clients share a bottleneck of 100Mbps. The envelope of highest and lowest rates and qualities are marked in red and black, respectively. It can be noted that the equal-rate allocation scheme results in fairly wild quality variations in the received video streams due to diversity in their contents. In contrast, the proposed joint optimization scheme achieves more fair video quality by allocating very different rates across clients and over time. The client-only optimization scheme achieves part of the performance gain, as it can only vary the video rates over time within each stream.

Figure 6 further shows the cumulative distribution function (CDF) of quality for all video segments in this experiment. Joint optimization yields significantly fewer percentages of low-quality video segments by shifting network resources away from video segments with extremely high qualities. As a result, it tends to equalize the quality amongst all video segments than equal-rate allocation. The performance gain from client-only optimization scheme is less substantial, due to the limitations that all clients are still allocated equal network bandwidth.

Figure 7 compares the three schemes with varying number of clients. Consistently, the proposed joint optimization scheme outperforms equal-rate allocation by 2-3.5 dB in PSNR in minimum video quality, measured at 5-th percentile of all video segments. This translates into supporting 75% more users at the same level of quality-of-experience (QoE). The intermediate scheme of client-only optimization achieves approximately half of that performance gain.

7. TESTBED EVALUATIONS

A. Testbed Setup

We now evaluate performance of SDN-based QoE optimization in a testbed environment. As illustrated in Figure 8, multiple HTTP-based adaptive video streaming clients are connected to their respective servers running on virtual machines hosted by a Cisco UCS C22 Server. All clients share a bottleneck of 100 Mbps as configured by the Cisco ASR1006 Router. Our Video QoE Optimization Application (VQOA) collects detailed transaction information on client activities and content meta-data (e.g., per-segment rate and quality scores for all profiles) from the Streamer. VQOA periodically updates per-client bandwidth allocation decisions and instantiates such changes on the ASR1006 Router via the OpenDaylight Controller (Linux Foundation, 2016).


31

Figure 5. Per-client video rate and quality trace for all three schemes: equal-rate allocation (left); client-only optimization (middle); and joint optimization (right)

Figure 6. Cumulative distribution function (CDF) of quality for all video segments from three schemes: equal-rate allocation (blue), client-only optimization (pink), and joint optimization (red)


32

Figure 7. Comparison of all three schemes with varying number of clients. The maximum and minimum video qualities are measured at the 95-th and 5-th percentiles, respectively, of the video quality distribution across all segments

Figure 8. Testbed topology


33

The testbed supports three types of HTTP-based adaptive streaming clients: VLC clients with its default rate adaptation logic (VideoLAN Organization, 2016); PANDA and AdapTech. The PANDA clients use the probe-and-adapt algorithm as described in (Li et al., 2014) for rate adaptation; the AdapTech clients use the rate adaptation algorithm as proposed in (Akhshabi et al., 2011). Both PANDA and AdapTech clients use more sophisticated rate adaptation logic than VLC to improve bandwidth utilization and QoE.

Our video sequence database consists of thirteen (13) different videos with various levels of motion activity and content complexity. They go through a two-pass encoding using x264 codec, followed by truncation in the HTTP Live Streaming (HLS) format into two second MPEG-2 transport streams using the FFmpeg (FFmpeg, 2016). Each of the video sequences have the following rate profiles: 400, 600, 800, 1200, 1600, 2400, 3200, 4400, 5600, 7000, 9000 Kbps in the master playlist. The independent media playlists advertise both the peak signal-to-noise ratio (PSNR in dB) and the perceptual quality metric SVQ ranging from 0 to 10 for each of the two-second transport streams. The encoding is done at a frame rate of 30 frames per second (fps). Consequently, each two-second video segment contains 60 frames.

We show the encoded rate and the corresponding quality of the first 50 segments of three video sequences from our dataset. The low motion sequence in Figure 9 is able to achieve high perceptual quality at low encoding rates. In contrast, the high motion sequence in Figure 10 requires a significant boost in encoding rate in order to improve from a video quality score around 4.5 (in SVQ) to a level around 9.5. Figure 11 corresponds to a sequence with content variation over time: some segments (e.g., 6, 33, and 34) can achieve high perceptual quality even at low rates while others (e.g. 13, 14, 22) require a significant boost in the encoding rate.

Figure 9. Rate and quality traces for Video No. 1: a low motion sequence where an architect explains a blue print


34

Figure 10. Rate and quality traces for Video No. 2: a dance sequence with frequent camera panning and zooming

Figure 11. Rate and quality trace for Video No. 3: a news report sequence containing many scene cuts


35

Figure 12 further plots the rate versus quality tradeoff curves for all the videos in our database over a 10 second window. It can be noted that while the top orange-colored sequence achieves a SVQ quality score of 8.2 at 400 Kbps, the bottom dark green-colored sequence needs 6Mbps to reach the same quality level. Consequently, different video sequences will need drastically different bandwidth allocations to reach quality fairness when sharing the same bottleneck link.

B. Competing Schemes

Our testbed implementation includes four variants of allocation schemes:

• Baseline “Laissez-Faire” Scheme: which does not impose any per-client bandwidth allocation at the router. This reflects the status quo of how multiple HTTP-based adaptive streaming clients share the Internet today (Akhshabi & Begen, 2012).

• Equal-Rate Allocation: the VQOA simply assigns an equal rate of C/N for each competing clients, where C is the total bandwidth at the bottleneck and N is the total number of clients. This scheme does not require video quality information.

• Quality-Aware Bandwidth Allocation: the VQOA periodically updates the bandwidth allocation based on aggregate rate and quality information of each client, according to the algorithm described in Section 5.A. This scheme does not require modification of the HTTP-based adaptive video streaming client.

• Joint-Optimization: the VQOA determines both network bandwidth allocation and profile selection decisions at the HTTP-based adaptive video streaming clients. It updates its decision on a periodical basis. The joint optimization scheme follows the algorithm in Section 5.B. For this scheme, the HTTP-based adaptive video streaming clients in VLC media player (VideoLAN Organization, 2016) is modified to follow per-segment profile selections recommended by VQOA.C. Comparison of Allocation Schemes

Figure 12. Rate-Quality tradeoff curves for all video sequences over a 10 second window (i.e., averaged over 5 segments)


36

We first evaluate performance of the proposed schemes in our testbed environment, where 16 VLC clients compete for a total bandwidth of 80Mbps, and each client streams one video sequence from our data set. The videos have different rate-quality characteristics. For example, some videos contain scenes with a lot of motion, which generally require a higher bitrate to achieve a certain quality than the videos with more static scenes. Thus one can expect the quality-aware bandwidth allocation scheme to achieve better performance than equal-rate since the former exploits diversity across different clients.

We run separate experiments for four schemes: baseline (laissez-faire), equal-rate allocation, quality-aware bandwidth allocation, and joint-optimization. For the joint-optimization scheme we used a modified VLC implementation, while the other three schemes used the VLC client as-is. The algorithm update interval is 10 second for quality-aware allocation and joint optimization; baseline and equal-rate schemes does not involve any periodic update.

Figure 13 exhibits the rate and video quality traces for each segment requested by the 16 clients sharing a 80Mbps bottleneck link, as the testbed configuration changes sequentially through four schemes: baseline equal-rate allocation, quality-aware bandwidth allocation, and joint optimization. Lines with different colors represent different clients. The instants at which the scheme changes are marked in the figures. Under the baseline scheme, the clients experience a lot of variations in both the rate choices and video quality. When we apply the equal-rate scheme we can immediately see the benefit -- there are few variations in rate choices, and the quality becomes higher with fewer variations as compared to the baseline scheme. However, there is room for improvement to achieve better video quality. Applying the quality-aware bandwidth allocation scheme results in even higher quality values with fewer variations. It can be observed that joint-optimization achieves the best overall quality among all schemes. Note that both quality-aware bandwidth allocation and joint-optimization result in more variations in rate choices than the equal-rate scheme. This is a consequence of exploiting video content diversity across time and across clients to improve aggregate video quality for all clients.

As a summary, Figure 14 exhibits the 5th percentile of per-segment video quality in SVQ under different schemes. This verifies our previous observations from Figure 13 that in terms of equalizing video quality across all clients and video segments, the performance ranking of the schemes is (from top to bottom): joint-optimization, quality-aware bandwidth allocation, equal-rate and the baseline.

Figure 13. Rate and quality traces for 16 competing VLC clients with different schemes: baseline, equal-rate allocation, quality-aware bandwidth allocation, and joint optimization


37

D. Impact of Different HTTP-based Adaptive Streaming Clients

Next, we examine impact of different HTTP-based adaptive streaming clients by carrying out the same experiment as in Section 7.C using two other clients: PANDA (Li et al., 2014) and AdapTech (Akhshabi et al., 2011). Figure 15 and Figure 16 show the rate and quality traces under the baseline scheme, when all 16 clients are PANDA and AdapTech respectively. Compared to the previous results using VLC clients (see Figure 13), PANDA and AdapTech clients achieve higher quality with fewer variations. This confirms the benefit of intelligent rate adaptation algorithms at the clients, and is consistent with the findings from (Li et al., 2014) and (Akhshabi et al., 2011).

Table 1 further summarizes the 5th percentile of quality experienced by different clients under equal-rate and quality-aware allocation schemes. Note that results from joint-optimization are irrelevant since the scheme overrides client rate adaptation behavior. Comparing the performance across different schemes for any given client, we can draw similar conclusions as previously in Section 7.C: equal-rate allocation outperforms the baseline scheme whereas quality-aware bandwidth allocation achieves the highest overall video quality. Comparing the results across different types of clients for a given scheme, it can be observed that the benefit of using a more sophisticated rate adaptation algorithm (e.g., using PANDA and AdapTech instead of VLC) becomes negligible for equal-rate and quality-aware bandwidth allocations. This implies that the SDN-based QoE optimization framework can improve the QoE of different types of clients to roughly the same level. This helps to reduce the need of complexity on the client side in terms of designing and implementing sophisticated rate adaptation algorithms.

E. Impact of Bandwidth Allocation Update Intervals

We now investigate performance of the proposed schemes under varying bandwidth allocation update intervals, i.e., how often we update the per-client bandwidth allocation decisions and program the ASR1006 router accordingly. Since no update interval is needed for baseline and equal-rate allocation schemes, we only consider quality-aware allocation and joint-optimization in our testbed experiments. The setup is similar to that in Sections 7.C and 7.D, except that 11 VLC clients share a total bandwidth of 55Mbps. Each client streams a different video from our video sequence dataset. Due

Figure 14. Overall video quality in terms of 5th percentile of SVQ scores of all video segments seen by VLC clients under different schemes


38

Figure 15. Rate and quality traces for PANDA client under the baseline scheme

Figure 16. Rate and quality traces for AdapTech client under the baseline scheme

Table 1. Overall video quality measured as 5-th percentile of SVQ scores for different HTTP-based adaptive streaming clients under different allocation schemes

VLC PANDA AdapTech

Baseline 8.35 8.76 8.61

Equal-rate allocation 9.00 9.01 8.97

Quality-aware allocation 9.38 9.30 9.41


39

to the choice of different video sequences, different number of clients, and a different total bandwidth, overall quality results from this subsection do not directly match those from previous sections.

Figure 17 plots the 5th percentile of quality against various update intervals for different schemes. We observe that joint-optimization always outperforms other schemes, and that equal-rate allocation always leads to lowest overall quality. Somewhat surprisingly, performance of the joint-optimization scheme does not change with varying update intervals. Although one may expect the joint optimization scheme to improve with longer update intervals (as it can optimize bandwidth allocation over a longer horizon), it turns out that such impact is minimal for update intervals exceeding 10 seconds.

Figure 17 also shows the 5th-percentile quality values from the quality-aware bandwidth allocation scheme: its performance degrades with increasing update intervals. The reason behind this is that the quality-aware scheme can adjust more frequently to temporal changes of video characteristics for shorter update intervals.

F. Varying Number of Clients

Figure 18 compares three schemes – baseline, equal-rate allocation, and joint-optimization -- in the testbed with varying number of clients. The shared bottleneck bandwidth is set at 100Mbps, same as in previous simulation studies. The range of video qualities are represented as 95-th and 5-th percentiles, respectively, of the video quality distribution across all segments. It can be noted that at the same 5-th percentile quality of 8.0 in SVQ, which corresponds to fairly good user viewing experience, joint optimization can support 35 clients whereas equal-rate allocation and baseline schemes can only support 25 and 15 clients, respectively. This translates to significant bandwidth

Figure 17. 5th percentile of quality for VLC clients under varying update intervals


40

savings while maintaining the same target QoE level for a given set of users, or the opportunity to support significantly more users at the same bottleneck bandwidth.

Finally, we carry out the same experiment while using PSNR as the video quality metric. It can be confirmed from Figure 19 that the performance gains of joint optimization over equal-rate allocation matches those from previous simulation-based results in Figure 7: joint optimization can support 35 clients at the same 5-th percentile quality of 38dB as achieved by equal-rate allocation for 20 clients.

8. CONCLUSION

We have, in this work, investigated a novel SDN-based solution for bandwidth management in HTTP-based adaptive streaming of video. Departing from the conventional paradigm of equalizing rate amongst all flows (within the same policy class), we show that the proposed SDN-based architecture can leverage the flexibility of an SDN infrastructure to build a more efficient and intelligent video delivery network. We propose to formulate the multi-client bandwidth allocation problem within the convex optimization framework, which is flexible enough to accommodate a wide variety of video quality metrics. Our solution tailors the bandwidth allocation to both the requested video content complexity and the playout buffer status of individual clients.

Extensive results from simulations and testbed-based experiments highlight the benefits of intelligent network control. While it is very challenging for competing HTTP-based adaptive streaming clients to attain efficiency, fairness, and stability via rate adaptation algorithms on their own, we show in our testbed that it is fairly straightforward for a network-based scheme to achieve the same goal via equal-rate allocation. In addition, the proposed joint optimization scheme can support up to 75% more users at the same level of quality-of-experience than conventional equal-rate allocations.

Figure 18. Testbed result with varying number of clients: SVQ as quality metric


41

Our future investigations will study how varying network conditions impacts performance of the optimization framework, as well as interactions between video streaming and background non-video traffic (e.g., file transfer) sharing the same network. We will also study distributed implementations of VQOA with loose coordination between network components and HTTP-based adaptive streaming clients.

ACKNOwLEDGMENT

The authors would like to thank Joel Schoenblum and Gene Halbrooks at Cisco Systems for their help with the SVQ video quality metric. Our acknowledgments to Zhili Guo from New York University for modifying the VLC player as part of his internship work at Cisco. The authors would also like to thank Gareth Bowen and Tankut Akgul at Cisco Systems for sharing their codes on the PANDA and AdapTech clients, as well as their help on implementing those clients in our testbed.

Figure 19. Testbed result with varying number of clients: PSNR as quality metric


42

REFERENCES

Akhshabi, S., & Begen, A. C. (2012, June). What Happens when HTTP Adaptive Streaming Players Compete for Bandwidth. Paper presented atACM Workshop on Network and Operating System Support for Digital Audio and Video (NOSSDAV’12), Toronto, Ontario, Canada. doi:10.1145/2229087.2229092

Akhshabi, S., Narayanaswamy, S., Begen, A. C., & Dovrolis, C. (2011). An Experimental Evaluation of Rate-Adaptive Video Players over HTTP. EURASIP Journal on Signal Processing and Image Communications, 27(4), 157–168.

Apple Inc. (2016). HTTP Live Streaming Overview. Retrieved from http://developer.apple.com

Boyd, S., & Vandenberghe, L. (2004) Convex Optimization. Cambridge, United Kingdom: Cambridge University Press. doi:10.1017/CBO9780511804441

Chen, C., Zhu, X., de Veciana, G., Bovik, A. C., & Heath, R. W. (2015). Rate Adaptation and Admission Control for Video Transmission with Subjective Quality Constraints. IEEE Journal of Selected Topics in Signal Processing, 9(1), 22–36. doi:10.1109/JSTSP.2014.2337277

Cisco Systems Inc. (2015). Cisco Visual Networking Index: Forecast and Methodology (White Paper). Retrieved from http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngn-ip-next-generation-network/white_paper_c11-481360.html

De Cicco, L., Mascolo, S., & Palmisano, V. (2011, February). Feedback Control for Adaptive Live Video Streaming. Paper presented atACM Multimedia Systems Conference (MMSys’11), San Jose, CA, USA. doi:10.1145/1943552.1943573

Essaili, A. E., Schroeder, D., Staehle, D., Shehada, M., Kellerer, W., & Steinbach, E. (2013). Quality-of-experience driven adaptive HTTP media delivery. Paper presented atIEEE International Conference on Communications (ICC’13), Budapest, Hungary (pp. 2480-2485).

FFmpeg. (2016). Retrieved from https://www.ffmpeg.org/

Houdaille, R., & Gouache, S. (2012). Shaping HTTP adaptive streams for a better user experience. Paper presented atACM 3rd Multimedia Systems Conference (MMSys ‘12), Chapel Hill, NC, USA. doi:10.1145/2155555.2155557

Hu, J., Choudhury, S., & Gibson, J. D. (2007, August) Perceptual quality constrained video user capacity of 802.11a WLANs with multipath fading. Paper presented atInternational Conference on Wireless Communications and Mobile Computing (IWCMC’07), Honolulu, HI, USA. doi:10.1145/1280940.1281005

Hu, S., Sun, L., Gui, C., Jammeh, E., & Mkwawa, I.-H. (2014, November). Content-Aware Adaptation Scheme for QOE Optimized DASH Applications. Paper presented atIEEE Global Communications Conference (Globecom’14), Austin, TX, USA. doi:10.1109/GLOCOM.2014.7036993

Jiang, J., Sekar, V., & Zhang, H. (2012). Improving fairness, efficiency, and stability in HTTP-based adaptive video streaming with FESTIVE. Paper presented at the8th international conference on Emerging networking experiments and technologies (CoNEXT ‘12), Nice, France (pp. 97-108). doi:10.1145/2413176.2413189

Joseph, V., V & de Veciana, G. (2014). NOVA: QoE-driven optimization of DASH-based video delivery in networks. Paper presented atIEEE Conference on Computer Communications (INFOCOM’14), Toronto, ON, Canada (pp. 82-90). doi:10.1109/INFOCOM.2014.6847927

Keimel, C., Oelbaum, T., & Diepold, K. (2010, March) Improving the Prediction Accuracy of Video Quality Metrics. Paper presented atIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’10), Dallas, TX, USA. doi:10.1109/ICASSP.2010.5496299

Li, Z., Zhu, X., Gahm, J., Pan, R., Hu, H., Begen, A. C., & Oran, D. (2014). Probe and Adapt: Rate Adaptation for HTTP Video Streaming at Scale. IEEE Journal on Selected Areas in Communications, 32(4), 719–733. doi:10.1109/JSAC.2014.140405

Linux Foundation. (2016) The OpenDaylight Platform. Retrieved from http://www.opendaylight.org

Liu, C., Bouazizi, I., & Gabbouj, M. (2011, February). Rate Adaptation for Adaptive HTTP streaming. Paper presented atACM Multimedia Systems Conference (MMSys’11), San Jose, CA, USA.

http://dx.doi.org/10.1145/2229087.2229092

http://developer.apple.com

http://dx.doi.org/10.1017/CBO9780511804441

http://dx.doi.org/10.1109/JSTSP.2014.2337277

http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngn-ip-next-generation-network/white_paper_c11-481360.html

http://www.cisco.com/c/en/us/solutions/collateral/service-provider/ip-ngn-ip-next-generation-network/white_paper_c11-481360.html

http://dx.doi.org/10.1145/1943552.1943573

https://www.ffmpeg.org/

http://dx.doi.org/10.1145/2155555.2155557

http://dx.doi.org/10.1145/1280940.1281005

http://dx.doi.org/10.1109/GLOCOM.2014.7036993

http://dx.doi.org/10.1145/2413176.2413189

http://dx.doi.org/10.1109/INFOCOM.2014.6847927

http://dx.doi.org/10.1109/ICASSP.2010.5496299

http://dx.doi.org/10.1109/JSAC.2014.140405

http://www.opendaylight.org


43

Ma, K. J., & Bartos, R., R. (2011). HTTP Live Streaming Bandwidth Management Using Intelligent Segment Selection, Paper presented atIEEE Global Telecommunications Conference (GLOBECOM’11), Houston, TX, USA. doi:10.1109/GLOCOM.2011.6133856

McKeown, N., Anderson, T., Balakrishnan, H., Parulkar, G., Peterson, L., Rexford, J., & Turner, J. et al. (2008). OpenFlow: Enabling Innovation in Campus Networks. ACM SIGCOMM Computer Communications Review, 38(2), 69–74. doi:10.1145/1355734.1355746

MPEG. (2012). ISO/IEC 23009-1:2012 Information technology – Dynamic adaptive streaming over HTTP (DASH) – Part 1: Media presentation description and segment formats.

Nam, H., Kim, K.-H., Kim, J. Y., & Schulzrinne, H. (2014, November). Towards QoE-aware Video Streaming using SDN. Paper presented atIEEE Global Communications Conference (Globecom’14), Austin, TX, USA.

Ramakrishnan, S., Zhu, X., Chan, F., & Kambhatla, K. (2015). SDN Based QoE Optimization for HTTP-Based Adaptive Video Streaming. Paper presented atIEEE International Symposium on Multimedia (ISM’15), Miami Beach, FL, USA. doi:10.1109/ISM.2015.53

Video LAN Organization. (2016). VLC Media Player. Retrieved from http://www.videolan.org/

Wang, Q., Xu, K., Izard, R., Kribbs, B., Porter, J., Wang, K.-C., . . . Ramanathan, P. (2014, January) GENI Cinema: A SDN-Assisted Scalable Live Video Streaming Service. Paper presented atIEEE 22nd International Conference on Network Protocols (ICNP’14), Raleigh, NC, USA. doi:10.1109/ICNP.2014.84

Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600–612. doi:10.1109/TIP.2003.819861 PMID:15376593

Zambelli, A. (2009). Smooth Streaming Technical Overview. Retrieved from http://www.iis.net/learn/media/on-demand-smooth-streaming/smooth-streaming-technical-overview

Zhu, X., Li, Z., Pan, R., Gahm, J., & Hu, H. (2013). Fixing Multi-Client Oscillations in HTTP-based Adaptive Streaming: A Control Theoretic Approach. Paper presented atIEEE International Workshop on Multimedia Signal Processing (MMSP’13), Pula, Sardinia, Italy (pp. 230-235). doi:10.1109/MMSP.2013.6659293

http://dx.doi.org/10.1109/GLOCOM.2011.6133856

http://dx.doi.org/10.1145/1355734.1355746

http://dx.doi.org/10.1109/ISM.2015.53

http://www.videolan.org/

http://dx.doi.org/10.1109/ICNP.2014.84

http://dx.doi.org/10.1109/TIP.2003.819861

http://www.ncbi.nlm.nih.gov/pubmed/15376593

http://www.iis.net/learn/media/on-demand-smooth-streaming/smooth-streaming-technical-overview

http://www.iis.net/learn/media/on-demand-smooth-streaming/smooth-streaming-technical-overview

http://dx.doi.org/10.1109/MMSP.2013.6659293


44

Sangeeta Ramakrishnan holds an MS degree in Electrical Engineering from the University of California, Santa Barbara, and a BE degree in Electronics and Communications Engineering from College of Engineering Guindy, Anna University, Chennai. Sangeeta Ramakrishnan is an architect at Cisco Systems and is responsible for developing the architecture for delivery of IP Video over DOCSIS and for leading Cisco’s efforts in the development of SDN Applications for Cable. Her current areas of interest include HTTP Adaptive Streaming, Video Quality of Experience Optimization, Software Defined Networking and Network Function Virtualization.

Xiaoqing Zhu is currently a Technical Leader at the Chief Technology and Architecture Office (CTAO) at Cisco Systems Inc. Her research interests span multimedia applications, networking, and wireless communications. At Cisco she has worked on various interesting problems, including HTTP-based adaptive streaming, congestion control for low-latency interactive video, fog computing, media delivery over vehicular networks, and video traffic modeling and synthesis. Zhu has published over 60 journal and conference papers, receiving the Best Student Paper Award at ACM Multimedia in 2007 and the Best Presentation Award at IEEE Packet Video Workshop in 2013. She is author of 4 granted U.S. patents and 16 pending applications. Within the research community, Zhu has served extensively for various journals, conferences, and workshops as reviewer, TPC member, and special session organizer. She served as guest editor for special issues in IEEE Journal on Selected Areas in Communications, IEEE Trans. Multimedia. She currently serves as Chair of the Multimedia Content Distribution: Infrastructure and Algorithms (MCDIG) Interest Group in IEEE Technical Committee on Multimedia Communications (MMTC). Zhu holds a BEng in Electronics Engineering from Tsinghua University, Beijing, China. She graduated from Stanford University, California, with both MS and PhD degrees in Electrical Engineering. Prior to joining Cisco, she interned at IBM Almaden Research Center in 2003, and at Sharp Labs of America in 2006.

Frank Chan received his BS degree in Computer Science from University of Wisconsin - Madison in 1999. He received his MS degree in Information Network from Carnegie Mellon University, Pittsburgh, PA, in 2000. He has been working in Cisco Systems, San Jose, CA, since 2001, primarily focus on IP Multicast Video delivery on DOCSIS network, and SDN-based solutions for cable service providers.

Kashyap K. R. Kambhatla (S’06-M’14) received the BTech degree with Honors in electrical and computer engineering (ECE) from Jawaharlal Nehru Technological University, Hyderabad, India, in 2004. He graduated magna cum laude with a MS degree in electrical and computer engineering from Clarkson University and PhD degree in ECE from Jacob’s School of Engineering at University of California, San Diego. His research interests are in the areas of video compression and processing for real-time cross-layer quality-of-service (QoS) aware video streaming. He also works on network optimization problems and developing efficient MAC protocols. Kambhatla is currently a video streaming engineer in the Chief Technology Architects Office at Cisco Systems Inc. in San Jose, CA. Previously, he worked on video telephony in 3G EV-DO Rev. A and B cellular networks as a research intern in the Next-Gen Wireless Technology Group at Sprint Advanced Technology Labs, Burlingame, CA. He has been a regular in the review panel for multiple IEEE journals in the communication and signal processing societies. Kambhatla is a member of Phi Kappa Phi inducted as a student for academic excellence.

Zheng Lu received his BE degree in Electronics Engineering from Tsinghua University in China in 2009. He received his MSE and PhD in Electrical and Computer Engineering from The University of Texas at Austin in 2011 and 2015 respectively. His research focuses on algorithms & architectures to enhance perceived video quality for video streaming, and resource allocation in Device-to-Device networks to optimize system & user perceived performance. He interned at Intel Labs, Hillsboro during summer 2013. And since 2015, he has been working at Cisco Systems in San Jose, CA.

Cindy Chan received her BS in Electrical Engineering and MS in Computer System Engineering from University of Massachusetts in 1986 and 1988 respectively. Since then, she has been working in the industry on various networking technology, and has earned multiple patents. Currently, she works at Cisco Systems developing software on Cable IPv4/IPv6 routing platforms, and SDN applications.

Bhanu Krishnamurthy received her BE degree in Electronics and Communication Engineering from Mysore University in 1992. She received her MS in Computer Science from Cal State Hayward in 1995. She has worked in the telecommunications industry focusing of embedded software development. The areas of expertise include DOCSIS, PacketCable, PCMM, IPv6, SDN. Currently working as a Senior Technical Leader developing SDN apps based on ODL.

Documents

Optimizing Quality-of-Experience for HTTP-based Adaptive ...ivms.stanford.edu/~zhuxq/papers/ijmdem2016.pdf · and different flavors of QoE optimization objectives. Extensive evaluations