13th International Conference on DEVELOPMENT AND APPLICATION SYSTEMS, Suceava, Romania, May 19-21, 2016

978-1-5090-1993-9/16/$31.00 ©2016 IEEE

Performance comparison of a WebRTC server on Docker versus Virtual Machine

Cristian Constantin Spoiala, Alin Calinciuc, Corneliu Octavian Turcu, Constantin Filote Faculty of Electrical Engineering and Computer Science

Stefan cel Mare University of Suceava Suceava, Romania

{cristian.spoiala, alin.calinciuc, cturcu, filote}@usv.ro

Abstract—Current developments in real-time technologies enable multiple companies to focus on a standard in order to bring real-time technologies to the web. The efforts lead to the API standard WebRTC that supports voice and video chat and P2P file sharing, without the need to install external plugins on browsers. As lots of services will switch to WebRTC we found useful to provide a comparison of a WebRTC server on Docker containers and virtual machines. As WebRTC server, we used Kurento Media Server, a powerful open source server with many advanced features. This paper is testing what kind of virtualization is more suitable for a multimedia application based on WebRTC. We tested Docker containers and KVM machines with a multimedia based test. As lots of services will switch to WebRTC we found useful to provide a comparison between two kinds of virtualization for a multimedia application based on WebRTC. Thus, we considered a WebRTC server on Docker containers and virtual machines. As WebRTC server, we used Kurento Media Server, a powerful open source server with many advanced features. We tested the Docker containers and KVM machines with a multimedia based test.

Keywords—Operating systems; Real-time systems; WebRTC; Open source software; Computer performance; Virtual machine monitors

I. INTRODUCTION

For a long time, cloud computing has been associated with virtual machines and Infrastructure as a Service (IaaS) providers, like Amazon and Microsoft, used them in order for customers to run their workloads. Moreover, most of the current cloud is powered by virtual machines and the technology matured in the past years and performance has been improving continuously [1].

In recent developments in the field of cloud computing, containers emerged as an alternative to virtual machines, providing a better utilization of hardware resources and, inherently, a decrease in operating costs [2]. Although Google uses these containers for years [3], the lack of a standard prevented their deployment on a large scale. That was the case until the emergence of the Docker platform.

Since due to security and performance issues, browser plugins, like Adobe Flash, are no longer used for chat and

video conferences, WebRTC [4] is filling this gap with a plugin-less approach. Moreover, a W3C standard is currently reaching its final stages of development, and a working draft was finalized in Jan 2016 [5].

We chose Kurento Media Server (KMS) as a WebRTC server, an open source WebRTC server with a powerful set of client APIs. KMS also provides advanced features like media processing capabilities involving computer vision, video indexing, augmented reality and speech analysis [6].

This paper aims to compare the performances of a WebRTC server on both virtual machines and Docker containers, in a scenario suitable for multimedia workload.

The purpose is to highlight the overhead of the hypervisor for virtual machines.

The main contributions of this work are:

• We made an up-to-date comparison of the Dockercontainers and virtual machines in a real-time userscenario and confirmed that the overhead ofvirtual machines is affecting the performance by5-10%.

• We identify use cases for best practices to useDocker or virtual machines.

• Analysis of the impact of the CPU performance ofWebRTC servers.

This paper is structured in 5 Sections. Section II describes, in more detail, both KVM based virtual machines and Docker containers.

Section III focuses on the methodology of the tests performed, and includes details regarding the hardware and tools used.

Section IV presents the test results, whereas Section V concludes the paper.

The test was conducted as part of NUBOMEDIA, which is a FP7 EU founded project for developing a PaaS for multimedia projects. On NUBOMEDIA, developers can develop, deploy and host real-time multimedia projects. From

295

their point of view, all infrastructure is hidden behind a single platform and developers can focus only on their multimedia application.

II. KVM AND DOCKER

A. KVM Kernel-based Virtual Machine (KVM) is a virtualization

infrastructure from Linux Kernel since version 2.6.20 that permits Linux to run virtual machines and acts as hypervisor [7]. KVM requires a processor with hardware-assisted virtualization (that supports Intel Virtualization Technology (VT-x). Moreover, in comparison to Docker containers, isolation between virtual machines is excellent, thus increasing security.

A single virtual machine is using a single set of hardware resources; in most real world scenarios, where applications are idling for long periods, this would mean a waste of resources. For KVM we assign and block resources that will idle, if the application is not utilized.

KVM virtual machines excels at security, but the hypervisor of the KVM technology adds an overhead and consequently, performance is affected, especially for the I/O operations.

OpenStack is an open source IaaS technology used to bring private and public clouds to anyone who can install and maintain it. It is mostly based on Python programming language and the dashboard is based on Django Framework.

OpenStack is supporting multiple hypervisors, including KVM-QEMU, the most stable one. KVM-QEMU is also the hypervisor that we are testing in our experiments presented in this paper.

B. Docker containers Linux containers (LXC) are an operating system-level

virtualization method that includes all the applications and

dependencies needed to run an application. Moreover, it shares the kernel with other containers. Resources are isolated using cgroups and run in isolated processes usually with the UID of the application [8]. This means that an application running as root (UID 0) will have root-level privileges on the underlying host when interacting with the kernel.

Docker is an open-source project that helps the use of linux containers by providing all the tools necessary to operate them. Docker is packaging tools and technologies like namespaces, union file system, libcontainer, cgroups and any application based on Docker will be able to run on any Linux server.

By adding low overhead over host machines [9], containers perform better than other methods of virtualizations based on hypervisors, such as KVM, Xen, etc. Unlike a virtual machine which runs the full operating system, a container can be a single process [10]. Even though a container does not have a full operating system, it behaves like one, as is running init, ssh, cron, etc [11].

One downside is that containers can’t run on different linux kernels. Other downside is security, as the degree of isolation is not as high as on virtual machines, where applications are running on independent OS instances that are not sharing the kernel.

The biggest advantage of containers is boot time [10], as a container starts in 1 second, and a virtual machine typically starts in more than 10 seconds.

III. TEST METHODOLOGY

This section describes the testing methodology, the tools used for the test and the exact hardware and software configuration used in order to be replicated by other peers.

All tests were performed on an OpenStack (version Kilo) with CentOS Linux 7.1, Docker 1.6.2, QEMU 1.5.3 and libvirt 1.2.8

Physical machines are based on an IBM BladeCenter. Each

Fig. 1 Architecture for testing

296

blade has the following Intel Xeon configuration:

•CPU: 2 x Intel(R) Xeon(R) E5345 @ 2.33GHz;

•RAM: 16GB PC2-5300 CL5 ECC DDR2 667MHz;

•LOCAL DISK: IBM 147GB SAS 10K HDD;

Network connectivity between servers is based on a Gigabit Ethernet Stack.

KMS Performance Tool (KPT) [12] is a tool used to measure the KMS performance on several environments. It is based on Selenium WebDriver and it is automating the web browsing [13] and use another KMS to simulate WebRTC users. In the present paper, we considered the 6.1.1 version of the KMS.

The KPT tool was run in multiple configurations; Fig. 1 displays the architecture of the test and the types of machines used. All these machines used to perform the test, were deployed on Ubuntu 14.04 LTS and are shown in the table below:

TABLE I. TYPES OF MACHINES USED

Type Numbers deployed

Configuration

KMS 5 4GB RAM, 4 vCPU

KPT 5 3GB RAM, 8 vCPU

Fake Client KMS 1 4GB RAM, 4 vCPU

As shown in Fig. 1, the test was started and managed through a Jenkins CI tool. All test results were saved in the CSV format on the Jenkins tool.

To add extra processing for the CPU, we performed both a test without any filter, and a test with an encoder filter. On this filter, the media is decoded to raw and encoded again afterwards. The encoding is performed for every output in the pipeline.

The test ran for 200 seconds, and the delay between clients was specified to 1000ms. For 50 fake clients and a delay of 1000ms means, it means that after 50 seconds all clients were started. Fig. 2 to Fig. 9, show that the ramp-up period until all fake clients are started is visible and that the chart stabilizes after 50 seconds.

IV. TEST RESULTS

Monitoring the memory usage, we discovered that the machines did not exceed 80% during the tests.

The charts from Fig. 2 to Fig. 5 show the CPU usage and latency of the KMS for the test without any media processing and 50 fake clients.

Fig. 2 to Fig. 5 shows clearly that the CPU usage is 5-10% lower for the Docker instances whereas the latency is more stable and a bit lower.

Fig. 2 CPU usage for KVM test without any filter with 50 fake clients

Fig. 3 Latency for KVM test without any filter and 50 fake clients

Fig. 4 CPU usage for Docker test without any filter with 50 fake clients

Fig. 5 Latency for Docker test without any filter with 50 clients

297

From Fig. 6 to Fig. 9 are shown the test results for KVM and Docker with an encoder filter and 15 fake clients. This is more intense for the CPU than the previous test.

Fig. 6 CPU usage for KVM and 15 fake clients

Fig. 7 Latency for KVM and 15 fake clients

Fig. 8 CPU usage for Docker test and 15 fake clients

Fig. 9 Latency for Docker test and 15 fake clients

Under heavy load, the latency on Docker machines is lower; this is helping multimedia applications to deliver a good service, even when lots of users are connected.

V. CONCLUSIONS

The conclusions drawn from this paper are that the Docker containers don’t have the same overhead as KVM based virtual machines. This supports the real-time performance to get more from the existing hardware. This way, the costs for maintaining the infrastructure are lowered and a better real-time experience is provided to applications based on the Docker containers.

From these benchmark tests, we concluded that the Docker performance is better than KVM, especially for latency, a critical metric for real-time applications. Moreover, Docker proved suitable as a hypervisor and, therefore, IaaS platforms should support it. It also proved suitable for the NUBOMEDIA project [14] and the consortium decided to use it as the solution for running multimedia media servers.

ACKNOWLEDGMENT

This work has been supported by the European Commission under projects NUBOMEDIA FP7-ICT-2013-1.6, GA-610576;

REFERENCES

[1] Mr. Ray J Rafaels, Cloud Computing: From Beginning to End,

CreateSpace Independent Publishing Platform, April 2015.

[2] W. Felter, A. Ferreira, R. Rajamony, J. Rubio “An Updated Performance Comparison of Virtual Machines and Linux Containers” IBM Research Division, July 2014.

[3] Joe Beda, Containers At Scale, GlueCon, May 2014.

[4] S. Taheri, L. A. Beni, A. V. Veidenbaum, A. Nicolau, R. Cammarota, J. Qiu, Q. Lu, M. R. Haghighat, "WebRTCbench: a benchmark for performance assessment of webRTC implementations", Embedded Systems For Real-time Multimedia, 2015 13th IEEE Symposium, October 2015.

[5] WebRTC 1.0: Real-time Communication Between Browsers https://www.w3.org/TR/webrtc/

[6] L. López-Fernández, M. Gallego, B. García, D. Fernández-López. F. J. López, "Authentication, Authorization, and Accounting in WebRTC PaaS Infrastructures: The Case of Kurento", IEEE Internet Computing, vol. 18, Issue 6, August 2014.

[7] IBM Corporation. Best practices for KVM, April 2012.

[8] Di Liu, Libin Zhao, "The research and implementation of cloud computing platform based on docker", 2014 11th International Computer Conference, December 2014.

[9] Adrian Mouat, Using Docker: Developing and Deploying Software with Containers, O'Reilly Media, January 2016.

[10] Karl Matthias, Docker: Up & Running, O'Reilly Media, July 2015.

[11] Roberto Morabito, "A Performance Evaluation of Container Technologies on Internet of Things Devices", IEEE Infocom, 2016.

[12] Kurento Media Server https://github.com/Kurento/kurento-media-server

[13] Raghavendra Prasad MG, Learning Selenium Testing Tools, 3rd Edition, Packt Publishing, February 2015.

[14] NUBOMEDIA: an elastic Platform as a Service (PaaS) cloud for interactive social multimedia www.nubomedia.eu

298