Monitoring and Analysis of CPU load relationships between

Thesis no: MEE-2015-NN

Monitoring and Analysis of CPU loadrelationships between Host and Guestsin a Cloud Networking Infrastructure

An Empirical Study

Krishna Varaynya Chivukula

Faculty of ComputingBlekinge Institute of TechnologySE–371 79 Karlskrona, Sweden

This thesis is submitted to the Faculty of Computing at Blekinge Institute of Technologyin partial fulfillment of the requirements for the degree of Master of Science in ElectricalEngineering. The thesis is equivalent to 20 weeks of full time studies.

Contact Information:Author(s):Krishna Varaynya ChivukulaE-mail: [email protected]

University advisor:Prof.Dr. Kurt TutschkuDepartment of Telecommunication Systems

Faculty of Computing Internet : www.bth.seBlekinge Institute of Technology Phone : +46 455 38 50 00SE–371 79 Karlskrona, Sweden Fax : +46 455 38 50 57

Abstract

Cloud computing has been a fast-growing business of the IT sector in the recentyears as it favors hardware resource sharing by reducing the infrastructure main-tenance costs and promising improved resource utilization and energy efficiencyto the service providers and customers. Cloud Service Providers, CSP, imple-ment load management techniques for effective allocation of resources based onneed, enabling them to maintain costs alongside meeting the SLAs. Understand-ing the impact and behavior of variable workloads in a cloud system is essentialfor achieving load management. CPU load is a principle computational resourcethat plays an important role in resource management.

This thesis work aims to monitor and analyze load in cloud infrastructure byapplying load collection and evaluation techniques. The aim is also to investigateCPU load relationship between host and guest machines under varying workloadconditions. In the thesis, along with a cloud environment built using OpenStack,we also consider a system with KVM hypervisor to achieve the goals.

The methodology applied is empirical, that is pure experimental examination.This study is about performing measurements to make an assessment about loadbehavior in the system. The experiments are designed to fulfill the objectives ofthe thesis. We also employ visual correlation analysis to understand the strengthof association between host and guest CPU load.

Results of the initial experimental study include distinction between CPU load ofOpenStack compute device and a device with KVM hypervisor. Further experi-mental runs are based on these observations. The succeeding results show quiteremarkable association between PM and VM under 100% workload conditions.However, few other variations in workload do not resemble similar results.

CPU load results obtained from cloud and a standalone virtualization systemdiffer, not drastically though. 100% workload situations have shown negligi-ble distortion in the visual correlation and usually reported linearity. Lowerworkloads showed distortions in correlation analysis. It is anticipated that moreiterations can likely refine the observations. Further investigation of these rela-tionships by using other applications commonly used in cloud is potential.

Keywords: Cloud, CPU load, Measurements, OpenStack, Virtualization

i

Dedicated to my family

ii

Acknowledgements

I am forever indebted to my supervisor, Prof. Dr. Kurt Tutschku for his valuableguidance, patience and continuous support throughout my thesis. I could nothave imagined a better advisor and mentor for my master thesis.

I sincerely thank Dr. Patrik Arlos for his constant encouragement and suggestionsin between his busy schedule. I extend my heartfelt thanks to Dr. Dragos Ilie forhis invaluable guidance when approached. Words cannot express my gratitudefor Anders Carlsson, my father figure, who gave me never ending support andopportunities to excel.

I am grateful to Svenska Institutet for embracing me as a deserving candidate forscholarship and enabling me to fulfill my dream of Master’s education. I acknowl-edge City Network Hosting AB for letting us perform tests in their infrastructure.

I am thankful to god, my wonderful parents and sister for their immense supportand motivation. Without you, it would not be the same. Last but not least, ahuge thanks to Vida and all my friends who made this journey worthwhile.

iii

List of Abbreviations

NIST National Institute of Standards and Technology

IaaS Infrastructure-as-a-Service

SaaS Software-as-a-Service

PaaS Platform-as-a-Service

PM Physical Machine

VM Virtual Machine

SLA Service-Level Agreement

PCPU Physical CPU

vCPU virtual CPU

VMM Virtual Machine Monitor

KVM Kernel-based Virtual Machine

QEMU Quick Emulator

OS Operating System(s)

iv

List of Figures

3.1 Minimal architecture and services offered by OpenStack’s Con-troller node(left), Networking node(center) and Compute node(right) 13

3.2 Experimental Methodology portrayed as a sprial model. . . . . . . 15

3.3 Example of graph showing scatter plots and linear correlation as arelation between x and y attributes. . . . . . . . . . . . . . . . . . 16

3.4 Anticipated graphical plots of possibly attainable correlation be-tween host and guest in terms of CPU load . . . . . . . . . . . . . 16

3.5 Server, Virtualization and Network components that are related tocausing or effecting load in the system. . . . . . . . . . . . . . . . 18

3.6 example of output of “uptime” command on terminal. . . . . . . . 18

3.7 example of output of “top” command on terminal. . . . . . . . . . 19

4.1 An OpenStack environment can have “n” number of compute nodesbased on requirement and a controller manages the compute nodes. 22

4.2 Abstraction of hardware, software and virtualization layers in asystem. Nova-compute is not present in a normal virtualizationsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.3 Visual representation of the PMs, VMs and tools used in the im-plementation of experiments. The figure resembles the question asto what could be the relationship between load on host and guestmachines. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.4 Depiction of the experimental setup on OpenStack platform . . . 25

v

4.5 Stages of experiments performed with stress . . . . . . . . . . . . 26

4.6 Depiction of on-premise device setup . . . . . . . . . . . . . . . . 27

4.7 Depiction of on-premise device experimental setup for Single guest 29

4.8 Stress applied initially on 1 vCPU . . . . . . . . . . . . . . . . . . 30

4.9 Stess configured to load 2 vCPUs . . . . . . . . . . . . . . . . . . 31

4.10 Stess configured to load 3 vCPUs . . . . . . . . . . . . . . . . . . 31

4.11 Stess configured to load 3 or more vCPUs. The dotted lines indi-cate that the number of vCPUs being stressed is increased in eachexperimental run. . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.12 Stess-ng configured to impose load om 1 or more vCPUs with 10,20, 30, 40, 50 and 100% load. The dotted lines indicate that thenumber of vCPUs being stressed is increased in each experimentalrun. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.1 CPU load observed in OpenStack compute node and on-premisedevice in multiple guest scenario . . . . . . . . . . . . . . . . . . . 35

5.2 Scatter plots showing the relationship between CPU load averageon host and guest in different vCPU stress conditions. . . . . . . . 37

5.3 Scatter plots showing the CPU load relationships between host andguest in varying load conditions - uptime tool. . . . . . . . . . . . 39




5.7 Scatter plots showing the CPU load relationships between host andguest in varying load conditions - top tool. . . . . . . . . . . . . . 43

vi

List of Tables

3.1 Minimal hardware required to install a controller and a computenode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

4.1 Specifications of the OpenStack Compute Node used for experiments 24

4.2 Specifications of the on-premise device used for experiments . . . 27

5.1 Stress-ng and uptime results on centOS host . . . . . . . . . . . . 44

5.2 Stress-ng and uptime results on Ubuntu host . . . . . . . . . . . . 44

5.3 Stress-ng and top results on CentOS host . . . . . . . . . . . . . . 44

5.4 Stress-ng and top results on Ubuntu host . . . . . . . . . . . . . . 45

vii

Contents

Abstract i

Acknowledgements iii

List of Abbreviations iv

List of Figures vi

List of Tables vii

1 Introduction 1

1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Aims and Objectives . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.4 Expected Contribution . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Related Work 5

3 Methodology 8

3.1 Introduction to Underlying Technologies . . . . . . . . . . . . . . 8

3.1.1 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . 8

viii

3.1.2 Hypervisors . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3.1.3 Cloud Computing and OpenStack . . . . . . . . . . . . . . 10

3.2 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2.1 Experimental Research . . . . . . . . . . . . . . . . . . . . 14

3.2.2 Visual Correlation Analysis . . . . . . . . . . . . . . . . . 15

3.3 Measurement Tools . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4 General Experimental Setup 21

4.1 Experimental Modeling . . . . . . . . . . . . . . . . . . . . . . . . 21

4.2 Experimental setup . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2.1 OpenStack Cloud Test-bed . . . . . . . . . . . . . . . . . . 24

4.2.2 On-premise Test-bed . . . . . . . . . . . . . . . . . . . . . 25

4.2.3 Stress tests on Single Guest . . . . . . . . . . . . . . . . . 28

5 Results and Analysis 34

5.1 Results from OpenStack and on-premise Test-beds . . . . . . . . . 35

5.2 Results of Stress tests . . . . . . . . . . . . . . . . . . . . . . . . . 36

5.3 Results of Stress-ng tests . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.1 Uptime tool . . . . . . . . . . . . . . . . . . . . . . . . . . 38

5.3.2 Top tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

5.4 Discussions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

6 Conclusions and Future Work 46

6.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

ix

References 48

x

Chapter 1

Introduction

This thesis document consists of 6 chapters. Chapter 1 introduces thesis con-cepts, problems statements and motivation in the background section, aims andobjectives, research questions and expected contribution of the thesis in the latersections. Research work associated with the thesis in general is presented inChapter 2. Chapter 3 exhibits the main thesis concepts by briefly discussingthe underlying technologies like virtualization, cloud computing, standard toolsused in experimentation and the methodology adopted. Experimental modelingand setup is highlighted in Chapter 4. Chapter 5 presents the results obtainedfrom the experimental runs with a detailed analysis and discussions. Conclu-sions derived from the analysis along with intended future work are highlightedin Chapter 6.

1.1 Background

Cloud computing is a predominant phenomenon in telecommunication, whichallows sharing of hardware resources with scalability and flexibility by eliminat-ing the constraints on distance. According to NIST, cloud model is classifiedinto three service models based on the resources provided: Software-as-a-Service,SaaS, Platform-as-a-Service, PaaS and Infrastructure-as-a-Service, IaaS. Appli-cation or software itself is offered to the customer as service in SaaS model, whilea platform for building or developing customer’s applications is provided as ser-vice in PaaS. On the other hand, IaaS renders pools of computational, storageor network resources and permits the consumers to provision them as per need.These cloud solutions are utilized on a pay-per-use basis, thereby saving the initialinvestment and maintenance costs for the customers. [1,2,3]

1

Chapter 1. Introduction 2

Similar to traditional systems, monitoring system performance with re-spect to computational resources and applications in cloud computing is impor-tant. Performance monitoring and resource management in cloud infrastructureslies on a higher level of complexity due to lack of standards in these service models,where the customers do not have access to the underlying hardware machines.Cloud service providers, CSP, monitor the resources to ensure quality in theirservices as well as to bill their customers.[4,5]

The costs faced by CSP depend on the CPU utilization and the costsof the user are based on the time of lease of resources. Higher CPU utilizationrequires more electricity and cooling, which amount to around 40% costs in dat-acenters. Although cloud services promise improved resource utilization, it iscomplicated to determine the adequate amount of resources in order to satisfyvariable workload. Load management techniques address this issue of managingcomputational resources according to the varying workloads, thus help in avoidingheadroom or hotspot and minimizing the costs. Our study focuses on CPU loadas metric of monitoring and analysis in cloud networking infrastructures.[6,7,8,9]

CPU load is the demand for computational resources, in other words,is the number of processes running or waiting for the resources. CPU load isdetermined by adding the CPU utilization and saturation. Utilization is the timea processor is busy and is indicated in percentage. Saturation is the number ofprocesses that are waiting for the CPU at a position where the CPU is 100%utilized.[10,11]

Cloud computing is built on virtualization technologies that provide astructure where multiple instances can be run on one PM. Since the customerscannot access the hardware, the responsibility of monitoring the resources lieswith the service provider to meet the SLAs. It is however complex since thereis no possibility for the service provider or user to ensure that the resources areeither over used or under used, which is violation of SLAs [5,12]. In such acase, monitoring Virtual Machine, VM, data is a challenge since the CSP and itscustomer have a different perspective of system performance. In this thesis, weaim at investigating CPU load as viewed from both CSP and customer perspectiveto identify the relationship between CPU load on host to that on the guests. Thisrelationship can help the CSP in billing their customers based on time as wellas resource usage and can also be applied for initiating load prediction for loadmanagement techniques.

The thesis mainly focuses on modeling a framework for CPU load gen-eration, collection and evaluation and on analyzing the guest and host CPU loadrelationships. The experiments are conducted on OpenStack compute node and


an on-premise virtualization device using a set of standard Linux tools.

This research is carried out in collaboration with a second master thesis“An investigation of CPU utilization relationship between host and guests in cloudinfrastructure”. While this thesis solely focuses on methodologies for obtaining,monitoring and analyzing CPU load relationships; the second thesis focuses onmethodologies in obtaining CPU utilization. The experimental scenarios of boththe theses coincide, yet the tools used for measurements and contributions madefrom the observation and analysis of results, differ. [13]

1.2 Aims and Objectives

The aim of the thesis is to establish a framework for CPU load characterizationin federated cloud environment. The experiments outline the performance ofload as defined by a set of standard Linux tools that are assist in obtaining theCPU metrics. This is achieved by imposing well-known stress on the vCPUsand extracting the load values from the Physical Machine, PM and VMs, thereby identifying the relation between the PCPU and vCPU metrics by a visualcorrelation analysis. Study related to the working mechanism of these tools isbeyond the scope of this thesis.

Cloud operators need to scrutinize the resources available regularly inorder to ensure proper load management and meet the SLAs set with cloud cus-tomers.Main objectives of the thesis include:

• Study of commercially off the shelf performance evaluation tools

• Study of available tools or applications for generating workload

• Modelling of experimental platform in OpenStack

• Modeling of standalone virtualization test platform

• Implementation and experimental runs on both the platforms

• Use of standard tools for CPU load collection

• Iterations of the experiments to ensure robust result

• Analysis of the results

• Observation of correlation values

• Visual correlation analysis of obtained host and guest CPU load


1.3 Research Questions

As discussed in section1.2, our goal is to design a model for load collection andevaluation and to investigate host and guest CPU load relationships for betterload management. The following are the research questions formulated:

1. How to model ways of collecting physical and virtual CPU load from a cloudinfrastructure?

2. How do the physical and virtual systems react when load is changed?

3. How to identify relationship between host CPU load and Guest CPU load?

4. In what way is the relationship of host and guest useful in load management?

1.4 Expected Contribution

The expected outcome of this thesis comprises:

• Design, modeling and implementation of cloud as well as virtualization plat-forms for experiments.

• Data collection and observation of the measurements over iterations to en-sure robust results.

• Method for analyzing load relationships between PM and VM in cloud andvirtualization environment.

• A detailed mathematical and visual correlation analysis.

• Identifying association between physical and virtual CPU load for betterload management.

Chapter 2

Related Work

Relevant research work associated with this thesis in multiple aspects is intro-duced in this section. A comparative study is conducted to identify the researchgaps with proposals of other applicable methods and tools.

As mentioned in previous sections, monitoring resource usage for properload management in cloud is an important and on-going topic of research. In[14], Mour, et al., presented a novel approach towards load management in cloudby migrating VMs from a burdened hardware machine to other under loaded ma-chines to help achieve substantial improvement in performance. In their model,they have considered a cloud environment comprising diversified range of physicalas well as virtual machines. The model comprises of several modules handlingindividual tasks of load collection, load evaluation, load prediction, load balanc-ing and VM migration. In order to manage the load it is necessary to collectand evaluate the existing load and make a proper prediction based on currentload as well as load expected in near future. Our thesis concentrates on loadcollection and evaluation in OpenStack cloud and virtualization system aimingat finding a relationship between load on VM and PM, which can be of use inload management models.

Another paper, [15], characterized a dynamic resource management sys-tem for cloud computing with focus on VM CPU mapping to the physical CPUbecause PM capacity should be adequate to fulfill the resource needs of all VMsrunning on it. They tried to estimate future resource usages without lookinginside the VMs, using trend of resource usage patterns. The cloud environmentthey have used is based on Xen hypervisor.

Few research works include empirical study on VM performance in cloud,i.e.; deducing from practical observations rather than believing in theory [16].

5

Chapter 2. Related Work 6

One such work can be found in [17], where the authors have characterized andanalyzed server utilization with diverse workloads encountered in a real cloud sys-tems. Their analysis states that workload variability across different time spansin a cloud environment is relatively high and they further intended to carry outa more fine-grained analysis on the correlation and effects of workload types onserver. In [18], an empirical study on OpenStack cloud’s scheduling functionalityis portrayed. The paper aimed at evaluating behavior of OpenStack schedulerwith respect to memory and CPU cores of PM and VMs and their interactions.Their findings include that CPU cores requested by instances is an importantfactor at the time of resource allocation to VMs in all types of OpenStack sched-ulers. Acquiring the concepts and research gaps from these papers, we set out toperform empirical study applying black-box approach similar to theirs, in Open-Stack cloud networking infrastructures to identify load relationships between hostand guest.

Authors of [19] designed a system to measure CPU overhead in thevirtual computing system to obtain VM and physical CPU utilization to mapa relationship between them. However this does not deal with the impact onPCPU when a VM utilizes more than the allocated resources i.e.; vCPUs, whichis addressed in our experiments.

Corradi et al., in their work “VM consolidation: A real case based onOpenStack Cloud”, conclude that power consumption can be exceptionally re-duced with VM consolidation and they desire to investigate further in the di-rection of workload effects with either CPU or network to understand the VMconsolidation and role of SLAs in service decision process. They also wished todeploy larger OpenStack cloud for further testing [20]. This is related to our the-sis work from the perspective of VM consolidation in real cloud infrastructures,which is experimented and tested in our thesis on a real OpenStack cloud whilemultiple guests share resources.

In [21], CPU utilization while running web service application on cloudand a normal virtualization system is compared which would be helpful for theuser in specifying the capacity to server, based on computational needs. Theyperformed experiments on cloud and virtualization environments and the resultsshow that web service CPU utilization in cloud is higher than that of on-sitedevice. Their scenarios of experimentation are similar to ours, but our goal isto identify load relationships of host and guest in variable workloads rather thanrunning a single application.

These excerpts from the related work show that load management andworkload impact on PCPU and vCPUs are indeed current important topics ofresearch. After careful review and comparison of these works and future targets

Chapter 2. Related Work 7

that stand as motivation to our thesis in multiple ways, we proceed to performour experiments aiming at comparing and identifying PCPU relation with respectto varying workload on vCPUs which could be advantageous in load predictionand management techniques.

Chapter 3

Methodology

This chapter provides a brief description of the underlying technologies along withthe research methodology and measurement tools used in this thesis.

3.1 Introduction to Underlying Technologies

A concise description of fundamental technologies required to grasp the mainidea of the thesis is provided in this section. The fundamentals include virtu-alization, hypervisor and cloud computing principles and standard measurementtools, which detailed in the forthcoming sections respectively.

3.1.1 Virtualization

Virtualization facilitates abstraction of physical servers into virtual machines withtheir OS. The virtual machines can share the resources at the same time. Virtual-ization is efficient as it reduces the need for physical resources by hosting multipleservers on a single physical machine. Two main techniques of virtualization areOS virtualization and Hardware Virtualization.[10]

OS virtualization –In OS virtualization, the operating system is partitioned to multiple instances,which behave like individual guests. These guests can be run, rebooted, admin-istered independent to the host machine. These instances can be virtual serversof high performance to cloud customers and of high density to the operators.The disadvantage of this technique is that the guests cannot run different kernel

8

Chapter 3. Methodology 9

versions and is overcome in hardware virtualization technique.[22]

Hardware Virtualization –This technique of virtualization involves creation of virtual machines with entireoperating system including kernels. This means that they can run different ker-nel versions unlike OS virtualization. Hardware virtualization supplies an entiresystem of virtual hardware components where an OS can be installed. This inturn has the following types involved:

• Full virtualization – binary translations: The instructions passed to theguest kernel are translated during the run time.

• Full virtualization – hardware assisted: The guest kernel instructions arenot translated or modified and are operated by hypervisor running a VirtualMachine Monitor, VMM.

• Para virtualization: This type of virtualization provides a virtual systemwith an interface to virtual OS for using physical resources through hy-percalls. This is mostly prevalent in network interfaces and storage con-trollers.[10, 23]

3.1.2 Hypervisors

Hypervisor is a computer software or hardware or firmware that creates, pro-visions, runs and monitors virtual machines. The following are two types ofhypervisors:Type 1 –This hypervisor runs on the processor directly but not as a kernel software. Thesupervision is taken care of by the first guest on the hypervisor, which runs onring 0. This performs the administration work like creating and running newguests. This is also called as bare-metal hypervisor and provides scheduling forVMs. eg: Xen.Type 2 –Host OS kernel executes and supervises the hypervisor and the guests existingon it. This does not come with a scheduler of it’s own but uses the host kernelscheduler. Eg: KVM.[24]

KVM, Kernel-based Virtual Machine, is a type 2 open source hypervisorused widely in cloud computing. This hypervisor, coupled with a user processcalled QEMU – Quick Emulator, creates hardware assisted virtual instances [25].It is also used in Joyent public cloud and Google Compute Engine [26]. Guestsare at first provisioned by allocating CPU resources as vCPUs and then provided


scheduling by hypervisor. The vCPUs allocation is limited to the physical CPUresource. When it comes to observability, physical resource usage cannot beobserved from the virtual instances.

Hardware support is limited to 8 virtual cores for each physical core onthe virtual machine and once the maximum number of CPUs exceeds, QEMUprovides software virtualization to KVM. In case of multiple VMs hosted by aphysical machine, better performance can be attained by assigning 1 virtual coreper VM.[27]

3.1.3 Cloud Computing and OpenStack

As summarized in section 1.1, cloud computing is a popular technology supportingphysical resource sharing by multiple tenant servers. The services of cloud: IaaSprovides compute; storage, network resources and the consumers are allowed toprovision them based on their needs. PaaS allows users to run their applicationson provided platform and SaaS, on the other hand, allows the customer to utilizethe application via a user interface.[1]

In the Future Internet architectures, federation of such public and pri-vate clouds is an interesting feature. One such project working towards the goalof reaching a cloud federation is FI-PPP, Future Internet Public Private Part-nership framework. This framework consists of several smaller projects and XiFiis the project that concentrates on building and providing smart infrastructuresfor this cloud federation. These Infrastructures facilitate the deployment of sev-eral applications and instances in a unified market place, where, business logicis instantiated as VMs. BTH is one among the current operational nodes acrossEurope.[28]

These nodes of XiFi are interconnected through networking infrastruc-ture. These architectures are beneficial since they are not implemented at oneplace and hence are resilient. If hardware at a place is crashed or out of stor-age, the virtual instances can be moved to other places depending on the alreadyexisting load on them.

The XiFi nodes are heterogeneous clouds built on OpenStack and pro-vide tools and services such as Generic Enablers for the deployment of various ap-plications. They encompass the Cloud principles namely, on demand self-service,resource pooling, scalability and security.[29]

Another example for such federated platform is the infrastructure atCity Network hosting AB. City Network AB is a corporation that provides cloud


domain services and hosting to its customers. Similar to XiFi, the services aredelivered via OpenStack user interface, where the users can create and provisiontheir VMs and utilize the storage and other high quality services offered by CityNetwork. This web interface is called City Cloud and is similar to FIWAREcloud lab of XiFi, which is built on the OpenStack dashboard. However, unlikeXiFi, City Network upgrades regularly and keeps track the latest OpenStackreleases. Currently, they provide hosting on their datacenters in UK, Stockholmand Karlskrona.[30]

City Network architectures have considerable number of customers uti-lizing their services. Identifying and comparing the host and guest CPU load willbe of great value in these operational clouds for better customer service and loadbalancing.

Cloud Networking focuses on providing the VMs with static or dynamicIP addresses, firewalls to make them available to reach from elsewhere. Cloud Net-working provides control and security for the network functions and services de-livered as a service over global cloud computing infrastructure. The word global,here, resembles the federation of the local clouds through network infrastructures.Cloud Networking can be of two types – Cloud Enabled Networking (CEN) andCloud Based Networking (CBN). In the criterion of CEN, the management andcontrol characteristics are moved into the cloud while the network functions suchas Routing, Switching and Security services remain in the hardware. In the sec-ond principle – CBN, the requirement for physical hardware is abolished and thenetwork functions are transferred to the Cloud. Yet, the networking infrastruc-ture needed for fulfilling the networking demands of physical devices remains inthe hardware.[31,32]

OpenStack is open source cloud solution software that provides andmanages IaaS. The infrastructures offered by OpenStack include compute, stor-age and networking resources. OpenStack comes with a combination of core andoptional services that can be implemented in its cloud architecture. The minimalarchitecture of OpenStack consists of its core components that can be realizedin either three-node or two-node architectures. Figure 3.1 shows the core com-ponents and services in a three-node architecture. The two-node architecture issimilar to figure 3.1 eliminating the Network node, whose services are moved tocompute node instead.[33,34]

• Controller –Openstack comes with a cloud controller that controls or administers othercore and optional components or nodes of openstack. Controller node isbuilt to serve as central management system for OpenStack deployments.Controller can be deployed in a single node or various nodes depending on


the requirement. The main services managed by controller include authenti-cation and authorization services for identity management, databases, userdashboard and image services.[35]

• Compute –OpenStack’s compute is known as Nova by its project name. The computenode is the device that comprises the software to host virtual instances,thus providing IaaS cloud platform. Nova does not come with virtualiza-tion software but comprises drivers to interact with the virtualization layerunderneath. Its main components include object storage component calledas swift and block storage component called as cinder that provide storageservices.[36]

• Networking –This aims at providing network services to the instances and enables com-munication between the compute nodes and the virtual instances. It is notnecessary to have a separate node for networking and one of the computenodes can be utilized for the this purpose. The project name of “Network-ing” component in OpenStack is “Neutron”.[34]

• Dashboard –OpenStack provides a user interface for the users to create and provisionvirtual instances as per need. It is formally named as “Horizon” in Open-Stack.[35]

• Telemetry –As shown in figure 3.1, Telemetry is an optional service in OpenStack.Telemetry, or Ceilometer in OpenStack, monitors the OpenStack cloud en-vironment for providing billing services. Ceilometer has an agent to collectCPU and network metrics that are used for billing, auditing and capacityplanning. It has meters to collect duration of instance, CPU time used,number of disk io requests. The CPU utilization reported by this agent isbased on CPU ticks but not workload of the VMs.[37,38,39]

The hardware required for OpenStack installation depends upon thenumber of virtual instances needed or the types of services provided. Table3.1 displays the minimal requirements for a small cloud platform using Open-Stack.[40]

It is evident that cloud networking aims at increasing efficiency. Effi-ciency can be defined as the ability to do something without wastage of material,time and energy. Smart Load management can improve efficiency in cloud infras-tructures with focus on how best the resources can be shared. In cloud, we placeas many VMs as we can in the system at different locations to increase efficiency;


Figure 3.1: Minimal architecture and services offered by OpenStack’s Controllernode(left), Networking node(center) and Compute node(right)

��

�� .�%��$$��B��"��+��41��'��

#��%��-��

F

#�$��4��(��+��2�(��/�+�� '�/�� $5��67��&��,��

J (,��"��,��$��,�� &��,�� "��$��"�%�+��0$�� "��1��%��&��%'

J (��''��%��."��%��&��-�.*��&��-�*��*��+��*��-��"��"�%�1��"'�.��$%��$�"�%�%��%��&��%�%��%��L7"��1�%�-��!��,��-��"�� &��'�-�&�.'

��$$�-��$$��"��%��%��+�0$�� -��1��-��%��-�(�$�*��-�2��1�%�-��"�2��C��%%��%��&��%'�(��%��*��%��&�"��""��$�+��%�+��&��*��'

J (��%��)��+��*��%��'��%��%'�0��"�+��$�-��*��%�%�*/0��%��)��'��*��$%��&�%��%��,�� %��"��&�"�%�+��,�$$��)�!��%��&��%'�3��*��*��"�'

Table 3.1: Minimal hardware required to install a controller and a compute node

Processor RAM Hard DiskController 1 2GB 5GBCompute 1 2GB 10GB

on the other hand, we need quality and speed required for running the VMs. Thesystem is not efficient if it is not completely loaded and we need to know aboutload on the system to characterize efficiency [14,15]. Our hypothesis is to findand understand the relation between CPU load inside the VMs and outside theVMs (host machine), which can be applied to smart load management techniquesto ensure efficient usage of resources.


3.2 Methodology

In this section, we describe the methodology applied to our thesis. In order toreach the aim of our thesis, our methodology includes two subsequent stages -Experimental Research and Visual Correlation Analysis. Experimental Researchsection resembles our approach in performing the experiments. Visual CorrelationAnalysis is our analysis strategy to find relationship between host and guest CPUload.

3.2.1 Experimental Research

Prior to experimentation, we have conducted a thorough study of appropriateliterature and tools and modeled experiments. The approach in carrying outthis thesis is Experimental methodology, which is broadly used while evaluatingsolutions for questions. In an experimental research, the researchers take mea-surements and further make evaluations to answer the questions.[41]

In an experimental approach, it is important to implement the followingthree stages, which are followed in our approach as well.

1. Record keeping – a good documentation of the experiments and configura-tion is important in experimental work for future reference and relevance.

2. Experimental setup design – the platform for running experiments need tobe modeled and designed. At the end of this phase, researcher should doc-ument hardware and software components and the experimental findings.

3. Reporting experimental results – the results obtained should be statedclearly without distortion and discussed.[41]

We have adopted spiral methodology, a well-known method of software develop-ment, in performing our experimental research [42]. As resembled in figure 3.2,our spiral model consists of 4 dimensions – Configuration, experimentation, ob-servation and analysis. First, we make necessary configurations of PM and VMsand then perform the experiments in that configured scenario. The results arethen observed and analyzed. Based on the analysis, the complexity of the config-urations is increased for further experimentation and observation, justifying thespiral model approach.


Figure 3.2: Experimental Methodology portrayed as a sprial model.

!!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Configuration!

Experimentation!

Observation!

Analysis!

3.2.2 Visual Correlation Analysis

There are various possible analysis methods viz. statistical analysis but we startwith a simple approach, where we do a visual correlation analysis of the dataobtained from experimentation. Visual correlation analysis shows associationbetween a set of x and y attributes as depicted in figure 3.3. This visual represen-tation to identify any correlation between 2 sets of values is called as scatter plots[43]. Figure 3.3 is an illustration of the scattered plots and linear correlation linemapping the x and y axes values.

We have applied bivariate analysis to the results to determine empiricalrelationship between CPU load reported by host and guest in the form of scat-ter plots and correlation co-efficient table. Bivariate analysis is used to predictvalue of dependent variable when the value of independent variable is known [44].However, finding the reason for such a behavior is beyond the scope of this thesis.

We set out to find a graphical representation that shows the relationshipbetween host and guest CPU load. Since, we apply a black-box approach asdiscussed under chapter 2, we anticipate that the correlation we obtain from thebivariate analysis can be linear or exponential or highly varying curves as shownin figure 3.4. Identifying the behaviour of this relation is of interest here.

In case of linear correlation obtained, we identify the strength of therelationship from correlation coefficient. Correlation coefficient (ρ) is calculatedusing the following formula:

ρ(H,G) =cov(H,G)

σHσG(3.1)


Figure 3.3: Example of graph showing scatter plots and linear correlation as arelation between x and y attributes.

0 1 2 3 4 5 6 7 80

1

2

3

4

5

6

7

8

X attributes

Y a

ttrib

utes

Figure 3.4: Anticipated graphical plots of possibly attainable correlation betweenhost and guest in terms of CPU load

where:

cov(H,G) =

∑(H −H)(G−G)

n(3.2)

H: host

G: guest


cov: covariance

σH : Standard deviation of host

σG: Standard deviation of guest

As we have gone through the overview of our methodology, we proceedto the measurement tools used in our experiments in the next section.

3.3 Measurement Tools

There are several simple and standard tools that help obtain CPU load metrics ina Linux system. We have followed “Tools” methodology in system performance,which embraces the use of available performance tools, followed by checking themetrics they provide and finally interpreting the metrics obtained [10]. Variousstandard tools used for this experimental purpose are described in this section.The scope of this thesis is limited to the use of these tools for experiments and itdoes not include the study related to the mechanisms of these tools.

In a cloud environment, load can be caused or affected by the factors asdepicted in figure 3.5. Load exists in three dimensions – in physical server, in thenetwork and in the virtualization technology [45,46]. This thesis is about CPUload observation in VMs and the physical server hosting them. Hence, our focus ison the hardware and resource allocation on server and virtualization dimensionsrespectively.

CPU load is the number of processes utilizing the computational re-sources. It is calculated based on exponentially moving average and is updatedevery 1 min, 5 min and 15 min interval [47]. There are various CPU performanceanalysis tools in LINUX OS. Few such tools that provide CPU statistics are topand uptime that fetch the values from /proc/loadavg. Load average implies thedemand for CPU and is computed by summing the number of threads running andthe number that are waiting to run. When there are processes that are queuedfor compute resources, the CPU is said to be in saturation. The load averageshould between the numbers of the CPU cores and if load average is higher thanmaximum number of CPU cores, the CPU is in saturation state since there areprocesses that are queued.[48,49]


Figure 3.5: Server, Virtualization and Network components that are related tocausing or effecting load in the system.Load management

Server

Application

Database System

Operating System

Server Hardware

Virtualization

Network

Hardware

Protoc

ols

Func

tions

Virtualization type

Hypervisor

Resource allocation

Description of the measurement tools used:

UPTIME –This is a command tool used for printing the load averages of thesystem for the last 1-, 5- and 15-minute. As indicated above, load average is theamount of work waiting for the processor and we can compare these three loadaverages in order to identify the trends in CPU load [50]. Figure 3.6 shows theoutput of “uptime” command on Linux terminal.[10]

Figure 3.6: example of output of “uptime” command on terminal.

ptg13750475

224 Chapter 6 ! CPUs

in Chapter 2, Methodology, for more about this, and also Section 2.7, CapacityPlanning, of the same chapter for more on scaling.

6.6 Analysis

This section introduces CPU performance analysis tools for Linux- and Solaris-based operating systems. See the previous section for strategies to follow whenusing them.

The tools in this section are listed in Table 6.6.

The list begins with tools for CPU statistics, and then drills down to tools fordeeper analysis including code-path profiling and CPU cycle analysis. This is aselection of tools and capabilities to support Section 6.5, Methodology. See the doc-umentation for each tool, including its man pages, for full references of its features.

While you may be interested in only Linux or only Solaris-based systems, con-sider looking at the other operating system’s tools and the observability that theyprovide for a different perspective.

6.6.1 uptimeuptime(1) is one of several commands that print the system load averages:

Table 6-6 CPU Analysis Tools

Linux Solaris Description

uptime uptime load averages

vmstat vmstat includes system-wide CPU averages

mpstat mpstat per-CPU statistics

sar sar historical statistics

ps ps process status

top prstat monitor per-process/thread CPU usage

pidstat prstat per-process/thread CPU breakdowns

time ptime time a command, with CPU breakdowns

DTrace, perf DTrace CPU profiling and tracing

perf cpustat CPU performance counter analysis

$ uptime 9:04pm up 268 day(s), 10:16, 2 users, load average: 7.76, 8.32, 8.60

From the Library of Kurt Tutschku

TOP – Another popular tool for monitoring the top running processes is “top”.The top command, when issued, prints a system-wide summary and processesrunning on the terminal. The top results are updated at regular intervals onLinux. System-wide summary is the CPU statistics which includes Load average,percentage utilization of the user, system, nice, idle waiting, hardware interrupt,software interrupt and stealing levels. These values are obtained by taking themean across all CPUs. [10,51]

Top is a well-accepted tool by Beginners of performance analysis dueto its features. In spite of this significance it has a minor drawback due to itsimpact on the performance by applying more load. That is, when top command


is issued on a system, the process or thread that is running top is visible inthe %usr column. This implies that top command consumes the CPU itself forextracting the statistics, which is not favorable in performance analysis. Thisis due to the existence of open, read and close function system calls which havetheir own costs of extracting stats from /proc entries. Top also has another minordrawback of eliminating short-living processes, since the stats are provided aftertaking a snapshot of the /proc entities. Hence, it does not consider the processesthat lived in between the interval of these snapshots [10]. Figure 3.7 shows theoutput of top command on terminal.

Figure 3.7: example of output of “top” command on terminal.

ptg13750475

6.6 Analysis 231

This lists every process (-e) with full details (-f). ps(1) on most Linux- andSolaris-based systems supports both the BSD and SVR4 arguments.

Key columns for CPU usage are TIME and %CPU.The TIME column shows the total CPU time consumed by the process (user +

system) since it was created, in hours:minutes:seconds.On Linux, the %CPU column shows the CPU usage during the previous second

as the sum across all CPUs. A single-threaded CPU-bound process will report100%. A two-thread CPU-bound process will report 200%.

On Solaris, %CPU is normalized for the CPU count. For example, a single CPU-bound thread will be shown as 12.5% for an eight-CPU system. This metric alsoshows recent CPU usage, using similar decayed averages as with load averages.

Various other options are available for ps(1), including -o to customize theoutput and columns shown.

6.6.6 top

top(1) was created by William LeFebvre in 1984 for BSD. He was inspired by theVMS command MONITOR PROCESS/TOPCPU, which showed the top CPU-consumingjobs with CPU percentages and an ASCII bar chart histogram (but not columns ofdata).

The top(1) command monitors top running processes, updating the screen atregular intervals. For example, on Linux:

$ ps -efUID PID PPID C STIME TTY TIME CMDroot 1 0 0 Nov13 ? 00:00:04 /sbin/initroot 2 0 0 Nov13 ? 00:00:00 [kthreadd]root 3 2 0 Nov13 ? 00:00:00 [ksoftirqd/0]root 4 2 0 Nov13 ? 00:00:00 [migration/0]root 5 2 0 Nov13 ? 00:00:00 [watchdog/0][...]

$ toptop - 01:38:11 up 63 days, 1:17, 2 users, load average: 1.57, 1.81, 1.77Tasks: 256 total, 2 running, 254 sleeping, 0 stopped, 0 zombieCpu(s): 2.0%us, 3.6%sy, 0.0%ni, 94.2%id, 0.0%wa, 0.0%hi, 0.2%si, 0.0%stMem: 49548744k total, 16746572k used, 32802172k free, 182900k buffersSwap: 100663292k total, 0k used, 100663292k free, 14925240k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND11721 web 20 0 623m 50m 4984 R 93 0.1 0:59.50 node11715 web 20 0 619m 20m 4916 S 25 0.0 0:07.52 node 10 root 20 0 0 0 0 S 1 0.0 248:52.56 ksoftirqd/2 51 root 20 0 0 0 0 S 0 0.0 0:35.66 events/011724 admin 20 0 19412 1444 960 R 0 0.0 0:00.07 top 1 root 20 0 23772 1948 1296 S 0 0.0 0:04.35 init

From the Library of Kurt Tutschku

Although top has these minor drawbacks mentioned above, this stan-dardized tool is used in well-known performance monitoring applications like Na-gios [52]. We have chosen both uptime and top tools for our experiments afterkeenly studying them, to be able to make a comparison and validation of ourexperiments on CPU load of the system. This helps us find out the how theresults of top differ from uptime, even though both collect data from the /procentries [10]. Our aim however, is not to compare these tools but to use them forour experiments.

Stress is a popular Linux tool that is used to literally stress the CPU withproper computational workload. It comes with various other options of CPUcores, memory, time etc. In our experiments, we have stressed the vCPUs for aperiod of 5 minutes. Scripts for uptime and top commands are written to runafter every 5-minute interval. Stress comes with various options and we have used“-c” in this experiment. Option “-c” is for increasing the number of CPUs thatyou wish to be stressed. This is equal to vCPUs if used on VMs and CPUs ifused on host machine.[53,54]

Stress-ng is a tool similar to stress tool with additional load customizing option.


With stress-ng tool, we can impose desired amount of load in percentage using“-l” option [55].

In our experiments, we started with the well-known tool stress widelyused in system performance tests viz. [56] and moved on to smarter tool stress-ngfor a refined analysis of system behavior under varying workloads.

Chapter 4

General Experimental Setup

Now that we have given a clear direction of our methodology and tools selected,this chapter gives brief description of our experimental design and it is essentialto have a clear picture of basic concepts and our approach from the previouschapters, in order to be able to connect the dots and to interpret the motivationbehind our experimental model. The following sections explain our conceptualmodel and different experimental structures with figures.

4.1 Experimental Modeling

We have modeled our experimental test bed in an OpenStack cloud to observeload relationship between host and guest. As stated earlier, in OpenStack, novacompute is the agent that provisions and runs the VMs (refer section 3.1.3) anddepending on the requirement there can be multiple compute nodes. Figure 4.1is a notion of having multiple compute nodes that are managed my one controllerin an OpenStack environment.

A compute node requires a hypervisor for interacting and managing theVMs. Our OpenStack cloud environment uses KVM hypervisor, which is a type-2hypervisor. Type-2 hypervisors use the host’s kernel scheduler to process requestsfrom VMs and do not have their own schedulers. Figure 4.2 is an abstraction of thelayers involved in a virtualization system with and without OpenStack computeagent. As shown in the figure 4.2, nova compute interacts with the virtualizationmechanism using APIs but does not have virtualization mechanism of its own.[36]

Hence, our hypothesis is that simple virtualization system and a system

21

Chapter 4. General Experimental Setup 22

Figure 4.1: An OpenStack environment can have “n” number of compute nodesbased on requirement and a controller manages the compute nodes.

!Controller!

Compute!Node!1!

Compute!Node!N!

Compute!Node!3!

Compute!Node!2!

Figure 4.2: Abstraction of hardware, software and virtualization layers in a sys-tem. Nova-compute is not present in a normal virtualization system.

with OpenStack compute agent will show similar load behavior. Since we applyspiral development approach to our experimentation, we initially have similar ex-perimental setup on two devices – OpenStack compute node and an on-premisedevice with basic configurations to compare the results. If the observed resultssupport our hypothesis, we increase the complexity in our experiments and con-tinue them on our on-premise setup. The main reason for setting up an on-sitedevice is the difficulty in collecting information from the VMs as a CSP is devoidof access to VMs and also to eliminate the impact of other services running onoperational cloud, on our test environment and avoid inconvenience for customers.


Working accordingly towards our aim of finding relationship betweenhost and guest CPU load, we collect the CPU load from both PM and VMs asportrayed in figure 4.3. The main subject of interest lies in investigating if wecan interpolate and extrapolate from host to guest and vice-versa by observingload behavior on the PM with varying workloads on the VM.

Figure 4.3: Visual representation of the PMs, VMs and tools used in the imple-mentation of experiments. The figure resembles the question as to what could bethe relationship between load on host and guest machines.

!

Guest!1!

Guest!OS!!

Kernel!stress/stress/ng!

uptime!!/top!

Guest!2!

Guest!OS!!

Kernel!

uptime!!/top!

stress/stress/ng!

Guest!4!

Guest!OS!!

Kernel!

stress/stress/ng!

uptime!/top!

Host!

uptime!!/top!

Hypervisor!(KVM)!!

Host!Kernel!!

Host!OS!

CPU!

I/O! Memory!

?!

Guest!3!

stress/stress/ng!

uptime!/top!

Guest!OS!!

Kernel!

Apart from this, we have performed the experiments also on two differenthost OS - CentOS and Ubuntu, popularly used in cloud architectures. As KVMis type 2 hypervisor and runs on host OS as software, the VMs on KVM areprocesses that can be manipulated. OS scheduling is provided by host kernel andour hypothesis leads us to observe behavior of these two different kernel versionsof Linux distribution system. CentOS and Ubuntu are chosen to examine ourhypothesis.

The tools used for load generation are – stress and stress-ng and for CPUload collection are uptime and top. All experimental scenarios and configurationsare described in detail in the succeeding sections of this chapter along with figures.The observations of the experiments are presented in chapter 5.


4.2 Experimental setup

In the initial setup, we modeled the experimental platform on operational cloudInfrastructure and an on-premise device, as described below.

4.2.1 OpenStack Cloud Test-bed

This experiment was executed on one of many compute nodes of City Cloudinfrastructure owned by City Network AB [57]. Features of the compute nodeused for the experiments are listed in Table 4.1.

Table 4.1: Specifications of the OpenStack Compute Node used for experiments

Criteria Specification

CPUIntel ® Xeon ® CPU E5-2697 v2 @ 2.70GHz

48 coresRAM 32GB

Hard disk 2x300 GB SASHypervisor KVMHost OS CentOS 6.6Guest OS Ubuntu 14.04 LTS server

Four virtual machines were created on this compute node with CentOSas host machine. Ubuntu 14.04 LTS server is the OS used for all the 4 VMs onthis node. Each VM was provisioned with 2 vCPUs as KVM hypervisor uses corenumbers for provisioning [27]. The next step in the process is stress testing VMsand simultaneously collecting the CPU load metrics from host. There is no otherapplication running on the guests or hosts while stress tests are in proceeding,to make sure that the CPU is free of any other load infliction than produced bystress.

The setup is as presented in figure 4.4. The four virtual instances presenton the host device are up and running all through the experiment. At first, stresscommand is issued to impose load on 1 VM for 5 minutes while the other VMsare idle. The stress command is customized to stress 2 vCPU cores, which meansthat the VM is imposed with 100% load, as each VM has 2 vCPUs assigned toit. Simultaneously, the load values are obtained from the host with the help ofcommand line tool – uptime.

The experiment is conducted in similar manner for 2 VMs. This means,


Figure 4.4: Depiction of the experimental setup on OpenStack platform!

!!!!!!!!!

!

Guest!OS!!

VM!1!!

Kernel!

Guest!OS!!

VM!2!!

Kernel!

Guest!OS!!

VM!3!!

Kernel!

Guest!OS!!

VM!4!!

Kernel!

Hardware!(Processors)!!

Ubuntu!or!CentOS!!

KVM!/!Libvirt!!

Host!Kernel!!

OpenStack!NOVA!Compute!

2 vCPUs each of 2 VMs were stressed. In other words, 100% load was imposedon 2 VMs each and load metrics were collected once again from host. The rest 2VMs are idle while 2 guests are being stressed.

The next stage involves conducting similar stress tests on 3 VMs with1 VM idle. After collecting the load from host machine, all the 4 VMs areimposed with stress on their dedicated vCPUs, i.e, 8 vCPUs in total. This againis simultaneous with uptime running on host. Figure 4.5 provides a clear pictureof the 4 stages of this experimental run.

4.2.2 On-premise Test-bed

This test setup is installed on a standalone device with KVM hypervisor. Thissetup helps us in distinguishing between the OpenStack existent and non-existentcases for supporting our hypothesis as stated in section 4.1. This test bed is builton the attributes as listed in Table 4.2.

This experimental setup imitates the City Network’s compute node cho-sen for the previous experiment, with minor difference being the number of CPUcores. The hypervisor used is KVM as it is highly used by popular public and pri-vate clouds viz. joyant and google [10], which is also evident from City Network’sinfrastructure.


Figure 4.5: Stages of experiments performed with stress

(a) stage 1 - stress imposed on 1VM

!VM!1!

VM!2!VM!3!

VM!4!

stress!

!!

!!

VM!1!stress!

stress!VM!2!

VM!3!VM!4!

! !!!

!!

!!

VM!1!VM!1!VM!2!VM!2!


stress!stress!

stress!

stress!stress!

stress!stress!

CPU! CPU!

CPU! CPU!

(b) stage 2 - stress imposed on 2VMs

!VM!1!

VM!2!VM!3!

VM!4!

stress!

!!

!!

VM!1!stress!

stress!VM!2!

VM!3!VM!4!

! !!!

!!

!!



stress!stress!

stress!

stress!stress!

stress!stress!

CPU! CPU!

CPU! CPU!

(c) stage 3 - stress imposed on 3VMs

!VM!1!

VM!2!VM!3!

VM!4!

stress!

!!

!!

VM!1!stress!

stress!VM!2!

VM!3!VM!4!

! !!!

!!

!!



stress!stress!

stress!

stress!stress!

stress!stress!

CPU! CPU!

CPU! CPU!

(d) stage 4 - stress imposed on 4VMs

!VM!1!

VM!2!VM!3!

VM!4!

stress!

!!

!!

VM!1!stress!

stress!VM!2!

VM!3!VM!4!

! !!!

!!

!!



stress!stress!

stress!

stress!stress!

stress!stress!

CPU! CPU!

CPU! CPU!

Virtual machine provisioning in KVM is based on the cores availableon the PCPU [27]. For the experimental purposes, 4 guests were created andprovisioned with 2 vCPUs each. That is, out of the 8 total cores available in thePCPU, 2 vCPUs each are allocated to one VM each.

Figure 4.6 provides a pictorial representation of this test-bed setup. Thehost OS used is CentOS 6.6 and it should be noted that the guest OS is Ubuntu


Table 4.2: Specifications of the on-premise device used for experiments

Criteria Specification

CPUIntel ® Xeon ® CPU E3-1230 v2 @ 3.30GHz

8 coresRAM 8 GB

Hard disk 500 GB SASHypervisor KVMHost OS CentOS 6.6Guest OS Ubuntu 14.04 LTS server

14.04 LTS server in all the cases that are considered for experimentation. Also,similar to the previously described OpenStack test case, all the VMs are runningall through the experimentation.

Moving on to the performance tests, stress tool is used for generatingload on the guests. Stress command is first issued on VM 1 for producing 100%load by stressing the 2 vCPUs available on the VM. The CPU load is collectedfrom the Centos host using uptime.

Then, this is repeated for 2 VMs, stressed with 2 vCPUs each. Thehost CPU load behavior when 2 vCPUs are stressed fully is observed via uptimecommand.

Figure 4.6: Depiction of on-premise device setup

VM1$ VM2$ VM3$ VM4$

KVM$

Ubuntu$or$Centos$Hardware$(Processors)$

Host$Kernel$

kernel$ kernel$ kernel$ kernel$

Guest$OS$ Guest$OS$ Guest$OS$ Guest$OS$


The next step is to stress test 3 VMs, while 1 VM is still idle, and observeload characteristics of the hardware CPU through the host with the help of onceagain. The last run is stress testing 7 vCPUs of the 4 VMs and collecting theCPU load from the host machine.

Stress tests on these setups are performed for 5 min on each VM while CPU loadvalues are retrieved from the respective hosts as explained. We reiterated theexperiments ten times to ensure robustness in the results. The data obtained isstored for analysis later on and is produced in Chapter 5.

4.2.3 Stress tests on Single Guest

Way through our spiral approach, the results of this first stage of experimentsconvinced us to contiue our experimentation on the on-site virtualization test-bed with increased complexity. This second main framework of experimentationis described this section.

This experiment is set on the standalone device with virtualization en-vironment but no OpenStack platform. The structure of this experiment is quitedifferent from the previous experimental configuration except for the hardwareused, which is similar in both the cases. Besides, a notable difference is thenumber of VMs that are hosted by the hypervisor, KVM. The specification is asshown is Table 4.2 and is summarized below once again –

• Intel® Xeon® CPU E3-1230 v2 @ 3.30 GHz

• 8 cores CPU

• KVM hypervisor

• Ubuntu 14.04 server for guest OS

• Ubuntu 14.04 LTS Desktop and CentOS 6.6 for Host OS

This is performed on two different Linux distribution OS – Ubuntu andCentOS due to their wide usage in cloud infrastructures. In contrast to the formerarrangement, only 1 VM is created on KVM and is allocated only 2 vCPUs.In other words, out of the total available 8 cores of PCPU, only 2 vCPUs areassigned to the only existing VM. This VM is later stress tested in multiple waysfor recognizing the impact on system load. The experimental setup is as depictedin figure 4.7.


Figure 4.7: Depiction of on-premise device experimental setup for Single guest

Guest&OS&

VM&

Kernel&

Ubuntu&or&CentOS&Hardware&(Processors)&

Host&Kernel&KVM&

4.2.3.A. Testing with “Stress” tool

Initially, stress tool is operated for inserting load on 1 vCPU of the VM. Uptimecommand is run on both the host and guest, to understand the CPU behavior ofboth physical and virtual systems. Load information of the guest systems was notcollected earlier in the multiple (four) guest scenario. This experiment is also runfor 5 min. Figure 4.8 demonstrates this initial step clearly. In our experiments,we have used uptime command to collect cpu load average every 1 second in this5 min period.

Next, the load is applied to 2 vCPUs on the VM. To achieve this, stresson the VM is issued along with the option of 2 (v)CPUs to be stressed. At thesame time, uptime is once again passed to host and guest for collecting the loadvalues over this 5 min time period.

Likewise, stress is now given an option to load 3 vCPUs from the VM.Even though the VM is bound to run with only 2 vCPU resources, stress is atool that can be used for enforcing load on more CPUs than that are actuallyavailable on the compute machines [53]. It is possible to enter a number higherthan the available vCPUs and this would make the OS unstable. We chose todo this experiment to observe the impact on PM load behavior when the guestutilizes more than the allocated resources.

Henceforth, after generating workload on 1 and 2 vCPUs, the tool


Figure 4.8: Stress applied initially on 1 vCPU!!!!!!!!!!

Cores!!!0!!!!!!!!!!!!!1!!!!!!!!!!!!2!!!!!!!!!3!!!!!!!!!!4!!!!!!!!!5!!!!!!!!!6!!!!!7!!

VM!

!1VCPU!!2VCPU!

stress!

stresses 3 vCPUs, while the load retrieval from host and Guest is on going si-multaneously. Thereafter, the rest of the vCPUs are tested for stressing, oneafter the other sequentially including the former vCPUs, that means, 4 vCPUsstressed at once, 5 vCPUs at once, then 6 vCPUs followed by 7 vCPUs. All thesevCPUs are tested subsequently for 5 min and the respective load is retrieved forthat particular period of 5 min.

Figures 4.9, 4.10 and 4.11 present the architecture and test sequences ina clear manner.

Similar tests are conducted on a different host OS, both CentOS and Ubuntu andthe obtained load values are stored for further analysis. One should not confusethat these tools are used at the same time on the same experimental run. Theresults are discussed in the forthcoming Chapter while we move on to the nextand last test case in the upcoming section.

4.2.4.B. Tests with “Stress-ng” tool

The observations from the previous experiments gave us an impression that thereis need for furthur refinement in the experiments. Following our spiral model, wehave used the tool “stress-ng” for a fine-grain analysis of the system behavior undervarying workload conditions. This test case constitutes the same environment anddevice as the preceding experiment. Given, this time, the load infliction is not


Figure 4.9: Stess configured to load 2 vCPUs!!!!!!!!!! ! !!!!Cores!!!0!!!!!!!!!!!!!1!!!!!!!!!!!!2!!!!!!!!!3!!!!!!!!!!4!!!!!!!!!5!!!!!!!!!6!!!!!!7!!

!

VM!

!1VCPU!!2VCPU!

stress

!

Figure 4.10: Stess configured to load 3 vCPUs!!!!!!!!!! !!!!

!

!!!!Cores!!!0!!!!!!!!!!!!!1!!!!!!!!!!!!2!!!!!!!!!3!!!!!!!!!!4!!!!!!!!!5!!!!!!!!!6!!!!!!7!!!

VM!

!1VCPU!!2VCPU!

stress!

carried out by stress but by stress-ng. As described in section 3.3, stress-ngis an updated version of the stress tool and provides the facility of producingcustomized percentage of load with the help of option “-l”.

Provided that there is only 1 VM present on the host machine and thatthis VM is provisioned with 2 vCPUs, stress-ng tool is used for generating certain


Figure 4.11: Stess configured to load 3 or more vCPUs. The dotted lines indicatethat the number of vCPUs being stressed is increased in each experimental run.!

!!!!!!!!!!!!! !!!!Cores!!!0!!!!!!!!!!!!!1!!!!!!!!!!!!2!!!!!!!!!3!!!!!!!!!!4!!!!!!!!!5!!!!!!!!!6!!!!!!7!!

!

VM!

!1VCPU!!2VCPU!

stress!

varying amounts of load on the VM.

In the initial stage of this experimental run, the stress-ng command ismodified to generate 10% of load on 1 vCPU of the guest. The tool is set torun for 5 min; meanwhile uptime and top are used for fetching CPU load fromhost and guest. In similar fashion as the previous case, the number of vCPUsbeing stressed is increased to 2 and more in sequentially. That means, stressing 2vCPUs, 3 vCPUs and so on until 7 vCPUs, with 10% load through stress-ng andthe load values are retrieved in all the cases over the 5 min experimental run.

After the 10% load generation, the load is doubled to 20%. So, 1 vCPUof the guest is inflicted with 20% of load by stress ng and load behavior on host isobserved. After the experiment is conducted for the rest of the vCPUs in order,the load is now increased to 30%.

In simpler terms, this can be explained as – 1 vCPU is loaded by 30%,then 2 vCPUs on whole are loaded by 30%, after which, 3 vCPUs are loaded by30%, followed by 4vCPUs by 30%, 5 vCPUs by 30% load, 6 vCPUs by 30% ofload using stress-ng and lastly all the 7 vCPUs loaded by 30% using stress-ngtool. This is as usual for 5 min with a 5-minute interval between runs. Theload is retrieved simultaneously from the host and guest using top and uptime inseparate runs.

The fourth stage includes imposing 40% load following the same strategy


of 0 vCPU to 7 vCPUs sequentially. The last two stages involve workload of 50%and 100% respectively on the vCPUs. Refer to figure 4.12 for visual abstractionand a rather better understanding of this complex scenario.

Figure 4.12: Stess-ng configured to impose load om 1 or more vCPUs with 10, 20,30, 40, 50 and 100% load. The dotted lines indicate that the number of vCPUsbeing stressed is increased in each experimental run.!

!!!!!!!!!!!!!!!

!!!!Cores!!!0!!!!!!!!!!!!!1!!!!!!!!!!!!2!!!!!!!!!3!!!!!!!!!!4!!!!!!!!!5!!!!!!!!!6!!!!!!7!!!

VM!

!1VCPU!!2VCPU!

Stress!ng!!10/20/30/40/50/100!!

Once again, it is noteworthy to mention that individual runs are per-formed for each OS - CentOS and Ubuntu, each load collection tool - uptimeand top. On the whole, we have collected immense data from these experimentalruns, which, we anticipate will provide the relationship between host and guest.The results obtained were stored for further analysis and are discussed in the nextChapter.

Chapter 5

Results and Analysis

All the experiments were conducted in multiple scenarios as described in section4.2. At first, the experiments were run on compute node of operational OpenStackand the later scenarios were performed on a test bed with just virtualizationenvironment. The runs on the test bed were of different combinations of tools,VMs, vCPUs and load values to observe the change in load behavior on the PCPUwith respect to vCPUs. The experimental runs are for a time period of 5 min witha 5-minute interval between each run. The results that were stored for furtheranalysis are presented and discussed in this Chapter.

Summarizing the experimental runs, each run was for a period of 5 min.The 1st min and the last min of the experiments are considered as warm-up phaseand are eliminated from analysis. All experiments are iterated 10 times and anaverage of these values is plotted and considered for further analysis.

As we know from section 3.3, top and uptime present load-average ofthe system over 1-min, 5-min and 15-min period. 1-min load average values areretrieved during the experiment in order to have stable load values. One shouldnot confuse this 5-min load average with our 5 min experimental run time period.

Following sections present the results with appropriate analysis, fromeach scenario - OpenStack and on-premise setup, stress and stress-ng tests re-spectively.

34

Chapter 5. Results and Analysis 35

5.1 Results from OpenStack and on-premise Test-beds

In this experiment, 1-4 VMs on a cloud and on a device with KVM hypervisor arestress tested and the host CPU load values are collected from the OpenStack’scompute node and our on-site device respectively. Uptime tool is used for loadcollection. Before stressing the VMs, in order to determine the CPU performanceof the system, uptime is issued for 5 minutes. Then, VM 1 is stressed after a5-minute interval post termination of the previous uptime command. Then theexperiments are continued for rest of the VMs with a 5-minute between each run.In short, 5 min experimental runs with 5-minute interval between each run. Thehost OS in both cases is CentOS and each VM is assigned 2vCPUs. The resultsobtained from this experiment are presented in the graphs in figure 5.1.

Figure 5.1: CPU load observed in OpenStack compute node and on-premise de-vice in multiple guest scenario

(a) CPU load-average collected from compute node

60 80 100 120 140 160 180 200 220 2400

1

2

3

4

5

6

7

8

9

10

Time(s)

CPU L

oad Av

erage

CPU load OpenStack compute node

0VM 1VM 2VM 3VM 4VM

(b) CPU load-average collected from on-premise host

60 80 100 120 140 160 180 200 220 2400

1

2

3

4

5

6

7

8

9

10

Time(s)

CPU L

oad Av

erage

CPU Load Average in CentOS Host

0VM 1VM 2VM 3VM 4VM


The x-axis on the graphs in figure 5.1 represents the time of the exper-imental run in seconds, 300s (5 min) in our case. We present the results afterremoving warm up phase of 60 s at the start and end. The y-axis presents theCPU load average observed from compute node - figure 5.1(a) and our on-sitedevice - figure 5.1(b).

The results show that when we increase the load in each VM, the loadon the respective hosts increased. The CPU load average of our on-site deviceshows exponential behavior over the 60s to 240s time period - Figure 5.1(b),while compute node reported constant load over this period - figure 5.1(a). Whenobserved keenly, these graphs show that there is difference between the CPU loadsobserved on both the devices. The load average on the cloud device observed ishigher by a small margin when compared. This load difference is not a substantialdifference as we contemplate it to be due to the impact of several other servicesrunning in the operational cloud environment. This result, in a way, supports ourhypothesis that the load behavior on both systems should be similar as statedclearly above in section 4.1.

After analyzing this, we have decided to continue our experiments onthe on-site device, aiming to reach our goal of finding a relation between host andguest CPU load. The following sections present our observations in this context.

5.2 Results of Stress tests

This section presents the observations made as a result of experiments from ourstress tests on a single guest scenario. This setup was implemented in a virtual-ization device on our premises with two host OS - CentOS and Ubuntu. The hostmachine had only 1 VM and this VM was provisioned with 2vCPUs. The stresswas initially imposed on 1 vCPU, then on 2 vCPUs and so on until 7vCPUS.We collected the load from both PM and VM in this case to map a relationshipbetween them.

We have applied visual correlation analyis to these results to investigateCPU load relationships between host and guest. The scatter plots in figure 5.2show visual correlation between host and guest CPU loads on CentOS device infigure 5.2(a) and on Ubuntu device in figure 5.2(b). The figures 5.2 (a) and (b)are scatter plots that are used to show relation between x-axis - CPU load averageon guest and y-axis CPU load average on CentOS and Ubuntu host respectively.

The scatter plots above show that in case of CentOS host, figure 5.2(a), the trends in correlation plots between PM and VM CPU loads are distorted


Figure 5.2: Scatter plots showing the relationship between CPU load average onhost and guest in different vCPU stress conditions.

(a) Correlation between CPU loads on host and guest in CentOS device

0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5

3

Load Average on Guest

Load

Ave

rage

on H

ost

Correlation between CPU load of Host and Guest in CentOS

1 vCPU 2 vCPU 3 vCPU 4 vCPU 5 vCPU 6 vCPU 7 vCPU

(b) Correlation between CPU loads on host and guest in Ubuntu device

0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5

3


Load

Ave

rage

on H

ost

Correlation between CPU load of Host and Guest in Ubuntu

1 vCPU 2 vCPU 3 vCPU 4 vCPU 5 vCPU 6 vCPU 7 vCPU

with slight linearity observed, on the range of 1 - 7vCPUs that are stressed. Onthe other hand, plots on Ubunut host - figure 5.2 (b), report better correlationthan CentOS host until 6vCPUs while the trend falls in case of 7vCPUs. Thisbehavior is considered strange provided that the guest machines in both cases areUbuntu servers.

We believe that a refinement in the load observation by varying theworkloads is required and hence we proceeded to perform more experiments for


a fine-grain analysis of varying workload behavior.

5.3 Results of Stress-ng tests

In order to have deeper insights into the CPU load behavior, these experimentsare performed using stress-ng tool in varying load conditions ranging from 10, 20,30, 40, 50 and 100 %. This load variability is applied to 1 - 7 vCPUs on the onlyexisting VM, which is provisioned with 2 vCPUs in this setup. Uptime and topcommands are used for load collection from PM and VM in both CentOS andUbuntu host.

To analyze the relationship between PM and VM, we present the scatterplots mapping load average of VM and PM. The following sections show thescatter plots.

5.3.1 Uptime tool

Figures 5.3, 5.4, 5.5 and 5.6 show the uptime load average values obtained fromPM and VM in case of 1, 2, 3 and 7 vCPUs respectively. Their sub-figures (a)and (b) present the observations from CentOS and Ubuntu hosts respectively.In all the scatter plots, x-axis is the load average observed on guest and y-axisrepresents load average observed on host. The scatter plots of load ranging from10, 20, 30, 50 and 100 % are are plotted. As an inital approach of analyzingresults, a linear trendline as an overall interpolation and extrapolation referrence,in case of 100% load is drawn. This comes handy in showing the deviation of thescatter plots from linearity. However, the results of 4, 5 and 6 vCPUs are omitteddue to similarity in most cases but their correlation co-efficient values are shownin section 5.4.

The following are the observations made based on these graphs.

• In all the cases - figures 5.3 to 5.6, scatter plots on Ubuntu host followedthe linear trendline better than the CentOS scatter plots.

• The load behavior in the conditions below 100% load varies to a greatextent, even though most of the plots observed positive correlation.

• The systems correlated in a proper trend with slight deviation at 100% loadcompared to the conditions under 100% load.


Figure 5.3: Scatter plots showing the CPU load relationships between host andguest in varying load conditions - uptime tool.

(a) Correlation between CPU loads on host and guest - 1vCPU - CentOSdevice

0 0.2 0.4 0.6 0.8 1 1.20

0.2

0.4

0.6

0.8

1

1.2


Load

Avera

ge on

Host

Correlation between CPU load of Host and GuestCentOS Host, uptime, stress−ng on 1 vCPU

10% 20% 30% 40% 50% 100% linear

(b) Correlation between CPU loads on host and guest - 1vCPU - Ubuntudevice

0 0.2 0.4 0.6 0.8 1 1.20

0.2

0.4

0.6

0.8

1

1.2


Load

Avera

ge on

Host

Correlation between CPU load of Host and GuestUbuntu Host, uptime, stress−ng on 1 vCPU

10% 20% 30% 40% 50% 100% linear

However, the overall view shows that the plots do not completely obay the lineartrendline. In some cases, it is slightly exponential too - figure 5.5 (a), 5.6 (b).It is anticipated that there is need for more experimentation and research inthis regard. More iterations may be needed to make conclusions based on theseobservations. We can observe the top results from the 7vCPUs in the samescenario in the next section for comparison of the load behavior.




0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5


Load

Avera

ge on

Host


10% 20% 30% 40% 50% 100% linear


0 0.5 1 1.5 2 2.50

0.5

1

1.5

2

2.5


Load

Avera

ge on

Hos

t


10% 20% 30% 40% 50% 100% linear

5.3.2 Top tool

Figure 5.7 shows CPU load behavior as characterized by the tool “top”. Eventhough we have extracted top results in all the scenarios, we believe this onescenario would be sufficient for comparison and hence, omitted the other plots.The discussions section - 5.4, will present correlation co-efficient values from all




0 0.5 1 1.5 2 2.5 3 3.50

0.5

1

1.5

2

2.5


Load

Avera

ge on

Host


10% 20% 30% 40% 50% 100% linear


0 0.5 1 1.5 2 2.5 3 3.50

0.5

1

1.5

2

2.5


Load

Avera

ge on

Host


10% 20% 30% 40% 50% 100% linear

the cases.

As we can interpret from figures 5.6 (a) and 5.7 (a), the load averageobserved by top tool differs from uptime tool to a major extent. However, thebehavior shown in figures 5.6 (b) and 5.7 (b) on Ubuntu hosts highly coincides.

From the above presented results and observations, we understand that CPU




0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5


Load

Avera

ge on

Host


10% 20% 30% 40% 50% 100% linear


0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5


Load

Avera

ge on

Hos

t


10% 20% 30% 40% 50% 100% linear

load relationships between host and guest show likeliness towards a linear rela-tionship in few cases, while few cases show the opposite. However, it can be seenthat the gaps observed between the scatter plots are due to the interval betweenexperimental runs.


Figure 5.7: Scatter plots showing the CPU load relationships between host andguest in varying load conditions - top tool.


0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5


Load

Avera

ge on

Host

Correlation between CPU load of Host and GuestCentOS Host, top, stress−ng on 7 vCPU

10% 20% 30% 40% 50% 100%


0 1 2 3 4 5 6 7 80

0.5

1

1.5

2

2.5


Load

Avera

ge on

Hos

t

Correlation between CPU load of Host and GuestUbuntu Host, top, stress−ng on 7 vCPU

10% 20% 30% 40% 50% 100%

5.4 Discussions

As we have seen the visual correlation analysis of the various experimental scenar-ios in the previous section, we present correlation co-efficient of the relationshipvalues in this section to know the strength of the association. The strength ofcorrelation is said to be large if the correlation co-efficient lies between 0.5 to 1


for positive and -0.5 to -1 for negative correlations. The strength is medium ifthe co-efficient ranges between 0.3 to 0.5 and -0.3 to -0.5 for positve and negativecorrelations respectively.

Tables 5.1 and 5.2 present the correlation co-efficient values obtainedfrom uptime - CentOS and uptime - Ubuntu combinations respectively. Tables5.3 and 5.4 show the correlation co-efficient values of top - CentOS and top -Ubuntu Combinations.

Table 5.1: Stress-ng and uptime results on centOS host

1vCPU 2vCPU 3vCPU 4vCPU 5vCPU 6vCPU 7vCPU10% 0.763 -0.461 0.738 -0.562 -0.459 0.867 0.40820% 0.152 -0.703 0.349 -0.072 0.354 0.205 -0.58930% -0.411 -0.490 0.865 0.568 0.851 0.451 0.61240% -0.326 -0.239 0.886 0.135 0.778 0.203 0.62050% 0.722 0.391 0.025 0.826 0.561 0.557 0.842100% 0.970 0.959 0.965 0.985 0.972 0.963 0.953

Table 5.2: Stress-ng and uptime results on Ubuntu host

1vCPU 2vCPU 3vCPU 4vCPU 5vCPU 6vCPU 7vCPU10% 0.124 -0.172 -0.366 -0.319 -0.413 -0.663 -0.46320% 0.540 -0.549 -0.504 -0.048 0.638 0.835 0.59430% 0.416 -0.129 -0.421 -0.383 0.056 0.547 0.56740% -0.181 -0.150 0.357 0.760 0.135 0.589 0.44550% 0.347 0.484 0.339 0.040 0.585 0.206 0.656100% 0.973 0.987 0.981 0.998 0.989 0.952 0.969

Table 5.3: Stress-ng and top results on CentOS host

1vCPU 2vCPU 3vCPU 4vCPU 5vCPU 6vCPU 7vCPU10% -0.129 0.401 0.603 0.477 -0.488 -0.097 -0.55120% 0.510 0.953 0.752 -0.384 0.520 0.098 0.76130% 0.252 0.704 0.535 -0.253 0.167 0.622 -0.30040% 0.367 -0.031 0.162 0.230 0.417 0.163 -0.34750% 0.131 0.754 0.839 0.891 0.140 0.303 0.186100% 0.892 0.962 0.969 0.822 0.947 0.955 0.853

From these tables of correlation co-efficient values, we observe that whenthe system reaches 100% load and thereby becomes saturated, the strength of


Table 5.4: Stress-ng and top results on Ubuntu host

1vCPU 2vCPU 3vCPU 4vCPU 5vCPU 6vCPU 7vCPU10% 0.003 -0.259 0.485 -0.020 0.581 0.011 -0.04120% 0.157 0.026 -0.107 0.381 0.474 0.170 0.03230% -0.295 -0.551 0.057 0.521 0.806 0.167 0.14140% 0.241 0.207 0.737 0.509 -0.052 0.871 0.49150% 0.653 0.679 0.408 0.533 -0.595 0.774 0.572100% 0.943 0.742 0.979 0.963 0.996 0.972 0.993

association between CPU load of PM and VM is very high. The strength of thisassociation under 50% load condition is between medium to large in most of thecases. Other lower load conditions showed weaker relationships.

These differences observed can be either an inherent feature of the toolsthat needs to be investigated further or a measurement shortcoming that needsmore iterations to have a more robust observation. Since this thesis concentrateson investigating methods to load relationships only, we were successful in accom-plishing this task of our experimental research. These association strengths andmore experimental measurements can lead us to predict the load conditions in asystem accordingly.

Furthermore, in an experimental research, validity threats have to beaddressed to validate the experimentation. In this thesis, internal and externalvalidity threats addressed. Internal validity is the extent to which one can be surethat no other variables except the one we study causes the result. To ensure thatwe meet this, we have thoroughly understood the setup and design and performed10 iterations per experiment. External validity is the extent to which results ofa study can be generalized to other environments and we have addressed this byperforming experiments in multiple scenarios like – VMs, vCPUs, OS ensuringthat it can be generalized.[58]

Chapter 6

Conclusions and Future Work

6.1 Conclusions

The purpose of this research work is to study the CPU load behavior of hostand guest machines under complex workload conditions. This investigation isbased on experiments and measurements in various complex scenarios. We haveperformed numerous experiments that are rarely found elsewhere and tried todefine a methodology for doing this.

Initially, we have modeled the experiments in two scenarios – OpenStackcloud at City Network and an on-premise system. This initial setup is a multipleguest scenario with 4 VMs running on each host. We observed that the resultssupported our hypothesis about OpenStack not having huge impact on systemload when compared to a virtualization system. Convinced by these observa-tions, following our spiral model (refer to section 3.2.1), we further performedthe experiments in more complex scenarios in the virtualization system. Here,we observed CPU load values from both PM and VM. The application of thesescenarios is not feasible in the cloud environment since the CSP does not haveaccess to the VMs (refer to section 1.1) and biding to this, we have continued ourcomplex configurations and experiments on virtualization system using differenttools.

Working towards the aim of finding relationship between host and guestCPU loads, we collected the results from both PM and VM and presented avisual correlation analysis of the results as scatter plots. We also applied bivariateanalysis to study the strength of this association. Our results showed that thecorrelation between guest and host in terms of CPU load is stronger at higherload levels, especially near 100% load. Other load conditions showed weaker

46

Chapter 6. Conclusions and Future Work 47

correlation.

We contribute to the understanding of the CPU load relationships be-tween host and guest and defined methods for achieving it using standardizedtools. The task here is to use the available simple tools in identifying these rela-tionships but not the inherent mechanisms of these tools. The purpose of havingseveral experimental scenarios is not to have a comparison of the tools or OSbut to understand the load behavior. Understanding these load relationships canprove effective in load prediction and management in datacenters.

6.2 Future Work

This research work focused on measurements and analysis under varying workloadconditions using standard tools. There is potential scope in this area to performthese measurements while running real-time business critical applications usingsimilar load collection tools. Also, working towards a load prediction model usingsimilar approach and tools can be considered as future research interest. Apartfrom CPU load, memory and I/O are also the aspects considered in load man-agement techniques, which can also be covered in our future work.

References

[1] P. Mell and T. Grance, “The NIST definition of cloud computing,” 2011.

[2] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud com-puting and emerging IT platforms: Vision, hype, and reality for deliveringcomputing as the 5th utility,” Future Gener. Comput. Syst., vol. 25, no. 6,pp. 599–616, 2009.

[3] A. Lenk, M. Klems, J. Nimis, S. Tai, and T. Sandholm, “What’s inside theCloud? An architectural map of the Cloud landscape,” in Proceedings of the2009 ICSE Workshop on Software Engineering Challenges of Cloud Comput-ing, 2009, pp. 23–31.

[4] Q. Zhang, L. Cheng, and R. Boutaba, “Cloud computing: state-of-the-art andresearch challenges,” J. Internet Serv. Appl., vol. 1, no. 1, pp. 7–18, 2010.

[5] G. Katsaros, G. Gallizo, R. Kübert, T. Wang, J. O. Fitó, and D. Henriksson,“A Multi-level Architecture for Collecting and Managing Monitoring Informa-tion in Cloud Environments.,” in CLOSER, 2011, pp. 232–239.

[6] G. Vilutis, L. Daugirdas, R. Kavaliunas, K. Sutiene, and M. Vaidelys, “Modelof load balancing and scheduling in Cloud computing,” in Information Tech-nology Interfaces (ITI), Proceedings of the ITI 2012 34th International Con-ference on, 2012, pp. 117–122.

[7] M. Gusev, S. Ristov, M. Simjanoska, and G. Velkoski, “CPU Utilization whileScaling Resources in the Cloud,” presented at the CLOUD COMPUTING2013, The Fourth International Conference on Cloud Computing, GRIDs,and Virtualization, 2013, pp. 131–137.

[8] A. Beloglazov and R. Buyya, “Energy Efficient Allocation of Virtual Machinesin Cloud Data Centers,” in Proceedings of the 2010 10th IEEE/ACM Interna-tional Conference on Cluster, Cloud and Grid Computing, Washington, DC,USA, 2010, pp. 577–578.

48

References 49

[9] A. Berl, E. Gelenbe, M. Di Girolamo, G. Giuliani, H. De Meer, M. Q. Dang,and K. Pentikousis, “Energy-efficient cloud computing,” Comput. J., vol. 53,no. 7, pp. 1045–1051, 2010.

[10] B. Gregg, Systems Performance: Enterprise and the Cloud. Pearson Educa-tion, 2013.

[11] “http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html.”

[12] Q. Huang, L. Ye, X. Liu, and X. Du, “Auditing CPU Performance in PublicCloud,” in 2013 IEEE Ninth World Congress on Services (SERVICES), 2013,pp. 286–289.

[13] V. Ahmadi and K. Tutschku, “An investigation of CPU utilization relation-ship between host and guests in cloud infrastructure.”

[14] S. Mour, P. Srivastava, P. Patel, H. Ram, B. N. Gohil, and D. Patel,“Load management model for cloud computing,” in Internet Technology andSecured Transactions (ICITST), 2014 9th International Conference for, 2014,pp. 178–184.

[15] Z. Xiao, W. Song, and Q. Chen, “Dynamic resource allocation using vir-tual machines for cloud computing environment,” Parallel Distrib. Syst. IEEETrans. On, vol. 24, no. 6, pp. 1107–1117, 2013.

[16] G. Dodig-Crnkovic, “Scientific methods in computer science,” Proc. Conf.Promot. Res. IT New Univ. Univ. Coll. Swed. Skövde Suec., pp. 126–130,2002.

[17] P. Garraghan, P. Townend, and J. Xu, “An analysis of the server character-istics and resource utilization in google cloud,” in Cloud Engineering (IC2E),2013 IEEE International Conference on, 2013, pp. 124–131.

[18] O. Litvinski and A. Gherbi, “Experimental evaluation of openstack computescheduler,” Procedia Comput. Sci., vol. 19, pp. 116–123, 2013.

[19] G. Tong, H. Jin, X. Xie, W. Cao, and P. Yuan, “Measuring and AnalyzingCPU Overhead of Virtualization System,” in Services Computing Conference(APSCC), 2011 IEEE Asia-Pacific, 2011, pp. 243–250.

[20] A. Corradi, M. Fanelli, and L. Foschini, “VM consolidation: A real casebased on OpenStack Cloud,” Future Gener. Comput. Syst., vol. 32, pp.118–127, 2014.

[21] M. Gusev, G. Velkoski, S. Ristov, and M. Simjanoska, “Web service CPUoverutilization in the cloud,” in ICIT 2013 The 6th International Confer-ence on Information Technology. Cloud Computing, 8-10 May 2013, 2013,pp. 459–66.

References 50

[22] B. Golden, Virtualization for dummies. John Wiley & Sons, 2011.

[23] D. Marshall, Understanding Full Virtualization, Paravirtualization, andHardware Assist. Copyright, 2007.

[24] M. Portnoy, Virtualization essentials, vol. 19. John Wiley & Sons, 2012. [

[25] “http://www.linux-kvm.org/page/Main_Page.” .

[26] “ ‘Frequently Asked Questions,’ Google Cloud Platform. [Online]. Available:https://cloud.google.com/compute/docs/faq. [Accessed: 09-Sep-2015].” .

[27] “KVM Performance Limits for virtual CPU cores,” Open-Source Routing and Network Simulation. [Online]. Available:http://www.brianlinkletter.com/kvm-performance-limits-for-virtual-cpu-cores/. [Accessed: 09-Sep-2015].

[28] “XIFI-D1.5-Federated Platform Architecture v2.pdf. Available athttp://wiki.fi-xifi.eu/Main.” .

[29] “HP, Seven design principles for buildinga converged cloud. Retrieved 7January 2015.http://www8.hp.com/h20195/v2/GetDocument.aspx?docname=4AA4-6524ENW.” .

[30] “City Cloud - Cloud Computing,” City Network. .

[31] T. Benson, A. Akella, A. Shaikh, and S. Sahu, “CloudNaaS: A Cloud Net-working Platform for Enterprise Applications,” in Proceedings of the 2NdACM Symposium on Cloud Computing, New York, NY, USA, 2011, pp.8:1–8:13.

[32] “Cloud-based networking,” Wikipedia, the free encyclopedia. 05-Aug-2015.

[33] “ ‘OpenStack: The Open Source Cloud Operating System’ availiable athttp://www.openstack.org/software/.” .

[34] “ ‘Openstack installation guide’ availiable athttp://docs.openstack.org/juno/install-guide/install/apt/content/ch_overview.html.”.

[35] “ ‘ Designing for Cloud Controllers and Cloud Man-agement’ availiable at http://docs.openstack.org/openstack-ops/content/cloud_controller_design.html.” .

[36] “ ‘Nova Concepts and Introduction’ availiable athttp://docs.openstack.org/developer/nova/nova.concepts.html#nova-concepts-and-introduction.” .

References 51

[37] “OpenStack Metering Using Ceilometer - Pure Play OpenStack.” .

[38] “ ‘Openstack system architecture’ availiable athttp://docs.openstack.org/developer/ceilometer/architecture.html.” .

[39] “ ‘ Billing for openstack’ availiable at http://docs.openstack.org/developer/ceilometer/glossary.html#term-billing.” .

[40] “http://docs.openstack.org/icehouse/install-guide/install/yum/content/basics-prerequisites.html.” .

[41] J. N. Amaral, “About computing science research methodology,” 2011.

[42] B. W. Boehm, “A spiral model of software development and enhancement,”Computer, vol. 21, no. 5, pp. 61–72, 1988.

[43] “ ‘Bivariate analysis’ availiable at http://sociologyindex.com/bivariate_analysis.htm.”.

[44] E. Babbie, The practice of social research. Cengage Learning, 2015.

[45] “Physical Servers vs. Virtual Machines,” Backup Academy. [Online].Available: http://www.backupacademy.com/blog/physical-servers-vs-virtual-machines.html. [Accessed: 03-Oct-2015].

[46] “Network congestion,” Wikipedia, the free encyclopedia. 03-Sep-2015.

[47] R. Walker, “Examining Load Average,” Linux J, vol. 2006, no. 152, p. 5–,Dec. 2006.

[48] M. Richard, Solaris" Performance And Tools: Dtrace And Mdb TechniquesFor Solaris 10 And Opensolaris. Pearson Education India, 2007.

[49] T. Myer and J. Barnaby, “Tenex Executive Manual. Bolt Beranek and New-man,” Inc April, 1973.

[50] “ ‘ uptime’ Linux man page’ availaible athttp://linux.die.net/man/1/uptime.” .

[51] “ ‘ top’ Linux man page’ availaible at http://linux.die.net/man/1/top.” .

[52] “https://exchange.nagios.org/directory/Plugins/System-Metrics/CPU-Usage-and-Load.” .

[53] “http://people.seas.harvard.edu/ apw/stress/.” .

[54] “http://manpages.ubuntu.com/manpages/lucid/man1/stress.1.html.” .

[55] “http://manpages.ubuntu.com/manpages/utopic/man1/stress-ng.1.html.” .

References 52

[56] H. Cheng, Z. Chen, N. Sun, F. Chen, and M. Wang, “Evaluation frameworkof virtualization systems for cloud computing,” in 2012 IEEE Asia PacificCloud Computing Congress (APCloudCC), 2012.

[57] “High performance public OpenStack cloud!,” City Cloud. [Online]. Available:https://www.citycloud.com/. [Accessed: 03-Oct-2015].

[58] Steven Taylor and Gordon J.G. Asmundson, "Ineternal and External Valid-ity In Clinical Research", [Online].Available: http://www.sagepub.com/upm-data/19352_Chapter_3.pdf.

Documents

Monitoring and Analysis of CPU load relationships between