Network virtualizationas enabler for cloud networking

DANIEL TURULL

Licentiate Thesis in Information and Communication TechnologySchool of Information and Communication Technology

KTH Royal Institute of TechnologyStockholm, Sweden 2016

TRITA-ICT 2016:16ISBN 978-91-7595-968-9

KTH School of Information andCommunication Technology

SE-164 40 KistaSWEDEN

Akademisk uppsats som med tillstånd av Kungl Tekniska högskolan framlägges tilloffentlig granskning för avläggande av licentiatexamen i Informations- och kommu-nikationsteknik, Kommunikation: Tjänster och infrastruktur torsdagen den 2 juni2016 klockan 14:00 i Sal C, Electrum, Kungl Tekniska högskolan, Isafjordsgatan26, Kista.

© Daniel Turull, June 2016

Tryck: Universitetsservice US-AB

iii

Abstract

The Internet has exponentially grown and now it is part of our everydaylife. Internet services and applications rely on back-end servers that are de-ployed on local servers and data centers. With the growing use of data centersand cloud computing, the locations of these servers have been externalizedand centralized, taking advantage of economies of scale.

However, some applications need to define complex network topologiesand require more than simple connectivity to the remote sites. Therefore, thenetwork part of cloud computing, what is called cloud networking, needs tobe improved and simplified.

This thesis argues that network virtualization permits to fill the missinggap and we propose a network virtualization abstraction layer to ease the useof cloud networking for the end users. We implement a software prototype ofour ideas using OpenFlow. We also evaluate our prototype with state of theart controllers that has similar functionalities for network virtualization.

A second part of this thesis focuses on developing a tool for performancetesting. We have improved the widely used tool pktgen with receiver function-alities. We use pktgen to generate traffic for our experiments with networkvirtualization.

Keywords: network virtualization, OpenFlow, pktgen, cloud networking, cloud,network performance, benchmarking, datacenters, perfomance testing, Libnetvirt

v

Sammanfattning

Internet har vuxit exponentiellt och nu är det en del av vår vardag.Internet-tjänster och tillämpningar bygger på backend-servrar som är belägnapå lokala servrar och i datacenter. Med den ökande användningen av datacen-ter och molnbaserade datatjänster blir placeringen av dessa servrar alltmerhar externaliserad och centraliserad för att dra nytta av stordriftsfördelar.

Men vissa nätverkstillämpningar har mer komplexa nätverkstopologieroch kräver mer än bara enkla förbindelser till anslutningspunkterna. Därförbehöver nätverksdelen av molnbaserade tjänster, det som kallas molnbaseradenät, både förbättras och förenklas.

Denna avhandling lyfter fram tesen att nätverksvirtualisering kan fyl-la detta gap vi föreslår ett abstraktionslager för nätverksvirtualisering somunderlättar användandet av molnbaserade nät. Utgående från dessa idéer ut-vecklar vi en prototyp baserad på Openflow. Vi utvärderar denna prototypgenom jämförande studier med moderna styrenheter för nätverk, "controllers",som har liknande funktionalitet för nätverksvirtualisering.

Den andra delen av denna avhandling är inriktad mot utveckling av ettverktyg för prestandatester i nätverk. Vi har utökat det välkända verktygetpktgen med mottagare- funktionalitet. Vi använder pktgen som trafikgenera-tor i våra experiment med prototypen för nätverksvirtualisering.

Nyckelord: nätverksvirtualisering, OpenFlow, pktgen, molnbaserade nät, moln-baserade datatjänster, nätverksprestanda, benchmarking, datacenter, prestandaut-värdering, Libnetvirt

vii

Acknowledgements

I would like to thank Peter Sjödin and Markus Hidell for the help in improvingthis thesis as well as the papers that I had co-authored with them. Without theirsupport, I would not have finished this thesis. Also I will like to thank them forencouraging me to finish this thesis.

Voravit Tanyingyong, who I shared the office for a few years, and we becomegood friends. He also gave constructive comments about my papers and tested thepktgen receiver.

Robert Olsson, who is a source of inspiration and help related to the part of pk-tgen as well as performance testing. It was a pleasure to have technical discussionswith him about the Linux Kernel.

My former colleagues at NSlab, Raul Jimenez, Flutra Osmani and Pehr Soder-man, for all the good times and support that I got from them.

The partners in SAIL, specially Azimeh Sefidcon, Bob Melander and HareeshPuthalath, who gave me access to the Ericsson facilities to experiment with OpenFlowon the hardware.

And last but not least, to my parents, Josep M and Núria, who pushed me tofinish the thesis and all the love and opportunities that they are giving to me.

This thesis was financed by the European Union Frame Program (FP) 7 projectSAIL (2010-2013) under grant number 257448.

Contents

Contents ix

List of Figures x

I Thesis Overview 1

1 Introduction 31.1 Research Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Thesis contribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Thesis organization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Literature Review 72.1 Network virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . 72.2 OpenFlow and Software Define Networks (SDN) . . . . . . . . . . . 102.3 Cloud Computing and Cloud Networking . . . . . . . . . . . . . . . 142.4 Network performance tools . . . . . . . . . . . . . . . . . . . . . . . 15

3 Problem definition 173.1 Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.2 Research Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4 Thesis contribution 234.1 List of contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Relation between papers . . . . . . . . . . . . . . . . . . . . . . . . . 264.3 Relation to state of the art . . . . . . . . . . . . . . . . . . . . . . . 27

5 Conclusions and Future work 295.1 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

Bibliography 31

ix

x List of Figures

II Research Papers 43

List of Figures

1.1 High level view of an OpenFlow network . . . . . . . . . . . . . . . . . . 4

2.1 Example of tunneling encapuslation . . . . . . . . . . . . . . . . . . . . 92.2 L3VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.3 OpenFlow architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3.1 LibNetVirt architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.1 Relation between papers . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

List of Abbreviations

ATM Asynchronous Transfer Mode. 13

CE Customer Edge. 8

CPU Central Processing Unit. 10

FNS Flash Network Slice. 15, 18

ForCES Forwarding and Control Element Separation. 13

GRE Generic Routing Encapsulation. 9

IaaS Infrastracture as a Service. 14

ICMP Internet Control Message Protocol. 25

IEEE Institute of Electrical and Electronics Engineers. 8

IETF Internet Engineering Task Force. 13

IP Internet Protocol. 10, 24

IPsec Internet Protocol Security. 9

IPv4 Internet Protocol version 4. 7

IPv6 Internet Protocol version 6. 7

IT Information Technology. 3, 14

L1VPN Layer 1 Virtual Private Network. 9

L2TP Layer Two Tunneling Protocol. 9

L2VPN Layer 2 Virtual Private Network. 3, 9

L3VPN Layer 3 Virtual Private Network. 9

xi

xii LIST OF ABBREVIATIONS

MAC Medium Access Control. 8

MPLS Multi-Protocol Label Switching. 3, 9, 24

NAT Network Address Translation. 13

NCP Network Control Point. 13

NIST National Institute of Standards and Technology. 14

PE Provider Edge. 8

PPTP Point-to-Point Tunneling Protocol. 9

PPVPN Provider-Provisioned VPN. 8

SDN Software Defined Networks. 12

SP Service Providers. 8

TCP Transmission Control Protocol. 10, 25

UDP User Datagram Protocol. 24, 25

VLAN Virtual Local Area Network. 3, 8, 24

VPLS Virtual Private LAN Services. 9

VPN Virtual Private Network. 8, 15

VPWS Virtual Private Wire Service. 9

VXDL Virtual private eXecution infrastructure Description Language. 10

Part I

Thesis Overview

1

Chapter 1

Introduction

This chapter introduces our research motivation for this thesis work. Additionally,it summarizes the main contributions and outlines the organization of the remainingchapters.

1.1 Research Motivation

Nowadays, the cloud computing paradigm creates a tendency to externalize or con-centrate the Information Technology (IT) infrastructure from an organization intothe cloud (public or private). A cloud [1] is a set of resources that can be reused andscaled up or down without knowing their exact physical location. Cloud resourcesconsist of computing, storage, and network resources. Computing resources referto the execution of the applications and how the CPU is assigned to the differentexecution instances. Storage resources refer to how the data is stored in the cloud.Network resources refer to provisioning of network connectivity and capacity in acloud infrastructure and how it can be used to interconnect other resources, thatis, computing capacity and storage.

Different network virtualization techniques exist that mainly have to be config-ured by hand, such as L2VPN, MPLS, VLANs and tunneling. Recently, differentprojects have tried to generate frameworks to manage the creation and termina-tion of point-to-point connections [2–6], but a unique abstract interface for networkvirtualization management does not exist. In a cloud environment, users shouldbe able to easily describe and create networks to interconnect their resources ina dynamic and autonomous way, independently of the network operator’s cloudprovider. However, a well-defined uniform interface to manage networks in such away is lacking.

An architecture that is gaining much attention is OpenFlow [7, 8]. OpenFlowis an open network interface that decouples the forwarding of the packets from theforwarding decisions. Traditionally, both functions have been implemented in thesame device and they could not be controlled from an external unit. OpenFlow

3

4 CHAPTER 1. INTRODUCTION

creates an open interface that allows an external controller to modify how theswitch or the router behaves. A high level overview is illustrated in Figure 1.1.The controller is a piece of software that runs on a server, or a cluster of servers, andhas a global view of the network. In other words, OpenFlow offloads the logic of therouters or switches to an external server. Nowadays, several manufacturers buildOpenFlow enabled switches and several research projects work with OpenFlow.

Controller

Figure 1.1: High level view of an OpenFlow network

1.2 Thesis contribution

Our main goal in this thesis is to investigate how a unified abstraction for virtualnetworks can be developed in order to increase the dynamicity when deployingand/or reconfiguring virtual networks in the cloud. A cloud provides on-demandand automated access to computer resources. Those resources need to be intercon-nected. The creation of this interconnection needs to be dynamic, in other words,operators of the cloud should not have to be involved in setting up the interconnect,neither in its reconfiguration.

An abstraction is a standardized, common interface that allows the user touse any common operation without the need to be specific about the underlyingtechnology. In case of a network abstraction, it should allow the user to use anykind of network technology to implement the virtual network, as long as the networktechnology conforms to the abstraction.

This abstraction should help in the automation of the virtual networks in thecloud. Therefore, we should ask ourselves: How can we define an abstraction thatsimplifies the deployment of virtual networks? Which technologies can we use todemonstrate the usefulness of the network abstraction? Which performance canwe obtain with the selected technologies? What tools and metrics do we needto measure the performance of the selected technologies? How does the networkabstraction interact with other parts of the cloud?

1.3. THESIS ORGANIZATION 5

In order to answer the previous questions, we explore the definition of a networkabstraction. For this, we investigate different models and define the architecture anda set of basic operations to control the network. After describing the abstraction, weimplement a prototype with OpenFlow that implements the basic functionality ofthe model that we envision. Thereafter, we evaluate it in the context of OpenFlownetworks and define measurements to determine the performance of the applicationsthat can use the abstraction. In addition, we design a receiver module for pktgen,a popular packet generator tool in Linux, to do performance tests on high-speednetworks. This tool helps to measure different aspects of network performance inour experiments.

In this work, we follow an experimental approach to develop our ideas, wherewe have built several prototypes and conducted extensive experimental work.

1.3 Thesis organization

This licentiate thesis is a compilation thesis, with two parts. The first part providesan overview of the thesis and a description of the problem statements. The secondpart is a compilation of the published, peer-reviewed papers of the author of thisthesis.

The rest of the first part is organized as follows. Chapter 2 provides the neces-sary background to understand this thesis. Chapter 3 defines the hypothesis andresearch problem. Chapter 4 describes the author’s publications. Finally, Chapter5 concludes the thesis and provides directions for future work.

Chapter 2

Literature Review

This chapter describes the state of the art and introduces the necessary backgroundto understand the rest of the thesis. The literature review is complementary to thebackground sections in the published articles.

2.1 Network virtualization

Computer networks permit to exchange information among computers in a fast andreliable fashion. Networks are designed to exchange information between all nodesconnected to it. However, organizations might want to separate the traffic betweenthe different users. For instance, let us assume that inside an organization, we needto isolate the traffic between groups of users. In this case, we need a network for eachgroup, which increases the cost of managing the networks. Here is where networkvirtualization comes into play. Network virtualization is a set of technologies thatpermit to share physical network resources among different users while still keepingthe users isolated from each other.

Network virtualization [9] technologies separate the physical infrastructure fromthe logical view. Each group of users perceives the network as if is the only groupusing the network. The physical elements of the network are shared between groups,however every group has the perception that it has the network exclusively for itselfand does not see the network traffic from other groups. The key idea of networkvirtualization is the coexistence of different networks in the same infrastructure.

Network virtualization has been claimed [10, 11] to help solving the problem ofthe ossification of the Internet. This problem refers to the difficulty in changingthe architecture of the Internet to allow further growth. A clear example of thisis the deployment of IPv6 [12] which aims to solve the problem of exhaustion ofIPv4 addresses. IPv6 was approved in 1998 and it is still far from being globallydeployed even though IPv4 address exhaustion is now a fact [13].

Network virtualization is a useful mechanism to share network resources. Itpermits to share equipment while keeping traffic isolated. This allows for cost sav-

7

8 CHAPTER 2. LITERATURE REVIEW

ings, since less equipment is needed and maintenance costs are lower. In addition,network virtualization permits to adapt and modify the network without the needof physical changes, which could potentially increase the speed of deployment ofnew services.

Network virtualization could have an important role in cloud computing becauseit has the potential to personalize and isolate the different networks that intercon-nect computing and storage resources. Interconnection of the different resourcesis a key component when deploying complex topologies that use shared resourcesamong different users.

Khan et al. [4] make an analogy between network virtualization and computervirtualization, and argues that network virtualization may not bring new capabili-ties, but it permits to organize networks in a way to overcome network ossification.

The key idea in network virtualization is the coexistence of different networksin the same infrastructure. Two of these technologies are: Virtual Local AreaNetworks (VLANs) and Virtual Private Networks (VPNs).

VLANs

Virtual Local Area Networks (VLANs) were standardized in 1998 with the standardIEEE 802.1q [14]. This protocol aims to generalize several proprietary protocolsthat existed then, such as, Cisco’s ISL (Inter-Switch Link) [15] and the 3Com’s VLT(Virtual LAN Trunk). The objective is to limit the broadcast domain of Ethernetnetworks. Simple switches can distinguish VLANs by port and more advancedswitches use frame tagging to differentiate between VLANs. IEEE 802.1q addsa four byte field between the MAC address and the EtherType/length field. Itincludes 12 bits of VLAN identification.

IEEE 802.1q has scalability problems because of the limited number of VLANsidentifiers, since the 12 bits permits only up to 4096 VLANs. This problem ispartly solved with QinQ encapsulation [16]. QinQ allows multiple VLAN tags tobe inserted into a single ethernet frame.

VPNs

Virtual Private Networks (VPNs) [17] are networks that use public networks toextend private networks to interconnect hosts located in different and distant lo-cations, as illustrated in Figure 2.2. In the enterprise world, VPNs are provi-sioned by Service Providers (SP) and are then known as Provider-Provisioned VPNs(PPVPN) [18]. Each PPVPN had two kinds of elements: Customer Edge (CE) andProvider Edge (PE). Customer Edges are in the customer facilities, for instance,hosts or routers, and interconnect to the Provider Edge Routers. PPVPNs provideisolation between different customers of PPVPNs. The PEs forward the customertraffic to the CEs that belong to the same VPN though the service provider’s corenetwork.

2.1. NETWORK VIRTUALIZATION 9

PPVPNs exist at Layer 3 (L3VPN), Layer 2 (L2VPN) and Layer 1 (L1VPN).VPN solutions are usually based on tunneling techniques. Tunneling means en-capsulation of a packet as payload to another protocol packet. An example of anencapuslated frame is illustrated in Figure 2.1. Some of the encapsulation protocolsfor L3VPNs are MPLS [17], GRE [19], PPTP [20], L2TP [21] or IPsec [22].

Header O Payload

Header O Payload

Header

Payload

Original packet

Encapsulated packet

Figure 2.1: Example of tunneling encapuslation

When a customer packet arrives to the CE (source CE), the CE looks up thedestination network and matches it with a destination CE. Then the packet isencapsulated by the source CE and transmitted through the Service Provider Net-work to the destination CE that belongs to the same VPN. The destination CEdecapsulates the packet, and delivers it to the destination customer network.

Public Network

PE

PE

PE

CECE

CE

Figure 2.2: L3VPNs

L2VPNs [23, 24] differ from L3VPN in that they provide the illusion that bothsites are in the same LAN, which gives the advantage of being capable of supportingheterogeneous higher-layer protocols. There are two kinds of L2VPNs: point-to-point Virtual Private Wire Service (VPWS) and point-to-multipoint Virtual PrivateLAN Services (VPLS). VPWS extends a local ethernet network to another site in a

10 CHAPTER 2. LITERATURE REVIEW

point to point fashion, while VPLS is designed to extend a multicast or broadcastdomain to multiple remote sites.

Network mapping and network abstractionsVirtual Network Mapping [25, 26] is the procedure of mapping components of thevirtual network onto the actual physical substrate. Each node of a virtual networkis mapped onto a physical node, and each virtual link is mapped onto a physicalnetwork path in the substrate. This mapping tries to fulfill a set of constraints,for example, topology constraints, data rate, CPU capacity, and so on. Its goal isto solve the Virtual Network Mapping Problem as efficiently as possible. However,this problem is known to be an NP-hard problem [25]. Different authors proposedifferent heuristics in order to reduce the complexity focusing on a subset of con-straints [6, 10, 27–39]. Cong et al. [40] suggest that using a topology for requestingvirtual networks limits the efficiency of the mapping algorithms. They propose analternative that uses traffic matrices to specify virtual network requests in theirmapping algorithm.

In addition to the algorithms to solve the mapping problem, several high-levellanguages have been created to describe a virtual network topology.

Virtual private eXecution infrastructure Description Language (VXDL) [41] is alanguage that permits modeling and specify the interconnection of virtual resources.It permits to describe the virtual topology, virtual routers and the timeline of thevirtual network.

Another approach in describing networks is to use a network abstraction. Kellerand Rexford [42] propose the single router abstraction that can be used to describea network, and this router abstraction can have different properties to define therequired behavior. Not defining an internal topology provides more flexibility inallocating resources, reducing the constraints on the virtual network mapping prob-lem. The characteristics of the interconnected nodes are defined by policies. In ad-dition, it provides a simplified view towards the user, since it only needs to definethe desired connectivity.

Kang et al. [43] optimize the single router abstraction defined by Keller andRexford, by proposing an architecture with three different parts: a high level whichdefines end-to-end connectivity on top of the single node abstraction, a middlelayer that decides the routing policies, and a compiler that converts the policies toforwarding rules.

2.2 OpenFlow and Software Define Networks (SDN)

OpenFlow [44] is a protocol that separates the data plane from the control plane,allowing an external controller to manage how the flows are forwarded in the switch.For instance, a flow can define a TCP connection, or all traffic to a specific IPaddress or port. Figure 2.3 shows a general overview of the OpenFlow architecture.

2.2. OPENFLOW AND SOFTWARE DEFINE NETWORKS (SDN) 11

OpenFlow switch

Control Path OpenFlow

Data Path (Hardware)

OpenFlow controller

OpenFlow Protocol (SSL/TCP)

Figure 2.3: OpenFlow architecture

OpenFlow networks are managed by external controllers that communicate di-rectly with the switches. The OpenFlow controller is responsible for setting upthe flow entries in the switches and for reacting to different network events. Flowentries are composed by matching rules and actions. A matching rule contains aset of values to match with a set of packet header fields in the incoming packet toidentify a flow. Examples of actions taken for an identified packet are: droppingthe packet, tagging it or forwarding it to an output port. When a switch receivesa packet that does not match any of the current entries, the switch forwards thepacket to the controller, which decides how to process the packet.

The authors of OpenFlow claim that OpenFlow will enable a more rapid inno-vation in the field of networking, where the Internet is claimed to be suffering fromossification [11]. Ossification stops innovation because new ideas are difficult to im-plement in real networks. With a logically centralized controller, network operatorswill have a network-wide view, improving resource utilization and flexibility [7]. Acentralized controller also enables new types of network applications [45, 46].

OpenFlow promises a flexible and decoupled method to control how the packetsare forwarded in a network. Network equipment needs a control plane to decidehow to forward the packets and a data plane which implements this forwarding.Traditionally, both functions are implemented in a single closed system that only thevendor can modify and update. This creates a barrier for new players to enter themarket, and slows down the improvement of network equipment and developmentof new functions.

OpenFlow decouples the control plane from the data plane, providing an openinterface to modify the forwarding tables of a switch or router. Therefore, thenetwork control functionality can be located outside the switch on a general purposeserver (a controller), which can be programmed and modified in a straight-forwardfashion. There are two categories of methods to modify forwarding tables: reactive

12 CHAPTER 2. LITERATURE REVIEW

methods and proactive methods. In a reactive method, the controller inserts theforwarding decision or rule after receiving a new packet that has not been seen beforeby the control plane. On the other hand, in a proactive mode, the controller insertsthe forwarding rule before receiving the packet and it uses previously availableinformation to decide how to forward packets.

OpenFlow was the first component of a more general architecture of how net-working should evolve. This architecture is called Software Defined Networks(SDN) [47]. OpenFlow is one way to implement SDN. SDN aims to separatethe network hardware from its logic and to make possible for external applica-tions to control the behavior of network equipment. Casado et al. [48] argue thatnetwork-wide structures, distributed updates, modular composition, virtualizationand formal verification are key components for the SDN architecture. For an ex-tensive survey of SDN, the reader is referred to the work done by Kreutz et al. [47],where 580 references are provided. Also, Feamster [49] explains the evolution ofSDN and how the different technologies and ideas have evolved to arrive to thecurrent state of the art. There are other OpenFlow-oriented surveys, which arepresented in [50, 51].

In the literature, different OpenFlow controllers have been presented. NOX [52]was the first controller presented. It is written in C++ and Python. Trema [53] isan OpenFlow controller developed by NEC, written in C and Ruby. Floodlight [54]is an OpenFlow controller developed by Big Switch. Floodlight is a fork from theBeacon controller [55], written in Java, and can be configured though a RESTfulweb service. The previous controllers are deployed on a single machine, which canbe seen as a single point of failure.

ONIX [56] is the first controller designed having scalability as a goal and it is dis-tributed between different machines. DevoFlow [57] also proposes the distributionof the controllers to increase the scalability of OpenFlow in Wide Area Networks.Other distributed controllers are: HyperFlow [58], HP VAN SDN [59], Disco [60],Yanc [61], Pane [62], Smartlight [63], and Fleet [64].

OpenDayLigth [65] is an open platform that builds upon SDN to provide net-work programmability. Several vendors and operators participate in its deployment,which include vendor-driven features.

The Open Network Operating System (ONOS) [66] is another open source plat-form for SDN networks that is targeted to service providers and has abstractionsfor developing applications and network services.

Flowvisor [67] is an OpenFlow proxy that splits the OpenFlow network in slicesand forwards the OpenFlow packets to the controllers in charge of the slice. This al-lows multiple controllers to manage different parts or specific traffic in the network.The OpenFlow switches see Flowvisor as the only OpenFlow controller. The slicesare defined by a flow specification, which means that each flow can only belong toone slice.

Besides all the hype that OpenFlow first had and then SDN is receiving, afew researchers argue that SDN is not the end of the road when designing a newnetwork architecture that overcomes internet ossification [11]. Casado et al. [68]

2.2. OPENFLOW AND SOFTWARE DEFINE NETWORKS (SDN) 13

suggest Fabric, while discussing the shortcomings of SDN. They also propose anew architecture that combines MPLS and SDN, called Software-Defined InternetArchitecture (SDIA) [69].

Some of the criticism about OpenFlow is that it is too low-level. As an alterna-tive, Frenetic [70] proposes a high-level language for programming network switches.The language allows classifying and aggregating network traffic with a declarativequery language. A similar approach is taken by Hyojoon and Feamster [71] wherethey provide a high-level policy language to make the management of SDN net-works easier. Monsanto et al. [72] propose abstractions for network applicationscomposed of independent modules. They use policies and define a programmaticlanguage called Pyretic. Other high level abstractions are explored in [73–76].

An interesting approach to migrate from legacy networks to OpenFlow networksis to use RouteFlow [77]. RouteFlow uses open source IP network stacks to controla group of OpenFlow switches that together will appear to the user as a single IProuter.

PredecessorsOpenFlow was created in Stanford and it was first published in 2008 [44]. However,the idea of splitting the control plane is not new and it has been tried in the past.

Network Control Point (NCP) [78] was an early effort to separate between dataand control plane. It was introduced to improve telephone network management.

Tempest [79] is a framework that provides a programmable network environ-ment for ATM networks [80]. It allows multiple control architectures to controlsimultaneously a single network using the concept of switchlets.

Another effort to separate the control channel from the data path is IETFForCES (Forwarding and Control Element Separation) [81]. ForCES distinguishesbetween three elements: Network Elements (NE), Control Elements (CE) and For-warding Elements (FE). External entities only are able to see the Network Elements,which are seen as a monolithic piece of network equipment, such as routers, NAT,load balancers, firewalls, etc. Network Elements internally are composed of ControlElements and Forwarding Elements. This permits to implement both componentsindependently, and they can also be physically independent.

Tools to measure performance in OpenFlowPerformance is a key element when working with networks and OpenFlow is notan exception. The following are examples of tools to evaluate performance inOpenFlow switches.

OFLOPS [82] is a framework that permits to measure the capabilities and per-formance of OpenFlow-enabled switches. The authors of OFLOPS measure thebottlenecks between the switch and the remote controller using hardware instru-mentation combined with a software framework. They observe that the appliedactions and firmware affect the forwarding performance. Also Kuzniar et al. [83]

14 CHAPTER 2. LITERATURE REVIEW

measure and analyze the performance of three hardware OpenFlow switches. Theyalso find that there are differences between the OpenFlow specification and itsimplementation on the tested switches.

Cbench [84] has been used in [85] to measure the number of flow set-ups thata controller can handle. Cbench emulates a switch that communicates with acontroller and is used to measure the controller performance. It offers latency andthroughput measurements. Cbench is a general toolkit to run tests in a systematicway.

Jarschel et al. [86] propose an analytical model to predict the packet sojourntime and the probability of packet loss. Jarschel et al. [87] introduce a new toolfor performance analysis called OFCBenchmark, which includes more systematicmeasurements.

NICE [88] is another tool that permits to test OpenFlow applications and detectimplementation errors in the management applications before they occur.

Also the placement of the controller affects the performance of the OpenFlownetwork, as shown by Heller et al. [89].

2.3 Cloud Computing and Cloud Networking

Nowadays, there is a tendency in cloud computing to externalize or concentrate theIT infrastructure from organizations into the cloud (public or private). Multipledefinitions of cloud are proposed in [1, 90–92]. Taking the definition from NIST [1],a cloud is a pool of resources (computing, network and storage resources) whichcan be used and scaled up or down on demand without knowing the exact physicallocation of the resources. Some providers already offer Infrastructure-as-a-Service(IaaS), with the main focus on computing or storage. Generally, cloud resourcesare located in data centers.

Computing resources have become a utility for many organizations, thanks tothe popularization of data centers and computer virtualization. Computing cloudscan be classified as public, private or hybrid:

• Public clouds belong to an external company that rents out its resources ona pay-per-use or pay-as-you-go basis to other companies or entities. Thesecompanies are also known as cloud providers.

• Private clouds belong to a specific company for its internal use. A privatecloud is used only by internal users. Different departments can access acommon set of resources in order to obtain the benefits of cloud computing,but without exposing or moving the critical data outside the organization.

• Hybrid clouds are a mix of private and public clouds. When a private cloudneeds more resources that the ones available, it offloads some of the workloadto a public cloud.

2.4. NETWORK PERFORMANCE TOOLS 15

Cloud computing uses computer virtualization as an abstraction of the com-puting resources in order to facilitate portability and standardization. A usefultechnology used for virtualization is libvirt [93]. Libvirt provides an abstraction ofthe different computer virtualization technologies, such as KVM [94] or Xen [95],and provides a unique API to control them. Different tools use libvirt to provide auser interface for computer virtualization resources.

Cloud networking is focusing on the networking aspect of the cloud, which is inan earlier development stage compared with computing and storage. More dynamicfunctionalities and a common understanding of a network resource are needed. Inthis context, SAIL [96] has come up with a concept called Flash Network Slices(FNS). The goal of an FNS is to create Virtual Private Networks (VPNs) on de-mand, providing Quality of Service (QoS) and dynamic reconfiguration properties.

Amazon EC2 [97] was the first cloud provider to offer virtual machines on apay-per-use basis. It provides a set of tools and APIs to control and manage thecreation, access and removal of virtual machines, which many other platforms latertook as a de-facto standard.

Opennebula [98] is an open-source software to build and manage data centersand private clouds. It originates from a research project in 2005. Since 2010 it hascommercial support and a community behind it. Eucalyptus [99] is open sourcesoftware for building private and hybrid clouds.

Openstack [100] is an open-source software for building public and privateclouds. It was initiated by Rackspace [101] and NASA [102] but now it is aglobal project with collaborators all over the world and more than 450 companiesworldwide. The networking component is called Neutron [103], formerly knownas Quantum. It is in charge of the network-related components and provides anAPI to manage and create virtual networks in Openstack. Neutron is based on aplug-in architecture that permits to use different technologies underneath, whereOpenFlow is one of them.

Open Cloud Computing Interface (OCCI) [104] is an open interface that permitsto manage the resources of the cloud. It is an alternative to replace the de-factostandard from Amazon EC2.

Multiple challenges to coordinate the allocation in distributed clouds arise.Soares et al. [105] propose a resource allocation algorithm to allocate cloud andnetwork resources in an integrated way. Matias et al. [106] propose a network vir-tualization framework for the cloud based on an abstract model where each physicalelement has its own virtual counterpart. The framework is based on a layer 2 net-work virtualization approach.

2.4 Network performance tools

An important aspect when developing systems is the possibility of testing its overallperformance in worst-case scenarios. Therefore, it is necessary to have tools toevaluate such conditions. In the case of networking, packet generators are tools

16 CHAPTER 2. LITERATURE REVIEW

to inject traffic into the networks in order to observe how the systems behave at acertain load. Challenges in network performance measurements relate to generatingpackets at high speed as well as to making accurate measurements of parameterssuch as forwarding performance and latency.

Commercial packet generators based on dedicated hardware, such as, Ixia [107]and Smartbits [108], exist and provide wire speed packet generation as well as veryaccurate measurements. However, these are expensive and not always a realisticalternative for performance measurements in smaller scale (such as for experimenta-tion in research labs). Another approach is to use generic processors in commoditycomputers.

Several user-space tools exist in Linux, such as Netperf [109] and Iperf [110].Netperf permits to measure bandwidth and latency for TCP and UDP traffic. Ituses the socket API provided by the operating system. Iperf is a well-known toolthat also allows to benchmark links with TCP and UDP connections. However, theyare limited in performance because of the overhead introduced by doing contextswitching between user and kernel space in the operating system.

Another approach that is used by pktgen [111] is to integrate the packet gener-ation functionality with the kernel. Netmap [112] and Intel DPDK [113] follow yetanother approach: to bypass the network stack and to provide direct access to thebuffers of the network card.

Netmap [112] is proposed as an API to send and receive packets in a Linuxor FreeBSD system. It defines its own API through which applications get directaccess to the packet buffers of the network cards. It effectively bypasses packetprocessing in the Linux network stack and can therefore acieve higher performance.

Intel DPDK [113] is a development framework that provides direct access tothe network cards for the user space applications, disabling the interrupts andcompletely bypassing the default network stack. In this case, the application is infull control of the network card.

Chapter 3

Problem definition

This chapter presents the hypothesis of our thesis, defines the goal in our researchstatement and describes the methodology.

3.1 Hypothesis

A cloud provides on-demand and automated access to resources (computing capac-ity, network resources, and storage). The configuration of the resources should bedynamic and without operator intervention. This dynamicity requires standard-ized interfaces for managing the resources in order to automate the provisioning.An abstraction can be used to provide such an interface. Whereas the computerabstraction is well defined and allows automatic configuration and deployment, wedo not have the same abstractions for networking. The abstraction of the networkcomponents and their functionality will enable heterogeneous deployments withmultiple technologies and will make the automation needed by the cloud possible.An example in computer virtualization is libvirt, which provides a unique API and aset of functionalities to control different computer virtualization technologies. Lib-virt enables deployment of virtual machines with different computer virtualizationtechnologies.

Our hypothesis is that using the same approach as libvirt in virtual networks willenable the automation required by the cloud and will increase its dynamicity. Inaddition, one open question is how to select a technology that makes it possible tooperate and manage virtual networks. This technology should be flexible enough forexperimentation and have enough performance to support today’s network traffic.

3.2 Research Statements

The goal of this thesis is to study and investigate novel approaches to networkvirtualization in cloud computing. In particular, we want to study how a networkvirtualization abstraction can be defined to be suitable for the cloud. Once we

17

18 CHAPTER 3. PROBLEM DEFINITION

have defined the abstraction, we want to implement a proof of concept. SDNtechnologies provide mechanisms to remotely configure switches and experimentwith new control functionality and forwarding. Therefore we want to evaluate howdifferent SDN techniques can be used in the context of network virtualization.

Experimental performance evaluation is another key aspect when testing andevaluating new concepts and technologies. Once we have selected the technologyand implemented a proof of concept, we want to evaluate its performance andcompare it with existing solutions. Finally, in case we need to improve existingperformance tools for the network evaluation, we want to study how they can bedesigned to benefit from new hardware features.

We follow an experimental approach where we implement a software prototypeand evaluate it. This prototype implements the basic functionalities of the envi-sioned virtualization abstraction. We divide our work into three research problems,which are described in more detail below.

How can we define a network virtualization abstraction?Our first problem is to define a unified network virtualization abstraction that en-ables deployment of networks for cloud computing in an automated fashion. Tra-ditionally, requesting a network service requires manual intervention from the net-work operator, which implies that the user needs to wait, perhaps for several daysor weeks. Cloud computing makes it possible to deploy virtual machines in secondsor minutes, and therefore it is important that the time to deploy networks is of thesame order of magnitude.

Our approach to this problem is to define the network as a single node, wheredifferent cloud resources are interconnected. A resource can be a host, a virtualmachine or another network. We want to find an abstraction that can be mappedonto any kind of virtualization technology. The user can specify the services heneeds, at a high-level abstraction without technology-specific details. The use of anabstraction allows creating an open programmatic interface that permits networksto be deployed directly without human intervention, which makes the deploymentof virtual networks more agile.

This is a similar approach as when using virtualization of computers and storage.To be able to create services and applications independent from the physical devices,an abstraction that defines the function is used. An example is logical disk volumes,where higher levels have a unified view of the storage while the data could be storedon separate physical disks. We use what we call a Flash Network Slice (FNS) tointerconnect the resources.

What is the performance of OpenFlow networks when used fornetwork virtualization?OpenFlow is an important and promising technology for network virtualization be-cause it enables to control how packets are forwarded from a centralized logic. We

3.2. RESEARCH STATEMENTS 19

develop a proof of concept of an OpenFlow controller that uses network virtualiza-tion with the previously defined network abstraction (FNS).

Well-founded concerns about performance exist and therefore our next step isto investigate them. The propagation and processing delays between switch andcontroller could increase the overall transmission time and affect applications thatrequire low latencies. In addition, it can create bottlenecks because the data planeand control plane are not in the same physical machine.

Therefore, once we define our abstraction and implement a prototype usingOpenFlow, our next problem is how we can evaluate an OpenFlow network in areal-life scenario? Network traffic is complex and multiple unpredictable variablescan influence the results. For instance, the latency of a packet might be affected bythe load of the controller, a bandwidth limitation between switch and controller, apacket loss in a congested switch, or any combination of these.

In order to address this problem, we develop and perform a set of experimentsto evaluate and characterize the performance of the OpenFlow control plane.

How can network performance tools be designed to benefit fromnew technologies?Our third problem, which is related to the second, is how to efficiently designand implement a software tool for network performance evaluation of high-speednetworks taking advantage of modern features in commodity network cards.

Network performance evaluation is important when deploying network systems,services and protocols. It enables us to measure and predict how systems behaveunder heavy load. Commercial equipment to measure network performance, such asthroughput, latency or jitter is available, but depends on costly dedicated hardware.Another approach is to use regular computers with open source applications thatconvert the computer into a tool. We follow the approach of taking an existingtraffic generator solution, pktgen [111], to which we add the capabilities to receiveand analyze traffic.

One of the key differences between dedicated hardware and open-source appli-cations, especially at 10Gbit/s links and beyond, is the number of generated andprocessed packets. Therefore, our solution should:

• be open-source to make it highly available to the research community,

• achieve wire speeds at 10Gbit/s with state-of-the-art computers,

• be modular and easy to deploy in any Linux system,

• work closely with the hardware to try to obtain the maximum performancefrom it,

• run multiple network interfaces, and

• be able to use new features of modern systems that can boost its performance.

20 CHAPTER 3. PROBLEM DEFINITION

Some of the challenges in this problem are the complexity in the operatingsystem, and how the network interfaces interact with the applications. At 10Gbit/swire speed, one packet needs to be processed every 67.2 nanoseconds in the worst-case scenario when the rate is 14.88 Mpps (million packets per second). This addsa strong constraint on how packets are managed by the operating system. Anotherchallenge is that general-purpose operating systems are designed to be general,which adds overhead with functionality that is not strictly necessary for measuringnetwork performance.

3.3 Methodology

In this section we give an overview of how we approach the research statementspresented in Section 3.2.

Network abstraction

For the research statement “How can we define a network virtualization abstrac-tion?” we want to simplify how networks appear to the user. We follow the ideaproposed by Keller and Rexford [42] to define the network as a single node, whereall the network functionality such as polices and connectivity is defined internally.We define the endpoints, which are the termination of the network, with a set ofnetwork properties, such as, attached network switch and port. Additional fields,such as MPLS or VLAN IDs can also be used. We also characterize the networkin terms of the forwarding type, for example, L2 (based on MAC addresses) or L3(based on IP addresses).

Furthermore, the idea is to separate the abstract network operation of a virtualnetwork, such as to create, modify, and remove a network, from the specific com-mands that each technology requires. The main purpose of a virtual network is tointerconnect nodes. From a logical point of view, the end user should not managethe underlying topology. Therefore, we define our network virtualization abstrac-tion as a single node, to which the user connects its endpoints. Furthermore, wedefined an interface for the network virtualization abstraction as a set of networkoperations, through which the endpoints interact with the single node network.

We implement the abstraction as a library, which we call LibNetVirt, as an anal-ogy to libvirt. The LibNetVirt architecture is illustrated in Figure 3.1. LibNetVirtis based on a driver architecture, composed in two parts: a generic interface anddrivers. The generic interface provides a programmatic interface to interact withthe virtual network. The drivers contain the specific configuration details of eachtechnology. In this thesis we have implemented LibNetVirt drivers for OpenFlowand a commercial MPLS routing stack.

3.3. METHODOLOGY 21

LibNetVirt

OpenFlowL2VPN MPLS

Management

Application

XML File

Virtual Network

description

Generic Interface

Drivers

Figure 3.1: LibNetVirt architecture. LibNetVirt generic interface can process XMLfiles that contain the Virtual Network description. A third party managementapplication can use the API provided by LibNetVirt to operate virtual networks.

OpenFlowFor the research statement “What is the performance of OpenFlow networks whenused for network virtualization?”, OpenFlow offers the possibility to remotely con-trol the forwarding tables of a switch from a central entity. We explore howOpenFlow can be used in virtual networks, especially the ones that reside in asingle physical network. An OpenFlow controller is able to monitor traffic thatis received in the network and make decisions about how to forward the traffic.We use the OpenFlow forwarding rules to isolate the traffic of different users. Weuse OpenFlow to control the virtual networks and to control the behavior of thenetwork from the controller.

We want to evaluate OpenFlow for network virtualization. We design a setof performance measurements in which we use pktgen as a packet generator tool.We evaluate and compare our implementation with other similar approaches fornetwork virtualization using the designed measurements.

PktgenFor the research statement “How can network performance tools be designed tobenefit from new technologies? we use pktgen. Pktgen is part of the regular Linuxkernel and is used for network testing. We extend pktgen with the capabilitiesneeded to become a performance tool for network elements. In order to do this, weenhance pktgen with a receiver part that measures throughput, jitter, inter-arrivaltimes and latency, and we use pktgen to generate and receive traffic when testingour OpenFlow prototype.

Chapter 4

Thesis contribution

Our work has been published in several peer-review conferences. This chapterincludes the list of contributions, a summary of each paper, and a description ofthe contributions to each paper by the author of this thesis.

4.1 List of contributions

The research papers included in this thesis are:

• Paper A: D. Turull, M. Hidell, and P. Sjödin, “libNetVirt: the network virtual-ization library,” in Workshop on Clouds, Networks and Data Centers (ICC’12WS - CloudNetsDataCenters), (Ottawa, Canada), June 2012

• Paper B: D. Turull, M. Hidell, and P. Sjödin, “Using libNetVirt to controlthe virtual network,” in 2012 1st IEEE International Conference on CloudNetworking (IEEE CloudNet’12), (Paris, France), Nov. 2012

• Paper C: D. Turull, M. Hidell, and P. Sjödin, “Performance evaluation ofOpenFlow controllers for network virtualization,” in 2014 IEEE 15th Interna-tional Conference on High Performance Switching and Routing (IEEE HPSR’14), (Vancouver, Canada), July 2014

• Paper D: D. Turull, P. Sjödin, and R. Olsson, “Pktgen: Measuring perfor-mance on high speed networks,” Computer Communications, pp. 39–48, May2016. Volume 82, ISSN 0140-3664, Available at: http://dx.doi.org/10.1016/j.comcom.2016.03.003

Publications of the same author, not included in this work• D. Turull, M. Hidell, and P. Sjödin, “Evaluating OpenFlow in libNetVirt,”

in The 8th Swedish National Computer Networking Workshop 2012 (SNCNW2012), (Stockholm, Sweden), June 2012

23

24 CHAPTER 4. THESIS CONTRIBUTION

Paper A. LibNetVirt: the network virtualization libraryIn this paper, we create a library for network virtualization. We propose a networkvirtualization abstraction that uses the single node abstraction. We define theendpoints, where the virtual network terminates. An endpoint is defined as thecombination of physical switch, port and VLAN, with the possibility to add extraparameters, such as IP address or MPLS tag. We propose a set of basic operationsto manage the network, such as creation, addition, or removal of endpoints andvirtual networks.

We use libvirt [93] as a model to design our library. Libvirt is the libraryto manage computer virtualization, which provides a common API for differentvirtualization technologies. Our library, LibNetVirt, is divided into two parts: thegeneric interface and the drivers. The drivers are technology-dependent and arecalled by the generic interface. The generic interface uses the proposed abstraction.

After presenting the global framework, we further elaborate upon the designof an application in the OpenFlow controller, presenting design alternatives andimplementation decisions that we make. The user only needs to specify the end-points, and the library is in charge of controlling the network and installation ofthe required forwarding entries to create the illusion of a single node network.

Finally, we contextualize the use of our controller to a use case based on cloudnetworking and the use of Flash Network Slices.

Individual contribution

I am the main author of the paper and proposed the architecture of LibNetVirt as aframework for network virtualization. In addition, I designed and implemented thecode of the software library that uses OpenFlow. I wrote the paper together withmy supervisors Peter Sjödin and Markus Hidell. Finally, I presented the paper ata peer-reviewed scientific conference.

Paper B. Using LibNetVirt to control the virtual networkThis paper is a continuation of the work from paper A and goes a step further byanalyzing the performance of our solution in a testbed with software switches thatrun Open vSwitch. We evaluate LibNetVirt in an OpenFlow-enabled network withthree different tests: the setup time for a flow, the behavior of the system under aDenial of Service attack, and packet losses for high rate UDP flows.

The setup time is the time required for the first packet to reach the destination.We compare the obtained results with the Linux Bridge and OpenFlow acting asa learning switch. We observe that after the first packet, the remaining packets donot suffer any extra delay.

A second scenario that we explore is when a host that does not belong to anyvirtual network sends UDP packets to a switch. By doing so, we emulate a Denialof Service attack. We observe that the switch that receives the UDP traffic suffers adrop of performance even at low packet rates. That is because the malicious traffic

4.1. LIST OF CONTRIBUTIONS 25

is unknown by the OpenFlow switch, which then needs to send the packets to thecontroller for further processing. The controller drops the packets because they donot belong to any virtual network, but all the extensive OpenFlow traffic saturatesthe control channel. Due to the saturation, legitimate packets are also dropped.

Finally, with the UDP tests we observe that all the packet losses at very highrates occur at the beginning of the transmission when the rules are not installedyet and that the amount of losses increases with the incoming rate.

Individual contribution

I am the main author of the paper and I wrote the paper together with my su-pervisors. In addition, I also proposed the experiments and wrote the code andscripts to run them. Moreover, I presented the paper at a peer-reviewed scientificconference.

Paper C. Performance evaluation of OpenFlow controllers fornetwork virtualizationBuilding upon the previous two papers, Paper C presents a set of different perfor-mance tests with ICMP, TCP and UDP to compare different OpenFlow controllersthat run applications for network virtualization. In this case, we use an emulationenvironment called Mininet [119].

We compare our solution LibNetVirt, with the Trema [53] controller and Flood-light [54] controller and compare them with a baseline configuration with OpenvSwitch [120]. We explain how the different controllers use the OpenFlow messagesto install the flows.

The different measurements are designed to represent practical uses of the net-work. The TCP test measures the transfer time for different data size and the UDPtest measures the packet losses at different packet rates.

In addition, we introduce a metric called Equivalent Packet Loss to measure theUDP losses at the beginning of the communication.

Individual contribution

I am the main author of the paper and I wrote the paper together with my supervi-sors. Furthermore, I proposed and executed the experiments included in the paper.Finally, I presented the paper at a peer-reviewed scientific conference.

Paper D. Pktgen: Measuring performance on high speednetworksPaper D presents pktgen, a tool which we have partially used in the previouspapers. Pktgen is a packet generator tool that is implemented in the Linux kernel.Originally pktgen had only the packet generator part and we have extended it with

26 CHAPTER 4. THESIS CONTRIBUTION

a receiver part and a protocol that defines the packet format. The paper describesits design and provides a few examples of network performance measurements.

Pktgen takes advantage of features of modern systems, such as multi-queueand parallelism. It uses UDP packets and allows to set the packet rate with finegranularity and high packet rates.

We evaluate and compare pktgen with other network performance tools, suchas, Netperf, Iperf, DPDK and Netmap. We experiment and show baseline resultsfor throughput, latency and packet delay latency using 10 Gigabit packet networks.

Individual contribution

I am the main author of the paper. Robert Olsson and others in the Linux commu-nity wrote the code for the packet generator part. The first version of the pktgenreceiver was done for my master thesis. During the work with this thesis, I im-proved the pktgen receiver. In addition, I proposed and perform all the networkperformance tests presented in the journal paper and wrote the paper together withPeter Sjödin and Robert Olsson.

4.2 Relation between papers

In order to summarize the contributions in a common thread, we illustrate therelation between this thesis’ papers in Figure 4.1.

Paper A:

LibNetVirt architecture

Paper B:

Testing LibNetVirt/OpenFlow

Paper C :

Comparing LibNetVirt/OpenFlow

with similars tools

Paper D:

Pktgen

Figure 4.1: Relation between papers

Paper A introduces the framework of LibNetVirt with its design and functional-ity. Later, Paper B evaluates LibNetVirt’s OpenFlow driver through three differentperformance measurements. Two of the them use pktgen, the tool in paper D, asa packet generator for UDP packets. Paper C evaluates LibNetVirt’s OpenFlowdriver and compares it with other control applications for OpenFlow (Trema, Flood-light and OpenVSwitch). Here, we also use pktgen for testing.

4.3. RELATION TO STATE OF THE ART 27

4.3 Relation to state of the art

The background in Section 2 includes the relevant work to our thesis. Here, wediscuss the main differences between our work and the work of others.

Related to LibNetVirt (Paper A), Keller and Rexford (in [42]) discuss the con-venience of the single router abstraction, allowing users to focus on the applicationsthat run on top of the network, which is the desired goal in the cloud. We developtheir concept further in the virtual network view of the network. We clearly spec-ify the definition of endpoints and we implement a prototype with OpenFlow. Aparallel work is done by Kang et al. [43], where they also explain how to developthe concept. We use a straight-forward approach using shortest path routing whiletheir approach is to separate the problem in three different parts: a high level partwhich defines end-to-end connectivity on top of the single node abstraction, a mid-dle layer that decides the routing policies, and a compiler that converts the policiesto the forwarding rules in the hardware.

Casado et al. [121] propose an architecture for virtualizing the network forward-ing plane. They separate the logical forwarding from the physical. We go a stepfurther by abstracting the network topology from the user, and providing a genericinterface to interconnect the user’s resources and simplifying the view even moretowards the user.

Similar work is done in the Openstack community to create abstractions for thenetworks inside the data center, especially in the Neutron project [103]. However,their work is focused on the Openstack ecosystem while we envision our work beingused in other scenarios, where QoS is needed.

OpenDayLight [65] and ONOS [66] are similar to frameworks to LibNetVirtin the sense that they allow developers to use APIs to control networks. Theseare industrial initiatives to provide reliable open source platforms for network con-trol. Although there are similarities in spirit and approach, these initiatives areindependent of, and subsequent to, LibNetVirt.

Related to the OpenFlow evaluation in Paper B and Paper C, we propose novelmeasurements and comparisons between Openflow controllers, while the authors ofcbench [122] provide a testing framework.

Chapter 5

Conclusions and Future work

The Internet has changed our lives and we believe that cloud computing and net-working will expand the use of Information Technology and democratize its use.Network virtualization has existed for several years, but now it has the opportu-nity to help in the development of cloud infrastructures. Management of virtualnetworks is complex and requires manual operation from the network operators.The manual operation slows down the deployment and acts a bottleneck for thedynamicity that cloud computing promises to provide.

We propose a network virtualization abstraction to ease the management ofthe network to the end user. In this thesis, we have built a proof of conceptusing OpenFlow and observe that OpenFlow provides the flexibility required tomanage network virtualization. Our prototype is also compared with other researchcontrollers and it is available as open source1.

We show that OpenFlow controllers need to be properly designed in order toachieve maximal performance and compare different performance results for differ-ent network protocols, such as, ICMP, TCP or UDP.

In addition to the work with the network abstraction and OpenFlow, we improvepktgen, a network performance tool for high speed networks. We have developed areceiver part that is able to process packets at a high rate, which takes advantage ofmodern features in commodity computers. Despite the fact that we do not achievewire speed for all configurations due to driver limitations, we foresee pktgen as avaluable tool for testing new equipment. We have used pktgen in some of the UDPmeasurements to evaluate our prototype and similar OpenFlow controllers. Thesource code is freely available2.

Our hypothesis is that by using a similar approach as libvirt is using for com-puter virtualization, but focused on virtual networks, would ease the managementof virtual networks. We have designed a library that follows this idea and we ob-

1LibNetVirt source code is available at https://github.com/danieltt/libnetvirt2pktgen source code is available at: http://people.kth.se/~danieltt/pktgen and https:

//github.com/danieltt/pktgen

29

30 CHAPTER 5. CONCLUSIONS AND FUTURE WORK

serve that the state of the art has evolved in this direction, with commercial solu-tions from VMware (with the acquisition of Nicira) [123], NEC [124], Huawei [125],Openstack Neutron [103], OpenDayLight [65] and ONOS [66] and others.

We believe that making virtual networks easy to manage can have a positiverole in society, and in our humble opinion we believe that LibNetVirt is a step inthat direction. Providing a unified interface towards different network technologiesenables the creation of new services that can potentially add value to society. Thecloud provides means to democratize access to technology and to reduce costs,which is a necessary step in order to facilitate the creation of innovative servicesand applications. The cloud enables anyone with a computer to start creating newservices for people, without the need of an upfront investment. Managing virtualnetworks in an efficient and dynamic way enriches these services.

5.1 Future work

LibNetVirt uses NOX, which has been deprecated after finalizing our prototypes.A future task is to implement our design in other OpenFlow controllers. CurrentlyLibNetVirt provides a programmatic interface with a CLI application. A web in-terface with support for LibNetVirt can be implemented. Additionally, integrationwith Openstack Neutron can be a good step forward. Openstack Neutron is thenetwork component of the Openstack platform. Our library provides simple inter-faces to network management operations, although we currently use a somewhatsimplistic algorithm to compute paths. A clear future direction is to investigatenew algorithms to use within our network abstraction.

A relevant research problem for LibNetVirt is how to keep all the forwardingtables of the switches up to date, especially when multiple switches and involvedand large numbers of flows need to be installed. This is a nontrivial tasks and anobvious potential bottleneck in any centralized controller. Another challenge is howto achieve high availability and resilience works in the presense of large numberof flows and virtual networks with OpenFlow. Yet another challange is how todistribute the controller without affecting the response time and performance whileincresing its scalability.

Interesting future work based on the second part of the thesis is the investiga-tions of how to improve the performance in Linux drivers and Linux networking ingeneral. Some of the tools that circumvent the network stack are able to achievewire speed at 10Gb/s with small packets, while traditional applications cannot.Therefore, we believe there is room for improvements. In addition, 100Gb/s net-works can present new challanges, reducing even more the time to process packets.

Bibliography

[1] P. Mell and T. Grance, “The NIST definition of cloud computing,”tech. rep., National Institute of Science and Technology, 2011. Available at:http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf.

[2] “AutoBAHN: GEANT2 bandwidth on demand,” 2008. Available at:http://geant2.archive.geant.net/server/show/nav.756.html.

[3] “FEDERICA.” Available at: http://www.fp7-federica.eu/.

[4] A. Khan, A. Zugenmaier, D. Jurca, and W. Kellerer, “Network virtualization:a hypervisor for the Internet?,” IEEE Communications Magazine, vol. 50,pp. 136–143, Jan. 2012.

[5] X. Chen, Z. M. Mao, and J. Van der Merwe, “Pacman: a platform for au-tomated and controlled network operations and configuration management,”in Proceedings of the 5th international conference on Emerging networkingexperiments and technologies, CoNEXT ’09, (New York, USA), pp. 277–288,ACM, 2009.

[6] X. Chen, Z. M. Mao, and J. Van Der Merwe, “ShadowNet: a platform forrapid and safe network evolution,” in Proceedings of the 2009 conference onUSENIX Annual technical conference, USENIX’09, (Berkeley, CA, USA),p. 3, USENIX Association, 2009.

[7] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner, “OpenFlow: enabling innovation incampus networks,” SIGCOMM Comput. Commun. Rev., vol. 38, pp. 69–74,2008.

[8] T. A. Limoncelli, “Openflow: A radical new idea in networking,” Queue,vol. 10, pp. 40:40–40:46, June 2012.

[9] N. M. K. Chowdhury and R. Boutaba, “A survey of network virtualization,”Comput. Netw., vol. 54, pp. 862–876, April 2010.

31

32 BIBLIOGRAPHY

[10] T. Anderson, L. Peterson, S. Shenker, and J. Turner, “Overcoming the in-ternet impasse through virtualization,” Computer, vol. 38, pp. 34 – 41, April2005.

[11] J. Turner and D. Taylor, “Diversifying the internet,” in Global Telecommu-nications Conference, 2005. GLOBECOM ’05. IEEE, vol. 2, pp. 6 pp.–760,Dec 2005.

[12] E. Karpilovsky, A. Gerber, D. Pei, J. Rexford, and A. Shaikh, “Quantifyingthe extent of IPv6 deployment,” in Passive and Active Network Measurement,vol. 5448 of Lecture Notes in Computer Science, pp. 13–22, Springer BerlinHeidelberg, 2009.

[13] L. Smith and I. Lipner, “Free pool of IPv4 address space depleted,” 2011.Available at http://www.nro.net/news/ipv4-free-pool-depleted.

[14] IEEE, “IEEE standards for local and metropolitan area networks: Virtualbridged local area networks,” IEEE Std 802.1Q-1998, 1999.

[15] Cisco, “Inter-switch link and IEEE 802.1q frame format,” tech. rep., Cisco,2006. Available at: http://www.cisco.com/en/US/tech/tk389/tk689/technologies_tech_note09186a0080094665.shtml.

[16] IEEE, “IEEE standard for local and metropolitan area networks - virtualbridged local area networks, amendment 4: Provider bridges,” 2006.

[17] E. Rosen and Y. Rekhter, “BGP/MPLS IP virtual private networks (VPNs).”RFC 4364 (Proposed Standard), Feb. 2006. Updated by RFCs 4577, 4684,5462.

[18] A. Nagarajan, “Generic requirements for provider provisioned virtual privatenetworks (PPVPN).” RFC 3809 (Informational), June 2004.

[19] D. Farinacci, T. Li, S. Hanks, D. Meyer, and P. Traina, “Generic routingencapsulation (GRE).” RFC 2784 (Proposed Standard), Mar. 2000. Updatedby RFC 2890.

[20] K. Hamzeh, G. Pall, W. Verthein, J. Taarud, W. Little, and G. Zorn, “Point-to-point tunneling protocol (PPTP).” RFC 2637 (Informational), July 1999.

[21] D. Harrington, R. Presuhn, and B. Wijnen, “An architecture for describingSNMP management frameworks.” RFC 2261 (Proposed Standard), Jan. 1998.Obsoleted by RFC 2271.

[22] S. Kent and K. Seo, “Security architecture for the Internet protocol.” RFC4301 (Proposed Standard), Dec. 2005. Updated by RFC 6040.

[23] L. Andersson and E. Rosen, “Framework for layer 2 virtual private networks(L2VPNs).” RFC 4664 (Informational), Sept. 2006.

BIBLIOGRAPHY 33

[24] W. Augustyn and Y. Serbest, “Service requirements for layer 2 provider-provisioned virtual private networks.” RFC 4665 (Informational), Sept. 2006.

[25] M. Melo, J. Carapinha, S. Sargento, L. Torres, P. Tran, U. Killat, andA. Timm-Giel, “Virtual network mapping - an optimization problem,” in Mo-bile Networks and Management, vol. 97 of Lecture Notes of the Institute forComputer Sciences, Social Informatics and Telecommunications Engineering,pp. 187–200, Springer Berlin Heidelberg, 2012.

[26] J. Lischka and H. Karl, “A virtual network mapping algorithm based on sub-graph isomorphism detection,” in Proceedings of the 1st ACM Workshop onVirtualized Infrastructure Systems and Architectures, VISA ’09, (New York,NY, USA), pp. 81–88, ACM, 2009.

[27] G. Schaffrath, C. Werle, P. Papadimitriou, A. Feldmann, R. Bless, A. Green-halgh, A. Wundsam, M. Kind, O. Maennel, and L. Mathy, “Network virtu-alization architecture: Proposal and initial prototype,” in Proceedings of the1st ACM Workshop on Virtualized Infrastructure Systems and Architectures,VISA ’09, (New York, NY, USA), pp. 63–72, ACM, 2009.

[28] N. Chowdhury, M. Rahman, and R. Boutaba, “Virtual network embed-ding with coordinated node and link mapping,” in INFOCOM 2009, IEEE,pp. 783–791, 2009.

[29] F. Yang, Z. Wang, J. Chen, and Y. Liu, “VLB-VNE: a regionalized valiantload-balancing algorithm in virtual network mapping,” in Wireless Commu-nications, Networking and Information Security (WCNIS), 2010 IEEE Inter-national Conference on, pp. 432–436, IEEE, June 2010.

[30] Y. Yu, C. Shan-zhi, L. Xin, and W. Yan, “RMap: An algorithm of virtualnetwork resilience mapping,” in Wireless Communications, Networking andMobile Computing (WiCOM), 2011 7th International Conference on, pp. 1–4,IEEE, Sept 2011.

[31] S. li Zhang, X. song Qiu, and L.-M. Meng, “Virtual network mapping al-gorithm for large-scale network environment,” in Communications and Net-working in China (CHINACOM), 2011 6th International ICST Conferenceon, pp. 765–770, IEEE, Aug 2011.

[32] J. Fan and M. Ammar, “Dynamic topology configuration in service over-lay networks: A study of reconfiguration policies,” in INFOCOM 2006. 25thIEEE International Conference on Computer Communications. Proceedings,pp. 1–12, IEEE, April 2006.

[33] R. Ricci, C. Alfeld, and J. Lepreau, “A solver for the network testbed mappingproblem,” ACM SIGCOMM Computer Communication Review, vol. 33, no. 2,p. 65, 2003.

34 BIBLIOGRAPHY

[34] W. Szeto, Y. Iraqi, and R. Boutaba, “A multi-commodity flow based approachto virtual network resource allocation,” in Global Telecommunications Con-ference, 2003. GLOBECOM ’03. IEEE, vol. 6, pp. 3004–3008 vol.6, IEEE,Dec 2003.

[35] M. Yu, Y. Yi, J. Rexford, and M. Chiang, “Rethinking virtual network em-bedding,” ACM SIGCOMM Computer Communication Review, vol. 38, no. 2,p. 17, 2008.

[36] Y. Zhu and M. Ammar, “Algorithms for assigning substrate network re-sources to virtual network components,” in Proceedings IEEE INFOCOM2006. 25TH IEEE International Conference on Computer Communications,pp. 1–12, IEEE, 2006.

[37] J. Nogueira, M. Melo, J. Carapinha, and S. Sargento, “Virtual network map-ping into heterogeneous substrate networks,” in Computers and Communica-tions (ISCC), 2011 IEEE Symposium on, pp. 438–444, June 2011.

[38] G. Alkmim, D. Batista, and N. da Fonseca, “Mapping virtual networks ontosubstrate networks,” Journal of Internet Services and Applications, vol. 4,no. 1, p. 3, 2013.

[39] I. Houidi, W. Louati, and D. Zeghlache, “A distributed virtual network map-ping algorithm,” in Communications, 2008. ICC ’08. IEEE InternationalConference on, pp. 5634–5640, May 2008.

[40] C. Wang, S. Shanbhag, and T. Wolf, “Virtual network mapping with trafficmatrices,” in Communications (ICC), 2012 IEEE International Conferenceon, pp. 2717–2722, June 2012.

[41] G. P. Koslovski, P. V.-B. Primet, and A. S. Charão, “VXDL: Virtual re-sources and interconnection networks description language,” in Networks forGrid Applications, vol. 2 of Lecture Notes of the Institute for Computer Sci-ences, Social Informatics and Telecommunications Engineering, pp. 138–154,Springer Berlin Heidelberg, 2009.

[42] E. Keller and J. Rexford, “The "platform as a service" model for networking,”in Proceedings of the 2010 internet network management conference on Re-search on enterprise networking, INM/WREN’10, (Berkeley, CA, USA), p. 4,USENIX Association, 2010.

[43] N. Kang, Z. Liu, J. Rexford, and D. Walker, “Optimizing the "one bigswitch" abstraction in software-defined networks,” in Proceedings of the NinthACM Conference on Emerging Networking Experiments and Technologies,CoNEXT ’13, (New York, NY, USA), pp. 13–24, ACM, 2013.

BIBLIOGRAPHY 35

[44] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner, “OpenFlow: enabling innovation incampus networks,” SIGCOMM Comput. Commun. Rev., vol. 38, pp. 69–74,Mar. 2008.

[45] M. Jarschel and R. Pries, “An openflow-based energy-efficient data center ap-proach,” in Proceedings of the ACM SIGCOMM 2012 conference on Applica-tions, technologies, architectures, and protocols for computer communication,SIGCOMM ’12, (New York, NY, USA), pp. 87–88, ACM, 2012.

[46] F. Hao, T. V. Lakshman, S. Mukherjee, and H. Song, “Enhancing dy-namic cloud-based services using network virtualization,” SIGCOMM Com-put. Commun. Rev., vol. 40, pp. 67–74, Jan 2010.

[47] D. Kreutz, F. Ramos, P. Esteves Verissimo, C. Esteve Rothenberg, S. Azodol-molky, and S. Uhlig, “Software-defined networking: A comprehensive survey,”Proceedings of the IEEE, vol. 103, pp. 14–76, Jan 2015.

[48] M. Casado, N. Foster, and A. Guha, “Abstractions for software-defined net-works,” Communications of the ACM, vol. 57, pp. 86–95, Sept. 2014.

[49] N. Feamster, J. Rexford, and E. Zegura, “The road to SDN,” Queue, vol. 11,no. 12, p. 20–40, 2013.

[50] A. Lara, A. Kolasani, and B. Ramamurthy, “Network innovation usingOpenFlow: A survey,” Communications Surveys Tutorials, IEEE, vol. 16,pp. 493–512, First 2014.

[51] B. Nunes, M. Mendonca, X.-N. Nguyen, K. Obraczka, and T. Turletti, “Asurvey of software-defined networking: Past, present, and future of pro-grammable networks,” Communications Surveys Tutorials, IEEE, vol. 16,pp. 1617–1634, Third 2014.

[52] N. Gude, T. Koponen, J. Pettit, B. Pfaff, M. Casado, N. McKeown, andS. Shenker, “NOX: towards an operating system for networks,” SIGCOMMComput. Commun. Rev., vol. 38, pp. 105–110, July 2008.

[53] “Trema.” Available at: https://github.com/trema.

[54] “Floodlight OpenFlow Controller.” Available at:http://floodlight.openflowhub.org/.

[55] “Beacon.” Available at: http://www.beaconcontroller.net/.

[56] T. Koponen, M. Casado, N. Gude, J. Stribling, L. Poutievski, M. Zhu, R. Ra-manathan, Y. Iwata, H. Inoue, T. Hama, and S. Shenker, “Onix: a distributedcontrol platform for large-scale production networks,” in Proceedings of the9th USENIX conference on Operating systems design and implementation,OSDI’10, (Berkeley, CA, USA), pp. 1–6, USENIX Association, 2010.

36 BIBLIOGRAPHY

[57] J. C. Mogul, J. Tourrilhes, P. Yalagandula, P. Sharma, A. R. Curtis, andS. Banerjee, “Devoflow: cost-effective flow management for high performanceenterprise networks,” in Proceedings of the 9th ACM SIGCOMM Workshopon Hot Topics in Networks, Hotnets-IX, (New York, NY, USA), pp. 1:1–1:6,ACM, 2010.

[58] A. Tootoonchian and Y. Ganjali, “HyperFlow: a distributed control plane forOpenFlow,” in Proceedings of the 2010 internet network management confer-ence on Research on enterprise networking, INM/WREN’10, (Berkeley, CA,USA), pp. 3–3, USENIX Association, 2010.

[59] “HP SDN controller architecture,” tech. rep., HP, September 2013. Availableat: http://goo.gl/TrMZiF.

[60] K. Phemius, M. Bouet, and J. Leguay, “Disco: Distributed multi-domain sdncontrollers,” in Network Operations and Management Symposium (NOMS),2014 IEEE, pp. 1–4, May 2014.

[61] M. Monaco, O. Michel, and E. Keller, “Applying operating system principlesto SDN controller design,” in Proceedings of the Twelfth ACM Workshop onHot Topics in Networks, HotNets-XII, (New York, NY, USA), pp. 2:1–2:7,ACM, 2013.

[62] A. D. Ferguson, A. Guha, C. Liang, R. Fonseca, and S. Krishnamurthi, “Par-ticipatory networking: an API for application control of SDNs,” in Proceed-ings of the ACM SIGCOMM 2013 conference on SIGCOMM, SIGCOMM ’13,(New York, NY, USA), pp. 327–338, ACM, 2013.

[63] F. Botelho, A. Bessani, F. M. V. Ramos, and P. Ferreira, “On the designof practical fault-tolerant SDN controllers,” in Third European Workshop onSoftware Defined Networks, 2014.

[64] S. Matsumoto, S. Hitz, and A. Perrig, “Fleet: Defending SDNs from maliciousadministrators,” in Proceedings of the Third Workshop on Hot Topics in Soft-ware Defined Networking, HotSDN ’14, (New York, NY, USA), pp. 103–108,ACM, 2014.

[65] J. Medved, R. Varga, A. Tkacik, and K. Gray, “Opendaylight: Towards amodel-driven sdn controller architecture,” in World of Wireless, Mobile andMultimedia Networks (WoWMoM), 2014 IEEE 15th International Sympo-sium on a, pp. 1–6, June 2014.

[66] P. Berde, M. Gerola, J. Hart, Y. Higuchi, M. Kobayashi, T. Koide, B. Lantz,B. O’Connor, P. Radoslavov, W. Snow, and G. Parulkar, “ONOS: Towardsan open, distributed SDN OS,” in Proceedings of the Third Workshop on HotTopics in Software Defined Networking, HotSDN ’14, (New York, NY, USA),pp. 1–6, ACM, 2014.

BIBLIOGRAPHY 37

[67] R. Sherwood, G. Gibb, K.-K. Yap, G. Appenzeller, M. Casado, N. McKe-own, and G. Parulkar, “Can the production network be the testbed?,” inProceedings of the 9th USENIX conference on Operating systems design andimplementation, OSDI’10, (Berkeley, CA, USA), pp. 1–6, USENIX Associa-tion, 2010.

[68] M. Casado, T. Koponen, S. Shenker, and A. Tootoonchian, “Fabric: a retro-spective on evolving SDN,” in Proceedings of the first workshop on Hot topicsin software defined networks - HotSDN ’12, (New York, New York, USA),p. 85, ACM Press, Aug. 2012.

[69] B. Raghavan, M. Casado, T. Koponen, S. Ratnasamy, A. Ghodsi, andS. Shenker, “Software-defined internet architecture: Decoupling architecturefrom infrastructure,” in Proceedings of the 11th ACM Workshop on Hot Top-ics in Networks, HotNets-XI, (New York, NY, USA), pp. 43–48, ACM, 2012.

[70] N. Foster, R. Harrison, M. J. Freedman, C. Monsanto, J. Rexford, A. Story,and D. Walker, “Frenetic,” ACM SIGPLAN Notices, vol. 46, p. 279, Sept.2011.

[71] H. Kim and N. Feamster, “Improving network management with softwaredefined networking,” Communications Magazine, IEEE, vol. 51, pp. 114–119,Feb 2013.

[72] C. Monsanto, J. Reich, N. Foster, J. Rexford, and D. Walker, “Composingsoftware-defined networks,” in Proceedings of the 10th USENIX Conferenceon Networked Systems Design and Implementation, nsdi’13, (Berkeley, CA,USA), pp. 1–14, USENIX Association, 2013.

[73] C. Monsanto, N. Foster, R. Harrison, and D. Walker, “A compiler and run-time system for network programming languages,” SIGPLAN Not., vol. 47,pp. 217–230, Jan. 2012.

[74] A. Voellmy, H. Kim, and N. Feamster, “Procera: a language for high-levelreactive network control,” in Proceedings of the first workshop on Hot topicsin software defined networks, HotSDN ’12, (New York, NY, USA), pp. 43–48,ACM, 2012.

[75] T. L. Hinrichs, N. S. Gude, M. Casado, J. C. Mitchell, and S. Shenker, “Prac-tical declarative network management,” in Proceedings of the 1st ACM work-shop on Research on enterprise networking, WREN ’09, (New York, NY,USA), pp. 1–10, ACM, 2009.

[76] A. Voellmy and P. Hudak, “Nettle: taking the sting out of programming net-work routers,” in Proceedings of the 13th international conference on Practicalaspects of declarative languages, PADL’11, (Berlin, Heidelberg), pp. 235–249,Springer-Verlag, 2011.

38 BIBLIOGRAPHY

[77] M. R. Nascimento, C. E. Rothenberg, M. R. Salvador, C. N. A. Corrêa,S. C. de Lucena, and M. F. Magalhães, “Virtual routers as a service: therouteflow approach leveraging software-defined networks,” in Proceedings ofthe 6th International Conference on Future Internet Technologies, CFI ’11,(New York, USA), pp. 34–37, ACM, 2011.

[78] D. Sheinbein and R. P. Weber, “800 service using SPC network capability,”The Bell System Technical Journal, vol. 61, Sep. 1982.

[79] J. E. van der Merwe, S. Rooney, L. Leslie, and S. Crosby, “The tempest - apractical framework for network programmability,” Network, IEEE, vol. 12,no. 3, pp. 20–28, 1998.

[80] ITU-T, “I.326: Functional architecture of transport networks based on ATM,”tech. rep., ITU-T, 3 2003. Available at: http://www.itu.int/rec/T-REC-I.326-200303-I/en.

[81] L. Yang, R. Dantu, T. Anderson, and R. Gopal, “Forwarding and ControlElement Separation (ForCES) Framework.” RFC 3746 (Informational), Apr.2004.

[82] C. Rotsos, N. Sarrar, S. Uhlig, R. Sherwood, and A. W. Moore, “OFLOPS:an open framework for OpenFlow switch evaluation,” in Proceedings of the13th international conference on Passive and Active Measurement, PAM’12,(Berlin, Heidelberg), pp. 85–95, Springer-Verlag, 2012.

[83] M. Kuzniar, P. Peresini, and D. Kostic, “What you need toknow about SDN control and data planes,” tech. rep., EPFL,2014. Available at http://infoscience.epfl.ch/record/199497/files/switches-tr-oct14_1.pdf.

[84] R. Sherwood, “Cbench Controller Benchmarker.” Available at:http://www.openflow.org/wk/index.php/Oflops.

[85] A. Tootoonchian, S. Gorbunov, Y. Ganjali, M. Casado, and R. Sherwood, “Oncontroller performance in software-defined networks,” in Proceedings of the2nd USENIX conference on Hot Topics in Management of Internet, Cloud,and Enterprise Networks and Services, Hot-ICE’12, (Berkeley, CA, USA),pp. 10–10, USENIX Association, 2012.

[86] M. Jarschel, S. Oechsner, D. Schlosser, R. Pries, S. Goll, and P. Tran-Gia,“Modeling and performance evaluation of an openflow architecture,” in Tele-traffic Congress (ITC), 2011 23rd International, pp. 1 –7, sept. 2011.

[87] M. Jarschel, F. Lehrieder, Z. Magyari, and R. Pries, “A flexible openflow-controller benchmark,” in Software Defined Networking (EWSDN), 2012 Eu-ropean Workshop on, pp. 48 –53, oct. 2012.

BIBLIOGRAPHY 39

[88] M. Canini, D. Venzano, P. Perešíni, D. Kostić, and J. Rexford, “A nice wayto test openflow applications,” in Proceedings of the 9th USENIX conferenceon Networked Systems Design and Implementation, NSDI’12, (Berkeley, CA,USA), pp. 10–10, USENIX Association, 2012.

[89] B. Heller, R. Sherwood, and N. McKeown, “The controller placement prob-lem,” in Proceedings of the First Workshop on Hot Topics in Software DefinedNetworks, HotSDN ’12, (New York, NY, USA), pp. 7–12, ACM, 2012.

[90] L. M. Vaquero, L. Rodero-Merino, J. Caceres, and M. Lindner, “A break inthe clouds: towards a cloud definition,” SIGCOMM Comput. Commun. Rev.,vol. 39, pp. 50–55, Dec. 2008.

[91] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, “Cloud com-puting and emerging it platforms: Vision, hype, and reality for deliveringcomputing as the 5th utility,” Future Gener. Comput. Syst., vol. 25, pp. 599–616, June 2009.

[92] J. Geelan, “Twenty one experts define cloud computing virtualization: elec-tronic magazine,” 2009. Available at: http://www.sys-con.com/node/612375/.

[93] “libvirt —- the virtualization API.” Available at: http://libvirt.org.

[94] A. Kivity, Y. Kamay, D. Laor, U. Lublin, and A. Liguori, “KVM: the linuxvirtual machine monitor,” in Proceedings of the Linux Symposium, vol. 1,pp. 225–230, 2007.

[95] P. Barham, B. Dragovic, K. Fraser, S. Hand, T. Harris, A. Ho, R. Neugebauer,I. Pratt, and A. Warfield, “Xen and the art of virtualization,” ACM SIGOPSOperating Systems Review, vol. 37, no. 5, pp. 164–177, 2003.

[96] P. Murray et al., “D-D.1 Cloud Networking Architecture Description,” tech.rep., SAIL FP7 Project, 2011.

[97] E. Amazon, “Amazon elastic compute cloud (amazon ec2),” Ama-zon Elastic Compute Cloud (Amazon EC2), 2010. Available at:http://aws.amazon.com/ec2/.

[98] B. Sotomayor, R. S. Montero, I. M. Llorente, and I. Foster, “Virtual infras-tructure management in private and hybrid clouds,” IEEE Internet Comput-ing, vol. 13, pp. 14–22, 2009.

[99] D. Nurmi, R. Wolski, C. Grzegorczyk, G. Obertelli, S. Soman, L. Youseff,and D. Zagorodnov, “The eucalyptus open-source cloud-computing system,”in 2009 9th IEEE/ACM International Symposium on Cluster Computing andthe Grid, pp. 124–131, IEEE, 2009.

40 BIBLIOGRAPHY

[100] “OpenStack.” Available at: http://www.openstack.org/.

[101] “Rackspace. The Open Cloud Company.” Available at:http://www.rackspace.com/.

[102] NASA, “NASA’s Nebula Cloud Computing Technology To Play Key RoleIn New Open Source Initiative.” Available at: http://www.nasa.gov/home/hqnews/2010/jul/HQ_10-172_Nebula_Initiative.html.

[103] “OpenStack Neutron.” Available at: http://wiki.openstack.org/Neutron.

[104] “OCCI: Open Cloud Computing Interface.” Available at:http://occi-wg.org/.

[105] J. Soares, J. Carapinha, M. Melo, R. Monteiro, and S. Sargento, “Resource al-location in the network operator’s cloud: A virtualization approach,” in Com-puters and Communications (ISCC), 2012 IEEE Symposium on, pp. 800–805,July 2012.

[106] J. Matias, E. Jacob, D. Sanchez, and Y. Demchenko, “An openflow basednetwork virtualization framework for the cloud,” in Cloud Computing Tech-nology and Science (CloudCom), 2011 IEEE Third International Conferenceon, pp. 672–678, Nov 2011.

[107] IXIA, “IxNetwork Datasheet.” Available at: http://www.ixiacom.com/products/ixnetwork.

[108] Spirent, “Smartbits.” Available at: http://www.spirent.com/Products/Smartbits.aspx.

[109] R. Jones, “Netperf: a network performance benchmark.” Available at:http://www.netperf.org/netperf/.

[110] “Iperf. TCP and UDP bandwidth performance measurement tool.” Availableat: https://code.google.com/p/iperf/.

[111] R. Olsson, “pktgen the linux packet generator,” in Linux Symposium 2005,(Ottawa, Canada), 2005.

[112] L. Rizzo, “Revisiting network I/O APIs: the netmap framework,” Commu-nications of the ACM, vol. 55, pp. 45–51, Mar. 2012.

[113] Intel, “Intel DPDK: data plane development kit.” Available at:http://dpdk.org/.

[114] D. Turull, M. Hidell, and P. Sjödin, “libNetVirt: the network virtualizationlibrary,” in Workshop on Clouds, Networks and Data Centers (ICC’12 WS -CloudNetsDataCenters), (Ottawa, Canada), June 2012.

BIBLIOGRAPHY 41

[115] D. Turull, M. Hidell, and P. Sjödin, “Using libNetVirt to control the virtualnetwork,” in 2012 1st IEEE International Conference on Cloud Networking(IEEE CloudNet’12), (Paris, France), Nov. 2012.

[116] D. Turull, M. Hidell, and P. Sjödin, “Performance evaluation of OpenFlowcontrollers for network virtualization,” in 2014 IEEE 15th International Con-ference on High Performance Switching and Routing (IEEE HPSR ’14), (Van-couver, Canada), July 2014.

[117] D. Turull, P. Sjödin, and R. Olsson, “Pktgen: Measuring performance on highspeed networks,” Computer Communications, pp. 39–48, May 2016. Volume82, ISSN 0140-3664, Available at: http://dx.doi.org/10.1016/j.comcom.2016.03.003.

[118] D. Turull, M. Hidell, and P. Sjödin, “Evaluating OpenFlow in libNetVirt,”in The 8th Swedish National Computer Networking Workshop 2012 (SNCNW2012), (Stockholm, Sweden), June 2012.

[119] N. Handigol, B. Heller, V. Jeyakumar, B. Lantz, and N. McKeown, “Repro-ducible network experiments using container-based emulation,” in Proceed-ings of the 8th international conference on Emerging networking experimentsand technologies - CoNEXT ’12, (New York, New York, USA), p. 253, ACMPress, Dec. 2012.

[120] U. Pettit, J. Gross, B. Pfaff, M. Casado, and S. Crosby, “Virtual switchingin an era of advanced edges,” in 2nd Workshop on Data Center - Convergedand Virtual Ethernet Switching (DC CAVES), Sept. 2010.

[121] M. Casado, T. Koponen, R. Ramanathan, and S. Shenker, “Virtualizing thenetwork forwarding plane,” in Proceedings of the Workshop on ProgrammableRouters for Extensible Services of Tomorrow, PRESTO ’10, (New York, NY,USA), ACM, 2010.

[122] C. Rotsos, N. Sarrar, S. Uhlig, S. Rob, and A. W. Moore, “Oflops: An openframework for openflow switch evaluation,” in Proceedings of Passive andActive Measurements Conference (PAM ’12), March 2012.

[123] VMware, “The vmware nsx network virtualization platform.”Available at: http://www.vmware.com/files/pdf/products/nsx/VMware-NSX-Network-Virtualization-Platform-WP.pdf.

[124] “NEC programmable flow.” Available at: http://www.necam.com/SDN/.

[125] “Huawei softcom.” Available at: http://www.huawei.com/en/solutions/softcom/.

Part II

Research Papers

43