1

A Policy based Security Architecture for SoftwareDefined Networks

Vijay Varadharajan, Senior Member, IEEE, Kallol Karmakar, Student Member, IEEE, and UdayTupakula, Member, IEEE and Michael Hitchens

Abstract—As networks expand in size and complexity, they posegreater administrative and management challenges. SoftwareDefined Networks (SDN) offer a promising approach to meetingsome of these challenges. In this paper, we propose a policydriven security architecture for securing end to end servicesacross multiple SDN domains. We develop a language basedapproach to design security policies that are relevant for securingSDN services and communications. We describe the policylanguage and its use in specifying security policies to controlthe flow of information in a multi-domain SDN. We demonstratethe specification of fine grained security policies based on avariety of attributes such as parameters associated with usersand devices/switches, context information such as location androuting information, and services accessed in SDN as well assecurity attributes associated with the switches and Controllersin different domains. An important feature of our architectureis its ability to specify path and flow based security policies,which are significant for securing end to end services in SDNs.We describe the design and the implementation of our proposedpolicy based security architecture and demonstrate its use inscenarios involving both intra and inter-domain communicationswith multiple SDN Controllers. We analyse the performancecharacteristics of our architecture as well as discuss how ourarchitecture is able to counteract various security attacks. Thedynamic security policy based approach and the distributionof corresponding security capabilities intelligently as a servicelayer that enable flow based security enforcement and protectionof multitude of network devices against attacks are importantcontributions of this paper.

Index Terms—Software Defined Networking (SDN) Security,Security Policies, Security Architecture, Inter-domain Security

I. INTRODUCTION

AS networks expand in size and complexity, they posegreater administrative and management challenges. In-creasingly, current networks are highly heterogeneous withmany different devices, from small sensors and appliancesto network devices such as routers to many different clientsand servers and peripherals. Furthermore, these devices usedifferent network technologies such as fixed, wireless andmobile networks. In such a complex heterogeneous envi-ronment, management of network devices (such as switchesand routers), the mobility of users and devices, the dynamicvariation in networks (due to failure of devices and networklinks), as well as the dramatic increase in security attacks areposing serious challenges. Software Defined Networks (SDN)[1] offer a promising approach to meeting some of these

Vijay Varadharajan is with the Faculty of Engineering, The University ofNewcastle, Australia. E-mail: vijay.varadharajan@newcastle.edu.au

Kallol Karmakar and Uday Tupakula are with the School ofElectrical Engineering and Computing, The University of Newcastle,Australia. E-mail: kallolKrishna.Karmakar@newcastle.edu.au anduday.tupakula@newcastle.edu.au

Michael Hitchens is with the Dept of Computing, Faculty ofScience and Engineering, Macquarie University, Australia. E-mail:michael.hitchens@mq.edu.au

challenges. SDN is rapidly emerging as a disruptive tech-nology, poised to change communication networks much thesame way cloud computing is changing the “compute” world.It is altering the texture of modern networking, movingaway from the current control protocols dominant in theTCP/IP Internet stack, towards something more flexible andprogrammable. It is potentially changing the way networkingwill be conducted in the future, by enabling devices that areopen and controllable by external software, unlike today’sproprietary network equipment that has protocols embeddedinto them by the vendors. SDN opens up new avenues ofresearch to realize network capabilities that were impossible orextremely cumbersome before, thereby helping to make futurenetworks more manageable and practicable. The separationof the control plane from the data plane in SDN results inthe network switches becoming simpler forwarding deviceswith the more sophisticated control logic implemented insoftware in a logically centralized Controller. This decouplingin SDN enables the design of new and innovative networkfunctions and protocols. First, it is simpler and less error-prone to modify network policies through software, than vialow-level device configurations. Second, a control programcan automatically react to spurious changes in the networkstate and thus maintain up to date high-level policies. Third,the centralization of the control logic in a Controller withnetwork domain wide knowledge can help to simplify thedevelopment of sophisticated network functions. AlthoughSDN offers such advantages to deal with complexities incurrent networks, a critical issue in SDN at present is thatof security; the current state of the art in SDN security is notmature [2]. Securing networks is becoming more challengingto businesses, especially with bring your own devices (BYOD),increased cloud adoption and the Internet of Things (IoT).The main causes of security concerns probably lie in SDN’smain benefits, namely programmability of networks and thecentralization of control logic. These capabilities introducenew security threats and attack surfaces, which do not exist intraditional networks. Ironically, the closed (proprietary) natureof network switches together with the heterogeneity of vendorsoftware and integrated control functions previously offerednatural layers of defense in traditional networks. That is, anattack against a network device from a specific vendor willoften not work against another network device from anothervendor. In some sense, the diversity of network devices andprotocols provides a certain level of security, just like secrecyassociated with proprietary systems. On the other hand, SDNwith its standardized interfaces and protocols (e.g. OpenFlow[1] protocol between the Controller and the switches) canprovide a focused target for the attackers; furthermore, anypotential logical security faults in compliant implementations

arX

iv:1

806.

0205

3v1

[cs

.CR

] 6

Jun

201

8

2

of the SDN protocols and software can increase the securityrisks. e.g. an attack similar to Stuxnet [3], which targeted theoperation of many devices in specific networked infrastruc-tures by automatically modifying their control programs andconfigurations, could have dramatic consequences in a highlyprogrammable SDN network. In adddition, the centralizationof functionality in the SDN Controller presents another pointof security weakness.In a SDN environment, the control plane specifies the policiesfrom which the flow rules are derived and enforced in thedata plane at the SDN switches and network devices. Inthis paper, we consider the design of security policies forthe establishment of secure end to end communication pathsin a distributed SDN environment. In large networks withmultiple autonomous system domains crossing organizationalboundaries, paths must be selected according to policy relatedparameters involving security attributes in addition to thetraditional parameters of connectivity, congestion and costs.The ability to distribute security capabilities intelligently asa service layer and to have a dynamic security policy basedapproach to securing a multitude of devices against threatsare important contributions of this paper. For instance, wecan have policies that enforce the requirement that certaintraffic must pass through certain switches which are moresecure than others. In this case, a specific set of switchesare obligatorily traversed for this traffic. Another exampleis that to counteract particular threats, certain switches maybe forbidden to be in the paths of certain defined traffic.Our policy based architecture combined with the Controller’snetwork domain wide visibility offers a powerful approach toenforcing security mechanisms to counteract security attacksin SDN. This capability is achieved in both the intra and inter-domain contexts with multiple domains managed by differentSDN Controllers.The main contributions of this paper are as follows: Wepresent the design of a security architecture with specificaccess policy constraints on communications between endusers/devices in a distributed SDN environment with multipledomains. Our method is based on the use of a security policylanguage based approach enabling secure flow of packetsand secure management of paths in a distributed SDN. Thesecurity policies are specified at a fine granular level, whichwhen enforced help to detect different types of attack flows,counteracting various threats in the SDN. In our architecture,the security policies use the context associated with the flows,such as location, routing information, services accessed as wellas security labels associated with the switches and Controllers,to securely manage the flows. For instance, suppose due to aDDoS attack, traffic from an end host is not able to get throughthe network. Our SDN security architecture is able to detectthe DDoS attack efficiently and establish an alternative path forthe traffic from the end host to reach the required destination.Moreover, we can enforce path and flow based policies such ascertain communications should go through a path with certainsecurity attributes. Such path based policies are critical whensecuring data from sensitive applications but are also usefulfor applications with different quality of service requirements.For instance, video traffic requiring certain bandwidth needs to

Application Plane

Control Plane

Data Plane

 Applications 

Network Services

Router Switch WAP

Northbound Interface 

Southbound Interface (e. g OpenFlow )

WAP‐ Wireless Access Point

Fig. 1: Standard SDN Architecture

take a path where the switches and channels have the necessarycapabilities (compared to just supporting audio traffic). Anovel feature of the proposed security architecture involves theuse of the visibility of the network domain and connectivityto manage flow and path based security policies to achievesecure communications and efficient provision of SDN ser-vices across multiple domains. Combined with this feature, ourarchitecture has the potential to dynamically update securitypolicies based on the distributed network state and detection ofattacks, which is particularly important to counteract emergingsecurity threats. We believe such architectural features are vitalfor achieving resilient network systems, especially in securingcritical infrastructures.The paper is organized as follows: After first outlining a briefintroduction to SDN, in Section II, we describe the threatmodel for our proposed security architecture. We considerthe various security threats in a SDN environment such asthreats to the Controller, the network switches as well asthe communications between the switches and the Controller,and the communications between the Controllers in multipledomains. We then focus on the security threats that our securityarchitecture is designed to address and outline the benefitsof our approach. Section III describes our proposed policybased security architecture for a distributed SDN. First wegive an overview of the proposed architecture. Then, wedescribe the specification of security policies, which is a keycomponent of the proposed policy based architecture. Thenwe provide a brief walk-through of the security architectureusing an inter-domain scenario with multiple domains andSDN Controllers. Section IV describes the implementation ofthe security architecture and its various components. SectionV presents the implementation and the performance results.We discuss security policies in both intra and inter domaincontexts and illustrate them with example scenarios. Securityanalysis of some of the implemented scenarios is given inSection VI. In Section VII, we review the relevant relatedworks and compare our architecture with existing solutions.Finally Section VIII concludes the paper and outlines somefuture work.

II. SDN ARCHITECTURE AND THREAT MODEL

A. Basic SDN Architecture

Figure 1 shows an overview of the SDN architecture withdifferent components in the control plane and data plane. Thecontrol plane consists of a logically centralized Controller(which itself can be distributed in practice) with native appli-cations for management of devices in a SDN network. Therecan be several third party applications that can be hosted(or access different services) in the Controller. The interface

3

between the Controller and the applications is referred to asthe North Bound Interface. The data plane consists of thenetworking devices and the interface between the Controllerand the networking devices is referred to as the South BoundInterface. OpenFlow [4] is the most commonly used protocolfor communication between the Controller and the networkingdevices. Other protocols include sFlow and SNMP that canbe used for communication between the Controller and thenetwork devices. In the SDN terminology, a switch refers toany networking device that operates in Layers 2 to 7 in theOSI model.The Controller manages the flow-entries in the flow tablesof the switches through a secure channel (that exists in theswitch itself). This management process might be done bothreactively (in response to packets) as well as proactively.There are several SDN Controllers [5] available using differentprogramming languages and environments. For example, NOXis based on C++ and Python programming languages; POX isbased on Python; and Beacon and Floodlight are based onJava.An OpenFlow compatible switch in the data plane containsthree parts: i) A Flow Table, with an action linked with eachflow entry, which tells the switch how to process the flow; ii)A channel that joins the switch to a remote control process (theController), allowing communication between a Controller andthe switch; communication instructions and packets to be sentbetween a Controller and the switch; using the OpenFlowprotocol; iii) The OpenFlow protocol, which provides an openand standard way for a Controller to interconnect with aswitch.B. Our Network Architecture and Threat ModelOur network architecture consists of multiple AutonomousSystem (AS) domains, with each domain having packet for-warding devices such as routers, gateways and switches andend hosts (which are connected to users). In general, there canbe multiple SDN Controllers but for simplicity, we assumethat each domain contains one SDN Controller. End hostsare connected to the forwarding devices. For clarity, we willassume that each AS has a separate entry and exit gateway.These are OpenFlow supported forwarding devices. Thoughwe assume OpenFlow switches, our solution will work withlegacy switches, as long as there exists a path which hassome OpenFlow switches. (Furthermore, migration solutionsare being developed which allow incremental migration tosoftware defined networks from current networks, e.g. [6]).The traffic generated by the end hosts and forwarded by thenetwork devices is subjected to security policies specified inthe Controller. As there are multiple SDN domains, we haveboth intra-domain as well as inter-domain communications.In an intra-domain communication, the traffic from sourceto destination passes through devices within a single SDNdomain and the requested services are provided by the serversand devices in the same domain. In this case, the trafficand service requests are subjected to security policies in theSDN Controller of that domain. Inter-domain communicationsinvolving multiple domains require cooperation between SDNControllers, as the communications are subject to securitypolicies in multiple SDN Controllers.

There are different types of security attacks that are possible ina SDN. There can be attacks on the control plane communica-tions between the switches and the Controller in a domain aswell as attacks on the data plane communications. In general,threats in SDN domains can be categorized into:

1) Threats against a SDN Controller2) Threats against the networking devices (switches)3) Threats against communications between the Controller

and the networking devices:4) Threats against communications between different SDN

Controllers in different AS domains1. Threats against a SDN Controller: As the Controller is atthe heart of a SDN domain, if the Controller is compromised,then the whole network under that SDN Controller is madevulnerable. A Controller can be compromised in three ways:i) due to malicious software errors or bugs in the Controllersoftware system such as the operating system;ii) due to threats arising from malicious or compromisedapplications running on top of or in the Controller;iii) due to threats from the underlying network devices suchas the OpenFlow switches.The threats and attacks arising in the first two categories (i)and (ii) are somewhat similar to those that occur in softwareand operating systems. Such attacks can include common onessuch as cross-site scripting, SQL injection, command injectionand buffer overflow. For instance, if a SDN application usesa web-interface, then it can be used by an attacker to installa malicious script which can bypass the authentication mech-anism in the Controller. This will enable the attacker to gainaccess to the Controller and carry put further attacks such aseavesdropping on the device traffic and delete/modify flows inthe flow entry tables. Security techniques to counteract thesethreats and attacks are similar to those used in software systemsecurity. These attacks are not the focus of this paper. Securitymechanisms described in works such as [7] and [8] discusssome such attacks in the Controller.Our security architecture addresses the threats against theController in (iii), which occur via the switching fabric. Theconnections that a switch can establish are based on thepolicies specified in our security architecture, thereby helpingto detect attack flows and counteract them. For instance,malicious hosts connected to the switches can launch floodingattacks by forging IPs [2]. These communication requests fromthe forged IPs will look like legitimate requests to the switches.In a proactive mode, they will be dropped immediately. In thereactive mode, since they are new flows, a switch will generatea request to the Controller. This can lead to denial of serviceattacks against the Controller. Our security architecture hasfine grained policies which are able to detect malicious hostand flooding attacks, and drop any further requests from thishost by installing drop rules to discard further flooding fromthe switches connected to that host.2. Threats against Network Devices: Forwarding networkdevices are usually physically closer to the attackers. Oftenit is the less sophisticated forwarding devices that are themain targets for attacks. Each device contains some numberof flow rules in a flow table which are used to route the

4

packets. For instance, an attacker who knows the IP range ofthe Controller domain can flood a network device with forgedpackets. Also, if an attacker can access multiple hosts, s/hecan launch denial of service (DoS) attacks by continuouslysending random IP packets. An attacker may also be able tolisten on the links between any of the forwarding devices. Thisis possible because often there is no protection on packets orpolicies to route the packet through specific switches (therebypreventing access to the flows) [9]. This can also lead tosubsequent attacks by spoofing the address of any of the hostsand launching DoS and Man-in-the-Middle attacks as wellas illegal modification of the flow table entries. Our securityarchitecture will help to detect attack traffic from malicioushosts towards the switches, and block them by dynamicallyinstalling policy rules in the switches.

3. Threats against the Communications between the Controllerand the Network Devices: The protocol between the SDN Con-troller and the switches need to be protected against securityattacks. Typically the Controller may establish a secure linkwith the forwarding devices using security protocols such asTransport Layer Security (TLS). Some Controllers may notuse the TLS support option in the OpenFlow [9]. In thiscase, it makes it vulnerable to traditional security attacks suchas eavesdropping, unauthorized modification of traffic andmasquerading. In this paper, we do not address confidentialityand related key management issues. The security architectureproposed in this paper deals with access and informationflow attacks in SDNs. We are currently extending the securityarchitecture with encryption mechanisms and associated keymanagement authorities, thereby counteracting such attacksas well as for providing on demand protection of traffic.We mention this in the Conclusion. Hence we will not beaddressing these communication threats in this paper. It isworth pointing out that the techniques used to achieve theconfidentiality and key management services are somewhattraditional and well known.

4. Threats against the Communications between SDN Con-trollers in different AS domains: In practice, large networksmay require more than one SDN Controller per AS domainand there can be many AS domains. Hence there can be attackson the communications between different SDN Controllers ina single AS or in multiple AS domains [2]. Furthermore, thereis a need to take into account legacy networks. Interceptionof traffic between Controllers can lead to many attacks suchas information theft, spoofing and flooding attacks. Hence theneed for a secure routing model for Controller to Controllercommunications.

Our security architecture is concerned with authorized flowsacross multiple domains by enforcing security policies speci-fied in different SDN Controllers. These policies detect attackssuch as flooding and spoofing, and ensure only secure andauthorized traffic flows occur between domains. The focus ofthis paper is the design of configurable security policies thatcan enforce authorized information flows in a distributed SDNenvironment with multiple domains managed by different SDNControllers. Our architecture enables secure virtual partitionof the network to achieve separation of flows and services,

thereby reducing the attack surface in SDN. It addresses threatsarising from malicious traffic generated by end hosts leadingto attacks against switches and the Controllers; it detectsand prevents unauthorized flows and unauthorized access toservices in a distributed SDN.

Attack Scenarios

In particular, our security architecture can deal with thefollowing specific attack scenarios:

• (A1) Unauthorized communications between end hostswithin a domain as well as between end hosts residing indifferent domains: Attacks such as worms are often suc-cessful by exploiting the default permit communicationbetween the hosts in current networks. The infected ma-chine uses a random source address to scan for vulnerablemachines and spreads the attack.

• (A2) A malicious end host attempting to access serviceson the network for which it is not authorized: Insiderthreat is one of the difficult challenges when it comes tosecuring networks in organisations. Often this involvesmalicious users making use of their machines to accessservices for which they are not authorized.

• (A3) Attacks originating from end hosts residing incertain end locations: Wirelsss LANs are often used inproviding free access to any user. Attackers can misusesuch free access networks for generating attacks. Sim-ilarly, there can be malicious ISP/AS domains that areused in the generation of attacks.

• (A4) A malicious end host generating malicious trafficattacking the switches and/or flows through certain pathsin the network: An attacker can generate malicious trafficthat is destined to the victim switch and/or flood certainpaths in the network with malicious flows.

• (A5) Attacks that chain malicious flows coming fromdifferent locations at different times: A bot master whohas control over several hundred compromised machinescan generate attacks on the victim networks or ASdomains using compromised machines from differentlocations at different times, with possibly different typesof attack traffic. Such situations can occur in inter-domain scenarios using attacks such as Crossfire [10] andCoremelt [11]. In Crossfire, a small set of bots directs lowintensity flows to a large number of publicly accessibleservers. These flows flood carefully chosen small set oflinks, effectively disconnecting the target servers from theInternet. In Coremelt, attackers only send traffic betweeneach other to clog the network, rather than towards avictim host. It is difficult to deal with these attacks asthe attack traffic appears as legitimate communicationsbetween the end hosts.

We discuss the security analysis of these attack scenarios insection VI using our proposed architecture.The deployment of our proposed security architecture willprovide system managers of complex distributed networkswith facilities to configure dynamically security policies whichcan detect and counteract different types of attack flows, andachieve secure provisioning of SDN services.

5

C. Security Requirements

This section discusses the security requirements that are ad-dressed in the design of our policy based security architecture.• (R1): Security requirements can vary for different types of

flows in a network. There can be security constraints ondifferent flow parameters such as the time of flow, loca-tion of devices that are generating or receiving the flows,and delay and bandwidth requirements for the flows. Forexample, critical applications may have stringent require-ments for the delivery of messages. Sensitive applicationsmay require that the traffic is transferred through a securechannel involving secure switches. Hence, there is a needto ensure that these specific flow requirements are pro-visioned correctly in a multi domain SDN environment.This requires the ability to handle flow specific securitycharacteristics in inter domain communications in SDNs.

• (R2) There is a need to secure normal SDN operations,as the attackers can exploit the weaknesses in the SDNoperations to generate different types of attacks. Forexample, currently Controllers do not validate the flowrequests before establishing the routes to enable commu-nication between the end hosts. For instance, the attackssuch as the spread of worms in traditional networks canalso happen in SDNs. A malicious host scan for randomaddresses to find vulnerable machines and spread theattacks, establishing routes to destination hosts.

• (R3) As a Controller has the visibility of its networkdomain topology and devices, there is a potential todevelop secure northbound applications making use ofthe information available at the Controller for achievingend to end security within a domain.

• (R4) In a multi-domain situation, the end hosts areinvolved in communication with hosts connected to dif-ferent networks. Hence, there is a need for to developtechniques for achieving end to end security over mul-tiple domains. However as a Controller has visibilityonly over its domain, achieving secure inter domaincommunications requires secure co-operation between theControllers in different domains.

We discuss how these requirements have been achieved by oursecurity architecture in Section VI

III. POLICY BASED SDN SECURITY ARCHITECTURE

A. Security Architecture Overview

First, we will consider a Policy based Security Architecture forintra-domain interactions and then we will address the inter-domain communications enabling end-to-end SDN servicesacross multiple domains. We will present a high level overviewof the architecture and then describe each component in detail.Figure 2 shows the Policy based Security Architecture forsecuring communication in a SDN domain. The Policy basedSecurity Architecture can either form part of the SDN Con-troller or can run as a Security Application on top of the SDNController. We have designed and developed the Policy basedSecurity Architecture as a Security Application running on topof a SDN Controller for flexibility reasons. We will refer tothis as Policy based Security Application (PbSA). PbSA is

Fig. 2: Policy based Security Architecture for SDN

implemented in the north bound interface of the Controller.As PbSA is designed in a modular fashion, the componentsof PbSA can be implemented on a single host or distributedover multiple hosts.We assume that each AS domain is controlled by a SDNController. Each Controller has a Policy Server. The PolicyServer has five main components the Topology and PolicyRepositories, a Policy Manager, a Policy Evaluation Engine,a Policy Enforcer and a Handle Creator. Each Controllermaintains and updates a Topology Repository and a PolicyRepository. The Topology Repository contains the networktopology information derived using traceroute mechanismmentioned below. The Policy Repository contains the PolicyExpressions and specifications which are expressed using asimple language based template described in Section III-Bbelow. The Policy Manager as the name implies manages everysingle operation of the security system. An Evaluation Engineis used to evaluate the incoming network traffic against therelevant security policies for that specific traffic. Followingthe evaluation, the Policy Manager determines the flow ruleswhich are then conveyed to the Enforcer module. The Enforcermodule not only fetches the required information from thesouth bound interface connected to the switches but alsoenforces the flow rules obtained from the Policy Manager. APacket Handle Creator module creates the necessary handlesfrom the visited Controller which is piggy backed with thepayload from the Policy Manager. These handles are used tocheck the authenticity of the packet and the enforcement ofpolicies at the switches.With intra-domain communications, the traffic from sourceto destination passes through devices within a single SDNdomain and the requested services are provided by the serversand devices in the same domain. In this case, the traffic andservice requests are subjected to security policies in the PbSAin the SDN Controller of that domain. The routing processbegins from the host that is generating the packets and therequest, which is the source of the communication. This sourcehost could be any client, such as a mobile device. The initialpacket header from the source host is sent by the switch (towhich this host is connected) to the SDN Controller in theAS domain. The header contains all the usual network andservice parameters such as the source address, the packet type.The PbSA application in the Controller extracts the relevantparameters from the incoming packets and uses the PolicyRepository and the Policy Manager to determine whetherthe relevant Policy Expressions are satisfied. If the PolicyExpressions are valid for the incoming packets, then PbSA willenforce the specified actions as flow rules in the appropriate

6

data plane devices such as switches to transfer the packets.

Inter-domain communications involving multiple domains re-quire cooperation between SDN Controllers, as the commu-nications are subject to security policies of multiple Con-trollers. Hence inter-domain routing of traffic requires an SDNController in one AS domain to have knowledge of otherControllers in other AS domains. To create a topological mapof a distributed SDN environment, we have used the traceroutemechanism. Note that although there are other alternatives[12] for topology discovery such as Border Gateway Protocol(BGP) and Internet Routing Registries, each one of them hasits own issues. For the purpose of our prototype, traceroutewas found to be sufficient and the easier one to use. It is not acritical part of our design and it is mainly an implementationissue. Each SDN Controller in an AS domain sends an ICMPsignal using a different TTL level (1-6) to make a topologicalmap of the AS domains. From the architecture point of view,each Controller has a Topology Repository to store the map-ping of the topology information. SDN Controllers in each ASkeep this Repository updated by running traceroute at differenttimes. We have included an additional security attribute, aSecurity Label, in the ICMP response message from eachAS SDN Controller. The intention of the Security Label isto reflect the level of security associated with that particularController. In our current architecture, this Security Label ishardwired (static) and is specified at the time of installationof the Controller based on the reputation of the manufacturerof the Controller. In the next stage of the development of thearchitecture, we will develop a meta-level security protocolthat will enable secure and dynamic updating of the SecurityLabel depending on the behaviour of the Controller over time.This will be done as part of a trust model which we are inthe process of developing for distributed SDN environment.We have modified the ICMP response messages to attach theSecurity Labels. Hence each AS domain SDN Controller nowhas the ability to discover the topological information as wellas the levels of security associated with all the neighboringAS domain Controllers in this Repository.

Consider the distributed SDN environment shown in Figure 3and the associated Topology Repository tables are shown in thefigure too. Each hexagon in Figure 3 represents an AS domain.We have represented the AS Gateways using Gateway Open-Flow Switches. In this case, all the controllers use tracerouteby varying TTL level from (1-4). When TTL becomes 0, theControllers responds with an ICMP TTL exceeded message,which contains additional information about the AS domain(Sender IP, ID, Security Label). This information is storedin the Topology Repository for future use by the respectiveSDN Controller. Each Table in Figure 3 shows the TopologicalRepository for the respective domains SDN controller. AS IDis the identity of a particular AS, Sec Label is the securitylabel of the AS domain and Hops is the distance from source todestination AS. The edge OpenFlow Switches are representedusing the notation ([source AS ID]SW[destination AS ID]), e.g. switch connecting AS1 to AS2 is represented as 1SW2.

In the inter-domain setting, our architecture introduces twoadditional mechanisms which have conceptual significance.

10.0.0.200:00:00:00:00:01

10.0.0.0/24

172.16.10.0/24

192.168.10.0/24

192.168.52.0/24

192.168.52.7200:00:00:00:01:01

2SW3X

Y

No AS ID Sec_Label Hop1 AS1 SL2 22 AS3 SL3 23 AS4 SL4 2

No AS ID Sec_Label Hop1 AS1 SL2 22 AS2 SL3 23 AS4 SL4 2

No AS ID Sec_Label Hop1 AS2 SL3 22 AS3 SL3 23 AS1 SL2 4

No AS ID Sec_Label Hop1 AS2 SL3 22 AS3 SL3 23 AS4 SL4 4

QC

C

C

C

C = SDN controller VM; Mininet = Mininet VM; Apps = NBI; Q = Quagga VM (for static routes) & VM Pair = C+ Mininet

VM Pair1

VM Pair2

VM Pair3

VM Pair4

Fig. 3: Implemented Network with Routing Information.

Handle: The first mechanism is a Handle. PbSA creates aHandle and tags to each flow request. The Handle consistsof a list of visited AS domain IDs. The Packet + Handleis then transferred to the next AS Domain Controller. Asimilar process is repeated as the packet goes through all thetransit AS domain SDN Controllers until the packet reaches itsdestination. This Handle will be protected for integrity and itwill be used in the validation of flows across multiple domains.

Policy Transfer Token The second mechanism is a PolicyTransfer Token, which comprises policy constraints that aretransferred from one AS domain to the subsequent transitAS domains and which need to be satisfied by the flow,as the packets are transferred. These constraints need to betaken into account in addition to the policy constraints of thetransit domains. For instance, if there is a constraint that thetraffic should only pass through AS domains with securitylabel greater than a certain threshold, then this constraintneeds to be satisfied by subsequent transit domains. Supposean AS domain SDN Controller (with AS ID = 10) has aconstraint that packets should only be forwarded through apath of AS domain SDN Controllers that have a securitylabel greater than SL3. In distributed systems, in general,it is not possible for one domain Controller to know aboutpolicies of other domains. Hence there is a need to transferthe policy constraints, which are communicated via PolicyTransfer Tokens. The significance of the transfer token isthat the policy constraints that are transferred are only thosepolicies that are specific to flows and packets in that flow.Such a mechanism is useful as it enables partial delegation ofpolicies that are flow dependent. The next section describesin detail the specification of security policies. In this paper,we assume that the AS domain Controllers are secure andtrusted, and that if and only if the policies can be satisfied inthe domain, the receiving Controller will accept the packets.

In terms of the notation, we denote the Handle as HASki whichis tagged to the packet (flow request), where Hi is the Handlefor a particular communication i, and k is the ID of the ASdomain SDN Controller which created the Handle. Similarly,the Policy Transfer Token is denoted as PTTASki , where againi denotes the specific communication and k denotes the ID ofthe AS domain. Hence an AS domain SDN Controller createsthe augmented Packet using the original Packet as well as theHandle and the Policy Transfer Token.

7

B. Security Policy Specifications

A key component of the security architecture is the specifica-tion of security policies that are to be enforced on the SDNcommunications. The specified security policies are stored ina Policy Repository in the PbSA.We have adopted a simple language based approach to specifythe security policies. We have chosen the policy based routingsyntax specified in RFC1102 [13] as the basis for our securitypolicy specifications. Policies are rules that specify a particularpath or paths that packets must follow in the network and theconditions under which the packets follow these paths. In ourlanguage, we have Policy Expressions specifying a range ofattributes associated with the flow and the entities in the SDN.These include the following:

• Flow Attributes: Flow ID, sequence of packets associated with the flow,type of packets, security profile indicating the set of security servicesassociated with the packets in the flow;

• Autonomous System Domain Attributes: AS identities such as sourceAS and destination AS identities (AS Domain ID), sub-net address space(SRCSUB for source and DSTSUB for destination), identities ofentry (SRCENT ) and exit (DSTEXT ) gateway/switch to AS, AStype (e.g. Commercial domain, Government domain) (SRCType andDSTType ) and security label associated with the AS (SRCSL forsource and DSTSL for destination);

• Switch Attributes: Identities of the switches and security label of theswitches;

• Host Attributes: Identities of hosts - source host IP & MAC ( SRCIP &SRCMAC ) and destination host IP & MAC ( DST IP & DSTMAC);

• Flow and Domain constraints: Flow constraints (FlowCons) and Domainconstraints (DomCons) associated with a flow such as thresholds, attacksignatures etc.;

• Services - Services for which the Policy Expression applies;• Time Validity - The time period for which the Policy Expression is

valid and• Path (ASSEQ) - In the case of intra-domain, indicates a specific

sequence of switches whereas with inter-domain communications, itindicates the sequence of Autonomous Systems traversed by a flow.

The flow constraints are conditions that apply to a specificflow or a set of flows. For instance, a constraint may specifythat the flow of packets of a specific type (e.g video) shouldonly go through a set of switches that can provide a certainbandwidth; or from a security point of view, a constraint couldbe that a flow should only go through AS domains that areat a particular security level. Domain constraints apply to allflows within a domain. They are used to specify domain-widepolicies. For instance, there could be a domain wide securitypolicy which may specify that all flows should be protectedfor integrity, as part of the security profile. These constraintsare used as part of the actions associated with the PolicyExpressions. Packet (PKTATT ) and Time attributes (TPEi )can be integrated into the constraints. Here, flow attributesindicates attributes associated with the sequence of packetsin the flow such as type of the incoming packets based onport numbers, thresholds, security services associated with thepackets and attack signatures. Time attributes represents theduration time for which a particular Policy Expression is valid.Alternatively, it is also possible to enforce specific paths byexplicitly specifying the set of switches through which a flowmust go through or a specific set of AS domains that shouldbe traversed.The policy language has wild cards in its syntax enablingspecification of policies that can apply to sets or groups of

entities and services. When a Policy Expression is satisfied,then the associated action is performed which could be simpleas allow or deny the request. Hence using these policy terms,one is able to specify different sets of Policy Expressions todeal with a range of scenarios in both intra-domain and inter-domain communication in a distributed SDN. An action canalso have some attributes. For instance, destination exit switch(DSTEXT ) attribute associated with an action indicates theexit switch through which a flow should pass, once a policyis satisfied. Hence we are able to specify conditional policies,constraint and state dependent policies as well as obligationpolicies.In particular, the language can be used to specify policies thattake into account the context associated with the resourcesand the devices. For instance, it can be used to specifyprotection policies that take into account the attributes of thedevices through which the flow can occur or be displayed.For certain confidential information, the paths through whichthe packets are transferred and the devices/switches which canprocess them must have certain security attributes. Such policyexpressions can be specified using simple Boolean algebra onthe security labels. The policy engine evaluates the Booleanexpression to determine whether the condition on the securitylabels is satisfied or not.The language can also be used tospecify release policies associated with the end points throughwhich the traffic can be released, requiring certain securityattributes. These types of policies, namely protection andrelease policies, are common in the context of content basedsecurity, and are significant when it comes to the provision ofSDN services.Using the policy terms mentioned above, a simplified PolicyExpression template could be as follows:PE

ASki = < FlowID, SourceAS,DestAS, SourceHostIP,

DestHostIP, SourceMAC,DestMAC,User, F lowCons,

DomCons, Services, Sec− Profile, Path >:< Actions > (1)where i is the Policy Expression number and k is the ASID. This is a generic Policy Expression for both intra andinter-domain. In the following sections, we will explain theuse cases for both intra and inter-domain. For simplicity, wehave omitted the AS ID notation in intra-domain expressions.Also, in intra-domain, path indicates a set of switches withinthe domain while in inter-domain path refers to a set of ASdomains.In general, we have a number of Policy Expressions storedin the SDN Policy Repository. Such a template enables us tospecify a range of policies for different users (and hosts), fromdifferent locations, accessing different services using differentdevices following different paths. Later we will illustrate theuse of such policy expressions in both intra and inter domainenvironments when we discuss the results of our systemimplementation in Section V-A

C. Security Architecture Walk-through

Let us now give a brief walk-through of the operation ofthe proposed security architecture described above. We willuse the inter-domain scenario given in Figure 3 to illustratethe various steps involved. The Tables beside each of the ASdomains in Figure 3 represent the Topology Repository. Table

8

TABLE I: Stored Policy TermsAS ID Policy Expression

AS1 PEAS11 =< ∗, (10.0.0.0/24, EDU, SL2), (192.168.52.0/24), 10.0.0.2,

∗, ∗, ∗, ∗, ∗, SL2+ =, (80, 443), conf, ∗ >:< (1SW2, Allow) >

AS2 PEAS24 =< ∗, (10.0.0.0/24, EDU, SL2), (192.168.52.0/24), 10.0.0.2,

∗, ∗, ∗, ∗, ∗, SL2+ =, (80, 443), conf, (AS1) >:< Allow >

AS3 PEAS32 =< ∗, (10.0.0.0/25, EDU, SL2), (192.168.52.0/24),

∗, ∗, ∗, ∗, ∗, ∗, SL2+ =, (80, 443), conf, (AS1, AS2) >:< Allow >

AS4PEAS42 =< ∗, (10.0.0.0/25, EDU, SL2), (192.168.52.0/24),10.0.0.2, 192.168.52.72, ∗, ∗, ∗, ∗, ∗, (80, 443), conf,(AS1, AS2, AS3) >:< Allow >

I shows the policies in each of these AS domains stored inthe Policy Repositories of PbSA.

The initial packet header from the source host is sent bythe switch (to which this host is connected) to the SDNController in that AS domain. The PbSA application in theController extracts the relevant parameters from the incomingpackets and uses the Policy Repository and the Policy Managerto determine whether the relevant Policy Expressions aresatisfied. If the Policy Expressions are valid for the incomingpackets, then PbSA will enforce the specified actions as flowrules in the appropriate data plane devices such as switches totransfer the packets.In this scenario, the host machine X (with IP address 10.0.0.2)wishes to communicate with the host machine Y (with IP ad-dress 192.168.52.72). As X and Y reside in two different ASdomains, communication between them occurs via transit ASdomains. At first, packets from X go to the SDN Controllerof AS1. As the Policy Expression PEAS11 in AS1 matcheswith this particular network traffic, HTTP & HTTPS trafficoriginating from 10.0.0.2 are allowed to go to Y in the subnet192.168.52.0/24. However let us assume that there is a flowconstraint which specifies communications between X and Ymust occur only through domains which have security levelsgreater than or equal to SL2. This will result in the trafficrouted through the AS2 domain via the OpenFlow Switch1SW2 (Connected to AS1). A similar process occurs in AS2and the traffic is sent to Y in AS4.

We also have constraints such as a flow or flows betweentwo entities in a domain A and B should only go throughpaths whose security labels are greater than or equal to aspecific security label L. Satisfying this rule requires thePbSA to determine the labels of the various paths between thedevices, and then check whether the flow constraint is satisfied.The policy rules can have Boolean expressions such as thesecurity label of the switch Si should be less than or equalto the security label of the neighbouring switch Si+1. If theneighbouring switch meets this constraint, then the flow willbe directed through it; if not, than another neighbouring switchwill be selected. In general, flow and domain constraints canrequire some form of evaluation which is more than just policymatching.

In general, our architecture can be used in either reactive orin a proactive mode. In the reactive mode, the first packet ofthe flow received by switch will trigger the insertion of flowentries in switches in the network. This approach presents themost efficient use of existing flow table memory, but everynew flow incurs an additional setup time. In the proactiveapproach, the flow tables in the switches are pre-configuredby the Controller based on policies specified by the network

administrators. This approach has no additional flow setuptime because the forward rule is defined.

IV. SECURITY ARCHITECTURE IMPLEMENTATION

We have implemented and validated our Policy based SecurityArchitecture for SDN using Open Network Operating System(ONOS). We have modified the SDN-IP application in ONOS,and added extra modules for policy control for SDN inter-domain communications. We have used the Oracle VM Boxenvironment to create ONOS and Mininet VM pairs, whichact as inter-domains. Each inter-domain SDN Controller runsour PbSA over ONOS.

A. Implementation Components

Topology Repository: Topology is stored in the TopologyRepository and is implemented using the SDN-IP application.The repository contains topological information of the neigh-boring domains as well as the Security Labels associated withthe SDN Controllers in these domains. As mentioned earlier, inthe current version of the security architecture, these SecurityLabels are statically assigned based on the manufacturer ofthe SDN Controller and the reputation of the manufacturer.However in the next version of the security architecture design,we intend to update these Security Labels using a dynamictrust model for SDN.Policy Repository: Policies are represented as Policy Ex-pressions in our template based language specifications andstored in the JSON Policy Repository. We have used the Javaparser to update the JSON Policy Repository. Collections ofPolicy Expressions specify the various policies as mentionedin Section III and they control the interactions and packettransfers both within and between AS domains.

Policy Manager: Coordinates the AS policy routing systemand is responsible for the following major tasks:

• Receiving information from the edge OpenFlow switches via the PolicyEnforcer module.

• Aggregating the necessary information to match the Policy Expressions.• Separating the handles from the payloads in the packets.• After processing by the Policy Evaluation Engine module, sending the

information to the Packet Handle Creator to create the new handle withthe previous payloads.

• Creating the new packet using the newly created handle and payload.• Sending instructions to the Policy Enforcer to install the respective

flows.

Policy Evaluation Engine: This module is used for determin-ing the policies related to the flow. Relevant parameters areextracted from south-bound traffic with the help of Policy En-forcer module. The Policy Manager sends these parameters tothe Policy Evaluation Engine. The Engine module checks theseparameters against the Policy Expressions stored in the PolicyRepository. When an appropriate match for the entry is found,it enforces the corresponding Policy Expression(s) using thePolicy Enforcer module via the Policy Manager. Creation ofthe Topology Repository and sending information to/from theTopology Repository to the Policy Manager are also performedby the Policy Evaluation Engine. The implementation sectionbelow gives the schema of the JSON based Policy Repository.Policy Enforcer: It enforces the flow rules as specified by thePolicy Manager for the specific incoming south-bound traffic.

9

JSON Parser Repositories 

SDNIP

Administrator INPUT

BgpSessionManager

IntentService ArpService

Configuration

Evaluation Engine Enforcer

Handle Creator

Policy ManagerPbSA

SDN‐IP

Fig. 4: PbSA Software Modules

B. Network Setup

We have used ONOS as the SDN Controller in each ASdomain. The machine we are using for simulation purposes is aCore i7 - 4790 with 3.60 GHz CPU and 32 GB of RAM. EachSDN controlled AS is constructed by a pair of ONOS basedVM and mininet VM. Our network configuration is shownin Figure 3. We have added bridged mode Ethernet adaptersto the VM pairs within the VM Box to create communicationchannels. BGP route advertisement is done by QUAGGA [14],which is a BGP software router. This experimental networksetup follows the same approach as the one taken by theONOS technical team for their demonstration of SDN-IPapplication [15].1) Application Modules: Figure 4 shows the different mod-ules used in the construction of PbSA. We have combined ourapplication with SDN-IP (ONOS built-in application for BGPcommunications). Here we use SDN-IP BGP route selectionand update for maintaining session modules. PbSA uses thisinformation to create and update the topology in the TopologyRepository.SDN-IP core consists of a software router which captures thebest BGP routes from BgpSessionManager. BgpSessionMan-ager maintains sessions, updates and selects the best BGProutes. ArpService is used for MAC address resolution. ONOScore configurations are captured by the Configuration module.IntentService submits the requests in the form of intents tothe ONOS controller. Using these modules, SDN-IP listens toBGP requests and chooses the best route. SDN-IP has certainlimitations which are mentioned in the related work section(Section VII).As mentioned above, JSON Repository is used for storing thePolicy Expressions. To update, modify and read the storedJSON policies, we have used JAVA JSON parser module(a built-in module in Java). The Policy Manager is a corecomponent of the PbSA. It maintains all the modules in thePbSA. The Policy Enforcer acts as a bridging module betweenthe SDN-IP and our application. It extracts the topological in-formation from SDN-IP and updates the topological database.The Policy Manager is aware of the topological informationvia this database. Before setting up any new flows betweenthe available domains, the Policy Manager checks the JSONpolicy repository using Policy Evaluation Engine. The PolicyEvaluation Engine parses the particular Policy Expressions andprovides them to the Policy Manager. It compares the policiesand sends the particular action to the Policy Enforcer, whichforwards it to the intent services. These intent requests arecaptured by the ONOS Controller and appropriate action is

D172.56.16.08

60‐FF‐2F‐D2‐60‐CCFTP SERVER

A172.56.16.02

48‐2C‐6A‐1E‐59‐2FAlice

B172.56.16.04

48‐2C‐6A‐1E‐60‐FF

C172.56.16.06

56‐2D‐7F‐2E‐50‐FFHTTP SERVER

SW1 IP

SW2 IP

SW3 IP

SW4 IPSW5 IP

Fig. 5: Intra-Domain Example Scenario

taken. The Handle Creator is used to create the packets.

Listing 1: AS2 ONOS Policy Database Sample[{

” i d ” : ”21” ,” f l o w i d ” : ”∗” ,” s r c a s i d ” : ”∗” ,” s r c a s s u b ” : ” 1 0 . 0 . 0 . 0 / 2 5 ” ,” s r c a s t y p e ” : ”EDU” ,” s r c a s t r u l a b e l ” : ”SL2 ” ,” d s t a s i d ” : ”∗” ,” d s t a s s u b ” : ” 1 9 2 . 1 6 8 . 5 2 . 0 / 2 4 ” ,” d s t a s t y p e ” : ”EDU” ,” d s t a s t r u l a b e l ” : ”SL4 ” ,” s r c i p ” : ” 1 0 . 0 . 0 . 2 ” ,” d s t i p ” : ” 1 9 2 . 1 6 8 . 5 2 . 7 2 ” ,” s rcmac ” : ” 0 0 : 0 0 : 0 0 : 0 0 : 0 0 : 0 1 ” ,” ds tmac ” : ” 0 0 : 0 0 : 0 0 : 0 0 : 0 1 : 0 1 ” ,” u s e r ” : ” ” ,” f l o w c o n s ” : ”∗” ,” domcons ” : ”SL2 +=” ,” s e r v i c e s ” : ”∗” ,” s e c p r o f ” : ” con f ” ,” seq ” : ”AS1 , AS2 ” ,” a c t i o n ” : ” a l l o w ”}]

2) Policy Example in the Database: Listing 1 shows a singlePolicy Expression from AS2 (VM pair 2) Policy Repository(JSON file). This single Policy Expression (ID 21) statesthat, for any packets originating from the subnet 10.0.0.0,host 10.0.0.2 with the MAC address ending in : 01, whosedestination subnet is 192.168.52.0 must be routed through thedomains whose Security Label is greater than or equal to SL2.

V. RESULTS

A. Implementation Examples

This section presents specific implementation intra and interdomain scenarios using our architecture.1) Intra-domain Scenario: Path Based Policy for Services:This scenario considers specific paths for specific servicesrequested from the hosts. Consider, for example, HTTP andFTP servers running on two machines with IP addresses172.56.16.06 and 172.56.16.08 respectively. We have usedOracle VM box for this purpose. We have created an Open-Flow switch matrix with five switches as shown in Figure5. Flows from two hosts with IP addresses 172.56.16.02 and172.56.16.04 are guided through this switch matrix dependingon the services they are intending to access. The PolicyExpressions associated with these requests are as follows:PE3=< ∗, ∗, ∗, 172.56.16.04, 172.56.16.06, 48 : 2C : 6A : 1E : 60 :FF, ∗, ∗, ∗, ∗, 80, {Conf, Intg}, (SW1;SW5;SW4) >:< Allow > (2)PE4=< ∗, ∗, ∗, 172.56.16.02, 172.56.16.08, 48 : 2C : 6A : 1E : 59 :

10

Fig. 6: Flowdump of the Switch SW5

2F, ∗, ∗, ∗, ∗, (20; 21; 22; 23), Conf, (SW1;SW3;SW4) >:< Allow >(3)

Equation 2 (PE3) states that flows from host with IP172.56.16.04 and MAC 48 : 2C : 6A : 1E : 60 : FFaccessing the HTTP server on IP 172.56.16.06, should beprotected for confidentiality and integrity, and forwarded viaOpenFlow switch SW1->SW5->SW4.According to the Equation 3 (PE4), flows to the FTP serverrunning on IP 172.56.16.08 from host with the IP address172.56.16.02 and MAC address 48 : 2C : 6A : 1E : 59 : 2Fare to be forwarded via a OpenFlow switch path SW1->SW3->SW4. This Policy Expression also instructs the Controller toprotect the flow for confidentiality. Also, all HTTP packets areto be routed through SW1->SW5->SW4. Figure 6 showsthe flow dump of the SW5 switch. The first flow is usedfor network discovery (ARP). The second flow is for theresponse from the Web Server running in IP 172.56.16.06,which tells the Controller that any IP packet from the WebServer should be handled normally. The third flow is dedicatedfor HTTP packets and indicates that all HTTP traffic shouldtake an output path Port 2 (in the Open vSwitch). For the FTPtraffic towards the FTP server, similar type of flows are beinginstalled in Switches 3, 1 and 4. But this time FTP traffic isused in the flow rules instead of HTTP.2) Inter-domain Scenario: Policy specific to User, MAC,IP Addresses: In this scenario, specific users, devices and IPaddresses from one AS domain are allowed access to otherAS domains running our application over ONOS Controller.For instance, this scenario can correspond to users fromone organization visiting another organization and trying tocommunicate with their home organization corresponding toa roaming BYOD. This scenario is shown in Figure 3. Here,user Alice is an employee of an organization belonging todomain AS2, which is a restricted domain for only authorizedusers. We assume that her device’s MAC address and the userID are stored in a database, and that our policy applicationuses this policy database for checking user Alice’s validity.Let us now consider Alice roaming with her mobile deviceusing company X’s (AS1) Internet service. Alice requests toget access to HTTP employee server running in AS2. Sincethere is no policy permitting this flow to the restricted domain,it will be dropped at the edge router of AS2 domain.PEAS28 =< ∗, ∗, AS2, ∗, (172.16.10.66), (79 : c8 : 82 : b2 : 7b :1a), ∗, Alice, ∗, ∗, (80, 443), ∗, ∗ >:< allow > (4)Let us now introduce the Policy Expression 8 (PEAS28 ). ThisPolicy Expression tells the SDN Controller that any trafficspecific to Alice’s device (MAC : 79 : c8 : 82 : b2 : 7b : 1a)for HTTP traffic must be allowed in AS2 domain. In thiscase, when the packet comes to the edge OpenFlow router,it does not have any flow rule to guide the traffic; so itforwards the traffic to the SDN Controller. In PbSA, the Policy

Evaluation Engine checks the packet information against thepolicies in the Policy Repository. It finds a match with thePolicy Expression PEAS28 . Therefore, the Policy Managerasks the Policy Enforcer to install the flow rules in the specificswitches. Finally, Alice is able to communicate with theemployee web portal running in the AS2 domain.

3) Inter-domain Scenario: Policy based Routing for Un-known Communications: Restricted AS domains often getrequests from unknown domains to pass their traffic. Passingsuch traffic can often lead to threats in the transit domains.PbSA has a default deny policy for dropping all the flowswithout a matching permit policy. However, there is an optionin the PbSA to permit such unknown flows through switcheswith low security labels. This will enable separation of secureflows from unknown flows. Hence even if unknown trafficleads to attacks in the network, this will not impact thesensitive flows in the switches with high security labels.Such policies help to create secure channels in the networkdynamically based on policies. Consider the following:PEAS210 =< ∗, (10.0.0.0/25, EDU, SL1), ∗, ∗,∗, ∗, ∗, ∗, ∗, SL1, (80, 443), ∗, ∗ >:< allow > (5)PEAS314 =< ∗, (10.0.0.0/25, EDU, SL1), ∗, ∗,∗, ∗, ∗, ∗, ∗, SL1, (80, 443), ∗, (AS1, AS2) >:< allow > (6)Here AS2 and AS3 are transit domains. Within a domain,recall that the edge routers and switches are tagged withlowest security labels. In this scenario, SL1 represents thelowest security label. A Web Server is running in AS4.Some host in AS1 is trying to communicate with the WebServer. According to the topology repository, there is no directpassage to AS4 from AS1. AS1s packets need to go viatransit domains AS2 and AS3. Assume that the domains AS2and AS3 are restricted and commercial domains, and henceaccess through these domains is limited and controlled by thePbSA policies. Policy Expressions PEAS210 and PE

AS314 are

stored in the Policy Repositories of AS2 and AS3 domains.The first Policy Expression PEAS210 states that, any packetoriginating from domain AS1 (10.0.0.0/25) and destined toWeb Server in AS4 without a matching policy should be givenpassage through switches with lowest security label (SL1) inAS2. Hence in this domain, specific transit switches bearinga security label SL1 are used to route the traffic for thisparticular host. Similar situation occurs with the AS3 transitdomain. Hence the specific host can access the Web Server inAS4.4) Flooding Attacks from Malicious Hosts: In this case, anadversary connected to one of the OpenFlow switches in ournetwork shown in Figure 5 carries out a flooding attack. As-sume that the prime motivation of the adversary is to exhaustthe Controller by overflowing it with Packet in requests. Inour network testbed, this is achieved by replacing Alice’s VMwith a Kali Linux VM and launching for instance a SYNflooding attack. Without PbSA, all the SYN messages resultedin the establishment of routes. When the PbSA was activated,it detects requests above the specified threshold from a switch,and installs a block rule in the switch to drop the traffic fromthe malicious host carrying out the attack. In our architecture,we have used two techniques to counteract such attacks. The

11

first one involves a heuristic rule based mechanism usingthresholds. This enables us to determine a threshold for eachhost connected to the switch and a threshold for each switchconnected to the Controller. These thresholds are dependenton the number of hosts connected to a switch and the numberof switches connected to the Controller respectively. Here isa simple example that illustrates these thresholds. Let ’CC’be the total capacity of Controller that is connected to ’X’switches. Let ’CS’ be the capacity of the switch and ’Y’ is thenumber of hosts connected to each switch. Then threshold ’TS’for each switch is determined by TSw = CC/X and thresholdfor each end host is determined by Thost = TS/Y. For example,consider the case of a single Controller that is connected to10 switches which are further connected to 10 end hosts.If the Controller is capable of processing 1000 requests persecond then each switch is restricted to sending 1000/10 =100 requests per second and each end host is restricted tosending 100/10=10 requests per second. We dynamically varythe weights associated with the calculation of threshold basedon history of packets received; the threshold policy also variesdepending on the number of running instances of the Con-troller. For instance, the Controller services can be hosted onmultiple machines during peak hours. As a second technique,we have experimented with machine learning based techniqueto detect anomalies. In this case, the history of packets receivedis used with a machine learning algorithm to detect whether anattack is happening. We have used Bayesian algorithm for thispurpose. We are currently looking into extending this aproachusing other machine learning algorithms [16].Furthermore, note that with our PbSA architecture, as eachflow request is validated by the Controllers in the domainsbefore permitting the flows, attacks such as Crossfire [10]and Coremelt [11] are prevented at the source, which is moreefficient. Even if the attacker generates untrusted traffic withunknown applications to create these attacks, policies such asthose in Equations 5 and 6 help to route all the unknown anduntrusted domain traffic via the switches with lower securitylabels. PbSA uses policies to compartmentalize the forwardinglayer based on the security labels of the switches, therebyprotecting the trusted traffic from unknown traffic.

B. Performance Results

We have conducted extensive analysis of the proposed archi-tecture. In this paper, we only present a subset of these resultsdue to space restrictions. In particular, we will present theresults related to the secure establishment of routes in intra do-main and inter domain with the PbSA that we have discussedabove. The results vary for different types of hardware andthe configuration of the devices. Hence, first we present thebaseline result, which shows the performance of the Controllerwithout the PbSA. Then we enable the PbSA on the Controllerwith the same hardware and baseline settings to obtain theperformance results specific to PbSA.We first present results for latency, flow establishment rate,and memory overhead in intra-domain scenarios. Then wepresent the results related to the inter-domain scenarios. Thelatency is the time taken by the Controller to process thePacket in message and respond with the Packet Out message.

0

1

2

3

4

5

6

WithoutPbSA

With PbSA

WithoutPbSA

WithPbSA

WithoutPbSA

WithPbSA

Without PbSA

WithPbSA

Without PbSA

With PbSA

100SW 200SW 300SW 400SW 500SW

Latency(in m

s)

Fig. 7: Latency at the Controller

0

1000

2000

3000

4000

5000

6000

1000 2000 3000 4000 5000

flow_m

od/s

Flow requests/ sec

Without PbSA

100PE

200PE

300PE

400PE

500PE

Fig. 8: Flow establishment rate of the Controller

It is measured by sending only one Packet in message to theController without enabling PbSA to obtain the baseline resultsfor the Controller. Then we enable PbSA with the similarsetup to obtain the latency results. The process is repeated 10times for each scenario and then we present the average resultfor each case. We repeat this process with varying numberof switches. The results show that there is an increase inthe latency with the PbSA compared to the baseline (withoutthe PbSA). Also note that the latency increases linearly withthe increase in the number of switches with the PbSA. Flowestablishment rate is measured by analysing the number offlow request messages that can be processed by the Controllerper unit time with 500 switches in the network. We havetested the flow establishment rate by increasing the number offlow request messages and observing the number of flow modmessages generated by the Controller in the baseline modeand then compared with PbSA, while varying the number ofpolicy expresssions (PEs). As shown in Figure 8, there was100 percent flow establishment in the baseline mode and withPbSA, when the flow requests are generated at the rate of4000 requests per second. However, we see a variation in theflow establishment when the flow requests are generated at5000 requests per second. In this case, the baseline mode wassuccessful in generating 4980 flow mod messages and therewas a variation in the number of flow mod messages withPbSA. The results varied with the number of PEs stored in thePbSA. We have observed flow establishment rate of 4934 with100 PEs; 4872 with 200 s PEs; 4806 with 300 PEs, 4724 with400 PEs and 4611 with 500 PEs. Hence the administratorshave to opt for either upgrading the hardware or runningmultiple instances of Controller if flow requests are to begenerated at the rate of 5000 requests/second. Although thereis an overhead with the PbSA, note that the baseline modewill result in establishing the routes for all the flow requests(including malicious requests). However, with PbSA onlysecure routes will be established corresponding to security

12

620

640

660

680

700

720

740

760

780

800

820

Without PbSA

With PbSA

Without PbSA

With PbSA

WithoutPbSA

WithPbSA

Without PbSA

WithPbSA

WithoutPbSA

WithPbSA

100SW 200SW 300SW 400SW 500SW

Mem

ory Usage(in

 MB)

Fig. 9: Memory Overhead with PbSA

0

5

10

15

20

25

Without PbSA

WithPbSA

WithoutPbSA

With PbSA

Without PbSA

WithPbSA

Without PbSA

WithPbSA

Without PbSA

WithPbSA

C1+C2 C1+C2+C3 C1+C2+C3+C4 C1+C2+C3+C4+C5 C1+C2+C3+C4+C5+C6

Latency (in

 ms)

Fig. 10: Latency at the Controller for multi AS domains

policy specifications. Hence for instance, during an outbreakof a worm attack, when the compromised hosts randomlyscan for vulnerable machines for spreading the attack, thebaseline mode will result in successful establishment of theroutes for all the malicious flow requests. With PbSA all therandomly generated malicious flow requests which do not havecorresponding PEs permitting the flows will be dropped. Thisis a significant advantage for overcoming attacks in networks,especially as the malicious flow requests are dropped at thesource end.We have also analysed memory overhead with PbSA with 500

policy expresseions (PEs) with varying number of switches.As shown in Figure 9, there is an increase in the memoryusage with PbSA. Also the memory usage increases with theincrease in the number of PEs stored in the PbSA.

With multiple AS domains, we have considered the caseof sequentially connected Controllers between the source andthe destination hosts. When the source host sends an intentmessage to its Controller for communication with a destinationhost in another AS domain, the intent message is initiallyprocessed by the Controller in the source AS domain andthen sequentially forwarded to the Controller of a directlyconnected AS domain. The process is repeated until the intentmessage is forwarded and processed by the Controller towhich the destination host is connected. Similar to the intradomain situation, we have analysed the performance first in thebaseline mode without the PbSA and then with PbSA enabledon all the Controllers. In the case of a multi AS domainscenario, there are three hops for each AS domain, namelyingress, transit and egress hops. The ingress hop receivesa packet flow from the source end host or from an egressswitch of the previous AS domain. There is one transit switch

0

5

10

15

20

25

30

Without PbSA

With PbSA

WithoutPbSA

WithPbSA

Without PbSA

WithPbSA

Without PbSA

WithPbSA

Without PbSA

WithPbSA

C1+C2 C1+C2+C3 C1+C2+C3+C4 C1+C2+C3+C4+C5 C1+C2+C3+C4+C5+C6

end to end

 flow

 estab

lishm

ent (in m

s)

Fig. 11: End-to-End flow establishment time

0.108

0.267

0.529

0.201

0.403 0.400

0.242

0.399

0.0010.000

0.100

0.200

0.300

0.400

0.500

0.600

250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000 3250 3500 3750 4000 4250 4500 4750 5000

Flow

 Installatio

n Ra

te (in ms)

Packet_in (per ms)

Flow Installation Rate vs Packet_in Request 

With out PbSA With PbSA(Using Drop Rule)

With PbSA(Using Threshold Only)

Fig. 12: Flow installation rate with varying Packet in Request

connecting the ingress switch to the egress switch. The egressswitch forwards the flow request to the destination host or toan ingress switch of the next AS domain. We have analysedlatency and end-to-end flow establishment in such a scenario.Figure 10 shows the latency results for a multidomain scenario.In such a scenario, there is latency in each Controller. Thelatency results shown are obtained by adding the latency ateach Controller without the PbSA. Then we enabled PbSAon each Controller and obtained the results. Similar to theintra-domain scenario, there is an increase in the latency withthe PbSA. Also, the latency increases with the increase inthe number of AS domains in the baseline mode and withPbSA. However, note that with PbSA, the intent messagesare forwarded to another Controller only if it is permittedaccording to the policies specified in the PbSA. All themalicious intent messages which do not have correspondingPEs to permit the intent messages are dropped at the firstController itself. This leads to saving of resources in theControllers in other AS domains. Figure 11 shows the end-to-end flow establishment time with and without the PbSA.Although there is some additional delay with the PbSA, notethat such a delay is only applicable to valid flows. There is noneed for flow establishment if there is no corresponding PEpermitting the flow.Now we consider the attack scenario (4) outlined in Section

V-A4.First we used a single attacking host machine connectedto an OpenFlow switch flooding the Controller with DDoSPacket in attack requests and measured the flow installationrate. Figure 12 presents 3 curves representing 3 differentexperimental scenarios: 1) Without PbSA active; 2) WithPbSA active using a threshold policy and 3) With PbSA activeusing a drop rule policy.When PbSA is not active, as expected, the number of flow

13

installation rate increases with the increase in the number ofPacket in requests (shown by the blue curve in the Figure 12).For 250 Packet in request/ms, the flow installation rate isaround 0.025295 flows/ms and at 5000 Packet in requests/msthe flow installation rate is around 0.52904. This incursconsumption of SDN Controller resources.With PbSA active, with the threshold policy applied at 4000requests/ms, the Controller limits the packet requests to 4000.The PbSA detects the flooding and above the threshold,it maintains a constant flow installation rate as per policyexpression (to 4000 Packet in requests/ms). This is shown bythe continuing green line in the Figure 12. The Controller isnot overloaded by malicious Packet in requests above 4000,and the flow installation rate remains constant around 0.40001-0.402631 flows/ms. In the third scenario, we enforced a droprule policy above the threshold. In this case, the PbSA detectsthe flooding and it installs a drop rule (in the OpenFlowswitch) specific to the malicious host. Hence all the subsequentpacket requests from this malicious host are dropped. Howeverthere is a delay for this to occur and this corresponds tothe time taken to install the drop rule at the switch and forthis rule to be enforced. This is shown by the orange linein the Figure 12. For 4000-5000 Packet in requests/ms, theflow installation rate drops to 0.0012 flows/ms and it remainsconstant around 0.0012 - 0.0016 flows/ms.

VI. SECURITY ANALYSIS

First in this section we examine how the security requirementsthat we originally set out in Section II are achieved by theproposed security architecture.(R1) The core aspect of our security architecture PbSAinvolves security policy based authorization of flows indistributed SDN environment. These security policies usedifferent flow parameters to authorize flows in intra and interdomain communications in SDN. The policy administratorscan define fine grained security policies based on differentattributes such as users/devices/AS identities and servicesrunning on the end hosts, and context attributes such as timeand location of devices as well as path based specificationsbsaed on security labels of switches.(R2) PbSA is used for security management of devices inAS domains and enhancing the security of SDN operations.For instance, PbSA enforces a default deny policy and dropsall the flow requests that do not satisfy policies permittingthe flow. Threshold policies have been used to detect attackssuch as worms and drop malicious flow requests when theinfected machines generate flooding attacks to destinations.(R3) PbSA makes use of the network domain wide informationavailable at the SDN Controller and the security policiesstored in the policy repository for achieving end to endsecurity within a domain. PbSA is based on modular designto enable incremental implementation of the required securitymodules, making it easily extendable with additional securityfeatures. For instance, we have implemented modules forpath selection based on the security labels of the switches,and attack detection using signature and thresholds, and arecurrently developing key management modules for generating

and distributing keys for securing the communication betweennetwork devices.(R4) Our PbSA architecture can be used to secure end to endcommunication across multiple AS domains. A novel featureof the proposed security architecture involves the use of thedynamic visibility of the network connectivity to specifyflow and path based security policies to achieve securecommunications and efficient provision of services acrossmultiple SDN domains. The specific flow requirements arevalidated against the policies of all the AS domains (source,transit and destination) before the routes are established toenable authorized communication between the end hosts.For instance, suppose due to a DDoS attack, traffic froman end host is not able to get through the network. PbSAis able to detect the DoS attack efficiently and establishan alternative path for the traffic from the end host to therequired destination. Moreover, we have shown that PbSAcan enforce policies such as certain communications shouldgo through a path with certain security attributes. Such pathbased policies are critical when securing data from sensitiveapplications but are also useful for applications with differentquality of service requirements.Let us now consider how the proposed security architecturedeals with the attacks mentioned earlier in Section II.(A1) Our security architecture enforces default deny commu-nication between the hosts. If the end hosts are located withinthe same domain, then the communication is permitted if itmeets the policy requirements of the AS domain. In this case,the unauthorised communication requests are dropped at theSDN Controller within the domain. If the end hosts are locatedin different domains, then the communication is permittedif it meets the policy requirements of all the AS domainsinvolved namely the source AS, transit AS and destination ASdomains. For instance, the scan messages that are destined torandom destination addresses during the spread of worms willnot result in establishment of routes to the destination host.In this case the unauthorised communication requests will bedropped at the AS domains where authroization policies arenot satisfied.(A2) The policy repository in the PbSA has information aboutthe services hosted on the servers in the domain. The securitypolicy administrators can specify the usage policies based ondifferent parameters such as the users/devices/AS identitiesand the services running on the end hosts, time of flows andlocation of devices. When an end host attempts to accessan unauthorised service, the switch that is connected to themalicious end host will generate a packet˙in message to theSDN Controller. Since the flow request will not be authorisedby the PbSA, the malicious flow request will be denied.(A3) As security policy administrators can specify policiesbased on different parameters such as location of the devicesand AS domains, attacks originating from hosts residing incertain locations are prevented by PbSA.(A4) If an end host is directly targeting any of the switches,then we have shown that PbSA is able to detect such maliciousattacks. For example, a malicious end host attempting to flooda particular switch in the network will be prevented by policiesin PbSA detecting such attacks. Also, the attackers do not have

14

any control on the path taken by the malicious traffic since theroutes are established dynamically for each flow request basedon policy specifications. If there is a specific need for the endhost traffic to take a specific path, then this has to be conveyedin the intent message and satisfy the policies in the PbSA.(A5) In our current design, we have used domain constraintsto specify threshold policies over multiple domains. This hasbeen used to enforce restrictions on new flows. Consideran attack that chains malicious flows coming from differentlocations. Recall that each Controller has a global view ofall the established routes and active flows within its domain.Also, each flow request is validated by the Controllers inall the relevant domains before the flow is permitted. If thetraffic at any link is above the threshold specified in thepolicies, then the Controller of that domain can select analternative path to the destination. If there is no alternativepath to the destination, then the Controller will not accept thisnew flow request. Furthermore, if any of the end hosts withina domain is generating a flow request above the threshold,then the switch will drop the flows from the maliciuos hostand the Controller will dynamically configure rule in thatswitch. Hence, the malicious flow request will not result in theestablishment of the new routes, and the attack that attempts tochain malicious flows from different locations are prevented.However the use of such domain constraints across multipledomains is restrictive; in our future implementation, we willbe relaxing this restriction using a hierarchy of Controllersand using policy inheritance to better coordinate the policiesin Controllers in different domains.

VII. RELATED WORK

Inter-domain routing forms the main backbone in today’snetwork infrastructures such as the Internet. There are some50,000 autonomous domains. A breakthrough in recent timesin the networking world is the paradigm of SDN and the pro-grammability, flexibility, and manageability that SDNs offer.This paper has proposed a security policy based control forboth intra and inter-domain communications in a distributedSDN environment, which offers significant advantages both interms of specification of authorized flows as well as detectionand prevention of attacks in networks. In this section, we willdiscuss different related works that are relevant to our research.Kreutz et al. in [2] presented the seven threat vector inSDN space. Our threat model has taken this as the basisand enhanced it in the context of intra and inter-domaincommunications in SDNs. In particular, we have shown howa policy based security architecture can be used to counteractflow related specific threats discussed in [2]. Furthermore, [2]advocates the need to consider security and dependability insuch networks. Currently we are in the process of developinga trust model for SDN, which addresses trust relationshipsbetween devices, switches, controllers and their applications;the proposed policy based security architecture together withthe trust model will help to achieve security and dependabilityproperties in SDNs.Clark in RFC1102 described policy-based routing for legacyinter-domain networks [13]. He proposed a language based

approach to policy specification to control routing of packetswithin autonomous system domains. This work formed thebasis of our security policy language in our security archi-tecture. Tsudik et al. in [17] introduced security measuresto policy-based inter-domain routing. Their work was mainlyconcerned with key management using symmetric keys tosecure communications in AS domains. We have used Clark’spolicy language syntax and extended it to develop fine grainedsecurity policy specifications in SDN for both intra andinter domain communications. We have developed a securityarchitecture which enables the enforcement of these securitypolicies in the Open Flow switches on the traffic originatingfrom the end hosts and users connected to these switches.As mentioned before, we will be extending this securitypolicy based architecture with key management to securecommuncations using a similar approach to that suggested in[17]. As indicated earlier, this extension on key managementis traditional and well-known. The novelty of our proposedsecurity architecture is the use of policy based specificationsenabling fine grained path and flow based policies for multipledomains as well as the ability to capture dynamic aspectsof the network and flow context within these policy basedspecifications enabling the detection of attacks. This extendsour previous work on policy based approach for intra-domaincommunications [18] and develops a comprehensive securityarchitecture for end to end services across multiple domains.

Peter et al. in [19] discussed about BGP issues in SDN andproposed an application-based routing architecture. OpenDay-light [20] and ONOS [21] have followed this approach buthave limited features to establish communications betweenSDN Controllers in autonomous systems using BGP. Open-Daylight uses SDNi protocol [22] to exchange state informa-tion between domain Controllers. On the other hand, ONOSuses SDN-IP [23] to communicate with other autonomoussystem SDN Controllers. The core features of the BGP such asno explicit support for iBGP sessions, IPv6 and limitations onthe number of routes [23] are the major weaknesses of SDN-IP. Routing Control Platform (RCP) [24] for SDNs uses BGProute speakers which are different from IP forwarding plane.Our PbSA for SDNs also uses application-based approach andemphasizes the need for policy-based routing for both intra andinter-domain communications in SDNs.

Fresco is a modular security framework designed explicitly forNOX controllers [25]. It provides a development environmentwhich allows modules to be developed in terms of scripts thatcan help to detect specific attacks such as threshold basedattacks, and drop or quarantine the malicious packets. Theirfocus is mainly on the framework and development environ-ment, particularly aimed at a single domain. Our approach incontrast considers a security architecture that allows securitypolicy specifications for controlling flows in both intra andinter domain SDN environments as well as enabling authorizedaccess to services across multiple domains. Furthermore, ourarchitecture enables fine grained flow and path based policiesthat are relevant for critical applications. As part of ourarchitecture implementation, we will be exploring the use ofFresco Development Environment (FRESCO DE) to enable

15

automatic translation of our flow policies into rules in theOpenFlow enabled switches.SDN Controllers such as Maple [26], Nettle [27] and flow lan-guages like Flow-based Management Language (FML) [28],Frenetic [29] and Pyretic [30] provide ways to express high-level network policies using Haskell and Python to manageintra-domain SDN interactions. Lee et al. [31] uses SDNbased policy management functions to logically separate thenetwork services thereby decreasing the attack surface andthen combining them to create composite services. Our ap-proach is different from the one proposed in [31]; however,in principle, we can also take advantage of such an approachand partition our security policy specifications based on, forinstance, virtualised services. Sahay et al. in [32] proposeda dynamic policy enforcement framework that allows ISPs tospecify security policies to mitigate the