libVNF: building VNFs made easy - acmsocc.github.io · libVNF: building VNFs made easy Priyanka Naik, Akash Kanase, Trishal Patel, Mythili Vutukuru Dept. of Computer Science and Engineering

libVNF: building VNFs made easyPriyanka Naik, Akash Kanase, Trishal Patel, Mythili Vutukuru

Dept. of Computer Science and EngineeringIndian Institute of Technology, Bombay

SoCC’18 11th October, 2018

NFV ecosystem

Network address translator

Router

Load balancer

Firewall

VM

VNF

VM

VNF

VM

VNF

VM

VNF

Hypervisor

Orchestrator

NFV: Network Function Virtualization VNF: Virtual Network Function �2

NFV ecosystem


Router

Load balancer

Firewall

VM

VNF

VM

VNF

VM

VNF

VM

VNF

Hypervisor

● Will they give good performance ?

● Is it easy to build them?

�3NFV: Network Function Virtualization VNF: Virtual Network Function

Orchestrator

How to build VNF?

VNF code developed by VNF developer

�4

How to build VNF?


�4

38% EPC code → read/write packets

CORD Intel EPC: https://gerrit.opencord.org/ngic

How to build VNF?


VNF Processing logic

VNF Framework

�4

38% EPC code → read/write packets

CORD Intel EPC: https://gerrit.opencord.org/ngic

How to build VNF?


VNF Frameworknetbricks

OpenNF VPP

StatelessNF

�4

How to build VNF?


VNF Frameworknetbricks

OpenNF VPP

StatelessNF

�4

What is missing in these frameworks?

What is required from VNF frameworks?● Requirement 1: Support for both L3 and Transport VNF

● Requirement 2: Flexibility of network stack

● Requirement 3: Support for distributed state management

�5




�6

Support for L3 and transport VNFsLayer 3 VNFs


Layer 3 Load balancer

�7


N/W layer

Data link layer

Header manipulations



�7


N/W layer

Data link layer




Frameworks: netbricks, YANFF

�7


N/W layer

Data link layer





vEPC internet

Transport Layer VNFs

�7


N/W layer

Data link layer





vEPC internet


N/W layer

Transport Layer

Data link layer

Request processing

Connection termination

Connection initiation

�7


N/W layer

Data link layer





vEPC internet


N/W layer

Transport Layer

Data link layer

Request processing



Frameworks: mTCP, TLDK

�7Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 YANFF: https://www.openhub.net/p/yanff mTCP: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI’14 TLDK: https://wiki.fd.io/view/TLDK


N/W layer

Data link layer





vEPC internet


N/W layer

Transport Layer

Data link layer

Request processing



Frameworks: mTCP, TLDK

�7Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 YANFF: https://www.openhub.net/p/yanff mTCP: A highly scalable user-level tcp stack for multicore systems. In Proc. of NSDI’14 TLDK: https://wiki.fd.io/view/TLDK

Are these frameworks enough?

Event driven I/O

�8

Existing transport-layer frameworks are event-driven

Event driven I/O

Pros:

�8


Event driven I/O

Pros:

● Efficient for multi-core scalability

�8


Event driven I/O

Pros:


�8


Event driven I/O

Pros:


Cons:

�8


Event driven I/O

Pros:


Cons:

● Needs explicit request state storage

�8


Need to maintain request state

A B C1 2

34

�9


State at B to process A’s request

A B C1 2

34

A’s request

C’s reply

Connection identifiers

�9


DPDK and netmap layer (packet)

network stack (mTCP) (connection)

VNF processing layer (abstraction ?)


A B C1 2

34

A’s request

C’s reply


�9






A B C1 2

34

A’s request

C’s reply


�9

Existing frameworks do not provide this support

What is required from VNF frameworks?● Requirement 1: Support for both Layer 3 and Transport VNF



�10

Flexibility of network stack

vNIC

Kernel network stack

Application VNF

Kernel Stack

�11


vNIC

DPDK/netmap

Application VNF + userspace stack

vNIC


Application VNF

Kernel Stack Kernel Bypass Stack

�11


vNIC

DPDK/netmap

Application VNF + userspace stack

vNIC


Application VNF

Kernel Stack Kernel Bypass Stack

�11

Easy switch between stacks




�12

Support for distributed state management

VM

VNF 1

VM

VNF 2

VM

VNF 2

VM

VNF 3

Hypervisor

Orchestrator

�13


VNF 2

VNF 2

�13

State Synchronization


VNF 2

VNF 2

State Migration openNF, split/merge

�13



VNF 2

VNF 2


Data StoreRemote store statelessNF

�13


Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14


VNF 2

VNF 2


Data StoreRemote store statelessNF

�13


None of above support transport layer VNFs

Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14

Summary of VNF FrameworksRequirement/

Framework netbricks Flick StatelessNF Split-Merge/

OpenNFlibVNF

Layer 3 + App-layer support

no yes no no yes


no no no no yes

Distributed State

Management

no no yes yes yes

�14

Netbricks: Taking the v out of nfv. In Proc. of OSDI’16 Flick: Developing and running application-specific network services. In Proc. of USENIX ATC’16 Stateless network functions: Breaking the tight coupling of state and processing. In Proc. of NSDI’17 Split/merge: System support for elastic execution in virtual middleboxes. In Proc. of NSDI’13 Opennf: Enabling innovation in network function control. In Proc. of SIGCOMM’14

libVNF Design Goals


Support for network and transport layer VNF

Distributed State Management

VNF processing logic Handled by VNF developer

Handled by libVNF

R3

R2

R1

R: Requirement�15

libVNF overviewVNF code

libVNF APIAPI Calls

�16

libVNF overview

mTCP+ netmap/DPDK initialization

Kernel stack initialization

VNF code

libVNF APIAPI Calls

Stack initialization

�16

libVNF overview



VNF code

libVNF APIAPI Calls


�16

Per-core threads

libVNF overview



VNF code

libVNF APIAPI Calls

Per-core data structures


�16

Per-core threads

● Lock-free ● Cache optimized

libVNF API

Communication State ManagementRequest state

libVNF API

�17

libVNF API

State ManagementRequest state

libVNF API

�18

libVNF API

State ManagementRequest state

libVNF API

�18

Communication

Communication APIVNF code

libVNF API

Per-core packet pool


�19

Pre-allocated memory pools (Per-core packet pools)


libVNF APIregisterCallback(socket, fn)



Store mapping

�19



libVNF APIregisterCallback(socket, fn)



Store mapping

�19

Packet arrives on socket

fn(packet)



libVNF API



getPktBuf

�19



libVNF API



getPktBuf

Buffer to write packet

�19


VNF Design Requirements

Communication State Management

libVNF API

�20


Communication State Management

libVNF API

�20

Request state

(abstraction ?)

Need for request state





A B C1 2

34

A’s request

C’s reply


�21

(abstraction ?)

Need for request state



VNF processing layer REQUEST OBJECT


A B C1 2

34

A’s request

C’s reply


�22

Request object

Request Object API

A B C1 2

34

libVNF API

Per-core request pool

�23

Request Object API

A B C1 2

34

libVNF API


allocReqObj(A connection_id)

Allocate request object block for A’s

request

�23

A’s request

C’s reply


Request Object API

A B C1 2

34

libVNF API


allocReqObj(A connection_id)

Allocate request object block for A’s

request

�23

A’s request

C’s reply



Request Object API

A B C1 2

34

libVNF API


linkReqObj(C connection_id)

Link to the existing A request object

�23

A’s request

C’s reply




Communication Request state

libVNF API

�24


Communication Request state

libVNF API

�24

State Management

State Management APIVNF code

libVNF API

�25


libVNF API

Local data store pool

�25


libVNF API


setData( )LOCAL

Store in local datastore

�25


libVNF API


libVNF data store wrapper

Redis KV storeRemote Data store

setData( )LOCAL

Store in local datastore

�25


libVNF API


libVNF data store wrapper

Redis KV storeRemote Data store

setData( )REMOTE

Cache locally

Store in remote data store

�25

Evaluation

● Overhead of libVNF

● Scalability with cores

● Benefits of libVNF

�26

Setup

�27

A B C1 2

34

Setup

�27

VNF A VNF C

S/W switch (on kernel)

A B C1 2

34

Setup

�27

VNF A VNF C


VNF B

S/W switch (like netmap-vale)

Physical NIC

NIC Queue

A B C1 2

34

Setup

�27

VNF A VNF C


VNF B

S/W switch (like netmap-vale)

Physical NIC

NIC Queue

A B C1 2

34

VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied

Evaluation




�28

Overhead check

�29

Overhead check

�29

<5% overhead of libVNF DPDK~ netmap performance

Evaluation




�30

Core scalability

�31

Core scalability

�31scales linearly with cores

Evaluation




�32

Building VNFs

VNF Performance Overhead of libVNF

LoC Saved

IMS (IP Multimedia Subsystem)

3.4% 42%

EPC (LTE-Evolved Packet Core )

5.5% 38%

Layer 3 Load Balancer 14% 52%

�33

Building VNFs

VNF Performance Overhead of libVNF

LoC Saved

IMS (IP Multimedia Subsystem)

3.4% 42%

EPC (LTE-Evolved Packet Core )

5.5% 38%

Layer 3 Load Balancer 14% 52%

�33

Low overhead in app-layer VNF Higher overhead in L3 VNF

Summary● Library to ease building of VNFs

● Expressive to build L3 and App-layer VNF

● Supports multiple network stacks

● Low performance overhead

�34

https://github.com/networkedsystemsIITB/libVNF [email protected]

�35

Thank You

Setup

�36

VNF A VNF C


LB VNF VNF B VNF B Data store VM

S/W switch (vale on netmap)

Physical NIC

NIC Queue

A B C1 2

34

Setup

�36

VNF A VNF C


LB VNF VNF B VNF B Data store VM

S/W switch (vale on netmap)

Physical NIC

NIC Queue

A B C1 2

34

VNF A, C: 4 core, 4GB RAM VNF B: 4 GB RAM, cores varied Data Store VM: 6 core, 16GB RAM LB: 1 core, 4GB RAM

Documents

libVNF: building VNFs made easy - acmsocc.github.io · libVNF: building VNFs made easy Priyanka Naik, Akash Kanase, Trishal Patel, Mythili Vutukuru Dept. of Computer Science and Engineering