fd.io vpp overview Uwvi - Meetupfiles.meetup.com/19623913/Day2DevLab_VPP_Overview.pdf ·...

vpp overviewKeithBurns

krb@cisco.com

Agenda

• VPPOverview

• Structure

• Integrations

• Performance

VPPOverview

Introducing Vector PacketProcessor- VPP

Predictableperformance

NetworkIO

PacketProcessing:VPP

ManagementAgentNC/Y REST ...

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

Seealso:www.fd.io (TheFastDataProject)

Consistentperformance

Optimalperformance• 14+Mpps percore• Testedto480Gbps

• (bi-directional,24cores)• Multi-millionentryFIBs

• Directedgraphofnodes• Nodesareaconsistent,repeatablepattern

• Processesvectorofpackets

Introducing Vector PacketProcessor- VPP

NetworkIO

PacketProcessing:VPP

ManagementAgentNC/Y REST ...

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

Seealso:www.fd.io (TheFastDataProject)

Flexibilitywithoutsacrificingperformance

Introducing Vector PacketProcessor- VPP

• Multi-coreCPUsandleveragesDPDK

• RunsasaLinuxuser-spaceapplication

• Shipsaspartofbothembedded&serverproducts,involume

• Activedevelopmentsince2002

• RobustandcompleteAPIs• Approaching4th release• PackagedasDebian andRPMimagesfor:

• Ubuntu14.04,Ubuntu16.04,Centos7.2

16.06 16.091.0.0 17.01

Structurecodecomposition...plugins...API...whatmakesVPPtick...

VPPOverview

vppinfra• libraryofcomposable functionsfor

• vectors• memorymanagement• advanceddatastructures

• applicabletomanyproblemdomains• (seesimulated-annealing)

vnet

vlib

vppinfra

VPPOverview

vlib• VPPapplicationmanagement

• threading• main()• buffermetadatadefinition• CLI• ...more

vnet

vlib

vppinfra

VPPOverview

vnet• VPPpacketprocessingfeatures

API• AllfeatureshavefullAPIdefinedatthislayer

vnet

vlib

vppinfra

VPPOverview– vnet breakdown

• vnet codebrokenintofoldersperfeature

• mostfunctionalitycanbesummarizedbythesecategories

• eachdiscretefeatureimplementedasanodeinadirected-graph

Devices

Layer2

Layer3

Traffic Mgmt

Overlays

vnet

AF_PACKET

DPDK v16.11,HQOS

NETMAP

SSVM

QEMUvhost-user

multi-queue, indirectdescriptors

Ethernet,MPLSoverEthernet

HDLC,LLC,SNAP,PPP,SRP,LLDPVLAN- single/double tagging

MACLearningdefault limit of50kaddresses

BridgingSplit-horizon groupsupport/EFP FilteringVTR– push/pop/Translate (1:1,1:2,2:1,2:2)

Multimillion entryFIBs

SourceRPF

Thousands ofVRFsControlledcross-VRFlookups

Multipath– ECMPandUnequalCost

IPSec

IPv6

(Neighbor discovery,

RouterAdvertisement)

ARM32,

RaspberryPi

ProxyARP

ARPtermination

IRB– BVISupportwithRouter/MACassignment

Flooding

InputACLs

Interfacecross-connect

SPAN

BFD

TUN/TAPjumbosupport

CryptoDev

FIB2.0recursiveFIBlookup,failuredetectionIPMPLSFIBSharedFIBadjacencies

Mandatory Input Checks:TTLexpiration,Headerchecksum,L2length<IPlength,ARPresolution/snooping, perinterfacewhitelists

SNAT

MagLev-likeLoadBalancer

IdentifierLocatorAddressing(ILA)

Highperformanceportrangeingressfiltering

LawfulIntercept

Policer

GBP/SecurityGroupsclassifiersupport

Connection tracking

LISP

NSH

SegmentRouting

MAP/LW46

iOAM

DHCP

GREMPLS-GRENSH-GRE

VXLAN

VXLAN-GPE

L2TPv3

VXLANoverIPv6underlay

ARPproxy

SourceRPF

MultiplemillionClassifiers–ArbitraryN-tuple

Pluginsarefirstclasscitizens

• Pluginscanbein-tree:• SNAT,PolicyACL,L2Session,FlowPerPacket,ILA,IOAM,LB,SIXRD,VCGN

• Separatefd.io project:• NSH_SFC

ethernet-input

snat-out2in

vxlan-gpe-input

ip4-input

nsh-input

nsh-mapper

vxlan-gpe-input

APIcomposability

• Eachgraph-nodeandfeatureexposesabinaryAPI

• Accessibleviahigh-performanceshared-memoryring-buffer

• Implementedasasynchronouscallback

• Python*,Javabindings

vl_api_snat_add_address_range_t_handler()

vl_api_nsh_add_del_entry_t_handler()

vl_api_vxlan_gpe_add_del_tunnel_t_handler()

ExampleAPIcallfromuser

void add_l2_bridge (int bd_id, client_main_t *cm) { vl_api_bridge_domain_add_del_t *mp; ...mp->_vl_msg_id = ntohs (VL_API_BRIDGE_DOMAIN_ADD_DEL); mp->client_index = cm->my_client_index; mp->bd_id = ntohl(bd_id); ...mp->learn = 1; vl_msg_api_send_shmem (cm->vl_input_queue, (u8 *)&mp);

}

plugin

ethernet-input

snat-out2in

vxlan-gpe-input

ip4-input

nsh-input

nsh-mapper

vxlan-gpe-input

...butonlywhentheinstructionsareloadedin

here...

Howdoesitwork?

*approx.173nodesindefaultdeployment

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

Nodeswilldoprocessing ...

1

2

...onthesethings ...3

...butonlywhenthedataisloadedinhere.

4

cpu instruction cache cpu datacache

Howdoesitwork? P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

cpu instruction cache cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

ethernet-input

Getpointertovector

PREFETCH#3and#4

PROCESS#1and#2

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

Integrations

OPNFVFDS

16.09

https://wiki.opnfv.org/display/fds/FastDataStacks+Home

GBPRendererManager

GBPNeutronMapper

VPPrendererTopologyManager-

VBD

NeutronNorthBound

Neutron

...

Honeycomb(DataplaneAgent)

VPP

DPDK

System

Test(Yardstick)

System

Install(A

PEX)

Colorado

REST

NETCONFYANG

VPPML2

ML2MechanicalDriver

Neutron

...

VPP

DPDK

REST

17.01

ML2Agent

PythonAPI

REST

Containerintegration

CGO

CONTIV

VPP

DPDK

CAPI

17.01

GOLANG

containeragent

VPPdrivercontiv

FELIXv2

VPPdriver

PythonAPI

GOLANGAPI

ApplicationNetworking

Containernetworking

FIFO

PID1234

TCP

IP(routing)

device

send()

FIFO

PID4321

TCP

IP(routing)

device

recv()

kernel

glibc

Containernetworking

FIFO

PID1234

TCP

IP(routing)

device

send()

FIFO

PID4321

TCP

IP(routing)

device

recv()

FIFO

device

FIFO

device

VPP

af_packet

etc etcetcACL,SR,VXLAN-GPEip(4|6)-rewriteip(4|6)-inputethernet-input

dpdk

dpdk

device

af_packet

Whynotthis?PID1234 PID4321

recv()

VPP

FIFOFIFO

TCP

IP

DPDK

send()

DASHBOARD 60Gbpsround-trip

DASHBOARD40Gbpsoneway

VPPFIFO

vsLinuxbridge

iPerf

*reference:E5-2620v42.1GHz

Performance

VPP technology in a nutshell

• NDR = Non Drop Rate

• Read zero frame loss

• Not ITU 10^-7 packet loss ratio

• So simply put

• NDR here is better than ITU NDR J

OVSDPDK-vhost-user

VPP-vhost-user

0

1

2

3

4

5

6

7

NIC-to-VMVM-to-VM

OVSDPDK

VPP0

5

10

15

20

2MACs2kMACs

20kMACs

NDR rates for 2p10GE, 1 core, L2 NIC-to-NIC

[IMIX Gbps]

[IMIX Gbps]

NDR rates for 2p10GE, 1 core, L2 NIC-to-VM/VM-to-VM

• Virtual network infra benchmark of efficiency• All tests per connection only, single core• Higher performance with more connections, more cores• Latest SW: OVSDPDK 2.4.0, VPP 09/2015

OVSDPDK

VPP0.0

20.0

40.0

60.0

80.0

100.0

120.0

12routes 1kroutes100kroutes

500kroutes1Mroutes

2Mroutes

OVSDPDK

VPP0.0

10.0

20.0

30.0

40.0

50.0

60.0

6routes1kroutes

100kroutes500kroutes

1Mroutes2Mroutes

• VPPdataplanethroughputnotimpactedbylargeIPv4FIBsize

• OVSDPDKdataplanethroughputheavilyimpactedbyIPv4FIBsize

• VPPandOVSDPDKtestedonHaswell x86platformwithE5-2698v32x16C2.3GHz(Ubuntu14.04trusty)

NDRratesfor6port10GE,6cores,IPv4 NDRratesfor12port10GE,12cores,IPv4

[IMIX Gbps] [IMIX Gbps]

vNet-SLAbenchmarkingatscale:IPv4VPPandOVSDPDK

fd.ioFoundation27

UPDATEw/LATEST#s

Backupslides

...butonlywhentheinstructionsareloadedin

here...

Howdoesitwork?

*approx.173nodesindefaultdeployment

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

Nodeswilldoprocessing ...

1

2

...onthesethings ...3

...butonlywhenthedataisloadedinhere.

4

cpu instructioncache cpu datacache

Howdoesitwork? P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

register node

init node

dispatch fn()

trace formatter

cpu instructioncache cpu datacache

Howdoesitwork? P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

cpu instruction cache cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

ethernet-input

Getpointertovector

PREFETCH#3and#4

PROCESS#1and#2

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

Howdoesitwork?

cpu instruction cache cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

ethernet-input

PREFETCH#3and#4

Getpointertovector

PROCESS#1and#2

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

1

2

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

P2 P3

Howdoesitwork?

cpu instruction cache cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

ethernet-input

Getpointertovector

PREFETCH#3and#4

PROCESS#1and#2

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets 2

1

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

P2 P3P0 P1

Howdoesitwork?

cpu instruction cache cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

ethernet-input

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

P2 P3P0 P1

Getpointertovector

PREFETCH#3and#4

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

PROCESS#1and#2Checkandfix-upforany

packetsthataren’tgoing tothelast

packet’snext-node!

Howdoesitwork?

cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

2

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

P2 P3 P4 P5

cpu instruction cache

ethernet-input

Getpointertovector

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

1

PREFETCH#3and#4

PROCESS#1and#2

Howdoesitwork?

cpu datacache

dispatch fn()

ethernet-input

dpdk-inputaf-packet-input

vhost-user-input

mpls-inputlldp-input

...-no-checksum

ip4-input ip6-inputarp-inputcdp-input l2-input

ip4-lookup ip4-lookup-mulitcast

ip4-rewrite-transit

ip4-load-balance

ip4-midchain

mpls-policy-encap

interface-output

P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 P10

P2 P3 P4 P5

cpu instruction cache

ethernet-input

Getpointertovector

ASSUMEnext_node sameaslastpacket

Updatecounters,advancebuffers

Enqueue thepackettonext_node

<asabovebutsingleton>

whilepacketsinvector

while4ormorepackets

whileanypackets

1

PREFETCH#3and#4

PROCESS#1and#2

VPPHistory

16.06 16.091.0.0 17.01pre-OSS

Devices

Layer2

Layer3

Traffic Mgmt

Overlays

vnet

DASHBOARD

14Mpps

Participatingprojects

Code

-#+#

Contributors: xxx

Coverity warnings:xxx

Integrations

contiv

openstack ML2

HoneycombCSITNSH_SFCONE_LISP

14

17

10

41

8

• UnitTestinfrastructure• maketest

• IPFIXenhancements• PNDAintegration

• Outputfeaturesupport• post- lookup

• Pythonbindings forplugins• Featuresub-grapharcsupport• iOAM enhancements

VPPHistory

16.06 16.091.0.0 17.01pre-OSS

Devices

Layer2

Layer3

Traffic Mgmt

Overlays

vnet

DASHBOARD

14Mpps

Participatingprojects

Code

-#+#

Contributors: xxx

Coverity warnings:xxx

Integrations

HoneycombCSITNSH_SFCONE_LISP

7

16

8

30

8

• Dynamicallyorderedsubgraphs• Doxygen generateddocs.fd.io• Java/Python languagebindings

VPPHistory

16.06 16.091.0.0 17.01pre-OSS

Devices

Layer2

Layer3

Traffic Mgmt

Overlays

vnet

DASHBOARD

14Mpps

Participatingprojects

Code

-#+#

Contributors: xxx

Coverity warnings:xxx

Integrations

HoneycombCSITNSH_SFCONE_LISP

7

16

8

19

8

VPPHistory

16.06 16.091.0.0 17.01pre-OSS

Devices

Layer2

Layer3

Traffic Mgmt

Overlays

vnet

DASHBOARD

14Mpps/core

Feb2016

Participatingprojects

Code

-#+#

Contributors: xxx

Coverity warnings:xxx

HoneycombCSIT

5

16

7

15

6

2002