Technical Forum

Scaling the Software Driven Cloud Network

Autumn 2015

Technical Forum

VXLAN - L2overLayer3

Hosts Hosts

Goals of Network Virtualization

Apps deployed based on resource utilization not location

Optimisation of the resource pool

40%

VM

Any to any connectivity

Remove islands of service connectivity

VM

Relieves management burdens

Operational Efficiency

Automation of the virtual and the physical

Decrease & automate Deployment time

VM VM

Technical Forum

EOS Network Virtualization -VXLAN§ Ethernet in IP overlay

§ Abstract Virtual from physical

§ Reduces size of failure domain

§ VMWare NSX Integration

§ Enables service stitching• Orchestration inetgration

• Macro-segmentation

§ Scale >4K VLANs: 16M VNIs

§ Does not extend fault domain

§ Consistent physical and virtual workloads

§ Seamless scale-out

Any-to-any Layer2 across Layer3 Automated VXLAN without controller

Between Data Centers

Within a Data Center

Technical Forum

At scale manual configuration of HER flood-list can be arduous, potential for excessive traffic flooding during learning processes

VXLAN Control-Plane – Unicast Replication

Host 4

VTEP 4

VNI 5000

VTEP 1

Host 1 Host 2

VTEP 3

Host 3

VTEP flood list on VTEP 1VNI 5000 à VTEP 3VNI 5000 à VTEP 4

VTEP flood list on VTEP 3VNI 5000 à VTEP 1VNI 5000 à VTEP 4

VTEP flood list on VTEP 4VNI 5000 à VTEP 1VNI 5000 à VTEP 31

2

35 5

4 4

1. VTEP flood-list - manually configured on each VTEP for each VNI

2. BUM traffic received from a locally attached node on VTEP-1

3. VTEP-1 replicates the BUM traffic for each VTEP in the flood-list of the associated VNI

4. Individual unicasts frames are sent on the wire to each VTEP in the VNI

5. Remote VTEPs receive BUM traffic

6. Remote VTEP’s learn inner source MAC and map it to the outer SRC IP (remote VTEP of origin)

Leaf 3 Leaf 4Leaf 1Leaf 2

VTEP 2Leaf 2

Technical Forum

EOS Scaling Intra-Switch

§ Single binary architecture for all platforms

§ Abstracts platform hardware specifics

§ Presents multiple open interfaces upstream

§ Delivers decoupled state sharing architecture (Sysdb)

§ Provides highly stable platform with great feature velocity

§ Enables agility in hardware choice

CLI eAPI OVSDB SDK

Common binary/APIs for all platforms

Low Inter-Process Communication

Arista EOS Architecture

Efficient Subscribe/Publish

Linear Cloud Scaling

Publish

Notify

PIM

SNMP BGP

MLAG

STP eAPI

IGMPSysdbDriver

Technical Forum

Introducing CloudVision eXchange (CVX)

EOS EOS EOS

Single Interface to EOS devices

EOS publish/subscribe model

§ Part of EOS CloudVision Framework

§ Abstracts the physical switch infrastructure

• MLAG support VTEP resilience

§ Provides a single access point for real-time provisioning

• Orchestration and integration with multi-vendor controllers

§ Distributed EOS state: CVX mounts state from all switches (sysDBs) in the network

§ No new protocol needed – uses EOS framework, same openness

§ Management plane, not data plane

§ CVX runs as VM

EOS

CVX

CLI eAPI OVSDB SDK

Open Standards-based northbound APIs

State publish/subscribe

CloudVision

eXchange

MLAG resilience is abstracted by CVX

Technical Forum

Leaf 2

CVX – simplified provision and learningAutomated flood-list configuration and MAC address distribution

VXLAN Control-Plane – CVX

1. MAC learnt locally on VTEP 1 From generated host traffic

2. Local VXLAN states are mounted by CVX

3. CVX has a global view of each VTEP

- local VXLAN MAC address tables, VNI configured on each VTEP

4. Remote MACs for locally configured VNI Written to local VXLAN table

5. Remote MAC added to local VXLAN hardware tableHost 4, MAC D

VTEP 4

VNI 5000

VTEP 1

Host1, MAC A Host 2

VTEP 2 VTEP 3

Host 3

1

2

5 5

4

Network DatabaseVTEP 1: VNI 5000:MAC AVTEP 4: VNI 5000:MAC D

VXLAN tableVNI 5000 MAC A VTEP 1VNI 5000 MAC D VTEP 4

CloudVision

eXchange

3

Leaf 2Leaf 1 Leaf 3 Leaf 4

Technical Forum

VXLAN Deployment Solutions

VTEP-1

OpenstackNSX, Nuage, …

Automated VXLAN without 3rd party controller

Automation and integrationwith 3rd party controller

Small Scale DC and DCI solution

Head Replication (HER)• Manually configured VTEP-flood

list

• Traffic flooded via the defined flood-list.

• Flow-based MAC learning

• No need for Multicast in the IP fabric

• Suitable for DCI solutions and small scale intra-DC solution due to manual config

CVX standalone• CVX provides centralized database

of all VXLAN state.

• MAC address learning via the CVX, flow-based learning optional

• HER flood-list automatically populated by the CVX

• No need for Multicast in the IP fabric

• Scalable for intra-DC solutions where a level of automation is required

CVX + 3rd party integration• Centralized database of CVX

shared with third-party controller (NSX, OpenStack, Nuage, etc)

• Distributed MAC address learning between Software and hardware VTEPs.

• VNI provisioning via centralized controller

• Solution for scalable DCs with HW to SW VTEP automation

CloudVision

eXchangeCloudVision

eXchange

Technical Forum

INTERFACES AND APIsTO CVX

Technical Forum

3rd Party Integration with CVX

§ Third-party controllers can consume the state and share state using their preferred API

§ Provides the third-party controller with single automation point for the physical infrastructure

§ Provides the third-party controller with visibility of the control-plane from the physical infrastructure

CVX has a global view of the physical hardware infrastructure, with multiple programmable open APIs

Network wide view of the physical networkGlobal VXLAN Database

CLI/SNMP

State SyncState Sync State Sync

OVSDBeAPI SDK others

CloudVision

eXchange

SW-1: VNI-X: MAC-A,

VNI-Y: MACBSW-2: VNI-Y:MAC-D

VXLAN VNI, VTEP, MAC, LLDP

States example:

VXLAN DB example:

Technical Forum

CVX Workload Service #1: Full Physical Topology§ Leaf switch builds their local topology table using standard LLDP

§ Contains directly attached compute nodes, which will host the virtual machines

§ CVX mounts the local LLDP tables, providing a network wide view

§ CVX knows the physical location (switch and interface) each compute node is attachedeAPI

cvs-switch#show network physical-topology neighborsInterface Neighbor Intf Neighbor Host------------------ ------------------ --------------Ethernet1 Ethernet1 atf-spine1Ethernet2 Ethernet1 atf-spine2Ethernet3 eth1 atf-oshost1Ethernet4 eth1 atf-oshost2

Network wide Topology Table

cvs-switch#show network physical-topology hostsUnique Id Hostname--------------------- ---------------------0050.5686.ba66 atf-host10050.5686.4711 atf-host20050.5686.1184 atf-host3 Compute Nodes

Network wide topology visible from CVX eAPI to consume the info northbound

LLDP

LLDP

compute compute

et2

Network Topology Database

LLDP State

et1

LLDP

LLDP

compute compute

et2

LLDP State

et1

CloudVision

eXchange

Technical Forum

EOS EOS EOS

EOS

CVXCloudVision

eXchange

CVX Workload Service #2: VXLAN

• VXLAN Control Services (VCS)• Control plane that integrates

with the VXLAN data plane

• VCS is used to distribute• VXLAN-related configuration

• Network reachability state across the network

• Simplifies Unicast Head-End Replication (HER)

Propagates VXLAN Network State

VCS

VMmobilityAcrossLayer3subnets

Technical Forum

Same publish/subscribe model

‘BYOC’

Single Interfaceto EOS devices

CVX Workload Service #3: Third Party Integration

• Integration and provisioning point for :

• 3rd party controllers / services

• orchestration systems

• Controller-Agnostic approach

• Network-wide visibility via single access point

• Collective state presented to controllers

• Integration via OVSDB, eAPI, EOS SDK

• Most efficient Physical-to-Virtual (P-to-V) synchronization

EOS EOS EOS

EOS

CVXCloudVision

eXchange

MLAG resilience is abstracted by CVX

Technical Forum

NativeOpenstack

NativevSphere3.x,4.x,5.x,6.x

Universal Integration with CloudVision

Multiple controllers and virtualisation platforms can coexist and integrate with CVX

CloudVision

eXchange

Technical Forum

Platform for Automation and Visibility across the Network

Network-Wide Database

Aggregation of Network wide ‘Sysdb’

Abstraction of the physical network

Single integration point to the network

More scalable controller integration

Provisioning the logical topology

State-sync

Technical Forum

Interacting with Controllers via OVSDB

vSwitch Driver Vendor A Driver

Networking Layer

Vendor Z Driver

asdasdasdpSwitch asdasdasdpSwitchasdasdasdvSwitchasdasdasdvSwitch

asdasdasdvSwitchasdasdasdvSwitch

asdasdasdvSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

§ Controller controls all HW and SW endpoints individually

§ Must scale to combined host & network

• Provisioning, learning, monitoring

§ Must understand topology logic for HW infrastructure

§ Must implement network layer redundancy logic for every vendor

§ Every controller vendor must implement support for each vendor independently

§ Introduces version- and vendor-dependencies

VirtualisationController

Technical Forum

XX

XX

X

Introducing the Hardware Switch Controller

§ HSC is a vendor provided abstraction layer

§ Removes the need for detailed knowledge of the Physical network

§ Enables multi-vendor version and platform independence

§ Reduces time to onboard technology

§ Decouple infrastructure from controller

asdasdasdpSwitch asdasdasdpSwitchasdasdasdvSwitchasdasdasdvSwitch

asdasdasdvSwitchasdasdasdvSwitch

asdasdasdvSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

asdasdasdpSwitchasdasdasdpSwitch

vSwitch Driver

Vendor A HSC

Networking Layer

Vendor Z HSC

X

VirtualisationController

Technical Forum

Overlay Controller

Scaling Controller Integration

18

OVSDB

Overlay Controller

Network Layer

Controller Layer

10xImprovement

OVSDB SysdbState Sync

Topology/DeviceDependent

Topology/DeviceAbstraction

Traditional Approach

CloudVision Approach

Technical Forum

Thank You