Midokura Enterprise MidoNet (MEM) Overviewfiles.meetup.com/10602292/Midonet SDN.pdf · 2015-02-10 · 4 Midokura Enterprise MidoNet (MEM) Network Virtualization Platform v Any Application

Confidential

Midokura Enterprise MidoNet (MEM) Overview

Confidential

About the company

• Founded in 2010, Midokura is a global company with offices in Tokyo, San Francisco and Barcelona

• Pioneer in network virtualization – provides software for networking using overlay approach. Pedigree derives Amazon, Cisco, VMware and Google

• Received over $20M in funding from Innovation Network Corporation of Japan, NTT, NEC, and Fujitsu

• Named by CRN as amongst the top 10 networking stories of 2013 and also amongst 10 coolest startups in the world

1

• Won Nokia’s Silicon Valley Innovation Challenge – 2014

• Named AlwaysOn award winner for the second consecutive year

• Significant contributor to the OpenStack Networking (Neutron) Project

• First SDN vendor to be certified for Red Hat OpenStack environment

• Early member of the Open DayLight Project (ODP)

• Broad and deep technical partnerships with network switch vendors, software companies and solution providers

Confidential 2

Our Ecosystem

MidoNet Users

Technology Partners

3

With increase in usage of cloud applications, Networks have become complex and hard to manage

Load Balancer Firewall

• Under utilization of compute

• Dedicated appliances

• More power consumption

Costly

• Networks don’t scale with dynamic workloads

• Takes time to provision network services

• Poor quality of service

Inflexible

• Manual provisioning

• Fragmented management

• Higher latency

• User experience can be improved

Complex

4

Midokura Enterprise MidoNet (MEM) Network Virtualization Platform

v

Any Application

Midokura Enterprise MidoNet

Logical L2

Any Network Hardware

OpenStack, vSphere, Custom Platforms

Logical Firewall

Logical Layer 4 Load Balancer

KVM, ESXi, LXC, Docker

Logical L3

Logical Switching – Layer 2 over Layer 3, decoupled from the physical network Logical Routing – Routing between virtual networks without exiting the software container Logical Firewall – Distributed Firewall, Kernel Integrated, High Performance Logical Layer 4 Load Balancer – Application Load Balancing in software MidoNet API – RESTful API for integration into any Cloud Management Platform

Distributed Networking Services

5

Open Source – Same license as OpenStack. Appeals to trending preference for open software. Aims to be the default networking for OpenStack and Docker Vendor Neutral – Works with any networking gear. Brownfield, Greenfield, all OK. (Added features with Cumulus+Dell) Trusted Technology – Accessible, widely deployed, proven by the community. Enterprise Class Offering– MEM is hardened with SLA backed support for production environments.

A truly open SDN overlay option

midonet.org

6

SWIFT

OBJECT STORAGE

OpenStack Cloud Infrastructure

6

CINDER

BLOCK STORAGE

Software • Massive Performance and Scale

• Designed with Open Standards

• Amazon Cloud “like” self service

• Massive Agility

PHYSICAL CLOUD INFRASTRUCTURE

HEAT

ORCHESTRATION

NOVA

COMPUTE

NEUTRON

NETWORKING

KEYSTONE

IDENTITY

CLOUD ENABLED LINUX OPERATING SYSTEM

GLANCE

IMAGE CATALOG

CEILOMETER

TELEMETRY

Hardware • Scalable HA High Performance

Networking 10Gb/40Gb powered by

Active Fabric Manager or Cumulus

Linux L3 Fabric

• Micro to Hyper-scale Compute

Framework

• Dense Converged Capable

MIDONET

MANAGER

HORIZON

DASHBOARD

MIDONET

CLI

X86

X86

X86

X86

X86

X86

X86

40G TII

40G TII

X86

X86

X86

X86

X86

EXAMPLE HARDWARE

7

Customer Journey

Agility

Provide rapid provisioning of isolated

network infrastructure for labs and devops.

Logical Network

Provisioning

Automated Provisioning

Isolated Sandboxes

Control

Network admins can better secure, control &

view network traffic.

Single Pane of Glass

OpsTools

Enhanced Security

Enable Compliance

IaaS Cloud

Build multi-tenant clouds with visibility

into usage.

Tenant Control

Metering

Automated Self Service

Performance

Improve network performance using edge

overlay & complementary technologies.

Single Hop Virtual

Networking

VXLAN Hardware Gateway

Massive performance

with 40Gb Support

Scale

Add virtual network infra & services simply & resiliently without

hardware & bottlenecks.

Distributed Logical

Networking FW, LB, L2/3,

NAT

Limitless “VLANs”

Scale out L3 Gateway

Bridge legacy VLANs

IPv6

Solution for OpenStack Networking

Use MN to overcome limitations of Neutron for

OpenStack users.

Replaces OVS Plugin

Va

lue

Do it Bigger Do it Faster Do it Better

Confidential

Evolution of Network Virtualization

8

Virtual Network

Overlays

Decoupling hardware

and software

• Cloud-ready agility

• Unlimited scalability

• Open, standards-based

• No impact to physical network

PROACTIVE SOFTWARE OVERLAY

INNOVATION IN NETWORKING AGILITY

Reactive End-to-End

Requires programming

of flows

• Limited scalability

• Hard to manage

• Impact to performance

• Still requires tenant state in physical network

OPENFLOW REACTIVE APPOACH

VLAN configured

on physical switches

• Static

• Manual

• Complex

• Tenant state maintained in physical network

Manual End-to-End

VLAN APPROACH

8

Confidential

Architecture Overview

Kernel Kernel

Kernel

Confidential

Logical Topology – Overlay Networks

Confidential 11

VXLAN Gateway: MidoNet + Cumulus Linux

VxLAN Tunnel

Physical Connection

OVSDB

TCP/IP

Feature supported on:

Trident II based switches

Confidential

MidoNet for

vSphere

12

Confidential

Why MidoNet?

13

• Distributed controller for best performance, resiliency, and scalability • Single Virtual Hop = Better Performance • No SPOF = Production Grade • Fully Distributed = Massive Scale

• Additional distributed services like L4 Load Balancing • Floating IPs, Security Groups, Routing without the need for IP Tables, L3

Agent, etc. (few or none do this) • Distributed Stateful NAT (others do failover) • Fully distributed L3 GW (others do failover) • L4LB with health checks (no one has this) • VXLAN Gateway • Simple Architecture=Simple Ops (no service nodes, no active/standby) • Competitive and Simple Subscription Licensing ($1,899 per node per year)

Confidential

MidoNet Distributed Advantage: Comparing with OVS and Centralized Controller Approaches

14

Confidential 15

Private IP Network

SDN Controller

Active Gateway Standby Gateway

Internet

Service Node

Linux Kernel

Open vSwitch Agent

VM

IP Tables

SD N C ontroller centrally processes flow s, and

program s virtual sw itches rem otely

VM VM

Linux Kernel

Open vSwitch Agent

VM

IP Tables

VM VM

C entralized C ontroller M odel

Confidential 16

Private IP Network

Network State Database

Internet

M idoN et A gents act as distributed controller

M idoN et D istributed M odel

Network State DatabaseNetwork State Database

Linux Kernel

MidoNet Agent

VMVM VM

Linux Kernel

MidoNet Agent

VMVM VM

Active GatewayActive Gateway

Active Gateways

D istributed scale out G atew ays

Logical N etw ork topology stored in

distributed database

M idoN et A gent rem oves need for Service N odes and

IP Tables

Confidential 17

Private IP Network

SDN Controller

Service Node

Service node centrally responsible netw ork services

like N AT, routing, Load balancing

Linux Kernel

Open vSwitch Agent

VM

IP Tables

VM VM

C entralized C ontroller M odel

Confidential 18

Private IP Network


M idoN et A gent program s the K ernel to provide services like security groups, routing, load balancing, and floating IP s

Linux Kernel

VMVM VM

M idoN et’s D istributed Edge M odel

MidoNet Agent

Confidential 19

Private IP Network

SDN Controller

Active Gateway Standby Gateway

Internet

Linux Kernel

Open vSwitch Agent

VM

IP Tables

A ll outgoing flow s travel through the active gatew ay

node.

VM VM

Linux Kernel

Open vSwitch Agent

VM

IP Tables

VM VM

A ctive/Standby G W M odel

Confidential 20

Private IP Network

Active Gateway 1

Active Gateway 2

Internet

Linux Kernel

VM

MidoNet Agent

O utgoing and Incom ing flow s balanced across M idoN et D istributed G atew ays

VM VM

Linux Kernel

VM

MidoNet Agetnt

VM VM

Active Gateway 3

Network State DatabaseNetwork State Database


Fully D istributed G W M odel

Confidential

Why L3 Gateway?

21

• Static routes suck

• Provides HA out of the box

• Inbound distributed NAT, routing, L4LB,

and Firewalls

• Can provide VPC like multi-tenant BGP

capabilities

Confidential

Midokura Enterprise MidoNet Pricing

22

Confidential

MidoNet Q&A

23

Confidential

Thank you!

24

Confidential

Backup Slides

25

Confidential

OVS Overview

26

Confidential

OVS Open Source Plugin

27

Overlay Networking

GRE Tunnels

Uses Open vSwitch Project

Components:

• Neutron OVS Agent

• Neutron DHCP Agent

• Neutron L3 Agent

• IPTables

N eutron N etw ork N ode

Neutron-Server + OVS Plugin

L3 Agent DHCP Agent OVS Agent

N AT /Floating IPs

IP Tables / Routing

dnsmasqovsdb/

vswitchd

Linux Kernel / IP Stack

C om pute N ode

nova compute

OVS Agent KVM

VM VM


ovsdb/vswitchd

IP Tables

C om pute N ode

nova compute

OVS Agent KVM

VM VM


ovsdb/vswitchd

IP Tables

G R E Tunnels

IP U nderlayWAN

security groups security groups

Confidential

Challenges with OVS Plugin

28

Neutron Network Node is a SPOF

Need to use corosync, etc for active/standby failover.

Challenging at Scale

Since there’s a single network node, this becomes a bottleneck

fairly quickly.

Inefficient Networking

IPTables, L3 Agent, multiple hops for single flow are causing

unnecessary traffic and added latency on your physical network

Confidential

How MidoNet works

29

Confidential

Yo

ur E

xis

ting

Infra

stru

ctu

re

30

Load Balancer

Mid

oN

et

Gate

way

Cloud Networking

Can Be Complicated

Then We Add MidoNet Storage

and MidoNet Border Nodes

Then we Install

the MidoNet

Agent on all the

Hypervsior

Nodes

Overlay needs underlay devices connected over IP

Confidential

Now we can build your Logical Network

31

Confidential

MidoNet creates a Provider Router which connects to the External Network Each Tenant can create their own virtual Tenant Router Then the tenant can create VMs and Networks then attach those to the Tenant Router Various rules and subnets can be applied to the virtual infrastructure

32

Provider Router

Tenant Router

Tenant Network

192.168.5.2 192.168.5.3

Let’s Spin up two VMs

for a Single Tenant

Subnet 192.168.5.0/24

Address: 192.168.5.1 Allow incoming tcp/22

NAT 192.168.5.2 <-> 112.140.32.94

Confidential

All of the logical topology is stored in MidoNet’s Storage Nodes

33

Provider Router

Tenant Router

Tenant Network

192.168.5.2 192.168.5.3

Subnet 192.168.5.0/24


NAT 192.168.5.2 <-> 112.140.32.94

Mid

oN

et

Gate

way

Yo

ur E

xis

ting

Infra

stru

ctu

re

Confidential

Now let’s talk about what happens when we send traffic between the two VMs

34

Confidential

First the outbound packet from VM1 is intercepted by the MidoNet agent on the Hypervisor Next, the MidoNet Agent queries Network state database for the virtual topology Then the MidoNet agent simulates the packet moving through the virtual topology and actions that need to be performed on the packet

35

Mid

oN

et

Gate

way

Yo

ur E

xis

ting

Infra

stru

ctu

re

Provider Router

Tenant Router

Tenant Network

192.168.5.2 192.168.5.3

Subnet 192.168.5.0/24


NAT 192.168.5.2 <-> 112.140.32.94

Confidential

Mid

oN

et

Gate

way

Yo

ur E

xis

ting

Infra

stru

ctu

re

Now MidoNet can create a GRE tunnel between the required nodes, and send the packet on its way Subsequent packets follow the already established path, and can travel at near-line-speed. Finally, the packet is received by the target node and delivered to the VM.

36

GRE Tunnel

Confidential

The process is similar when sending packets to/from the External Network

37

Confidential

Mid

oN

et

Bo

rders

Y

ou

r Ex

istin

g

Clo

ud

Infra

stru

ctu

re

First the outbound packet from VM1 is intercepted by the MidoNet agent on the Hypervisor Next, the MidoNet Agent queries the Network State Databasefor the virtual topology Then the MidoNet agent simulates the packet moving through the virtual topology and actions that need to be performed on the packet Now MidoNet can create a GRE tunnel between the required nodes, perform the packet actions, and send the packet on its way Subsequent packets follow the already established path, and can travel at near-line-speed.

38

Provider Router

Tenant Router

Tenant Network

192.168.5.2 192.168.5.3

Subnet 192.168.5.0/24


NAT 192.168.5.2 <-> 112.140.32.94

Confidential

Mid

oN

et

Bo

rders

Y

ou

r Ex

istin

g

Clo

ud

Infra

stru

ctu

re

39

Provider Router

Tenant Router

Tenant Network

192.168.5.2 192.168.5.3

Subnet 192.168.5.0/24


NAT 192.168.5.2 <-> 112.140.32.94

The process is similar for packets starting from the Internet... ...only this time the Border Node queries the Storage Nodes for the virtual topology... ...and then simulates the packet moving through the virtual topology and actions that need to be performed on the packet Now MidoNet can create a GRE tunnel between the required nodes, perform the packet actions, and send the packet on its way As before, Subsequent packets follow the already established path, and can travel at near-line-speed.

Confidential

Deep Dive on

MidoNet OpenStack Implementation

Confidential 41

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink

Provider Virtual Router (L3)

Tenant AVirtual Router

Tenant BVirtual Router

VM6

Virtual L2 Switch B1

Virtual L2 Switch A1


TenantB office

Tenant BVPN Router

Office Network

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

42

Isolated tenant

network

(virtual data center)

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

43

L3 isolation

(similar to VPC and VRF)

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

44

Isolated L2 networks

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

45

Redundant, optimized and

fault-tolerant paths to the

Internet (e.g. via BGP)

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

46

Fault-tolerant

devices and links

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

47

NAT, LB, and

Filtering

NAT, LB, and

Firewalls

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

48

L3 (and

L2) VPNs

Requirements

Confidential

Tenant/Project A

Network A1

VM1 VM3

Network A2

VM5

Tenant/Project B

Network B1

VM2 VM4

uplink




VM6




TenantB office

Tenant BVPN Router

Office Network

49

Minimize ARP broadcasts

by exploiting CMS config RESTful API for CMS

integration and direct

tenant access

Solid integration with

leading open CMS:

OpenStack, CloudStack

DHCP, DNS and other

services

Requirements

Confidential 50

• Multi-tenancy

• Scalable, fault-tolerant

devices (or device-agnostic

network services).

• L2 isolation

• L3 routing isolation

• VPC

• Like VRF (virtual routing

and fwd-ing)

• BGP gateway

• Scalable control plane

• ARP, DHCP, ICMP

• Floating IP

• Stateful NAT

• Port masquerading

• DNAT

• ACLs

• Stateful (L4) Firewalls

• Security Groups

• LB health checks

• VPNs at L2 and L3

• IPSec

• REST API

• Integration with CMS

• OpenStack

• CloudStack

Requirements Recap

Confidential 51

VM

VM

Edge

Edge Edge

Edge Edge

Edge

IP encapsulation

provides isolation

Edge-to-Edge Overlays

Confidential 52

VM

VM

Edge

Edge Edge

Edge Edge

Edge

Virtual network

processing at

ingress host,

decoupled from

physical network


Confidential 53

VM

VM

Edge

Edge Edge

Edge Edge

Edge

Virtual network

changes don't affect

underlay state


Confidential 54

Distributed State

MidoNet REST API

Dashboard

MidoNet SDN Solution

Confidential 55

Distributed State

Linux Kernel + OVS KMOD

VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Lazy state

propagation


Confidential 56

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

VM sends first

packet; table miss;

NetLink upcall to

MidoNet


Confidential 57

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

MidoNet agent locally

processes packet (virtual

layer simulation); installs

local flow (drop/mod/fwd)


Confidential 58

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Packet tunneled to

peer host; decap;

kflow table miss;

Netlink notifies peer

MidoNet agent


Confidential 59

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

MN agent maps tun-

key to kernel

datapath port#;

installs fwd flow rule


Confidential 60

Distributed State


VM1 MidoNet

Ctrl

HW


VM2 MidoNet

Ctrl

HW

Host A Host B

Subsequent packets

matched by flow rules

at both ingress and

egress hosts


Documents

Midokura Enterprise MidoNet (MEM) Overviewfiles.meetup.com/10602292/Midonet SDN.pdf · 2015-02-10 · 4 Midokura Enterprise MidoNet (MEM) Network Virtualization Platform v Any Application