BRKDCT-3831

2009 Cisco Systems, Inc. All rights reserved. Cisco PublicBRKDCT-3831 1

Advanced Data Center Virtualization

BRKDCT-3831Carlos Pereira [email protected]


Housekeeping

We value your feedback- don't forget to complete your online session evaluations after each session & complete the Overall Conference Evaluation which will be available online from Thursday

Visit the World of Solutions Please remember this is a 'non-smoking' venue! Please switch off your mobile phones Please make use of the recycling bins provided Please remember to wear your badge at all times

including the Party


If you can see it and it is there

Its real

If you cant see it but it is there

Its transparent

If you can see it and it is not there

Its virtual

If you can not see it and it is not there

Its gone

Setting the stage:Whats the meaning of virtual?


Todays Data Center Virtualization journey


F

r

o

n

t

-

E

n

d

Virtual SANs/Unified IO

Virtual Storage

Virtual Network ServicesVirtual Firewall Context 1

Virtual SSLContext 3

Virtual Machines

Front-End Virtualization

Virtual Firewall Context 1


Virtual SLBContext 29



VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d

Data Center Virtualization Overview

Front-End Data Center Virtualization

Core LayerVDC

Aggregation LayervPCVSSServer Load BalancingSecurity Services

Access Layer Server Virtualization

Nexus 1000v Back-End Virtualization

SANHBAUnified IO (FCoE)Storage

Q&A

The journey in a nutshell: from the Network Core up to the Disks


After session goal: for us to avoid the virtualization


Agenda

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d

Data Center Virtualization Overview

Front-End Data Center Virtualization

Core LayerVDC

Aggregation LayervPCVSSServer Load BalancingSecurity Services




Q&A


VDC 2

VDC 4

Device Partitioning

Virtualized Interconnect

Device Pooling

VDCs

VLANs

L3 VPNs MPLS VPNs, GRE, VRF-Lite, etc.

L2 VPNs - AToM, Unified I/O, VLAN trunks, PW, etc.

VSS, Stackwise, VBS,

Virtual Port Channel (vPC)

HSRP/GLBP

FW,ACE context

VRFs

1 : n1 : n1 : n1 : n n : 1n : 1n : 1n : 1n : mn : mn : mn : m

Network Virtualization Building Blocks


DC Core

CBS 3xxx Blade

Nexus 5000 & Nexus 2000Rack

Nexus 7000End-of-Row

Gigabit Ethernet10 Gigabit Ethernet10 Gigabit DCE

4/8Gb Fiber Channel

Nexus 5000Rack

DC Access

Nexus 700010GbE AggCisco Catalyst 6500DC Services

MDS 9500Storage

Cisco Catalyst 6500End-of-Row

FC Storage

IP+MPLS WAN Agg Router

10GbE and 4Gb FC Server Access

CBS 3xxxMDS 9124eNexus blade (*)

10GbE and 4/8Gb FC Server Access10Gb FCoE Server Access

10 Gigabit FCoE/DCE

1GbE Server Access

Nexus 700010GbE Core

Cisco Catalyst 650010GbE VSS AggDC Services

DC Aggregation

Virtualized Data Center Infrastructure

FC

WAN

SAN A/BMDS 9500Storage Core

(*) future


Agenda Data Center Virtualization

Overview Front-End Data Center

VirtualizationCore Layer

VDCAggregation Layer

vPCVSSServer Load BalancingSecurity Services




Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Virtual Device Contexts Provides Virtualization at the Device Level Allowing Multiple Instances of the Device to Operate on the Same Physical Switch at the Same Time

Kernel

Infrastructure

Protocol Stack (IPv4/IPv6/L2)

L2 Protocols

VDC1

VLAN Mgr

Nexus 7000 Physical Switch

VDCnProtocol Stack (IPv4/IPv6/L2)

L3 Protocols

UDLD

VLAN Mgr UDLD

LACP CTS

IGMP 802.1x

RIB

OSPF GLBP

BGP HSRP

EIGRP VRRP

PIM SNMP

RIB

L2 Protocols

VLAN Mgr

L3 Protocols

UDLD

VLAN Mgr UDLD

LACP CTS

IGMP 802.1x

RIB

OSPF GLBP

BGP HSRP

EIGRP VRRP

PIM SNMP

RIB

Virtual Device Contexts at Nexus 7000 VDC Architecture


Virtual Device Contexts VDC Fault Domain

KernelInfrastructure

Protocol StackVDCA

Nexus 7000 Physical Switch

VDC A

P

r

o

c

e

s

s

A

B

C

P

r

o

c

e

s

s

D

E

F

P

r

o

c

e

s

s

X

Y

Z

Protocol StackVDCB

VDC B

P

r

o

c

e

s

s

A

B

C

P

r

o

c

e

s

s

D

E

F

P

r

o

c

e

s

s

X

Y

Z

Process DEF in VDC B Crashes

Process DEF in VDC A Is Not Affected and Will Continue to Run Unimpeded

A VDC Builds a Fault Domain Around All Running Processes Within That VDCShould a Fault Occur in a Running Process, It Is Truly Isolated from Other Running Processes and They Will Not Be Impacted

ABCD

AB

C D


Linecard 1 Linecard 2 Linecard 3

V

D

C

3

0

V

D

C

2

0

V

D

C

2

0

V

D

C

2

0

Virtual Device Contexts VDC Resource Utilization (Layer 2)

Switch Fabric

MAC Table MAC Table MAC Table

V

D

C

1

0

V

D

C

1

0

V

D

C

3

0

1/1 1/2 1/3 1/4 2/1 2/2 2/3 2/4 3/1 3/2 3/3 3/4

MAC Address A

MAC A MAC A

X

MAC A Is Propagated to Linecard 2 and 3 but Only Linecard 2 Installs MAC Due to Local Port Being In VDC 10

Layer 2 Learning with Multiple Active VDCs Also Has an Impact on Resource UtilizationMAC Addresses Learnt in a VDC Are Only Propagated to Other Linecards When That Linecard Has a Port in That VDC


FIB TCAM

Size 128K

ACL TCAM

Size 64K

FIB TCAM

Size 128K

FIB TCAM

Size 128K

FIB TCAM

Size 128K

VDC-1

IP routes: 20K

ACL entries: 10K

VDC-2

IP routes: 100K

ACL entries: 50K

ACL TCAM

Size 64K

VDC-3

IP routes: 100K

ACL entries: 50K

ACL TCAM

Size 64K

ACL TCAM

Size 64K

Linecard 1 Linecard 2

Linecard 3Linecard 4

Virtual Device Contexts Separate Resource Allocation Domains (Layer 3)


VDC Use Case ExamplesSecurity Partitioning

Appliance Model Service Module Model

Firewall

VDCFirewall

Some Infosec departments are still reluctant about collapsed infrastructure

Concerns around change management Infrastructure misconfiguration could bypass

policies

Ideally they want to have physically separately infrastructure.

Not cost effective in larger deployments.

VDCs provide logical separation simulating air gap

Extremely low possibility of configuration bypassing security path Must be physically bypassed

Model can be applied for any DC services Inside

VDC

Outside

Outside Inside


VDC Use Case ExamplesHorizontal Consolidation Preface: Lead with separate physical boxes as they provide the

most scalable solution. VDCs are useful in certain situations! Objective: Consolidate lateral infrastructure that delivers similar

roles for separate operational or administrative domains. Benefits: Reduced power and space requirements, can maximize

density of the platform, easy migration to physical separation for future growth

Considerations: Number of VDCs (4), Four VDCs != Four CPU Does not significantly reduce cabling or interfaces needed.

core1

core2

agg2agg1

acc2acc1

agg4agg3

accYaccNacc2acc1 accYaccN

corecore

Core

Aggregation VDCs

Core Devices

Aggregation Devices agg VDC 1agg VDC 2

agg VDC 1agg VDC 2

agg VDC 1 agg VDC 2Admin Group 1 Admin Group 2 Admin Group 1 Admin Group 2


core1

core2

agg4agg3

accYaccN accYaccN

core VDC

agg VDC

Core VDCs

Aggregation VDCs

Core Devices

Aggregation Devices

core VDC

agg VDC

VDC Use Case ExamplesVertical Consolidation Preface: Lead with separate physical boxes as they provide the most

scalable solution. Large Three Tier designs should remain physical.Smaller Two Tier designs can leverage VDCs for common logical design with three tier.

Objective: Consolidate vertical infrastructure that delivers orthogonal roles to the same administrative or operational domain.

Benefits: Reduced power and space requirements, can maximize density of the platform, provides smooth growth path, easy migration to physical separation in future

Considerations: Number of VDCs (4), Four VDCs != Four CPU Intra-Nexus7000 cabling needed for connectivity between layers.


VDC Use Case ExamplesVertical & Horizontal Consolidation Preface: Lead with separate physical boxes as they provide the most

scalable solution. Combined vertical & horizontal consolidation in small to medium designs (2 aggregation blocks or less) Power, cooling and real estate optimization for multiple layers Maximize the benefits of a high-density platform Simplified growth migration path

core1

core2

agg2agg1

acc2acc1

agg4agg3

accYaccNacc2acc1 accYaccN

corecore

aggagg

Core VDCs

Aggregation VDCs

Core Devices

Aggregation Devices


VDC Use Case ExamplesConsolidated Infrastructure Simplified Growth & Migration

Preface: Lead with separate physical boxes since the beginning as they provide the most scalable solution.

Growth to physical core & aggregation is a simplified migration Configuration is portable Logical topology remains the same

acc2acc1 accYaccN

corecore

aggagg

Core VDCs

Aggregation VDCs

core1

core2

agg2agg1

acc2acc1

agg4agg3

accYaccN

Core Devices

Aggregation Devices










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Virtual Port-Channel (vPC)Feature Overview

Allow a single device to use a port channel across two upstream switches

Separate physical switches independent control and data plane

Eliminate STP blocked ports. Uses all available uplink bandwidth

Dual-homed server operate in active-active mode

Provide fast convergence upon link/device failure

Available in NX-OS 4.1 for Nexus 7000. Nexus 5000 availability planned for CY09.

Logical Topology without vPC

Logical Topology with vPC


vPC Terminology vPC peer a vPC switch, one of a

pair vPC member port one of a set of

ports (port channels) that form a vPC vPC the combined port channel

between the vPC peers and the downstream device

vPC peer-link Link used to synchronize state between vPC peer devices, must be 10GbE

vPC ft-link the fault tolerant link between vPC peer devices, i.e., backup to the vPC peer-link

CFS Cisco Fabric Services protocol, used for state synchronization and configuration validation between vPC peer devices

vPC

vPC peer-link

vPC peer

non-vPCdevice

vPC ft-link

vPC member

port

vPCvPC

member port

CFS protocol


Virtual Port Channel Properties

Standard Port Channel on Downstream

Switches


Switches


Switches

vPC on vPC peers with

local forwarding

L2L3


Multi-level vPC

SW4SW3

vPC_PLvPC FT-Link

SW2SW1

vPC_PLvPC FT-Link

SW4SW3

vPC_PLvPC FT-Link

SW2SW1

vPC_PLvPC FT-Link

Physical View

Logical View

Up to 16 links between both sets of switches: 4 ports from sw1-sw3, sw1-sw4, sw2-sw3, sw2-sw4

Provides maximum non-blocking bandwidth between sets of switch peers Is not limited to one layer, can be extended as needed


vPC peer interaction

vPC SecondaryvPC Primary

STP rootHSRP Active

PIM DR

STP backupHSRP

Standby

vPC Primary Primary is manually defined, with

manual failback in case of system failure

STP root highest priority HSRP active highest priority PIM DR highest priority

vPC Secondary STP root lower priority HSRP standby PIM DR standby

STP is used for backup in case of mis-configuration

STP, HSRP, PIM primary/secondary configuration should follow vPC primary/secondary to simplify debug

STP/HSPR/PIM failover to secondary/standby is not forced by vPC, follows standard failover operation










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Storage10GbE and 4Gb FC Server Access10GbE and 4/8Gb FC Server Access10Gb FCoE Server Access

1GbE Server Access

CBS 31xx Blade

Nexus 5000 &Nexus 2000 Rack

Nexus 7000End-of-Row

Gigabit Ethernet10 Gigabit Ethernet10 Gigabit DCE

4/8Gb Fiber Channel

Nexus 5000Rack

DC Access

Nexus 700010GbE AggCisco Catalyst 6500DC Services

MDS 9500Storage

Cisco Catalyst 6500End-of-Row

IP+MPLS WAN Agg Router

CBS 31xxMDS 9124eNexus Blade (*)

10 Gigabit FCoE/DCE

Nexus 700010GbE Core

Cisco Catalyst 650010GbE VSS AggDC Services

DC Aggregation

FC

WAN

SAN A/BMDS 9500Storage Core

DC Core

Aggregation Services Design Options

(*) future

One-Arm Service SwitchesEmbedded Service Modules


Virtual Switch System Is a Technology Break Through for the Cisco Catalyst 6500 Family

Virtual Switch System (VSS)Concepts


Virtual Switch Domain

Switch 1Control Plane Active Switch 2Control Plane Hot Standby

Virtual Switch Domain

Switch 1Data Plane Active Switch 2Data Plane Active

In Virtual Switch Mode, While Only One Control Plane Is Active, Both Data Planes (Switch Fabrics) Are Active, and as Such, Each Can Actively Participate in the Forwarding of Data

Virtual Switch System (VSS) ArchitectureForwarding Operation


EtherChannel ConceptsMultichassis EtherChannel (MEC)

Regular EtherChannel on Single Chassis

Multichassis EtherChannel (MEC) Across Two VSL-Enabled Chassis

Virtual Switch Virtual Switch

LACP, PAGP, or ON EtherChannel Modes

Are Supported


One Physical DeviceMultiple Virtual Systems

(Dedicated Control and Data Path)

ACE Module: Virtual Partitioning System Separation for Server Load Balancing and SSL

Single configuration file Single routing table Limited RBAC Limited resource allocation

Distinct context configuration files Separate routing tables RBAC with contexts,

roles, domains Management and data

resource control Independent application rule sets Global administration and

monitoring Supports routed and bridged

contexts at the same time

25% 25% 20%15%15%100%

Cisco Application Infrastructure ControlTraditional Device


e.g., Three customers three security contextsscales up to 250 VLANs can be shared if needed (VLAN 10 on the right-hand side example) Each context has its own policies (NAT, access-lists, inspection engines, etc.) FWSM supports routed (Layer 3) or transparent (Layer 2) virtual firewalls at the

same time

Core/Internet

Cisco Catalyst 6500

FW SMVFW VFW VFW

MSFC

Core/Internet

Cisco Catalyst 6500

FW SMVFW VFW VFW

MSFC

VLAN 10 VLAN 20 VLAN 30

VLAN 11 VLAN 21 VLAN 31

VLAN 10

VLAN11 VLAN 21 VLAN 31

A B C A B C

Firewall Service Module (FWSM)Virtual Firewalls


Data Center Virtualized ServicesCombination Example

v5

v105

v6 v7

v107

v2081v2082v2083...

v206 v207

v206

BU-4BU-2 BU-3

v105

v108

BU-1

1

2

3

4

* vX = VLAN X**BU = Business Unit

VRF

VRF

VRFVRFVRF

v208

Front-End VRFs (MSFC)

Firewall Module Contexts

ACE Module Contexts

Back-End VRFs (MSFC)

Server Side VLANs

v207

3

4v8


Switch-1(VSS Active)

Switch-2(VSS Standby)

Virtual Switch System (VSS)

Data Plane Active

Control Plane Active

ACE Active

FWSM Standby

Data Plane Active

Control Plane Hot Standby

ACE Standby

FWSM active

VSL

Failover/State sync Vlan

VSS with ACE and FWSM ModulesActive / Standby Pair


Services can beattached using EtherChannelAppliance basedServices-chassis based

(standalone or VSS)

ASAACEAppliance

NAMAppliance

ServicesChassis

vPC

VSSNexus 7000 with vPC

Combining vPC with VSS for Services






VSSvPCServer Load BalancingSecurity Services




Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Top of Rack (ToR) Typically 1-RU servers 1-2 GE LOMs Mostly 1, sometimes 2 ToR switches Copper cabling stays within rack Low copper density in ToR Higher chance of East-West traffic

hitting aggregation layer Drives higher STP logical port count

for aggregation layer Denser server count

Middle of Row (MoR) (or End of Row) May be 1-RU or multi-RU servers Multiple GE or 10GE NICs Horizontal copper cabling for servers High copper cable density in MoR Larger portion of East-West traffic stays

in access Larger subnets less address waste Keeps agg. STP logical port count low

(more EtherChannels, fewer trunk ports) Lower # of network devices to manage

Data Center Access Layer Options


Catalyst 6500Catalyst 6500 Nexus 7000Nexus 7000

Many to 1 VirtualizationService Modules

Single Control Plane

1 to Many VirtualizationHigh Density (10/100/1000 & 10GE)

Distinct control planes while virtualized

VSS and MEC VDC and vPC

Middle of Row (MoR) (or End of Row)Virtual Switch (Nexus 7000 or Catalyst 6500)


Nexus 2000 combines benefits of both ToR and EoR architectures

Physically resides on the top of each rack but Logically acts like an end of row access device

Nexus 2000 deployment benefits Reduces cable runs Reduce management points Ensures feature consistency across hundreds of

servers

Enable Nexus 5000 to become a high density 1GE access layer switch

VN-Link capabilities

ToR @ 1GE: Nexus 2000, the Nexus 5000 virtual linecard


Nexus2000

Nexus 2000 (Fabric Extender - FEX)


Central Point of Management

FE4x 10G uplinksfrom each rack

Rack-1 Rack-2 Rack-3 Rack-4 Rack-N

AccessLayer

Servers

Nexus 5020

AggregationLayer

CoreLayer

L3L2

VSS

N2K

Rack-5

Nexus 5020

N2K N2K N2K N2K N2K

Nexus 2000 implementation examplePhysical Topology Logical Topology

Central Point of Management

AccessLayer

Servers

AggregationLayer

CoreLayer

L3L2

VSS

Nexus 5020Nexus 5020

12 x Nexus 2000

Rack-1 Rack-N Rack-1 Rack-N

12 x Nexus 2000


Blades: Cisco Virtual Blade Switching (VBS) Up to 8 Switches acts as Single VBS Switch

Distributed L2/ MAC learningCentralized L3 learning

Each switch consists ofSwitch FabricPort Asics (downlink & uplink ports)

One Master Switch per VBS1:N Resiliency for MasterL2/L3 reconvergence is sub 200 msec

High Speed VBS Cable (64 Gbps)

Example Deployment:16 servers per enclosure X

2 GE ports per server X4 enclosures per rack = 128GE

2 x 10GE uplinks = 20GE 128GE / 20GE = 6.4:1 oversubscription


Aggregation LayerAccess Layer (Virtual Blade Switch)

Single Switch / Node (for Spanning Tree or Layer 3 or Management)

Spanning-Tree Blocking

Cisco Catalyst Virtual Blade Switch (VBS)with Non-vPC Aggregation


Aggregation Layer

Access Layer (Virtual Blade Switch)


Spanning-Tree Blocking

Cisco Catalyst Virtual Blade Switch (VBS)with Non-vPC Aggregation


Aggregation LayerNexus vPC



All Links Forwarding

Cisco Catalyst Virtual Blade Switch (VBS) with Nexus vPC Aggregation


Aggregation Layer (Nexus vPC)



All Links Forwarding

Cisco Catalyst Virtual Blade Switch (VBS) with Nexus vPC Aggregation










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Microsoft Hyper-V: Networking configuration

Virtual Switch ConfigurationParent Partition LAN Configuration


Hyper-V Setup: Networking & iSCSI


Windows Server 2008

VM 2VM 1

Designed for Windows Server Hardware

Windows hypervisor

VM 3

Parent Partition Child Partitions

User Mode

KernelMode

Ring -1MgmtNIC 1MgmtNIC 1

iSCSI NIC 2

iSCSI NIC 2

VSPVSPVSPVSP

VSwitch 1NIC 3

VSwitch 1NIC 3

VSwitch 2NIC 4

VSwitch 2NIC 4

Applications Applications Applications

VM ServiceVM ServiceWMI ProviderWMI Provider

VM Worker Processes

Windows Kernel VSC Windows Kernel VSC

LinuxKernel VSC

VMBusVMBus VMBusVMBus VMBusVMBusVMBusVMBus

Hyper-V Setup: Networking & iSCSI


dom0

OracleVMAgent

domU domU domU

Hypervisor

Host Hardware CPU, Network, Memory, Disk

Java-based Management Server

Web Browser-based User Interface

Oracle VM Server

Oracle VM Manager

Oracle VMManager

Repository

Oracle VM / Xen Architecture


Xen Virtual Ethernet interfaces

Xen creates, by default, seven (07) pair of "connected virtual ethernetinterfaces" for use by dom0.

Think of them as two ethernetinterfaces connected by an internal crossover ethernet cable.

veth0 is connected to vif0.0, veth1 is connected to vif0.1, etc., up to veth7 -> vif0.7.

source: wikipedia

Xen networking (1/2)


Xen networking (2/2)Every time a domU instance is created, it is assigned a new domain id number. The first domU will be id #1, the second will be #2, etc.

For each new domU, Xen creates a new pair of "connected virtual ethernet interfaces", with one end in domU and the other in dom0.

For example, domU #5's eth0 is attached to vif5.0.

The default Xen configuration uses bridging within domain 0 to allow all domains to appear on the network as individual hosts.

source: wikipedia


VMs

vmnic0

vmnic1

vNIC

vNICVirtual Ports

VM_LUN_0007

VM_LUN_0005

vSwitch0

vSwitch

VMNICS =Uplinks

Per ESX Server Configuration

VMware ESX 3.x Networking Components


VM1 VM2 ServiceConsole

VMkernel

VMkernelNIC VSwitch A VSwitch B

ESXServer

PhysicalSwitches

Physical NICs

Virtual NICs

XNo Loop

XNo LoopIn ESXWithout a bridging VM

XNo TrunkBetween vSwitch

How is it like a switch:

- MAC addrforwarding VLAN segmentation

How is it different:- No need to learn

MAC addresses it knows the address of the connecting vNICs

- No participation in spanning tree

ESX vSwitch Overview Software implementation of an Ethernet switch


VN-Link (or Virtual Network Link) is a term which describes a new set of features and capabilities that enable VM interfaces to beindividually identified, configured, monitored, migrated and diagnosed.

VN-Link requires platform support for Port Profiles, Virtual Ethernet Interfaces, vCenter Integration, and Virtual Ethernet mobility.

Cisco VN-Link

The term literally refers to a VM specific link that is created between the VM and Cisco switch. It is the logical equivalent & combination of a NIC, a Cisco switch interface and the RJ-45 patch cable that hooks them together.

HypervisorVNIC VNIC

VETH VETH


VN-Link With the Cisco Nexus 1000V

Cisco Nexus 1000VSoftware Based

VMW ESX

VM#1

VM #4

VM #3

ServerVM #2

Nexus 1000V

NIC NIC

LAN

Nexus1000V

Industrys first third-party ESX switch Built on Cisco NX-OS Compatible with switching platforms Maintain vCenter provisioning model

unmodified for server administration but also allow network administration of Nexus 1000V via familiar Cisco NX-OS CLI

Policy-Based VM Connectivity

Non-DisruptiveOperational Model

Mobility of Network and Security Properties

Announced 09/2008

Shipping 1H09(ESX 4)


Policy-Based VM Connectivity

Non-DisruptiveOperational Model

Mobility of Network and Security Properties

Nexus Switch with VN-LinkHardware Based

Allows scalable hardware-based implementations through hardware switches

Standards-based initiative: Cisco & VMware proposal in IEEE 802 to specify Network Interface Virtualization

Combines VM and physical network operations into one managed node

Future availability

VMW ESX

VM #4

VM #3

ServerVM #2

VM #1

VN-Link

Nexus

http://www.ieee802.org/1/files/public/docs2008/new-dcb-pelissier-NIC-Virtualization-0908.pdf

VN-Link with Network Interface Virtualization (NIV)


VMW ESXVMW ESXVMW ESX

Server 2Server 2


Server 1Server 1

Cisco Nexus 1000V Industry First 3rd Party Distributed Virtual Switch

VM #5VM VM #5#5

VM #8VM VM #8#8

VM #7VM VM #7#7

VM #6VM VM #6#6

VM #4VM VM #4#4

VM #3VM VM #3#3

VM #2VM VM #2#2

VM #1VM VM #1#1

VMware vSwitchVMware VMware vSwitchvSwitch VMware vSwitchVMware VMware vSwitchvSwitchNexus 1000VNexus 1000VNexus 1000VNexus 1000VNexus 1000VNexus 1000V Nexus 1000V DVSNexus 1000V DVSNexus 1000V DVS

VM #8VM VM #8#8

VM #7VM VM #7#7

VM #6VM VM #6#6

VM #4VM VM #4#4

VM #3VM VM #3#3

VM #2VM VM #2#2

VM #5VM VM #5#5

Nexus 1000V provides enhanced VM switching for VMware ESX

Features Cisco VN-Link:Policy Based VM ConnectivityMobility of Network & Security

PropertiesNon-Disruptive Operational

Model

Ensures proper visibility & connectivity during VMotion

Enabling Acceleration of Server Virtualization Benefits

VM #1VM VM #1#1


Virtual Supervisor Module (VSM) Virtual or Physical appliance

running Cisco OS (supports HA) Performs management,

monitoring, & configuration Tight integration with VMware

vCenter

Cisco Nexus 1000V Architecture

vCentervCenter


Server 1Server 1

VMware vSwitchVMware VMware vSwitchvSwitchVMW ESXVMW ESXVMW ESX

Server 2Server 2

VMware vSwitchVMware VMware vSwitchvSwitchVMW ESXVMW ESXVMW ESX

Server 3Server 3

VMware vSwitchVMware VMware vSwitchvSwitch

VM #1VM VM #1#1

VM #4VM VM #4#4

VM #3VM VM #3#3

VM #2VM VM #2#2

VM #5VM VM #5#5

VM #8VM VM #8#8

VM #7VM VM #7#7

VM #6VM VM #6#6

VM #9VM VM #9#9

VM #12VM VM #12#12

VM #11VM VM #11#11

VM #10VM VM #10#10

Nexus 1000V

VSM

Nexus 1000VNexus 1000V

VSMVSM

VEMVEMVEM VEMVEMVEM VEMVEMVEMNexus 1000V DVSNexus 1000V DVSNexus 1000V DVS

Virtual Ethernet Module (VEM) Enables advanced networking

capability on the hypervisor

Provides each VM with dedicated switch port

Collection of VEMs = 1 DVS

Cisco Nexus 1000V Enables: Policy Based VM Connectivity Mobility of Network & Security

Properties Non-Disruptive Operational Model


1. Nexus 1000V automatically enables port groups in vCenter via API

2. Server Admin uses vCenter to assign vnic policy from available port groups

3. Nexus 1000V automatically enables VM connectivity at VM power-on

1.

VMW ESX

Server 1

Nexus 1000V - VEM

VM #1

VM #4

VM #3

VM #2

Available Port Groups

WEB Apps HR

DB Compliance

2.

Nexus 1000V

VSMvCenter

3. WEB Apps: PVLAN 108, Isolated Security Policy = Port 80 and 443 Rate Limit = 100 Mbps QoS Priority = Medium Remote Port Mirror = Yes

Example: Port Profiles Propagation


Example: Port Profile (Nexus 1000v VSM view)


Example: Port Profile (vCenter View)


Example: Port Profile (VM View)










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Virtual Servers

Virtual Fabrics / Unified IO

Virtual Storage

Virtual HBAsFCoE CNA

Pools of storage resources

V

i

r

t

u

a

l

i

z

a

t

i

o

n

VHVH

VH

BackupVSAN

EmailVSAN

OLTPVSAN

Optimizes resource utilization

Increases flexibility and agility

Simplifies management

Reduces TCO

End-to-End Back-End Virtualization


Virtual Storage Area Network (VSAN) Deployment

Consolidation of SAN islandsIncreased utilization of fabric ports with just-in-time provisioning

Deployment of large fabricsDividing a large fabric in smaller VSANsDisruptive events isolated per VSANRBAC for administrative tasksZoning is independent per VSAN

Advanced traffic managementDefining the paths for each VSANVSANs may share the same EISLCost effective on WAN links

Resilient SAN extension Standard solution

(ANSI T11 FC-FS-2 section 10)

SAN Islands

Department A

Department B Department C

Virtual SANs (VSANs)

Department A

Department B

Department C


N-Port ID Virtualization (NPIV) Mechanism to assign

multiple N_Port_IDs to a single N_Port

Allows all the access control, zoning, port security (PSM) be implemented on application level

Multiple N_Port_IDs are so far allocated in the same VSAN

Application Server

File Services

N_PortID-3

Web

N_PortID-2

E-Mail

N_PortID-1

F_PortF_Port F_Port

E-MailVSAN_3

WebVSAN_2

File and PrintVSAN_1

E_Port

E_Port


NPIV Usage ExamplesIntelligent Pass-ThruVirtual Machine Aggregation

FC FC FC FC

NP_Port

F_PortF_Port

FC FC FC FC

FC

NPIV-Enabled HBA

NPV Edge Switch


FC

Storage Array(LUN Mapping and Masking)MDS9000

Zone FC Name Server

pWWN-P

Single Login on a Single Point-to-Point Connection

Virtual Servers Share a Physical HBA A zone includes the physical HBA

and the storage array Access control is demanded to storage

array LUN masking and mapping, it is based on the physical HBA pWWN and it is the same for all VMs

The hypervisor is in charge of the mapping, errors may be disastrous

H

W

H

y

p

e

r

v

i

s

o

r

V

i

r

t

u

a

l

S

e

r

v

e

r

s

pWWN-P

Mapping

FC


H

W

H

y

p

e

r

v

i

s

o

r

V

i

r

t

u

a

l

S

e

r

v

e

r

s

pWWN-P

Mapping Mapping Mapping Mapping

FC FC FC FC

FC

Storage ArrayMDS9000

Virtual Server Using NPIV and Storage Device Mapping

Virtual HBAs can be zoned individually LUN masking and mapping is based on

the virtual HBA pWWN of each VMs Very safe with respect to

configuration errors Only supports RDM Available in ESX 3.5

pWWN-PpWWN-1pWWN-2pWWN-3pWWN-4

Multiple Logins on a Single Point-to-Point Connection FC Name Server

pWWN-1 pWWN-2 pWWN-3 pWWN-4To pWWN-1

To pWWN-2

To pWWN-3

To pWWN-4FC


Standard HBAs

VM1 VM2 VM3 VM2

All LUNs must be exposed to every server to ensure disk access during live migration (single zone)

VM1 VM3 VM3VM1 VM2

All configuration parameters are based on the World Wide Port Name (WWPN) of the physical HBA

WWPN

ST ATUS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

WS-X9 016

1/2 Gbps FC Module

FCFC

VMotion LUN Migration without NPIV


HBAswith NPIV

VM1 VM2 VM3

Centralized management of VMs and resourcesRedeploy VMs and support live migration

No need to reconfigure zoning or LUN maskingDynamically reprovision VMswithout impact to existing infrastructure

WWPN1WWPN2WWPN3

FCFC

ST ATUS

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

WS-X9 016

1/2 Gbps FC Module

VMotion LUN Migration with NPIV


Blade Switch/Top-of-Rack Domain ID Explosion

Domain ID used for addressing, routing, and access control

One domain ID per SAN switch

Theoretically 239 domain ID, practically much less supported

Limits SAN fabric scalability

Tier 1 Tier 2 Tape Farm

Blade Switches Increase Domain IDs, Increase Fabrics

MDS9500

Theoretical Maximum: 239 Domain IDs per SAN

Blade Switch


Cisco MDS Network Port Virtualization (NPV) Eliminates edge switch

Domain ID Edge switch acts as an

NPIV host Simplifies server and

SAN management and operations

Increases fabric scalability

Tier 1 Tier 2 Tape Farm

NPV-Enabled Switches Do Not Use Domain IDs

Supports Up to 100 Edge Switches

MDS9500

Edge Switch Acts as a NPIV Host

NPV NPV

Blade Switch

NPV NPV NPV NPV


FC1/1vPWWN1

Before

After

Flex Attach (Virtual PWWN) Assign virtual PWWN

on NPV switch port Zone vPWWN to storage LUN masking is done

on vPWWN Reduce operational overhead

Enables server or physical HBA replacementNo need for zoning modificationNo LUN masking change

Automatic link to new PWWN No manual relinking to new PWWN is needed

PWWN1

FC1/1vPWWN1

PWWN2

pwwn1 pwwnX vpwwn1 pwwnX

pwwn2 pwwnX vpwwn1 pwwnX


Enhanced Blade Switch Resiliency F-Port PortChannels

Bundle multiple ports in to 1 logical linkAny port, any module

High-Availability (HA)Blade Servers are transparent if a cable, port, or line cards fails

Traffic ManagementHigher aggregate bandwidthHardware-based load balancing

F-Port Trunking for Blade SwitchPartition F-Port to carry traffic for multiple VSANs

Extend VSAN benefits to BladeServers

Separate management domainsSeparate fault isolation domainsDifferentiated services: QoS, Security

Storage

B

l

a

d

e

S

y

s

t

e

m

F-Port Port Channel

Blade 1

Blade 2

Blade N

F-Port Port Channel

F-PortN-Port

Core Director

NPV

SAN

Storage

B

l

a

d

e

S

y

s

t

e

m

F-Port Trunking

Blade 1

Blade 2

Blade N

Core Director

VSAN 1

VSAN 2

VSAN 3

F-Port Trunking

NPV

F-PortN-Port

SAN

SAN-OS

4.1










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


FC TrafficFC HBAFC HBA

Unified I/O (FCoE) Fewer CNAs (Converged Network adapters) instead of

NICs, HBAs and HCAs Limited number of interfaces for Blade Servers

All traffic goes over

10GE

CNACNA

CNACNA

FC TrafficFC HBAFC HBA

NICNIC LAN Traffic

NICNIC LAN Traffic

NICNIC Mgmt Traffic

NICNIC Backup Traffic

IPC TrafficHCAHCA


Fiber Channel over Ethernet (FCoE):How It Works

Direct mapping of fiber channel over Ethernet

Leverages standards-based extensions to Ethernet to provide reliable I/O delivery

Priority flow control (PFC)Data Center Bridging Capability eXchange Protocol (DCBX)

MACPHY

FCoE Mapping

FC-0

FC-1

FC-2

FC-3

FC-4

FC-2

FC-3

FC-4

FC Frame

Ethernet Header

Ethernet Payload

Ethernet FCS

S

O

F

E

O

F

C

R

C

(a) Protocol Layers (b) Frame Encapsulation

10GE LosslessEthernetLink

FCoE Traffic

Other NetworkingTraffic


FCoE Enablers

10 Gbps Ethernet Lossless Ethernet

Matches the lossless behavior guaranteed in FC by B2B credits

Ethernet jumbo framesMax FC frame payload = 2112 bytes

E

t

h

e

r

n

e

t

H

e

a

d

e

r

F

C

o

E

H

e

a

d

e

r

F

C

H

e

a

d

e

r

FC Payload

C

R

C

E

O

F

F

C

S

Same as a Physical FC Frame

Control Information: Version, Ordered Sets (SOF, EOF)

Normal Ethernet Frame, Ethertype = FCoE


Priority Flow ControlPriority Flow Control

Enables lossless Fabrics for each class of service

PAUSE sent per virtual lane when buffers limit exceeded

Transmit QueuesEthernet Link

Receive Buffers

EightVirtualLanes

OneOne OneOne

TwoTwo TwoTwo

ThreeThree ThreeThree

FourFour FourFour

FiveFive FiveFive

SevenSeven SevenSeven

EightEight EightEight

SixSix SixSixSTOP PAUSE

Data Center BridgingCapability eXchange Protocol

Data Center BridgingCapability eXchange Protocol

Handshaking Negotiation for:CoS BW ManagementPriority Flow Control (PFC)Congestion Management (BCN/QCN)Application (user_priority usage)Logical Link Down

Nexus5000

LossLess Ethernet >> Data Center Ethernet (DCE) PFC & DCBX


SAN BSAN ALAN

FCoEEthernetFC

Today

Unified I/O Use Case

Management


SAN BSAN ALAN

FCoEEthernetFC

Unified I/O Use Case

Unified I/O Reduction of server adapters Fewer cables Simplification of access

layer and cabling Gateway-free implementation

fits in installed base of existing LAN and SAN

L2 multipathing accessdistribution

Lower TCO Investment protection

(LANs and SANs) Consistent operational model One set of ToR switches

Unified I/O

FCoESwitch

Management


Converged Network Adapters (CNA)


CNAs: View from Operating System

Standard drivers Same management Operating system sees:

2 x 10 Gigabit Ethernet adapter2 x 4 Gbps fiber channel HBAs


Fabric AFabric A Fabric BFabric B

SAN Fabric

L2

L3

L3

L2

CoreCore

AggregationAggregation

AccessAccess

A B ED

Storage Storage ArraysArrays

Enet

FC

FCoE

CNALAN Access

SAN EdgeB

SAN EdgeA

N7KN7K N7KN7K

N5KN5K N5KN5K N5KN5K N5KN5K

N7KN7K N7KN7K

MDS9500MDS9500 MDS9500MDS9500 MDS9500MDS9500 MDS9500MDS9500

Unified IO Server Farm Pod EnvironmentConverged Edge Infrastructure: Unified/IO using ToR at the edge, and CNA at the hostsToR 10GE Unified/IO Server EnvironmentsLeverage Ethernet and Storage Clouds to reach traditional LAN/SAN services

C6KC6K C6KC6K

VF_Ports

VN_Ports

Unified IO Deployment - Unified IO POD



SAN Fabric

L2

L3

L3

L2

CoreCore


AccessAccess

A B ED


Enet

FC

FCoE

CNALAN Access

SAN EdgeB

SAN EdgeA

N7KN7K N7KN7K

N5KN5K N5KN5K N5KN5K N5KN5K


Unified IO Server Farm using vPC at Aggregation LAN cloudAccess Switches remain as single logical instanceStorage connectivity is unchanged

C6KC6K C6KC6K

4 4

4

N7KN7K

4 44

4

Unified IO Farm - Phase 1: vPC @ Aggregation



SAN Fabric

L2

L3

L3

L2

CoreCore


AccessAccess

A B ED


Enet

FC

FCoE

CNALAN Access

SAN EdgeB

SAN EdgeA

N7KN7K N7KN7K

N5KsN5Ks N5KsN5Ks


Unified IO Server Farm using vPC at Aggregation LAN cloudAccess Switches provide vPC for LAN connectivityStorage connectivity is unchanged (different physical paths for SAN Fabric A and B)

C6KC6K C6KC6K

4 4

4

N7KsN7Ks

8 8

Unified IO Farm - Phase 2: vPC @ Aggregation and Access










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Storage Volume Virtualization

Adding more storage requires administrative changes Administrative overhead, prone to errors Complex coordination of data movement between arrays

Target

SANFabric

Initiator

Initiator Target


SANFabric

Storage Volume Virtualization

A SCSI operation from the host is mapped in one or more SCSI operations to the SAN-attached storage

Zoning connects real initiator and virtual target or virtual initiator and real storage

Works across heterogeneous arrays

Virtual Volume2

Virtual Target 1VSAN_10

Virtual Volume1


Virtual Initiator VSAN_30


Initiator VSAN_20

Initiator VSAN_10


Sample Use: Seamless Data Mobility

Works across heterogeneous arrays Nondisruptive to application host Can be utilized for end-of-lease storage migration Movement of data from one tier class to another tier

Tier_2 ArraySANFabric

Virtual Volume2


Virtual Volume1




Initiator VSAN_20

Initiator VSAN_10 Tier_2 Array










Q&A

F

r

o

n

t

-

E

n

d


Virtual Storage



Virtual Machines







VSSVLAN VRF VPNsVDC

vHBAVSANs FCoECNA

B

a

c

k

-

E

n

d


Meet The Expert

To make the most of your time at Cisco Networkers 2009, schedule a Face-to-Face Meeting with a top Cisco Expert.

Designed to provide a "big picture" perspective as well as "in-depth" technology discussions, these face-to-face meetings will provide fascinating dialogue and a wealth of valuable insights and ideas.

Visit the Meeting Centre reception desk located in the Meeting Centre in World of Solutions


Source: Cisco Press

Recommended ReadingBRKDCT-3831

Documents

BRKDCT-3831