Upload
buituong
View
284
Download
9
Embed Size (px)
Citation preview
Designing MPLS in Next Generation Data Center: A Case Study BRKMPL-2108
Khalil Jabr – Distinguished Engineer [email protected]
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Session Goals
At the end of the session, the participants should:
Understand the design requirements
Understand the technical building blocks
Understand the selected design and reasoning behind it
Understand the lessons learned
3
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Agenda
Design Requirements
Technology Involved
– The Fabric
– The Service Layer
– The WAN Connectivity
Design Options
Selected Design
Lessons Learned
4
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
5
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Strategy
Virtual FW/LB per tenant
Flexible placement
Incremental capacity
Multi-tenancy
Security and
Separation
Traffic Eng
Scalable
Network
Topology
(DC & WAN)
Virtualized
L4-7
Services
Network Virtualization
Flexible topology
Minimize oversubscription
Scale out and scale up
No spanning tree
Incremental scale
6
Background Info: The Building Blocks
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
8
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Virtual Network
Cust2
Network Virtualization
Giving one physical network the ability to support multiple virtual networks
Customer requirement: 60-80 virtual networks
Separation between:
Line of business
Customers
App layers
9
Actual Physical Infrastructure
Alpha Network
Virtual Network Virtual Network
Cust2
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Provider Edge VRFs (PE VRFs)
802.1q
VRF
VRF
VRF
VPN LSP/Tunnel Logical or Physical
Int (Layer 3)
MPLS/Tunnel Labels
and Route Targets
PE Router
IP Switching Label Switching (MPLS) or
Tunneling (L2TPv3)
10
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS in Data Center
11
Business drivers Business Goal Solution
L3
VPN
MVPN TE 6PE /
6VPE
Multi-tenant hosting
Mergers / acquisitions
Network consolidation
Shared services
Compliance
Segmentation
Bandwidth provisioning Capacity
Planning
High Availability
Path Diversity
Ensure SLAs
IPv4 address depletion
IPv6 readiness / migration
Expansion
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS and VDCs
Secure and flexible way of software process partitioning
All MPLS features are VDC aware
Each VDC operates as separate MPLS router (LER / LSR):
No internal communication between VDCs
Multiple logical P / PE routers can be configured
Each VDC has independent label space for prefix labels: LDP, VPN, TE
Note: per-VRF / aggregate VPN labels - globally significant for whole chassis, all others are locally significant to VDC
12
VDC 1
VDC 2
VDC 3
VDC 4
Kernel
Infrastructure
VDC 8
…
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS and VDCs
Vertical consolidation – collapse layers of P/PE routers
Horizontal consolidation – collapse PE’s from several PODs
13
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
P1
(VDC 3)
PE7
(VDC 2)
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
PE5
(VDC 4)
P2
(VDC 3)
PE8
(VDC 2)
PE6
(VDC 4)
PE1
(VDC 2)
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
Serv
er
PE3
(VDC 3)
PE2
(VDC 2)
PE4
(VDC 3)
MPLS
Core
POD 1 POD 2
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
14
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Number of Spine Switches
Need f
or
HA
Spine-Leaf DesignChanges to the Approach to Structured High Availability
Spine
Leaf
15
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
DC Fabric w/FabricPath
Externally the Fabric looks like a single switch
Internally, ISIS adds Fabric-wide intelligence and ties the elements together.
Provides in a plug-and-play fashion: Optimal, low latency connectivity any to any
High bandwidth, high resiliency
Open management and troubleshooting
ISIS for multipathing and reachability
FabricPath FabricPath
16
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Layer 2 with FabricPath
Allows extending vlans with no limitation (no risks of loop)
Devices can be attached active/active to the fabric using IEEE standard port channels using LACP and without resorting to STP
17
FabricPath
A
s3 s8 s7
B
s5
VLAN X VLAN Y VLAN Z
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Edge Device Integration
Hosts see a single default gateway
The fabric provide them transparently with multiple simultaneously active default gateways
Allows extending the multipathing from the inside to the fabric to the L3 domain outside the fabric
Equal Cost Multipathing (ECMP) from the upstream network to the servers
18
Hosts can leverage multiple L3 default gateways
FabricPath
A
s3
dg dg L3
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Split VLANs
Some polarization
Inter-VLAN traffic can
be suboptimal
VLAN
100-200 VLAN 300-
400
FabricPath
GLBP
Host is pinned to a
single gateway
Less granular load
balancing
VLAN 100-400
FabricPath
Anycast HSRP
All active
Available in NX-OS 6.2
release
VLAN 100-400
FabricPath
spine spine
Default Gateway Options
19
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Cisco Data Center Fabric
FabricPath
Spine Layer
Default Gateway
Elastic East/West
Performance
Low Latency Security
Multipathing
Any VLAN Anywhere
Efficient Performance
Open
20
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Hardware Support
All MPLS functionality supported on all M-series I/O modules
F1 and F2e* modules support FabricPath and
MPLS via proxy-mode
21
* - Software support in NX-OS 6.2
N7K-M108X2-
12L
N7K-M148GS-11
N7K-M148GS-11L N7K-M148GT-11
N7K-M148GT-11L
N7K-M132XP-12
N7K-M132XP-12L
N7K-M224XP-23L
N7K-M202CF-22L
N7K-M206FQ-23L
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS Layer 3 VPNs - I/O modules
F1 / F2e* I/O modules – mixed chassis design with M I/O modules
F2 / F2e I/O modules – separate VDC design with VRF-lite
22
L3
L2
MPLS
VR
F-L
ite
M M M M
M M
F2/F2e F2/F2e
F2/F2e
F2/F2e F2/F2e F2/F2e F2/F2e
* - NX-OS 6.2
Required
L3
L2 F1/F2e/M F1/
F2e
M
MPLS
M M
F1/
F2e
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
F2e-M for FabricPath + MPLS Designs
F2e + M2 (or M1-XL) at the aggregation in the same VDC *
Layer 3 routing performed by the M-Series. F2e in Layer 2 mode
FabricPath towards the Access
MPLS towards the Core
MPLS
FabricPath
* Requires NX-OS 6.2
23
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
24
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Firewall Options
Three options to consider
– Virtual Firewall (ASA 1000v) and VSG Virtualized services
High scale
Leverages vPath technology
– IOS Zone Based Firewall Router based
Native routing
– ASA Purpose built hardware
Advanced firewall and security features
Next slides explore the ASA and ZBFW options
25
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
ASA FW + Fusion Router
Fusion router:
– Inter-VPN connectivity
– Shared resource connectivity
Internet, servers, etc.
ASA contexts:
– VPN isolation / protection
– Per VPN policies: ACL, NAT …
– 256 contexts per FW
– Map to VLANs
I-Net
FW Contexts
Shared Services
VPN A
VPN B
VPN C
VPN D
Fusion VDC
Context functionality available on the ASA
26
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Virtual Firewall per VRF
VDC or VRF Sandwich Design
Virtual firewalls assigned to VRF by VLAN association
One pair of physical or virtual firewall per VRF
Each firewall requires two VLANs; inside and outside
Firewall in transparent or routed mode
Can be made simpler by delegating default gateway functionality to the firewall
VDC-Agg
Active/Standby
VDC-Sub-Agg
VDC-Agg
VDC-Sub-Agg
VRF A
VRF B VRF C
27
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Sample VRF w/ASA
Default Gateway
28
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Traffic Flow w/Server Load Balancer
VDC-Agg
VDC-Sub-Agg
VLAN 5
VLAN 12
VLAN 10
Transparent
802.1q trunk
VLAN 4
VLAN 13
Transparent
VLAN 11
S-NAT
VDC-Agg
VDC-Sub-Agg
VLAN 5
VLAN 12
VLAN 10
Transparent
802.1q trunk
VLAN 4
VLAN 13
Transparent
VLAN 11
Source NAT
from LB to
guarantee
symmetry
Bypass LB
when
needed
29
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
ASA Option: ASA Cluster Solution
Cluster up to 8 ASA appliances
• One unit is designated as master and rest are slaves
• All of them have a dedicated interface for Cluster Control Link (CCL)
• Keepalive/CP/DP messages, forwarded traffic and RPC are sent over CCL
Load balancing approach
• Stateless load balancing by external switch(EtherChannel) or router(Policy Based Routing, Equal-Cost Multi-Path Routing)
• Conn-rebalance between cluster units over CCL
Clu
ste
r C
ontr
ol Lin
k
30
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
ASA Option: Solution Overview (cont)
Fully distributed data-path so there is no single point of failure
• All units are active
Coordinated data-path to locate the owner and forward packets through the cluster to accomplish stateful firewall inspection
• Multiple roles for a connection in data-path among units (owner, director, forwarder etc.)
• Consistent hashing algorithm to redirect packets within cluster
In-Cluster High Availability
• Connection state sharing/backup between units
• N + 1 redundancy
• Hitless upgrade
Centralized management and monitoring
31
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Zone Based Firewall w/ASR1000
Hardware Based Performance
IOS Based
Zone-pair
VRF-aware
Fusion VRF (Gray VRF in later slides)
Native MPLS Connectivity
Per Zone Firewall Policy
I-Net
FW Zone-Pairs
Shared Services
VPN A
VPN B
VPN C
VPN D
Fusion VRF
VRF-Aware ZBFW
Internal Link
32
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Cisco ASR1000: VASI Feature
A point to point virtual link
Internal to the router
Connects two VRFs together
Allows for direct peering (IGP/BGP)
Allows for ACLs, NAT, WCCP etc
VRF aware firewall applied prior to traversing the virtual link
33
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
34
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Extend VPN services over multiple independently managed MPLS domains
Fast geographic service coverage expansion
Two MPLS VPN Providers peering to cover for a common customer base
Build MPLS VPN networks on original multi-domain network
IGP isolation with service continuity
Interconnect BGP confederations with different IGPs in the same AS
Two available as described in RFC 4364 : 1.Carrier Supporting Carrier (CSC)
2.Inter-Autonomous Systems (I-AS)
MPLS Inter-AS Use Cases
AS3 DC2
WAN Core (AS2)
AS1 DC1
Cust1 Cust1
Cust2 Cust2
35
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Extending MPLS with Inter-AS
VPN-R1 VPN-R2
PE22
CE2 CE1
AS #1
MPLS AS #2
MPLS
PE11
MP-eBGP for VPNv4
(Option B)
Multihop MP-eBGP
between RRs
(Option C)
MP-eBGP+Labels
Back-to-Back VRFs
(Option A) ASBR1 ASBR2
Option C: Most interesting since we offload the VPN routes from ASBRs
36
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Requirements – NG Data Center
Multi-tenancy – Public SaaS
Highly Scalable DC Architecture
L2 Connectivity Between Racks
Optimized for East/West as well as North/South
Minimize Oversubscription
Scalable L4-7 Service Layer
Highly available WAN
Scalable WAN Architecture
Some DCs connect via Internet
Simplicity!
37
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Deployment & Implementation Scenarios
P-to-P tunneling (MPLS MPLS)
IP WAN Transport
IPSEC Option for security
P to P Tunnel
Looks like an MPLS Link
Drawbacks:
– Cumbersome with multiple sites (MPLSoMGRE is an alternate solution)
– MTU
IGP Label
VPN Label
IP Payload
GRE Header
IGP Label’
VPN Label
IP Payload
MPLSoGRE
IP NetworkMPLSDC1
MPLSDC2
PE1 PE2P1 P2
IGP Label
VPN Label
IP Payload
38
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Pulling the Building Blocks Together
39
MPLS
FabricPath
Firewalls
WAN
Design Options
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS Layer 3 VPN – Multi-POD
Requirement:
Secure Segmentation for Hosted / Enterprise Data Centers or Campus networks via MPLS VPNs
Solution:
One MPLS network infrastructure for all services
MPLS PE boundary in POD EoR/ToR access/ aggregation layer
Below MPLS boundary: L2 or L3 (VRF-lite with PE-CE)
Direct PE-PE or PE-P-PE networks
Scaling POD architecture without operational overhead using Fabric Extenders
POD POD POD
L3 L2
Internet
Global Interconnect Campus
/WAN Edge
MP
LS
Laye
r-2
41
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Starting Point
42
Default Gateway
FabricPath
Spine Layer (N7k)F2e
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Design Option Leveraging FabricPath Zone Based Design
43
• Segmentation by separating default gateway
• Each segment considered a Zone
• Each Zone has unique FWs and LBs
• Can leverage VDCs
• Simple
Default GatewayZone1
FabricPath
Spine Layer (N7k)FabricPath Only
CORE
Default GatewayZone2
LB
LB
vPC or FP
LB
LB
vPC or FP
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Design: Firewall Placement w/Virtualization
44
Default Gateway
FabricPath
Spine Layer (N7k)
LB
LB
COREOption1 Option2
Default Gateway
FabricPath
Spine Layer (N7k)F2e
MPLS
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option1: Traffic Flow
45
Default Gateway
FabricPath
Spine Layer (N7k)
LB
LB
CORE
Default Gateway
FabricPath
Spine Layer (N7k)
LB
LB
CORE
Intra VRF
Inter VRF
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option1: Solution w/ASA Cluster
46
Default Gateway
FabricPath
Spine Layer (N7k)
LB
LB
CORE
Inside Outside
• Use ASA cluster for firewalling
• One ASA context per virtual segment
• Scale up by growing ASA cluster and add additional clusters
• VRF or VDC sandwich design
• Core layer is simple. No VRFs.
• Traffic symmetry is automatically handled by ASA cluster
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: Traffic Flow
47
Default Gateway
FabricPath
Spine Layer (N7k)F2e
MPLS
Default Gateway
FabricPath
Spine Layer (N7k)F2e
MPLS
Intra VRF
Inter VRF
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Comparing Options
Option1: ASA Firewall
– Scales up by way of distributing customers to firewalls and leveraging clusters
– Stateful HA
– Purpose built hardware
– Management tools
– Inter-VRF traffic flow leverages spine layer
Option2: ASR1k ZBFW:
– MPLS attached
– Additional services like NAT and WCCP
– Hardware forwarding
– No concerns about trunking VLANs
There is absolutely nothing wrong with going with either option. The choice is dependent on many factors such as requirements, comfort level with product, management and operations etc.
48
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: Zone Based Firewall (ZBFW)
49
QFP
MPLS
QFPZBFW ZBFW• ASR1k Hardware Performance
• Native MPLS Attachment
• VRF-Aware
• Attach Anywhere with MPLS Reachability
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: ZBFW w/VASI Details
50
GrayVRF
VRFs 100-199
vasi
MPLS
LDP
vasi
BGP or OSPFOver VASI
ASR1k
Per-VRF Security Policy Applied Before
Traversing VASI
• Native MPLS termination
• Gray VRF interconnects tenant VRFs
• Leverage VASI
• Each ‘tenant’ gets a security policy zone-pair
• NAT possible and WCCP Possible on VASI
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: Firewall Design w/Zone Based Firewall
51
• Redundancy by way of routing
• Active/Standby
• Leverage metrics
• Limiting factors:
• Throughput
• Number of connections
• Number of conn/sec
GrayVRF
GrayVRF
VRFs 100-199 VRFs 100-1
99
vasi
MPLS
vasi
Services VRF
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: Firewall Design w/Zone Based Firewall
52
• Per-VRF loadbalancing
• N+1 redundancy
• Very scalable design
• Grow as you go
• Scalability is additive
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Option2: Firewall Design w/Zone Based Firewall
53
• Second Gray VRF for further segmentation
• Same logic as before
• Per-vrf loadbalancing
• Grow as you go
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Inter-DC Flow Connectivity
54
Inter-DC: Same VRF
WAN Core
DC1 DC2
Cust1_VRF1 Cust1_VRF1
• Symmetric traffic flow is critical
Inter-DC: Different VRFs
WAN CoreFW-DC1
FW-DC2
DC1 DC2
Gray VRF Gray VRF
Cust1_VRF1 Cust1_VRF2
Supernet or Defaultr oute
Supernet or Defaultr oute
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Customer Selected
55
• Spine/leaf architecture
• FabricPath for L2 multi-pathing
• No spanning tree
• Default gateway at spine layer
• ASR1ks w/ZBFW for firewall layer
• Nexus 5k/2k at the access Default Gateway
FabricPath
Spine Layer (N7k)F2e
QFP QFP
MPLS
Inter-VRF Firewalls
ALG ALG
ASR9kASR9k
ASR9k ASR9k
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
How do I Scale Up?
56
QFP QFPQFP QFP QFP
FabricPath FabricPath FabricPath
MPLS MPLSMPLS
MPLS
P-Layer
QFP QFPRoute Reflector Route Reflector
Firewall Layer
PE LayerPE Layer
WAN Design
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
WAN Requirements
Highly available
IGP reconvergence or instability should not affect other DCs
Minimize state in the WAN
Add/remove data centers without network outage
Connect DCs with fiber, leased lines and encrypted tunnels
Traffic engineering
58
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
A WAN Core Layer – Dual Plane
59
• IGP Isolation between each plane
• Isolate topology changes
• Flexible topology
• Highly redundant
• Similar to two provider environments
• Traffic Engineering
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Back to Design
60
• WAN Core routers are co-located in major DCs
• DC Core routers connect directly to WAN core routers
• No connection between WAN core routers
Default Gateway
FabricPath
Spine Layer (N7k)F2e
QFP QFP
MPLS
Inter-VRF Firewalls
ALG ALG
ASR9kCore1
ASR9kCore2
ASR9kWAN1
ASR9kWAN2
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
A WAN Core Layer – With Inter-AS
DCs connect using dark fiber, GRE, or leased lines
The IGP used in the WAN core is separate
DCs peer to the WAN core using eBGP
Inter-AS option C
– Only feed infra routes to WAN Core
– VPN exchanged between RRs at each DC
Advantages:
– Scale & Flexibility
– IGP Isolation
– Adding/removing DCs is seamless
– High level of HA
61
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Summary
CLOS Architecture for Scale and Flexibility
FabricPath for any VLAN Anywhere in the DC
Spine layer with Integrated MPLS PE
Firewalls Native Attached to MPLS
Scalable Architecture
Grow as you Go
Highly Flexible WAN that Scales and Highly Redundant
Flexible Growth with Multiple DCs
62
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Lessons Learned - 1
FabricPath Scale
– MAC, SVI and VLAN limits
– Topology size (number of switch-IDs) and links
Active/Active HSRP
– Requires either vPC or GLBP today
– Anycast HSRP in the 6.2 release. Requires a new release on the N5k (roadmap)
F2e/M2 mix mode with proxy routing mode
– Requires 6.2 release of NX-OS
– All routing including SVI routing done on the M2
– Relationship to oversubscription ratio
– Increases MAC address scale to 128k
Firewall design
– Asymmetric routing challenges with ASR1k. Requires BGP metric
– DC to DC flows with symmetry. Requires supernet routes 63
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Lessons Learned - 2
Inter-AS
– Option C not supported on N7k yet (roadmap)
GRE: MTU requirement
Routing over VASI:
– OSPF and iBGP were possible options over VASI initially
– eBGP support with local-AS/Remote-AS support in 3.7.2 release on the ASR1k
– Deciding on which routes to advertise from Gray VRF requires BGP filters
MPLS PE placement
– VRF-lite harder to manage and operate
– Using the N7k F2e/M2 mix makes the design way simpler
Virtual firewalls, like the ASA1000v, would make an interesting solution
64
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
R25
SP
AS100
172.16.81.0/24
E0/3
E0/1
172.16.80.0/24
LTRMPL-3102 (Advanced MPLS Lab)
65
R1
R5
R6
R2
R4
R3
10.1
.14.
0/24
10.1.15.0/24
10.1.16.0/24
10.1
.17.
0/24
10.1.18.0/24
10.1.19.0/24
E0/3E0/3
E1/0
E0/3
E0/3
E1/0
E1/0
E1/0
R7
R8R13
R15
R16
R14
FW1
Host1
Host2
AS65001DC1
10.1
.1.0
/24
10.1.2.0/24
10.1.3.0/24
192.168.1.0/24
192.168.2.0/24
10.1.10.0/24
10.1.11.0/24
10.1
.12.
0/24
10.1.13.0/24
E0/0
E0/0
E0/0
E0/0
E0/0
E0/3
E0/0
E0/2
E0/0
E1/0
E0/0
E0/2
E1/0
E0/1
E0/2
E0/3
E1/0
E0/3
E0/2
PE1
R9 R10
R17
R19
R20
R18
FW2
Host3
Host4
AS65002DC2
10.1
.4.0
/24
10.1.5.0/24
10.1.6.0/24
192.168.3.0/24
192.168.4.0/24
10.1.20.0/2410.1.21.0/24
10.1.22.0/24
E0/0
E0/0
E0/0
E0/3 E0/3
E0/0 E0/2
E0/0
E0/1
E1/0 E1/0
E0/2
E0/2 E0/0
E0/0
E0/2
E0/0 E0/2
E0/3
E1/0
PE2
R12
R11R21
R23
R24
R22
FW3
Host5
Host6
AS65003DC3
10.1.7.0/24
10.1.8.0/24
10.1
.9.0
/24
192.
168.
5.0/
24
192.168.6.0/24
172.16.82.0/24
172.16.83.0/24
E0/0
E0/0
E0/0
E0/0E0/3
E0/3E0/0
E0/0
E0/0
E1/0
E1/0
E0/1
E0/2
E0/2
E0/3
E1/0
PE3
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
R25
SP
AS100
172.16.81.0/24
E0/3
E0/1
172.16.80.0/24
R1
R5
R6
R2
R4
R3
10.1
.14.
0/24
10.1.15.0/24
10.1.16.0/24
10.1
.17.
0/24
10.1.18.0/24
10.1.19.0/24
E0/3E0/3
E1/0
E0/3
E0/3
E1/0
E1/0
E1/0
R7
R8R13
R15
R16
R14
FW1
Host1
Host2
AS65001DC1
10.1
.1.0
/24
10.1.2.0/24
10.1.3.0/24
192.168.1.0/24
192.168.2.0/24
10.1.10.0/24
10.1.11.0/24
10.1
.12.
0/24
10.1.13.0/24
E0/0
E0/0
E0/0
E0/0
E0/0
E0/3
E0/0
E0/2
E0/0
E1/0
E0/0
E0/2
E1/0
E0/1
E0/2
E0/3
E1/0
E0/3
E0/2
PE1
R9 R10
R17
R19
R20
R18
FW2
Host3
Host4
AS65002DC2
10.1
.4.0
/24
10.1.5.0/24
10.1.6.0/24
192.168.3.0/24
192.168.4.0/24
10.1.20.0/2410.1.21.0/24
10.1.22.0/24
E0/0
E0/0
E0/0
E0/3 E0/3
E0/0 E0/2
E0/0
E0/1
E1/0 E1/0
E0/2
E0/2 E0/0
E0/0
E0/2
E0/0 E0/2
E0/3
E1/0
PE2
R12
R11R21
R23
R24
R22
FW3
Host5
Host6
AS65003DC3
10.1.7.0/24
10.1.8.0/24
10.1
.9.0
/24
192.
168.
5.0/
24
192.168.6.0/24
172.16.82.0/24
172.16.83.0/24
E0/0
E0/0
E0/0
E0/0E0/3
E0/3E0/0
E0/0
E0/0
E1/0
E1/0
E0/1
E0/2
E0/2
E0/3
E1/0
PE3
66
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Related Sessions
67
Session-ID Session Name
BRKMPL-1100 Introduction to MPLS
BRKMPL-2100 Deploying MPLS Traffic Engineering
BRKMPL-2102 Deploying MPLS-based IP VPNs
BRKMPL-2109 MPLS in Multi-Tenant Data Centers
LTRMPL-2102
LTRMPL-3102
Enterprise Network Virtualization using IP and MPLS
Technologies (intro and adv)
BRKDCT-2081 Cisco FabricPath Technology and Design
BRKDCT-2121 Virtual Device Context (VDC) Design and Impl.
BRkSEC-2021 Firewall Architectures
© 2013 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Maximize your Cisco Live experience with your
free Cisco Live 365 account. Download session
PDFs, view sessions on-demand and participate in
live activities throughout the year. Click the Enter
Cisco Live 365 button in your Cisco Live portal to
log in.
Complete Your Online Session Evaluation
Give us your feedback and you could win fabulous prizes. Winners announced daily.
Receive 20 Cisco Daily Challenge points for each session evaluation you complete.
Complete your session evaluation online now through either the mobile app or internet kiosk stations.
68