View
222
Download
1
Category
Preview:
Citation preview
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Global MPLS WAN Redesign Case Study BRKMPL-2108
2
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Assumptions and Disclaimers
Participants should have ‒ A solid base knowledge of IP routing over a WAN
‒ Basic knowledge of VRFs and IP Tunnels
‒ Basic understanding of MP-BGP, MPLS control/forwarding plane
ASN Numbers depicted are for representative purposes only
This discussion will not cover the encryption devices, except to note their existence in the topology
While the diagrams depict Internet connectivity, again this is for representative purposes only
3
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Agenda
Introduction
Legacy Network Infrastructure
Change Impetus and Requirements
WAN Virtualization Options Considered
End to End Design
Proof-of-Concept Testing
Migration Strategy
Conclusion and Lessons Learned
4
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Who and Where?
Government institution ‒ World-wide private network
‒ 50+ nodes on Private WAN
Cisco Advanced Services ‒ Dedicated team on-site for over 7 years delivering Network Optimization Service
(NOS)
‒ Performing Design Review, Software Strategy, Integration Testing, Network Troubleshooting
Adam Callis ‒ CCIE #18125 Service Provider / Routing & Switching
‒ MPLS, BGP, VoIP, and Telepresence
5
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
What?
Global MPLS WAN Redesign
How Cisco helped with the Private WAN redesign ‒ Reviewed legacy design against current and future network requirements
‒ Reviewed new network virtualization requirements
‒ Proposed MPLS over GRE solution
‒ Designed and executed proof of concept testing
‒ Identified required hardware upgrades
‒ Creation and testing of the migration strategy
‒ Development of the baseline production configurations
‒ Support of production network migrations
6
Legacy Network Infrastructure
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
7609
Evolution of the WAN
8
2621XM
7507
7513
3600
7513
7507
2621XM
Internet
OC3
DS3 DS3 DS3
DS3
DS1 DS1
DS1
DS1
Private Line Core Circa 1999-2004
8950
8950
8950
8850
7609
7609
7606
7609
7606
7609
7609
7606
Internet
Backup
OC192
OC192
OC192
OC48
OC48
Primary
ATM MGX Core Circa 2004-2009
FastEth
Internet
Provider MPLS VPN
7609
7606 7606
7606
Provider MPLS Core Circa 2009 - Present
GRE
GRE
GRE GRE
FastEth
FastEth
GigE
GigE
FastEth
GigE
GigE
GigE
GigE
GigE
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Private Line Core – Key Points
No network virtualization deployed ‒ One of over thirty built by customer
‒ Costly dedicated hardware and circuits
‒ Requires independent NMS tools
‒ Simplify security posture
Dedicated bandwidth between sites ‒ Doesn't allow for sharing bandwidth between
networks
‒ Simplifies QoS policies per network
9
2621XM
7507
7513
3600
7513
7507
2621XM
Internet
OC3
DS3 DS3 DS3
DS3
DS1 DS1
DS1
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
ATM MGX Switched Core – Key Points
Virtualization of core bandwidth ‒ High bandwidth core circuits (OC192,OC48)
‒ Share core circuits between WANS
‒ ABR VC's allow bursting when available
Cisco 7600 with ATM SPAs used to create VRF-Lite private IP transport core ‒ Provides GigE / FastEth handoff
‒ Supports QoS on ATM Links
IPSec VPN Device (IVD) required ‒ IVD tunnels over ATM Core
‒ IVD needs static routes loaded (Management pain point)
10
8950
8950
8950
8850
7609
7609
7606
7609
7606
7609
7609
7606
Internet
Backup
OC192
OC192
OC192
OC48
OC48
Primary
FastEth
FastEth
FastEth
GigE
GigE
FastEth
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Provider MPLS Core – Key Points
Ethernet IP handoff ‒ Lower cost interfaces
‒ Allows for 802.1q virtualization
‒ Up to 10 Gig per port supported
‒ Compatible with existing IVD's
Provider QoS SLA ‒ Utilizes Diffserv Code Points to classify traffic
‒ Non-realtime bandwidth contract 50Mbps (per site)
‒ Mark down packets when exceeding contract rate
‒ Best effort queue available for large file transfers
7609
Internet
Provider MPLS VPN
7609
7606 7606
7606
GRE
GRE
GRE GRE
GigE
GigE
GigE
GigE
GigE
11
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Initial Virtualization (VRF-Lite)
New user community requirement for connectivity
‒ Quick turnaround was required
‒ GRE Tunnels in new VRF built to key nodes requiring access
‒ FR Encapsulation on leased lines deployed to enable DLCI separation
VRF-Lite shortcomings identified
‒ Network outages when provisioning new paths on IVD's
‒ Time consuming process for tunnel deployment
12
7606
7609
7606
7606
7606
Internet AS 701
7609
3945
1941
1841 1841
7606
3845 1841
Global GRE
VRF-A GRE
DS3 FR
DS3 FR
DS1 FR
DS1 FR
GigE 802.1q
GigE 802.1q
GigE 802.1q
Provider MPLS VPN
Change Impetus and Requirements
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
New Virtualization Needs
New user community needing widespread network connectivity
Cost savings edict from CIO ‒ No more building parallel networks!
‒ Utilize as much existing hardware as possible
Security ‒ Must maintain data separation
‒ Must maintain control plane separation
Scalability ‒ Rapid deployment of new VRF's as new communities of interest want to join the
network
14
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Capitalizing on the Technology Shift
Migrate from leased lines to IP based transport where available ‒ Cost Savings
Less hub equipment required to terminate service
Circuits now remain local to SP network
‒ Increased network efficiency by sharing bandwidth
Shift from ATM to Ethernet Service ‒ Ethernet interfaces widely available and typically lower cost then ATM card
‒ Improved network efficiency by removing the ATM "Cell Tax"
15
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
ATM MGX Equipment EoS Announcement
16
End of Sale Notices Entire Deployed MGX Core announced EoS 30/Jul/2010
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Network Improvement Requirements
The "Hand Grenade" Test ‒ Fully redundant hardware configuration
‒ Redundant network paths (for core sites)
Easy to manage
Support for line rate transfers (100M and 1G)
Support for full MTU packets (1500 bytes)
Utilize tunneling protocol to limit source / destination IP addresses
QoS capable of shaping and queuing per destination
Symmetric routing to/from Active/Active firewalls
BGP routing for any inter-as peering
17
WAN Virtualization Options Considered
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Policy Based Isolation
Well understood solution
Easy to implement
No capital expenditure required
19
Private VLANS in Campus layer 2 network
Shared layer 3 routing table
Access Control Lists (ACL) applied to each router
Scalability
‒ ACLs managed on 50+ routers not feasible
Lack of Virtualization
‒ IP Addresses cannot overlap
Prone to human error
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
VRF Lite Isolation
Improves security and stability by creating separate control plane per vrf
Widely supported across platforms Simple configuration Familiar CLI command structure
20
Creates isolated routing table per VRF
Utilizes 802.1q, Frame-Relay, or ATM for Layer 2 separation
IP Tunnels (GRE / DMVPN) extend VRF over IP cloud
Scalability ‒ May cause high CPU utilization for Per
VRF routing processes (eg: OSPF) ‒ Excessive configuration when adding new
nodes ‒ Elongated provisioning time for new VRFs
No support for Layer 2 VPN
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS over GRE – Point to Point Tunnels
Extends MPLS over IP Core Supported on most platforms Per-Tunnel H-QoS supported
‒ Shape tunnel ‒ Prioritize shaped traffic
21
Static GRE Tunnels built over IP Core between PEs
OSPF and LDP enabled on tunnel interfaces
IPv4 + VPNv4 iBGP sessions built over tunnels
Complexity ‒ Layered troubleshooting effect on uptime ‒ New CLI commands to learn
Scalability ‒ Requires unique loopbacks per tunnel ‒ Mesh of GRE tunnels as nodes are added
Hardware Dependency for 7600's ‒ SIP-400 w/ 2x1GE SPA or ES+ Required
Static GRE Tunnels
PE
PE PE
PE
IP Core
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
MPLS over mGRE
Extends MPLS over IP Core Scalability
‒ No tunnel related configuration (IGP, LDP) ‒ Adding nodes only requires iBGP neighbor ‒ Simplified troubleshooting resulting in
more uptime Supported on most platforms
22
No GRE Tunnels configured
All MPLS signaling exchanged over iBGP mesh
Packets auto encapsulated with VPN Label and GRE Header based on BGP Next-Hop
Hardware Dependency for 7600's ‒ SIP-400 w/ 2x1GE SPA or ES+ Required
No Per-Tunnel H-QoS GRE packets always sourced from
BGP Loopback ‒ Creates corner case limitation for our
Customer scenario
iBGP Full Mesh Peering
PE
PE PE
PE
IP Core
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Decision to Use Static GRE Tunnels
The WAN virtualization direction came down to Static GRE tunnels vs mGRE. Static GRE tunnels were selected for the following reasons
Software support
‒ At the time of testing and implementation, mGRE wasn't supported on code deployed to legacy ISR routers
‒ Customer testing requirements create 9 month wait for new software deployment
Corner case implementation requirement to integrate with IVD's ‒ Requirements dictate redundant IVDs at core sites ‒ IVD's can only have a single next-hop per destination ‒ To support redundancy with these IVD's, multiple loopbacks for GRE endpoints
are required. This is NOT supported in the current mGRE implementation
23
End to End Design
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Design Overview Provider provisioned MPLS VPN
provides core IP transport
Redundant uplink and IVD to Provider MPLS Service
Point to point GRE tunnel overlay through external IVD's
Consolidation of leased lines to tail sites onto 3945's at core sites
Customer provisioned MPLS VPN over GRE and leased lines
CE devices all layer 2 switches
Redundant firewalls to Internet
25
3945 P
1841 PE 1941 PE
7609 P/PE
Provider MPLS VPN Service
eBGP
GigE
GigE
7609 P/PE
DS1
GRE
eBGP
3945 P
GigE
1841 PE 1941 PE
DS1 DS1
DS1
CE Switch CE Switch
FastEth GigE
7609 P/PE
GRE
eBGP
3945 P
GigE
1841 PE 1941 PE
DS1 DS1
CE Switch CE Switch
FastEth GigE
CE Switch CE Switch
FastEth GigE
GRE
7606 7606
Internet
GRE eBGP eBGP
eBGP eBGP
eBGP eBGP
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Provider MPLS VPN Service Overview
Use of service is mandated where available
Similar to a private IP service commercially available ‒ OC-192 Core
‒ Supports only Ethernet (1 Gig or 10 Gig) handoff
Standard Layer 3 VPN service
Multicast VPN Support
QoS SLAs guarantee bandwidth based on agreed upon DSCP markings ‒ Non-Realtime
‒ Realtime
‒ Best Effort (Discard Eligible)
26
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Core Site Design
Re-Use of 7600's as WAN Edge router ‒ Minimal hardware investment (Line cards only)
‒ Highly available chassis
‒ Provides user access ports via Catalyst cards
‒ PE node for local servers / clients
‒ P node for downstream serial aggregation and tail sites
3945 used to aggregate leased line circuits (Leased Line Tail Circuits) ‒ Reduced per port cost
‒ Frees up high speed slots in 7600 for ES+ modules
27
3945 P
1841 PE 1941 PE
7609 P/PE
Provider MPLS VPN Service
GigE
GigE
DS1 DS1
CE Switch CE Switch
FastEth GigE
Serial Aggregation Router
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Leased Line Tail Site Design
Layer 2 Switch deployed as CE ‒ VLAN Separation between user communities
‒ PE Node will act as default gateway for user subnets
‒ Saves on power, space, cooling, and cost
Small Branch Office (Up to 25 People) ‒ 1941 ISR-G2 router deployed as PE
‒ DS1 Uplink to closet serial aggregation router
Larger Campus Offices (Over 25 People) ‒ 3945 ISR-G2 router deployed as PE
‒ Multiple DS1 (Bonded with ML-FR) or DS3 Uplink
28
3945 P
1941 PE 3945 PE
DS1 DS3
CE Switch CE Switch
GigE GigE
Serial Aggregation Router
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Provider MPLS VPN Service
GRE Tunnels over Provider MPLS VPN
Jumbo Frame Support
‒ Avoid fragmentation
‒ Increase performance
RFC 1918 Address Assignment
‒ Conserve routable IPv4 space
‒ Require unique loopbacks per GRE tunnel for 7600
H-QoS policy applied to GRE tunnel
‒ Shaper to prevent overrunning remote site
‒ Queuing to protect real time traffic (Voice / Video)
LDP Enabled on GRE tunnels that can support MPLS
7609
7609
Interface IP
Loopback500 10.255.0.1/32
GigabitEthernet0/0/0 172.16.0.1/30
Tunnel500 10.127.0.1/30
Interface IP
Loopback500 10.255.0.2/32
GigabitEthernet0/0/0 172.16.0.5/30
Tunnel500 10.127.0.2/30
eBGP
eBGP GRE
29
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
IGP Design Principles
OSPF with single area 0.0.0.0 deployed network wide ‒ Simple to configure ‒ Single area required to support traffic engineering tunnels (future need) ‒ OSPF database not large as it only contains routes to loopback addresses
OSPF Network type set to "point-to-point" on /30 Ethernet interfaces ‒ Avoids DR election reducing network convergence time
Passive interface default and MD5 authentication deployed ‒ Ensures only authorized devices can establish OSPF adjacency
OSPF interface costs set based on round trip latency measured (utilizing ping) at installation ‒ Customer applications are highly sensitive to latency, goal is to deliver lowest
latency possible
30
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
BGP Design Principles
Full iBGP mesh between four core sites Core sites enabled as route reflectors for downstream tail sites
‒ Eliminates the need to full mesh every node in iBGP network
MD5 Authentication enabled on all peers ‒ Prevent someone from spoofing BGP session
Peer groups configured for core and route reflector clients ‒ Eliminates repetitive configuration ‒ Ensures conformity of configuration
Symmetry is required through firewalls ‒ No state information shared between firewalls
Enable use of Standard and Extended Communities ‒ Standard communities utilized in route decision process ‒ Extended communities required for MPLS VPN service
31
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
BGP Community Usage – Default Route
Inbound from Internet
‒ Only default route necessary from Internet
Reduces BGP Table size
‒ Communities set to denote exit point and firewall learned through
‒ MPLS Routers peer with both east and west coast route reflectors
‒ MPLS Routers match east or west firewall community and adjust local preference accordingly to prefer that exit path
32
RR - P/PE
3945 P
GigE
1841 PE 1941 PE
DS1 DS1
CE Switch CE Switch
FastEth GigE
INET-GW
Internet
eBGP Per-VRF
eBGP
0.0.0.0/0
0.0.0.0/0 701:1
0.0.0.0/0 701:1 65500:1
Set Local Pref 120
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
BGP Community Usage – Internal Routes
Outbound to Internet
‒ PE router advertises route with community denoting which firewall traffic should return through
‒ INET-GW adjusts local preference to prefer one firewall over another
‒ INET-GW removes private AS
‒ INET-GW aggregates routes (if possible)
‒ INET-GW adjusts the AS-PATH length to influence return traffic from Internet
33
RR - P/PE
3945 P
GigE
1841 PE 1941 PE
DS1 DS1
CE Switch CE Switch
FastEth GigE
INET-GW
Internet
eBGP
eBGP
192.168.10.0/24 65501:1
192.168.10.0/24 65501:1
192.168.10.0/24 AS PATH Adjustment
Proof of Concept Testing
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Lab Baseline
Cisco Equipment
Cisco 7606
‒ SUP720-3BXL
‒ ES+ WAN Interface
‒ SIP-400 w/2x1GE V2 SPA WAN Interface
Cisco ISR (1841, 3845)
Cisco ISR-G2 (1941)
Cisco ASA 5505
Software Baseline
12.2(33)SRE2 (7600)
15.1(2)T2 (1941, 3845)
12.4(22)T3 (1841)
Other Tools
Agilent Test Set (Not shown)
35
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Test Case Overview
Basic Functionality ‒ Basic end to end connectivity between workstations within VRF
Failover Testing ‒ Testing the failure of uplinks and redundant hardware to validate network
redundancy
Configuration Complexity ‒ No Pass/Fail Criteria, just documented for future reference
Migration strategy validation ‒ Passing Criteria: User migration can be achieved with minimal downtime
36
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Test Case Sample: Basic Functionality Testing
37
PE PE IP Core
Single GRE tunnel between two PE nodes
Establish OSPF, LDP, iBGP over GRE tunnel
Create a VRF on both PEs
Generate ICMP traffic between workstations
0% Packet Loss
Latency consistent with underlying transport (< 5 ms for lab)
Migration Strategy
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Pre-Migration Requirements
All egress interfaces from 7600 that carry GRE packets must be either ‒ SIP-400 w/2x1GE v2 SPA
‒ ES+ Module
All 7600 routers running 12.2(33)SRE2 or later
3rd Party IVD's must have routes for remote destinations programmed in advance
New circuit paths must be tested and accepted ‒ Large Packets with DF-Bit to confirm MTU support
‒ Extended ping to validate packet loss
‒ Extended ping to confirm QoS settings from provider VPN
39
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Migrating the Network – The Plan
Two maintenance windows This is to allow validation of the IVD behavior
prior to actual cutover ‒ Initial Configuration (Over several days)
Building GRE Tunnels Establishing OSPF adjacencies Configuring and validating OSPF costs Document Latencies of each GRE tunnel
‒ Actual Migration (One night) Enable LDP, VPNv4 BGP peering Disable Per-VRF IPv4 BGP peering
40
7609 7609 Global GRE
GigE GigE
7609 7609 GigE GigE
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Migrating the Network – The Reality
Situation ‒ Core sites missing proper hardware
SIP-400 w/2x1GE SPA or ES+ ‒ IOS Code not yet upgraded on core routers ‒ Network virtualization requirements cannot be
delayed ‒ ISR / ISR-G2 can support MPLS AS-IS
Work around ‒ Extend legacy VRF-Lite peering between key
core routers GRE tunnel per VRF
‒ Move forward with MPLS deployment on leased line circuits to tail sites
Implication ‒ Migrations timeline significantly extended
41
7609 7609 Global GRE
GigE GigE
7609 7609 GigE GigE
VRF A - GRE
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Next-hop Limitations Deploying Workaround
Non-MPLS nodes need BGP next-hop reachable within VRF. ‒ Next-hop self doesn't apply to iBGP relationships ‒ Achieved by route-map next-hop manipulation
MPLS nodes need BGP next-hop to be in
global table for their island ‒ Achieved by route-map next-hop manipulation
VPNv4 Peering must be contained within island
‒ Prevents PE from attempting to impose VPN label that cannot be reached
‒ No LSP path between islands
7609 7609 Global GRE
GigE GigE
7609 7609 GigE GigE
VRF A - GRE
3945
802.1q
1941
DS1 MPLS
3945
1941
DS1 MPLS
802.1q
MPLS Islands
VRF-Lite
42
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Network Migrations – Where Are We Now? Where Are We Going Next? Where are we now?
‒ All new tail sites being deployed are utilizing MPLS for network virtualization
‒ Most existing tail sites have been migrated to MPLS
Primarily sites that didn't require virtualization are left unvirtualized
‒ Some core sites still not migrated
Lack of funding to procure proper SIP/SPA combo required
Where are we going next?
‒ Working with IVD manufacturer to overcome limitation that forced static GRE
‒ Migrating any tail site requiring virtualization
‒ Once hardware and software installed on 7600's, completion of core configurations
43
Lessons Learned
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Deploy MPLS over mGRE Instead of Static Point to Point Tunnels Common Issues with static tunnels
‒ Static tunnels add configuration complexity (eg: OSPF, LDP, etc) ‒ It's easy to inadvertently mis-configure static tunnels and cause 7600 into
software forwarding state ‒ Static tunnels require manual MTU adjustment, a simple omission of this
configuration can cause detrimental performance impacts on network ‒ Adding a new node requires significant configuration
Problems we faced ‒ Several tunnel mis-configurations caused packet fragmentation and reassembly
to be done in software on 7600, eventually causing protocol adjacencies to fail due to high CPU utilization
‒ When adding a new node a typo caused the GRE tunnel to be punted to software and sent the CPU to 100%
45
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Software Upgrade Lessons Learned
Procure spare CF cards and load them with proper IOS in centralized location, then ship to site for installation with return envelope for old card ‒ Ensures consistency of image being deployed
‒ Ensures sufficient flash space will be present at time of upgrade
‒ Allows limited skill set personnel to be your remote hands onsite
Utilize internal CF adapter on SUP720 ‒ Reduces possibility of local site tech "borrowing" your CF Card needed for IOS
Image
Perform IOS Upgrades well in advance of network migration ‒ Any problems such as insufficient memory can be identified and remedied before
the upgrade is actually required
46
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
General Design Lessons Learned
DNS will save you a lot of time ‒ Adding the point to point subnets to the DNS server allowed made traceroute
more useable when troubleshooting paths
Utilize RFC 1918 space for internal addresses (Loopbacks / P2P) ‒ Conserves your public space ‒ Allows you to more easily make the IP addresses mean something to the network
administrator (ex: Encoding Building Number into 3rd Octet)
Expect your "temporary workaround" to end up being semi-permanent ‒ Once virtualization requirement was met, remote engineers became focused on
next big requirement leaving network in a state of migration
Plan for "never" ‒ Each time I heard "We will never need X" I took a note and planned on supporting
it as inevitably we would need to support those features.
47
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
BGP Design Lessons Learned
Document and publish your BGP Communities in use ‒ While it made perfect sense what you were doing in the lab, you will forget why
your setting that community when your troubleshooting.
Enable soft-reconfiguration inbound on all peers ‒ During troubleshooting it is often valuable to soft clear BGP sessions
When mixing VRF-Lite and MPLS, pay close attention to next-hop
Don't forget that in order to utilize MD5 on BGP through a firewall requires special configuration
48
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Network Management Lessons Learned
Centralize management of network to small group of qualified engineers ‒ Having technicians that are less then qualified dispersed around the country leads
to confusion, configuration errors, and ultimately unplanned downtime
‒ A small qualified group can be trained and highly familiar with the proper configurations. This will help prevent unplanned outages
Ensure your NMS platforms are included in your global routing table and are the first to be migrated ‒ Some router management functions are not well supported if inside a VRF.
Placing all management (TACACS, NTP, Syslog, SNMP, etc) in global table ensures maximum support
‒ Routers respond very slowly when they cannot reach TACACS+ for command authorization (if configured) exasperating the migration time
49
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Migration Lessons Learned
Wait to migrate until ENTIRE network can be migrated ‒ Software incompatibilities and hardware availability forced us to migrate from the
outside (tail sites) inward toward the core creating MPLS islands
‒ Created many next-hop resolution issues
‒ Massive confusion among NOC engineers when troubleshooting
Validate ALL network paths, even previously known good paths ‒ A provider MTU had been mis-configured preventing our full frame packets from
transmitting when the DF-Bit was set
‒ A QoS policy shaper was mis-configured on provider causing all traffic to be treated as best-effort by default vs non-realtime by default
50
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Related Sessions
MPLS ‒ BRKMPL-2102 – Deploying MPLS-based IP VPNs
‒ BRKMPL-2109 – MPLS Solutions for Cloud Networking
‒ BRKMPL-3101 – Advanced Topics and Future Directions in MPLS
WAN Virtualization ‒ BRKRST-2045 – Network Virtualization Design Concepts over the WAN
51
Questions?
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Complete Your Online Session Evaluation Give us your feedback and you
could win fabulous prizes. Winners announced daily.
Receive 20 Passport points for each session evaluation you complete.
Complete your session evaluation online now (open a browser through our wireless network to access our portal) or visit one of the Internet stations throughout the Convention Center.
Don’t forget to activate your Cisco Live Virtual account for access to all session material, communities, and on-demand and live activities throughout the year. Activate your account at the Cisco booth in the World of Solutions or visit www.ciscolive.com.
53
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Final Thoughts
Get hands-on experience with the Walk-in Labs located in World of Solutions, booth 1042
Come see demos of many key solutions and products in the main Cisco booth 2924
Visit www.ciscoLive365.com after the event for updated PDFs, on-demand session videos, networking, and more!
Follow Cisco Live! using social media: ‒ Facebook: https://www.facebook.com/ciscoliveus
‒ Twitter: https://twitter.com/#!/CiscoLive
‒ LinkedIn Group: http://linkd.in/CiscoLI
54
© 2012 Cisco and/or its affiliates. All rights reserved. BRKMPL-2108 Cisco Public
Recommended