Advanced enterprise campus design. routed access (2015 milan)

© 2015 Cisco and/or its affiliates. All rights reserved.BRKCRS-3036 Cisco Public

Some loops are fun ...

Advanced EnterpriseCampus Design: Routed Access

BRKCRS-3036

Mark Montañez, CCIE #8798Architecture Lead, Enterprise Segment

Distinguished Consulting Engineer

@MarkMontanez or [email protected]


Agenda - Enterprise Campus Design: Routed Access

• Introduction

• Cisco Campus Architecture Review

• Campus Routing Foundation and Best Practices

• Building a Routed Access Campus Design

• Routed Access Design and VSS

• Impact of Routed Access Design for Advanced Technologies

• Summary

4

Start with the Core

! "#$%&' ( $#

Add in theDistribution Layer …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

Traditional Multi-Layer Distribution …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

VSS-basedDistribution …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

Add in theAccess Layer …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Multi-Layer Access …L3 terminated at Dist.

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Routed Access …L3 terminated at Access

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Converged Access …Wired / Wireless

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Instant Access …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Add inWired clients ...

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Add inAccess Points …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

… and someWireless clients …

! "#$%&' ( $#

) *+,#*- . / "0%&' ( $#

122

344$++&' ( $#

Add in a CampusServices Layer …

! "#$%&' ( $#

) *' #$+! ' , - . /) $#012$/

31/4#15. 6"7%&' ( $#

8) )

922$//&' ( $#

… with some WirelessLAN Controllers (WLCs)

! "#$%&' ( $#

) *' #$+! ' , - . /) $#012$/

31/4#15. 6"7%&' ( $#

8) )

922$//&' ( $#

! "# ! "#

… and some Firewalls

! "#$%&' ( $#

) *' #$+! ' , - . /) $#012$/

31/4#15. 6"7%&' ( $#

8) )

922$//&' ( $#

! "#

$%&' ( ) **

! "#

$%&' ( ) **

Form the WLCs intoa Mobility Group …

! "#$%&' ( $#

) *' #$+! ' , - . /) $#012$/

31/4#15. 6"7%&' ( $#

8) )

922$//&' ( $#

! "#

$%&' ( ) **

! "#

$%&' ( ) **

Create the CUWN CAPWAP overlay …

! "#$%&' ( $#

) *' #$+! ' , - . /) $#012$/

31/4#15. 6"7%&' ( $#

8) )

922$//&' ( $#

! "#

$%&' ( ) **

! "#

$%&' ( ) **

Add in Converged Access to the mix …… and add in theData Center for the siteInternet access, dual-homed, with RA VPNGuest wireless access,terminated in DMZNow, let’s move outto the WAN …First, we may haveMAN connectivity …We may also have atraditional WAN (T1, etc)

We may have an SP-provided MPLS serviceWe may be using DMVPN over InternetWe may be using GET VPN over WAN/MPLS …… or we may be using DMVPN over 3G/4G/SatBranches may be single-attached to the WAN …Or branches may be dual-WAN-attachedAdd in remote teleworkers …We may have an second, backup Data Center …… using a variety of DCI options for connectivityFinally, all of this may be virtualized “N” times …Complexityin Today’s Solution


Access

Dist.

Core

VLAN 22 WLAN10.1.22.0/24

VLAN 11 Voice10.1.11.0/24

Trunk

HSRP

VLAN 10 Data10.1.10.0/24

VLAN 21 Voice10.1.21.0/24

Layer 2

VLAN 20 Data10.1.20.0/24

MultilayerSOME VLANS Span

GLBP

VLAN 31 Voice10.1.31.0/24

VLAN 30 Data10.1.30.0/24

VLAN 41 Voice10.1.41.0/24

VLAN 40 Data10.1.40.0/24

Layer 3

MultilayerNO VLANS Span

VLAN 51 Voice10.1.51.0/24

P-to-P Link

No FHRP Needed

Layer 3

VLAN 50 Data10.1.50.0/24

VLAN 61 Voice10.1.61.0/24

VLAN 60 Data10.1.60.0/24

RoutedAccess

VLAN 70 Data10.1.70.0/24

VLAN 71 Data10.1.71.0/24

VLAN 72 Voice10.1.72.0/24

No FHRP

Needed

VSS

OSPFEIGRPBGP

SummarizationRoute redistribution

Route filtering …

CustomTopologies

VLAN 80 Data10.1.80.0/24

VLAN 81 Data10.1.81.0/24

VLAN 82 Voice10.1.82.0/24

OSPFEIGRP

OSPFEIGRP

Many Options – All with some benefits and challenges


Enterprise CampusCollaboration and Video Evolution

• IP Telephony (IPT) is now a mainstream technology

• Ongoing evolution to the full spectrum of Unified Communications

• High Definition Video Communications requires stringentService-Level Agreement (SLA)– Reliable Service – High Availability Infrastructure

– Application Service Management – End-to-End QoS

© 2007 Cisco Systems, Inc. All rights reserved. Cisco Confidential


One Time Zone—Real Time

Enterprise Campus21st Century Business Realities

Rapid Collaborative Decisions

Strict Governance for Compliance and Risk Reduction

Workers, Customers, and Partners Operate Anywhere

Resources Must be Leveraged to Their Maximum



• Introduction






• Summary

9


Building BlockWAN Internet

SiSi SiSi SiSi SiSi SiSi SiSi

SiSi SiSi

SiSi SiSi

SiSi SiSiSiSi

SiSi

Access

Distribution

Core

Distribution

Access

• Offers hierarchy—each layer has specific role

• Modular topology—building blocks

• Easy to grow, understand, and troubleshoot

• Creates small fault domains—clear demarcations and isolation

• Promotes load balancing and redundancy

• Promotes deterministic traffic patterns

• Incorporates balance of both Layer 2 and Layer 3 technology, leveraging the strength of both

• Can be applied to both the multilayerand routed campus designs

Hierarchical Network DesignWithout a Rock Solid Foundation the Rest Doesn’t Matter


L2

Multilayer Campus Network DesignLayer 2 Access with Layer 3 Distribution

• Each access switch hasunique VLAN’s

• No layer 2 loops

• Layer 3 link between distribution

• No blocked links

• At least some VLAN’s span multiple access switches

• Layer 2 loops

• Layer 2 and 3 running over link between distribution

• Blocked links

SiSi SiSi SiSi SiSi

Vlan 10 Vlan 20 Vlan 30 Vlan 30 Vlan 30 Vlan 30

L3


Multilayer Campus Network DesignWell Understood Best Practices

• Mature, 10+ year old design

• Evolved due to historical pressures

– Cost of routing vs. switching

– Speed of routing vs. switching

– Non-routable protocols

• Well understood optimization of interaction between the various control protocols and the topology

– STP Root and HSRP primary tuning to load balance on uplinks

– Spanning Tree Toolkit (RootGuard, LoopGuard, …)

– etc, …

SiSi SiSi

SiSi SiSi

BRKCRS-2031 – Multilayer Campus Architectures and Design Principals

Root

Bridge &

HSRP

Active

HSRP

Standby

CISF, BPDU Guard

LoopGuard

RootGuard


0

2

4

6

8

10

250 msec 3 secs

Multilayer Campus Network DesignGood Solid Design Option

• Utilizes multiple Control Protocols– Spanning Tree (802.1w, …)

– FHRP (HSRP, VRRP, GLBP…)

– Routing Protocol (EIGRP, …)

• Convergence is dependent on multiple factors– FHRP - 900msec to 9 seconds

– Spanning Tree - 400msec to 50 seconds

• FHRP Load Balancing– HSRP/VRRP – Per Subnet

– GLBP – Per Host

Tim

e t

o r

esto

re V

oIP

data

fl

ow

s (

seco

nd

s)

HSRP Hello Timers

FHRP Convergence


3/2 3/2

3/1 3/1Switch 1 Switch 2

DST MAC 0000.0000.4444

DST MAC 0000.0000.4444

Multilayer Campus Network DesignLayer 2 Loops and Spanning Tree• Campus Layer 2 topology has sometimes proven a operational or

design challenge

• Spanning tree protocol itself is not usually the problem, it’s the external events that triggers the loop or flooding

• L2 has no native mechanism to dampen down a problem:– L2 fails Open, as opposed to L3 which fails closed

• Implement physical L2 loops only when you have to



• Introduction






• Summary

15


Best Practices—Campus RoutingLeverage Equal Cost Multiple Paths

• Use routed pt2pt links and do not peer over client VLANs, SVIs.

• ECMP used to quickly re-route around failed node/links while providing load balancing over redundant paths

• Tune CEF L3/L4 load balancing hash to achieve maximum utilization of equal cost paths (CEF polarization)

• Build triangles not squares for deterministic convergence

• Insure redundant L3 paths to avoid black holes

• Summarize distribution to core to limit event propagation

• Utilized on both Multi-Layer and Routed Access designs

Data CenterWAN Internet

Layer 3 Equal

Cost Link’sLayer 3 Equal

Cost Link’s SiSiSiSi

SiSiSiSi

SiSi SiSiSiSiSiSi



Routed Interfaces Offer Best Convergence Properties

• Configuring L3 routed interfaces provides for faster convergence than a L2 switchport with an associated L3 SVI

21:32:47.813 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/1, changed state to down

21:32:47.821 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet2/1, changed state to down

21:32:48.069 UTC: %LINK-3-UPDOWN: Interface Vlan301, changed state to down

21:32:48.069 UTC: IP-EIGRP(Default-IP-Routing-Table:100): Callback: route, adjust Vlan301

1. Link Down

2. Interface Down

3. Autostate

4. SVI Down

5. Routing Update

21:38:37.042 UTC: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet3/1, changed state to down

21:38:37.050 UTC: %LINK-3-UPDOWN: Interface GigabitEthernet3/1, changed state to down

21:38:37.050 UTC: IP-EIGRP(Default-IP-Routing-Table:100): Callback: route_adjust GigabitEthernet3/1

SiSiSiSi

L2

SiSiSiSi

L31. Link Down

2. Interface Down

3. Routing Update

~ 8 msec loss

~ 150-200 msec

loss


Best Practice—Build Triangles Not SquaresDeterministic vs. Non-Deterministic

• Layer 3 redundant equal cost links provide fast convergence

• Hardware based—fast recovery to remaining path

• Convergence is extremely fast (dual equal-cost paths: no need for OSPF or EIGRP to recalculate a new path)

Triangles: Link/Box Failure Does Not

Require Routing Protocol Convergence

Model A

Squares: Link/Box Failure Requires

Routing Protocol Convergence

Model B

SiSi

SiSiSiSi

SiSiSiSi

SiSiSiSi

SiSi


0

0.5

1

1.5

2

2.5

3

3.5

500 1000 5000 10000 15000 20000 25000

Co

nve

rge

nc

e (

se

c)

ECMP ECMP (SXI2) MEC

CEF ECMP—Optimize ConvergenceECMP Convergence Is Dependent on Number of Routes

• Until recently, time to update switch HW FIBwas linearly dependent on the number ofentries (routes) to be updated

• Summarization and Filtering will decreaseRP load as well as speed up convergence

Number or Routes in Area – Sup720

SiSi

SiSi

SiSi

Time for ECMP

Recovery

Time for ECMP/MEC Unicast Recovery


CEF Load BalancingUnderutilized Redundant Layer 3 Paths

• The default CEF hash ‘input’ is L3source and destination IP addresses

• Imbalance/overload could occur

• CEF polarization: in a multihopdesign, CEF could select the same left/left or right/right path

• Redundant paths are ignored/underutilized

• Two solutions:

1. CEF Hash Tuning

2. CEF Universal ID

Redundant

Paths

Ignored

SiSiSiSi

SiSi SiSi

SiSi SiSi

L

L

R

R

Distribution

Default L3 Hash

Core

Default L3 Hash

Distribution

Default L3 Hash

Access

Default L3 Hash

Access

Default L3 Hash

70%

load

30%

load


SiSiSiSi

SiSi SiSi

SiSi SiSi

CEF Load Balancing1. Avoid Polarization with CEF Hash Tuning

• With defaults, CEF could select the same left/left or right/right paths and ignore some redundant paths

• Alternating L3/L4 hash and default L3 hash will give us the better load balancing results

• The default is L3 hash—no modification required in core or access

• In the distribution switches use:

– mls ip cef load-sharing full

to achieve better redundant path utilization

RL

RDistribution

L3/L4 Hash

Core

Default L3 Hash

Distribution

L3/L4 Hash

L

RL

Left Side

Shown

Access

Default L3 Hash

Access

Default L3 Hash

All Paths

Used

L


CEF Load Balancing2. Avoid Polarization with Universal ID

• Cisco IOS uses “Universal ID” concept (also called Unique ID) to prevent CEF polarization– Universal ID generated at bootup (32-bit pseudo-random

value seeded by router’s base IP address)

• Universal ID used as input to ECMP hash, introduces variability of hash result at each network layer

• Universal ID supported on Catalyst 6500 Sup-32, Sup-720, Sup-2T

• Universal ID supported on Catalyst 4500 SupII+10GE, SupV-10GE and Sup6E

Hash using Source IP

(SIP), Destination IP (DIP)

& Universal ID

Original Src IP + Dst IP

Universal* Src IP + Dst IP + Unique ID

Include Port Src IP + Dst IP + (Src or Dst Port) + Unique ID

Default* Src IP + Dst IP + Unique ID

Full Src IP + Dst IP + Src Port + Dst Port

Full Exclude Port Src IP + Dst IP + (Src or Dst Port)

Simple Src IP + Dst IP

Full Simple Src IP + Dst IP + Src Port + Dst Port

Catalyst 4500 Load-Sharing Options Catalyst 6500 PFC3** Load-Sharing Options

SiSi SiSi

SiSi SiSi

SiSi

* = Default Load-Sharing Mode



• Introduction






• Summary

23


Routed Access Design Layer 3 Distribution with Layer 3 Access: no L2 Loop

• Move the Layer 2/3 demarcation to the network edge

• Leverages L2 only on the access ports, but builds a L2 loop-free network

• Design Motivations: simplified control plane, ease of troubleshooting, highest availability

Data 10.1.20.0/24 2001:DB8:CAFE:20::/64

Voice 10.1.120.0/24 2001:DB8:CAFE:120::/64

EIGRP/OSPF EIGRP/OSPF

GLBP Model

SiSiSiSi

Layer 3

Layer 2

Layer 3

Layer 2EIGRP/OSPF EIGRP/OSPF

SiSi SiSi

Data 10.1.40.0/24 2001:DB8:CAFE:40::/64

Voice 10.1.140.0/24 2001:DB8:CAFE:140::/64


Routed Access AdvantagesSimplified Control Plane

• Simplified Control Plane– No STP feature placement (root bridge, loopguard, …)

– No default gateway redundancy setup/tuning (HSRP, VRRP, GLBP ...)

– No matching of STP/HSRP priority

– No asymmetric routing and unicast flooding

– No L2/L3 multicast topology inconsistencies

– No Trunking Configuration Required

• L2 Port Edge features still apply:– Spanning Tree Portfast

– Spanning Tree BPDU Guard

– Port Security, DHCP Snooping, DAI, IPSG

– Storm Control

– 802.1x

– QoS Settings ...

SiSi

SiSiSiSi

SiSi

L3 L3 L3 L3

L3

SiSi SiSi


Routed Access AdvantagesSimplified Network Recovery

• Routed Access network recovery is dependent on L3 re-route

• Time to restore downstream flows is based on a routing protocol re-route

– Time to detect link failure

– Time to determine new route

– Process the update for the SW RIB

– Update the HW FIB

• Time to restore upstream traffic flows is based on ECMP re-route

– Time to detect link failure

– Process the removal of the lost routes from the SW RIB

– Update the HW FIB

Upstream Recovery: ECMP

Downstream Recovery: Routing Protocol

SiSi

SiSiSiSi

SiSi

SiSi SiSi


0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

RPVST+

FHRP

OSPF EIGRP

Upstream

Downstream

Routed Access AdvantagesFaster Convergence Times

• RPVST+ convergence times dependent on FHRP tuning– Proper design and tuning can

achieve sub-second times

• EIGRP converges <200 msec

• OSPF converges <200 msecwith LSA and SPF tuning

Both L2 and L3 Can Provide Sub-Second Convergence

SiSiSiSi

SiSi SiSi


SiSi

Designated

Router

(High IP Address)

IGMP Querier

(Low IP address)

Designated

Router & IGMP

Querier

Non-DR has to

drop all non-RPF

Traffic

SiSiSiSi SiSi

SiSi

Routed Access AdvantagesA Single Router per Subnet: Simplified Multicast• Layer 2 access has two multicast routers per access subnet,

RPF checks and split roles between routers

• Routed Access has a single multicast router which simplifies multicast topology and avoids RPF check altogether


Routed Access Advantages Ease of Troubleshooting

• Routing troubleshooting tools– Consistent troubleshooting:

access, dist, core

– show ip route / show ip cef

– Traceroute

– Ping and extended pings

– Extensive protocol debugs

– IP SLA from the Access Layer

• Failure differences– Routed topologies fail closed—i.e.

neighbor loss

– Layer 2 topologies fail open—i.e. broadcast and unknowns flooded

SiSi

SiSiSiSi

SiSi

L3 L3 L3 L3

L3


Routed Access Design ConsiderationsDesign Constrains

• Can’t span VLANs across multiple wiring closet switches+ Contained Broadcast Domains

+ But can have the same VLAN ID on all closets

• RSPAN no longer possible– Can use ER-SPAN on Catalyst 6500

• IP addressing—do you have enoughaddress space and the allocation planto support a routed access design?–

SiSi

SiSiSiSi

SiSi

L3 L3 L3 L3

L3

SiSi SiSi


Routed Access Design ConsiderationsPlatform Requirements

• Catalyst Requirements

– Cisco Catalyst 3850 & 3650

– Cisco Catalyst 4500

– Cisco Catalyst 6500

• Catalyst IOS IP Base minimum feature set

– EIGRP-Stub – Edge Router

– PIM Stub – Edge Router

– OSPF for Routed Access

– 200 Dynamically Learned Routes

– Catalyst 3x00 Series IOS 12.2(55)SE

– Catalyst 4500 Series IOS 12.2(53)SG

– Catalyst 6500 Series IOS 12.2(33)SXI4

–

SiSi

SiSiSiSi

SiSi

L3 L3 L3 L3

L3

SiSi SiSi


Routed Access Design Migrating from a L2 Access Model

• Typical deployment uses Vlan/Subnet for different user groups

• To facilitate user mobility, vlans extend to multiple closets

DHCPDNS

10.1.20.0/24

10.1.30.0/24

...

10.1.120.0/24

VLAN 20

VLAN 30

...

VLAN 120

EIGRP/OSPF

GLBP ModelVLAN 20

VLAN 30

...

VLAN 120

VLAN 20

VLAN 30

...

VLAN 12020,30 ... 120

User

Groups

User

Groups

interface Vlan20

ip address 10.1.20.3 255.255.255.0

ip helper-address 10.5.10.20

standby 1 ip 10.1.20.1

standby 1 timers msec 200 msec 750

standby 1 priority 150

standby 1 preempt

standby 1 preempt delay minimum 180

interface GigabitEthernet1/1

switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 20-120

switchport mode trunk

switchport nonegotiate

10.5.10.20

SiSiSiSi

SiSiSiSi


DHCPDNS


• As the routing is moved to the access layer, trunking is no longer required

• /31 addressing can be used on p2p links to optimize ip space utilization

10.1.20.0/24

10.1.30.0/24

...

10.1.120.0/24

VLAN 20

VLAN 30

...

VLAN 120

EIGRP/OSPF

GLBP ModelVLAN 20

VLAN 30

...

VLAN 120

VLAN 20

VLAN 30

...

VLAN 12020,30 ... 120

User

Groups

User

Groups

interface Vlan20

ip address 10.1.20.3 255.255.255.0





standby 1 preempt



switchport

switchport trunk encapsulation dot1q

switchport trunk allowed vlan 20-120

switchport mode trunk

switchport nonegotiate

10.5.10.20

SiSiSiSi

L3

L3L3L3 L3

SiSiSiSi


description Distribution Downlink

ip address 10.120.0.196 255.255.255.254


DHCPDNS


• SVI configuration at the access layer is simplified

• Larger subnets used before can simply be split into smaller ones and assigned to new DHCP scopes

10.1.20.0/24

10.1.30.0/24

...

10.1.120.0/24

VLAN 20

VLAN 30

...

VLAN 120

EIGRP/OSPF

GLBP ModelVLAN 20

VLAN 30

...

VLAN 120User

Groups

User

Groups

interface Vlan20

ip address 10.1.20.3 255.255.255.0





standby 1 preempt


10.5.10.20

SiSiSiSi

L3

L3L3L3 L3

interface Vlan20

ip address 10.1.20.3 255.255.255.128


10.1.20.0/25

10.1.30.0/25

...

10.1.120.0/25

10.1.20.128/25

10.1.30.128/25

...

10.1.120.128/25

SiSiSiSi


Agenda - Enterprise Campus Design: Routed Access• Introduction




– EIGRP Design to Route to the Access Layer

– OSPF Design to Route to the Access Layer

– Other Design Considerations



• Summary

35


Deploying a Stable and Fast Converging EIGRPCampus Network

•The key aspects to consider are:1. Using EIGRP Stub at the access layer

2. Route Summarization at the distribution layer

3. Leverage Route filters

4. Consider Hello and Hold Timer tuning


EIGRP NeighborsEvent Detection

• EIGRP neighbor relationships are created when a link comes up and routing adjacency is established

• When physical interface changes state, the routing process is notified– Carrier-delay should be set as a rule because

it varies based upon the platform

• Some events are detected by therouting protocol– Neighbor is lost, but interface is UP/UP

• To improve failure detection– Use routed interfaces and not SVIs– Decrease interface carrier-delay to 0– Decrease EIGRP hello and hold-down timers*

• Hello = 1Hold-down = 3

– * Not recommended with NSF/SSO

SiSiSiSi


ip address 10.120.0.50 255.255.255.252

ip hello-interval eigrp 100 1

ip hold-time eigrp 100 3

carrier-delay msec 0

Hellos

RoutedInterface

SiSi

SiSi

SiSi

L2 Switchor VLAN Interface


EIGRP in the CampusConversion to an EIGRP Routed Edge

• The greatest advantages of EIGRP are gained when the network has an ip addressing plan that allows for use of summarization and stub routers

• EIGRP allows for multiple tiers of hierarchy, summarization and route filtering

• Relatively painless to migrate to a L3access with EIGRP

• Deterministic convergence time in very large L3 topology

• EIGRP maps easily to campus topology

10.10.0.0/1710.10.128.0/17

10.10.0.0/16

SiSi SiSi SiSi SiSi

SiSi SiSi


EIGRP Design Rules for HA CampusLimit Query Range to Maximize Performance

• EIGRP convergence is largely dependent on query response times

• Minimize the number of queries to speed up convergence

• Summarize distribution block routes to limit how far queries propagate across the campus– Upstream queries are returned immediately with infinite cost

• Configure access switches as EIGRP stub routers– No downstream queries are ever sent

SiSiSiSi

SiSiSiSi

router eigrp 100

network 10.0.0.0

eigrp stub connected

interface TenGigabitEthernet 4/1ip summary-address eigrp 100 10.120.0.0 255.255.0.0 5

router eigrp 100network 10.0.0.0distribute-list Default out <mod/port>

ip access-list standard Defaultpermit 0.0.0.0


EIGRP Query ProcessQueries Propagate the Event

• EIGRP is an advanced distant vector; it relies on its neighbor to provide routing information

• If a route is lost and no feasible successor is available, EIGRP actively queries its neighbors for the lost route(s)

• The router waits for replies from all queried neighbors before the calculating a new path

• If any neighbor fails to reply,the queried route is stuck inactive and the router resetsneighbor adjacency

• The fewer routers and routesqueried, the faster EIGRP converges; solution is to limit query propagation

SiSiSiSi

Query

SiSiSiSi

SiSiSiSi

Query

Query

Query

Query

Query

Query

Query

Query

Reply

Reply

Reply

Reply

Reply

Reply

Reply

Reply

ReplyAccess

Distribution

Core

Distribution

Access


No Queries to Rest of Network

from Core

Limiting the EIGRP Query RangeWith Summarization

• When we summarize from distribution to core for the subnets in the access we can limit the upstream query/reply process

• In a large network this could be significant because queries will now stop at the core; no additional distribution blocks will be involved in the convergence event

• The access layer is still queriedSiSiSiSi

SiSiSiSi

Query Query

Query ReplyReply

Reply

Reply∞Reply∞

interface gigabitethernet 3/1

ip address 10.120.10.1 255.255.255.252

ip summary-address eigrp 1 10.130.0.0 255.255.0.0

Summary

RouteSummary

Route


Limiting the EIGRP Query RangeWith Stub Routers

• A stub router signals (through hellos) that it is a stub and not a transit path

• Queries are not sent towards the stub routers but marked as if a “No path this direction” reply had been received

• D1 knows that stubs cannot be transit paths, so they will not have any path to 10.130.1.0/24

• D1 will not query the stubs, reducing the total number of queries in this example to one

• Stubs will not pass D1’s advertisement of 10.130.1.0/24 to D2

• D2 will only have one path to 10.130.1.0/24

D2D1 Query

Distribution

Access

SiSi SiSi

STUB

10.130.1.0/24

Hello, I’m a

Stub—

I’m Not Going to

Send You Any

Queries Since

You Said That

Stub Stub Stub

Reply


No Queries to Rest of Network

from Core

EIGRP Query ProcessWith Summarization and Stub Routers• When we summarize from distribution

into core we can limit the upstream query/reply process

• Queries will now stop at the core; no additional routers will be involved in the convergence event

• With EIGRP stubs we can furtherreduce the query diameter

• Non-stub routers do not query stub routers—so no queries will be sent to the access nodes

• Only three nodes involved in convergence event—No secondary queries

SiSiSiSi

SiSiSiSi

Query Reply

Reply∞Reply∞

Stub Stub

Summary

RouteSummary

Route


SiSiSiSi

SiSiSiSi

EIGRP Route Filtering in the CampusControl Route Advertisements

• Bandwidth is not a constraining factor in the campus but it is still advisable to control number of routing updates advertised

• Remove/filter routes from the core to the access and inject a default route with distribute-lists

• Smaller routing table in access is simpler to troubleshoot

• Deterministic topologyip access-list standard Defaultpermit 0.0.0.0

router eigrp 100network 10.0.0.0distribute-list Default out <mod/port>

Default

0.0.0.0

Default

& other

Routes


SiSiSiSi

SiSiSiSi

EIGRP Routed Access Campus DesignSummary

• Detect the event:

– Set hello-interval = 1 second and hold-time = 3 seconds to detect soft neighbor failures *

– Set carrier-delay = 0

• Propagate the event:

– Configure all access layer switches as stubrouters to limit queries from the distribution layer

– Summarize the routes from the distribution to the core to limit queries across the campus

• Process the event:

– Summarize and filter routes to minimize calculating new successors for the RIB and FIB

– * Not recommended with NSF/SSO

Summary

Route

Stub

Default

0.0.0.0

Stub Stub

Default

& other

Routes











• Summary

46


Deploying a Stable and Fast Converging OSPFCampus Network

• Key Objectives of the OSPF Campus Design:1. Map area boundaries to the hierarchical design

2. Enforce hierarchical traffic patterns

3. Minimize convergence times

4. Maximize stability of the network


OSPF Design Rules for HA CampusWhere Are the Areas?

• Area size/border is bounded by the same concerns in the campus as the WAN

• In campus the lower number of nodes and stability of local links could allow you to build larger areas however-

• Area design also based on address summarization

• Area boundaries should define buffers between fault domains

• Keep area 0 for core infrastructure do not extend to the access routers

Data CenterWAN Internet


SiSiSiSi

SiSiSiSi

SiSiSiSi

SiSiSiSi

Area 100 Area 110 Area 120

Area 0


Hierarchical Campus DesignOSPF Areas with Router Types

Data CenterWAN InternetBGP


SiSi SiSi

SiSi SiSi

SiSi SiSi

Area 0

Area 200

Area 20 Area 30Area 10

BackboneBackbone

ABR ABR

InternalInternal

Area 0

ABR

Area 100

ASBR

ABR

ABR

Area 300

Access

Distribution

Core

Distribution

Access

SiSiSiSi


OSPF in the CampusConversion to an OSPF Routed Edge

• OSPF designs that utilize an area for each campus distribution building block allow for straight forward migration to Layer 3 access

• Converting L2 switches to L3within a contiguous area is reasonable to consider as long as new area size is reasonable

• How big can the area be? – It depends

– Switch type(s)

– Number of links

– Stability of fiber plantArea 200Branches

Area 0Core

Area 10Dist 1

Area 20Dist 2

SiSi SiSi SiSi SiSi

SiSiSiSi


When a Link Changes State

• Every router in area hears a specific link LSA

• Each router computes shortest path routing table

Router 2, Area 1

Old Routing Table New Routing Table

Link State Table

LSA

Dijkstra Algorithm

ACKSiSi

Router 1, Area 1


OSPF LSA ProcessLSAs Propagate the Event• OSPF is a Link State protocol; it relies

on all routers within an area having the same topology view of the network.

• If a route is lost, OSPF sends out an LSA to inform it’s peers within the area of the lost route.

• All routers with knowledge of this route in the OSPF network will receive an LSA and run SPF to remove the lost route.

• The fewer the number ofrouters with knowledge of theroute, the faster OSPF converges;

• Solution is to limit LSApropagation range

SiSiSiSi

LSA 2

SiSiSiSi

SiSiSiSi

LSA 2

LSA 2

LSA 2

LSA 2

LSA 2

LSA 2

LSA 2

LSA 2

Area 0

Area 0

SPF

SPF SPF

SPF

SPF SPF

SPF SPF

SPF SPF

Access

Distribution

Core

Distribution

Access


SiSiSiSi

Backbone

Area 0

Area 120

OSPF Regular AreaABRs Forward All LSAs from Backbone

ABR Forwards theFollowing into an Area

Summary LSAs (Type 3)

ASBR Summary (Type 4)

Specific Externals (Type 5)

Access Config:router ospf 100

network 10.120.0.0 0.0.255.255 area 120

Distribution Configrouter ospf 100

area 120 range 10.120.0.0 255.255.0.0 cost 10

network 10.120.0.0 0.0.255.255 area 120

network 10.122.0.0 0.0.255.255 area 0

SiSiSiSi

External Routes/LSA Present in Area 120


SiSiSiSi

Backbone

Area 0

Area 120

OSPF Stub AreaConsolidates Specific External Links—Default 0.0.0.0

Stub Area ABR ForwardsSummary LSAs

Summary 0.0.0.0 Default


area 120 stub

area 120 range 10.120.0.0 255.255.0.0 cost 10

network 10.120.0.0 0.0.255.255 area 120

network 10.122.0.0 0.0.255.255 area 0

SiSiSiSi


network 10.120.0.0 0.0.255.255 area 120

Eliminates External Routes/LSA Present in Area (Type 5)


SiSi

Backbone

Area 0

Area 120

A Totally Stubby AreaABR Forwards

Summary Default

OSPF Totally Stubby AreaUse This for Stable—Scalable Internetworks


area 120 stub no-summary

area 120 range 10.120.0.0 255.255.0.0 cost 10

network 10.120.0.0 0.0.255.255 area 120

network 10.122.0.0 0.0.255.255 area 0


network 10.120.0.0 0.0.255.255 area 120

SiSi

SiSi

SiSi

Minimize the Number of LSAs and the Need for Any External Area SPF Calculations


SiSi

Backbone

Area 0

Area 120

Area Border Router

ABRs ForwardSummary 10.120.0.0/16

Summarization Distribution to CoreReduce SPF and LSA Load in Area 0


network 10.120.0.0 0.0.255.255 area 120


area 120 stub no-summary

area 120 range 10.120.0.0 255.255.0.0 cost 10

network 10.120.0.0 0.0.255.255 area 120

network 10.122.0.0 0.0.255.255 area 0

SiSi

SiSiSiSi

Minimize the Number of LSAs and the Need for Any SPF Recalculations at the Core


SiSiSiSi

SiSiSiSi

OSPF Design ConsiderationsWhat Area Should the Distribution Link Be In?

• Two aspects of OSPF behavior can impact convergence– OSPF ABRs ignore LSAs generated by other

ABRs learned through non-backbone areas when calculating least-cost paths

– In a stub area environment the ABR will generate a default route when any typeof connectivity to the backbone exists

• Ensure loopbacks are ‘not’ in area 0

• Configure dist to dist link as a trunk using 2 subnets one in area 0 and one in stub area when possible


SiSi

SiSi

OSPF Timer TuningHigh-Speed Campus Convergence

• OSPF by design has a number of throttling mechanisms to prevent the network from thrashing during periods of instability

• Campus environments are candidates to utilize OSPF timer enhancements

– Sub-second hellos*

– Generic IP (interface) dampening mechanism

– Back-off algorithm for LSA generation

– Exponential SPF backoff

– Configurable packet pacing ReduceLSA and SPF

Interval

SiSi

SiSi

Reduce Hello Interval

* Not recommended with NSF/SSO


Access Config:interface GigabitEthernet1/1dampeningip ospf dead-interval minimal hello-multiplier 4ip ospf network point-to-point

router ospf 100timers throttle spf 10 100 5000timers throttle lsa all 10 100 5000timers lsa arrival 80

Subsecond HellosNeighbor Loss Detection—Physical Link Up

• OSPF hello/dead timers detect neighbor loss in the absence of physical link loss

• Useful in environments where anL2 device separates L3 devices(Layer 2 core designs)

• Aggressive timers quickly detectneighbor failure

• Not recommended with NSF/SSO

• Interface dampening is recommended with sub-second hello timers

• OSPF point-to-point network type to avoid designated router (DR) negotiation.

OSPF Processing

Failure(Link Up)

A B

SiSi

SiSi

SiSi

SiSi


5.68

0.72

0.24

0

1

2

3

4

5

6

Default

Convergence

10 msec. SPF 10 msec. SPF

and LSA

OSPF Requires Sub-Second Throttling of LSATimers to Speed Convergence• OSPF has an SPF throttling timer designed

to dampen route recalculation

• After a failure, the router waits for the SPF timer to expire before recalculating a new route

• By default, there is a 500ms delay before generating router and network LSAs; the wait is used to collect changes during a convergence event and minimize the number of LSAs sent

• Propagation of a new instanceof the LSA is limited at the originator

• Acceptance of a new LSAs is limited by the receiver

• Make sure lsa-arrival < lsa-hold

Tim

e t

o R

es

tore

Vo

ice

Flo

ws

(se

c)

timers throttle spf 10 100 5000

timers throttle lsa all 10 100 5000

timers lsa arrival 80



timers lsa arrival 80


OSPF Design Rules for HA CampusLSA/SPF Exponential Back-off Throttle Mechanism

• Sub-second timers without risk 1. spf-start or initial hold timer controls how long to wait prior to starting the SPF

calculation

2. If a new topology change event is received during the hold interval, the SPF calculation is delayed until the hold interval expires and the hold interval is temporarily doubled

3. The hold interval can grow until the maximum period configured is reached

4. After the expiration of any hold interval, the timer is reset

timers throttle spf <spf-start> <spf-hold> <spf-max-wait>

timers throttle lsa all <lsa-start> <lsa-hold> <lsa-max-wait>

Time [ms]

Topology Change Events

SPF Calculations

200 1600 msec100 400 800 msec













• Summary

62


Routing Protocol Churn Can Be Reduced with IP Event Dampening

• Prevents routing protocol churn caused by constant interface state changes

• Dampening is applied on a system: nothingis exchanged between routing protocols

• Supports all IP routing protocols– Static routing, RIP, EIGRP, OSPF, IS-IS, BGP

– In addition, it supports HSRP and CLNS routing

– Applies on physical interfaces and can’t be applied on subinterfaces individually

Up

Up

Interface State Perceived by EIGRP or OSPF

Interface State


description Uplink to Distribution 1

dampening

ip address 10.120.0.205 255.255.255.254

Down

Up

Down

SiSi

SiSiSiSi

UpDown

Up

UpDown

Down


Using Redundant Supervisors at the Access Layer with SSO

1. Supervisor switchover event occurs

2. SSO maintains SSO-aware applications, including L2 tables, L2/L3 forwarding is maintained

3. Routing protocols will restart on the newly active Supervisor– L3 routes are purged stopping L3 forwarding

4. Routing neighbors lose adjacency with the restarting router– Routes to the lost neighbor are purged

5. Routing neighbors reestablish adjacencies, forwarding to and from non-directly connected L3 networks resumes

SiSiSiSi

SiSi SiSi

SSO alone is not enough with a Routed Access

do not run SSO w/o NSF in the RA design


NSF—Configuration and Monitoring

Switch(config)#router eigrp 100

Switch(config-router)#nsf

Router#sh ip ospfRouting Process "ospf 100" with ID 10.120.250.4Start time: 00:01:37.484, Time elapsed: 3w2dSupports Link-local Signaling (LLS)

<snip>Non-Stop Forwarding enabled, last NSF restart 3w2d ago (took 31 secs)

Router#sh ip protocol

*** IP Routing is NSF aware ***

Routing Protocol is "eigrp 100 100"

<snip

EIGRP NSF-aware route hold timer is 240s

EIGRP NSF enabled

EIGRP

Switch(config)#router ospf 100

Switch(config-router)#nsf

NSF-Capable

NSF-Aware

OSPF

Recommendation Is to Not Tune IGP Hello Timers. Use Default Hello and Dead

Timers for EIGRP/OSPF When Peering to a Device Configured for NSF/SSO


Using Redundant Supervisors at the Access Layer, Now with NSF/SSO

1. Supervisor switchover event occurs

2. SSO maintains SSO-aware applications,including L2 tables, L2/L3 forwarding is

maintained

3. NSF-capable router signals NSF-aware routing peers of a routing protocol restart

4. NSF-aware routers detect the restarting router– Assist in re-establishing full adjacency– Maintain forwarding to and from the

restarting router

5. NSF restart complete, traditional L3convergence event is avoided

2

SiSiSiSi

SiSi SiSi

1

4

3


SiSiSiSi

SiSiMasterAccess

S1 S2 S3

Single logical Switch

SiSiSiSi

Design Consideration with StackWise at the Access Layer

• Recommended Design:– Configure priority for master and its backup for

deterministic failures

– Avoid using master as uplink to reduce uplink related losses

– Use “stack-mac persistent timer 0” to avoid the gratuitous ARP changes for

• Best convergence

• Where GARP processing is disabled in the network, e.g. Security

• Where network devices/host do not support GARP, e.g. Phones

• Upstream traffic is not interrupted by master failure

• Downstream traffic is interrupted due to routing protocol restart and adjacency reset– Run 12.2(37)SE or higher for NSF support


Routed Access Does Not Require Switch Management Vlan

• In the L2 design it was considered a best practice to define a unique Vlan for network management

• In the routed access model, the best way is to configure a loopback interface

• The /32 address should belong to the summarized routed advertised from the distribution block

• The loopback interface should be configured as passive for the IGP

• ACLs should be used as required to ensure secure network management

SiSi

SiSiSiSi

SiSi

SiSi SiSi

SNMP Server

interface Loopback0

description Dedicated Switch Management

ip address 10.120.254.1 255.255.255.255



• Introduction






• Summary

69


Virtual SwitchCatalyst 6500 Virtual Switching System (VSS)• Virtual Switching System consists of two Catalyst 6500’s defined as members

of the same virtual switch domain running a VSL (Virtual Switch Link) between them

• Single Control Plane with Dual Active Forwarding Planes

• Extends NSF/SSO infrastructure to Two Switches

VSS

SiSiSiSi

Switch 1 + Switch 2 =

Virtual Switch Domain

Virtual Switch Link (VSL)


Virtual Switch SystemImpact to the Campus Topology

Physical network topology does not change

Still have redundant chassis

Still have redundant links

Logical topology is simplified as we now have a single control plane

Allows the design to replace traditional topology control plane with Multi-chassisEtherchannel (MEC)

No reliance on IGP Protocol to provide linkredundancy

Convergence and load balancing are based on Etherchannel

SiSiSiSi SiSiSiSi SiSiSiSi SiSiSiSi

BRKCRS-3035 – Advance Enterprise Campus Design: Virtual Switching System (VSS)

SiSiSiSi


VSS and Routed Access DesignLink Down Convergence Without VSS

• Downstream traffic recovery is dependent upon the Interior Gateway Protocol reroute to the peer distribution switch – Use Stub on the access devices, and

proper summarization from distribution– Tune IGP ... etc.

• Upstream traffic recovery is dependent upon updates to the Access Switch’s Forwarding Information Base removing the adjacency for the lost link (ECMP)

Downstream IGP rerouteUpstream CEF ECMP

SiSi

SiSi

SiSi

SiSiL3 ECMP


• Access layer switch has one neighbor

• Distribution switch has neighbor count reduced by half

• Upstream and Downstream traffic convergence now is an Etherchannellink event– No IGP reconvergence event– No Impact of number of routes/vlans

• Fast IGP Timers not needed nor recommended (only 1 IGP peer)

• Summarization rules still recommended

• Achieves sub-second failure and no L2loop on the topology

VSS and Routed Access DesignLink Down Convergence with VSS MEC

Downstream IGP rerouteUpstream CEF ECMP

SiSi

SiSiSiSi

SiSiL3 ECMPMEC



• Introduction





• Routed Access Design for IPv6


• Summary

74


Analyzing the Impact on Advanced Technologies

• Unified Communications Deployments work the same way. You still need to provision a voice vlan/subnet per wiring closet switch

• TrustSec (802.1x) solutions work the same: user vlan assigment still possible, as well as per user dACL (checkout BRKSEC-2005)

• Wireless LAN works seamlessly as well, since LWAPP works with UDP hence at L3.

• We will take a closer look at;

– Network Virtualization


• Access control techniques remain the same with a Routed Access Model

• Path Isolation techniques remain the same, but there are provisioning implications by running routing at the access layer

Network Virtualization Functional Architecture

Access Control Path Isolation Services Edge

WAN – MAN – Campus Branch – Campus Data Center – Internet Edge –Campus

EthernetVRFs

GREVRFs

MPLSVPNs

BRKCRS-2033 – Deploying a Virtualized Campus Network Infrastructure


VRFVRF

Global

Path IsolationFunctional Components

• Device virtualization–Control plane virtualization

–Data plane virtualization

–Services virtualization

• Data path virtualization–Hop-by-Hop–(VRF-Lite End-to-End)

–Multi-Hop–(VRF-Lite+GRE, MPLS-VPN)

VRF: Virtual Routing and Forwarding

Per VRFVirtual Routing Table

Virtual Forwarding Table

IP

802.1q


Network Virtualization and Routed AccessPath Isolation Issues—VRFs to the Edge

• Define VRFs on the access layer switches

• One VRF dedicated to each virtual network (Red, Green, etc.)

• Map device VLANs to the corresponding VRF

• Provisioning is more challenging, because multiple routing processes and logical interfaces are required.

• The chosen path isolation technique must be deployed from the access layer devices

VRF-lite Ethernet

– VRF-Lite GRE– MPLS L3 VPNs

Campus Core

Layer 3

Links

SiSiSiSi

VLAN 21 Red

VLAN 22 Green

VLAN 23 Blue

VLAN 21 Red

VLAN 22 Green

VLAN 23 Blue

VRF Blue

VRF Green

VRF Red


Network Virtualization and Routed AccessPath Isolation Issues—VRFs to the Edge (Cont.)

• Catalyst 6500 supports all three path isolation techniques:– 802.1Q Ethernet VRF-Lite

– GRE with VRF-Lite

– MPLS VPN

• Catalyst 3000s and 4500s only support 802.1Q Ethernet VRF-Lite

• Convergence times increase– ~800ms for 9 VRFs + Global

– Increased load from multiple routing processes and logical interfaces

• Operational impact of managing multiple logical networks

Campus Core

Layer 3

Links

SiSiSiSi

VLAN 21 Red

VLAN 22 Green

VLAN 23 Blue

VLAN 21 Red

VLAN 22 Green

VLAN 23 Blue

VRF Blue

VRF Green

VRF Red

Network Virtualization--Path Isolation Design Guide

http://www.cisco.com/en/US/docs/solutions/Enterprise/Network_Virtualization/PathIsol.html#wp277205

http://www.cisco.com/en/US/docs/solutions/Enterprise/Network_Virtualization/PathIsol.html#wp277205



• Introduction





• Routed Access Design for IPv6


• Summary

82


= STP Blocked Link

STP-Based Redundant Topology

B

Routed Access Redundant Topology

SiSi SiSi

SiSi SiSi

SiSi SiSi

SiSi SiSi SiSi SiSi

SiSi SiSi

SiSi SiSi

SiSi SiSi

Routed Access Campus DesignEnd to End Routing: Fast Convergence and Maximum Reliability

B

BB

B

Q&A


Summary

• Traditional Layer 2 designs remain valid

• Routed Access Design:– Simplified Control Plane (no

dependence on STP, HSRP, etc.)– Increased Capacity: Provide flow-

based load balancing– High Availability: 200 msec or better

recovery– Simplified Multicast– No L2 Loops– Easy Troubleshooting

• Flexibility to provide for the right implementation for each network requirement

SiSi SiSi SiSi SiSi

SiSi SiSi


Campus Design GuidanceWhere To Go for More Information

http://www.cisco.com/go/srnd


Call to Action

• Visit the World of Solutions for

– Cisco

– Walk in Labs

– Technical Solution Clinics

• Meet the Engineer

• Lunch time Table Topics

• DevNet zone related labs and sessions

• Recommended Reading: for reading material and further resources for this session, please visit www.pearson-books.com/CLMilan2015

87

http://www.pearson-books.com/CLMilan 2015


Complete Your Online Session Evaluation

• Please complete your online sessionevaluations after each session.Complete 4 session evaluations& the Overall Conference Evaluation(available from Thursday)to receive your Cisco Live T-shirt.

• All surveys can be completed viathe Cisco Live Mobile App or theCommunication Stations

88


Interested in Learning about Next Gen Solutions?

• Have your account team setup a meeting @ Enterprise Segment Innovation Forum

• Requirements– Cisco Account Team Presence

– Cisco NDA in Place

• Please use the address if you have any queries…

[email protected]

• We are at MiCo - Milano Congressi, Piazzale Carlo Magno 1, 20149 Milano Italy, Meeting Village, North Building, Level 1

89


Continue Your Education

• Demos in the Cisco Campus

• Walk-in Self-Paced Labs

• Table Topics

• Meet the Engineer 1:1 meetings

90