IS-IS Dynamic Flooding in Data Center Networks White Paper · protocol convergence in data center networks. The feature is being developed as per the draft draft-ietf-lsr-dynamic-flooding,

© 2019 Cisco and/or its affiliates. All rights reserved. Page 1 of 20

IS-IS Dynamic Flooding in Data Center Networks

White Paper

Cisco public


Contents

What you will learn 3

Introduction 3

IS-IS flooding reduction 4

Dynamic flooding 4

Mode of operation 5

IS-IS TLVs 6

Configuration 7

Interoperability 18

Caveats and limitations 18

Efficiency of flooding algorithm 19

Summary 20


What you will learn

This paper discusses the IS-IS dynamic flooding solution, which will greatly help optimize IS-IS routing

protocol convergence in data center networks. The feature is being developed as per the draft draft-ietf-

lsr-dynamic-flooding, which addresses dynamic flooding in IS-IS and OSPF protocols. Cisco provides an

initial implementation of IS-IS dynamic flooding, based on the current IETF draft status. This feature is

available with Cisco NX-OS Release 9.3(1). The feature with complete conformance to the IETF RFC will be

present in an upcoming NX-OS release. This paper discusses how the solution works, along with

configuration details and scalability examples in a typical data center fabric topology.

Introduction

Modern day data centers have rapidly evolving trends and technologies that underscore the need for scale

and fast convergence. In order to efficiently route traffic within the data center, Interior Gateway Protocols

(IGPs) are deployed either in the network core or within the underlay network. The IGPs commonly used

today are Open Shortest Path First Protocol (OSPF) and Intermediate System to Intermediate System

Protocol (IS-IS), which are both link-state routing protocols. IS-IS is regularly deployed in service provider

networks but has a wide range of applications in data center and enterprise networks. While OSPF is a

Layer 3 protocol that runs on top of IP, IS-IS is an Open Systems Interconnection (OSI) Layer 2 protocol

that is protocol-agnostic. Due to this, IS-IS can easily support multiple protocols (IPv4, IPv6, etc.) and

optimizations through TLV extensions, as will be explored further in this document.

Traditional routing protocols were designed for networks where bandwidth was expensive and the

underlying infrastructure was unreliable, thereby involving a lot of redundancy and resiliency to guarantee

stable convergence. With dense topologies and large scale in data centers in recent years, their

performance is suboptimal.

Today’s data center networks support spine-leaf or bipartite topologies, CLOS networks that have full

mesh connectivity between layers with ECMP paths, and fat-tree topologies in which the link bandwidth

progressively increases toward the core of the network. These topologies follow a scale-out model in

MSDC environments. Conventional IGP protocols including IS-IS, OSPFv2, and OSPFv3 under-perform in

these topologies by redundantly flooding information throughout the dense topology. The redundant

information still needs to be processed at the routers’ control plane before being discarded, and may often

get queued ahead of significant updates. This overloading of the control plane could cause delays and

retransmissions, further exacerbating the performance to the point that the IGP can no longer efficiently

scale. Network operators have resorted to using BGP in certain cases, which is suboptimal due to a lot of

reasons, including configuration overhead. Various proposals are under discussion at the IETF that address

different aspects of the problem, such as what information needs to be flooded, where to flood the

information, and extensions to the protocols themselves.

This paper describes the solutions proposed to increase scalability and convergence of IS-IS in data center

fabric topologies.

https://tools.ietf.org/html/draft-ietf-lsr-dynamic-flooding

https://tools.ietf.org/html/draft-ietf-lsr-dynamic-flooding


IS-IS flooding reduction

IS-IS is commonly used as an underlay routing protocol for MSDC networks. In its current implementation,

a given IS-IS router within the CLOS topology would receive multiple copies of exactly the same LSP (link-

state packet) from multiple IS-IS neighbors. In addition, two IS-IS neighbors may send each other the

same LSP simultaneously. Ultimately, all of the redundant copies will be discarded, but only after they have

reached the control plane and been processed. The unnecessary link-state information flooding wastes the

precious process resources of IS-IS routers greatly because there are too many IS-IS neighbors for each

IS-IS router within the CLOS topology.

The reduction of IS-IS flooding is very beneficial to improve the scalability and convergence of MSDC

networks.

These problems are not new and were seen when the underlying link layer fabric presented the network

layer with a full mesh of virtual connections. This was addressed by reducing the flooding topology through

IS-IS mesh groups, but this approach requires careful configuration of the flooding topology.

Dynamic flooding

The dynamic flooding solution consists of de-coupling the “flooding topology” from the physical topology.

This means that prior to any topology changes, a separate flooding topology is established on all the nodes

that support the feature. Nodes within an IGP area would then only flood updates on the flooding topology.

Legacy flooding rules would apply to nodes that do not support the feature. This flooding topology is

typically a subset of the physical topology, but this could vary based on the nature of the topology (the

solution is more efficient if there is a greater number of ECMP paths, for example).

In the centralized mode of operation, the flooding topology is computed on the Area Leader and is then

encoded and distributed to all other nodes as part of the normal link-state database. In the distributed

mode of operation, the identifier of the dynamic flooding algorithm to be used is distributed within the Area

Leader Sub-TLV. Each of the nodes then computes the flooding topology based on this algorithm, in a

distributed fashion.

Note: The initial Cisco NX-OS implementation will support only a distributed mode of operation.

With dynamic flooding, nodes within such an IGP area would only flood on the flooding topology. On links

outside of the flooding topology, normal database synchronization mechanisms (that is, IS-IS CSNPs

[complete sequence number protocol data units]) would apply, but flooding may not. New link-state

information that arrives from outside of the flooding topology suggests that the sender has different or no

flooding topology information and that the link-state update should be flooded on the flooding topology as

well.


Mode of operation

All devices participating in dynamic flooding will elect a unique Area Leader. For redundancy, more than

one Area Leader may be configured.

In the distributed mode of operation in NX-OS, each device participating in dynamic flooding will compute

the flooding topology independently using the algorithm instructed by the Area Leader. The initial Cisco

NX-OS implementation supports an algorithm called “Cisco-Dual-SPT-V1”.

Once the final flooding topology is calculated, flooding will happen only on that topology and no longer on

all of the IS-IS physical links.

Cisco-Dual-SPT-V1 algorithm

The Cisco-Dual-SPT-V1 algorithm calculates a flooding topology with the Area Leader as root. Optionally,

a secondary Area Leader can be configured. In this case, there would be two flooding topologies, based

on the primary and secondary Area Leaders respectively as root.

The algorithm will attempt to find unique paths on each of the two flooding topologies. This will ensure that

we have a redundant path (if possible) to reach all the nodes in the IS-IS domain.

Each calculated flooding topology will include all the nodes in the IS-IS domain but may not include all the

links in the topology.

Since all the nodes participating in the dynamic flooding elect the same primary and secondary Area

Leader, the distributed calculation of the flooding topology will be exactly the same.

The final flooding topology is the sum of the two flooding topologies.

Figure 1.

Physical flooding topology

The flooding topology shown below covers the full set of nodes within the area but excludes some of the

links.


Figure 2.

Primary and secondary flooding topologies

Since the flooding topology is computed prior to topology changes, it does not factor in the convergence

time and can be done when the topology is stable.

During transients, it is possible that loops will form in the flooding topology. This is not problematic, as

legacy flooding rules would cause duplicate updates to be ignored. Similarly, during transients, it is

possible that the flooding topology may become disconnected, in which case temporary flooding may be

used.

IS-IS TLVs

In order to compute the flooding topology, the feature includes three new TLVs when the feature is used in

distributed mode. For the centralized mode of operation, an additional two TLVs are defined (IS-IS Area

Node IDs TLV and IS-IS Flooding Path TLV).

IS-IS Area Leader Sub-TLV

The Area Leader Sub-TLV allows a system to:

● Indicate its eligibility and priority for becoming Area Leader

● Indicate whether centralized or distributed mode will be used to compute the flooding topology in

the area

● Indicate the algorithm identifier for the algorithm that will be used to compute the flooding topology

in distributed mode

Nodes that do not advertise this Sub-TLV are not eligible to become Area Leader. The Area Leader is the

node with the numerically highest Area Leader priority in the area. In the event of ties, the node with the

numerically highest system ID is the Area Leader.


The Area Leader Sub-TLV is advertised as a Sub-TLV of the IS-IS router capability TLV-242.

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

| Type | Length | Priority | Algorithm |

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

IS-IS Dynamic Flooding Sub-TLV

The Dynamic Flooding Sub-TLV allows a system to indicate that it supports Dynamic Flooding, along with

the set of algorithms it supports for distributed mode.

IS-IS Flooding Request TLV

The Flooding Request TLV allows a device to request an adjacent node to enable flooding toward itself on

a specific link in the case where the connection to an adjacent node is not part of the existing flooding

topology.

Nodes that support dynamic flooding MAY include the Flooding Request TLV in their IIH (IS-IS Hello) PDUs.

Configuration

Note: The configuration commands and examples are based on the initial implementation of IS-IS

dynamic flooding in NX-OS 9.3(1).

The feature is enabled using the following statement within the “router isis” configuration mode.

[no] dynamic-flooding

This creates a placeholder with all the default configuration for dynamic flooding for that specific level.

Example

n9k-1(config-router)# dynamic-flooding

n9k-1(config-router-df)#

[no] algorithm algorithm-id <algorithm-id> algorithm-name <algorithm-name>

The above command creates the mapping for the algorithm ID to the user-defined name. In the initial

implementation, only algorithm ID 128 and algorithm name “cisco-dual-spt-v1” are supported.

Example

n9k-1(config-router-df)# algorithm algorithm-id 128 algorithm-name cisco-dual—spt-v1

[no] area-leader priority <priority> algorithm-id <algorithm id> [<level-1 | level-2>]

The above command enables the router to participate in the Area Leader election. The second command

configures the priority and the algorithm for the Area Leader election.

Example

The command below is used to enable the router in Area Leader election:

n9k-1(config-router-df)# area-leader priority 177 algorithm 128

The command below is used to disable the router in Area Leader election with specific level values:

n9k-1(config-router-df)# no area-leader priority 177 algorithm 128


A sample configuration from a router configured for dynamic flooding with a priority for Area Leader is

shown below. Please note the bold-faced configuration commands used to enable the feature and

configure Area Leaders.

feature isis

router isis 9964

net 49.0001.0000.0002.0005.00

address-family ipv4 unicast

dynamic-flooding

area-leader priority 40 algorithm-id 128

interface Ethernet1/1

isis network point-to-point

ip router isis 9964

Here, the priority can be defined as a value between 0 and 254. The algorithm-id field needs to be

configured with the value 128, which is assigned to the cisco-dual-spt-v1 algorithm from the private

distributed algorithm space of the Area Leader sub TLV for IS-IS.

Below is a sample configuration from a router configured as secondary Area Leader, based on the value

chosen above for the primary Area Leader:

feature isis

router isis 9964

net 49.0001.0000.0002.0002.00


dynamic-flooding

area-leader priority 10 algorithm-id 128



ip router isis 9964


Below is a sample configuration from a router configured as a node participating in the dynamic flooding

topology:

router isis 9964

net 49.0001.0000.0002.0004.00


dynamic-flooding



ip router isis 9964

Verification of dynamic flooding configuration

A new set of commands can be used to verify the configuration of the flooding topology, as shown below:

show isis [<isis-tag>] dynamic-flooding [detail]

Example

The following output shows the configuration on the Area Leader:

n9k-6# show isis 9964 dynamic-flooding

ISIS Process ID 9964, VRF default

Level 0

Dynamic flooding not enabled

Level 1

Dynamic flooding enabled

My dynamic flooding Info:

Area Leader capable: Yes

Algorithm: Dual SPT - V1/128

Priority: 61

Elected Primary Leader Info:

No primary leader for L1

Elected Secondary Leader Info:

No secondary leader for L1


Level 2



Area Leader capable: Yes

Algorithm: Dual SPT - V1/128

Priority: 61


L2 Algorithm: Dual SPT - V1/128

Priority: 2

SystemId: 1251.0010.0061


L2 Algorithm: Dual SPT - V1/128

Priority: 2

SystemId: 1251.0010.0060

A detailed CLI output is shown below:

n9k-3# show isis dynamic-flooding detail


Level 0

Dynamic flooding not enabled

Level 1


Reachablity Matrix for L1:

No Reachablity Matrix

Local FT interfaces:

Local temp FT interfaces:

Level 2


Reachablity Matrix for L2:

Src NodeId: 1251.0010.0060

Neighbor NodeId 1251.0010.0061

Used overall/tree-1/tree2: 0/0/1 Metric: 4261412864

No valid interface



No valid interface



No valid interface




No valid interface

Src NodeId: 1251.0010.0061



No valid interface



No valid interface

Src NodeId: 1251.0010.1111



No valid interface



No valid interface



No valid interface

Src NodeId: 1251.0010.2222



No valid interface



No valid interface

Src NodeId: 1251.0010.3333



Interface ID: 4 Name: Ethernet1/29







Src NodeId: 1251.0010.4444



No valid interface




No valid interface

Local FT interfaces:

Ethernet1/9

Ethernet1/29

Ethernet1/49

Local temp FT interfaces:

In the above output, “Local FT interfaces” refers to flooding topology interfaces, which are the interfaces

that will be used during dynamic flooding. “Local temp FT interfaces” will show the temporary interfaces

that will be used to flood IS-IS updates during a reconvergence of the flooding topology, such as when

there is a link failure.

In addition to the above command outputs, traditional IS-IS commands will have outputs that reflect the

flooding topology feature.

show isis [<isis-tag>] protocol

Example

n9k-1# show isis DYN-FLOOD protocol

ISIS process : DYN-FLOOD

Instance number : 2

UUID: 1107296536

Process ID 405

VRF: default

System ID : None IS-Type : L1-L2

SAP : 413 Queue Handle : 14

Maximum LSP MTU: 1000

Stateful HA enabled

Graceful Restart enabled. State: Inactive

Last graceful restart status : successful

Start-Mode Complete

BFD IPv4 is globally disabled for ISIS process: 1

BFD IPv6 is globally disabled for ISIS process: 1

Topology-mode is base

Metric-style : advertise(wide), accept(narrow, wide)

Area address(es) :

None

Process is disabled because :

NET is not specified

VRF ID: 1

Stale routes during non-graceful controlled restart

Enable resolution of L3->L2 address for ISIS adjacency

SR IPv4 is not configured and disabled for ISIS process: 1


SR IPv6 is not configured and disabled for ISIS process: 1

Interfaces supported by IS-IS :

Topology : 0

Address family IPv4 unicast :

Number of interface : 0

Distance : 115



Distance : 115

Topology : 2



Distance : 115



Distance : 115

Level1

Auth type:MD5

Auth check set

Level2

No auth type and keychain

Auth check set

L1 Next SPF: Inactive

L2 Next SPF: Inactive

Level 1 Dynamic flooding enabled

L1 Area Leader capable: Yes

L1 Algorithm: Dual SPT - V1/128 Priority: 61

Level 2 Dynamic flooding enabled

L2 Area Leader capable: Yes

L2 Algorithm: Dual SPT - V1/128 Priority: 61

Attached bits

MT-0 L-1: Att 0 Spf-att 0 Cfg 1 Adv-att 0

MT-0 L-2: Att 0 Spf-att 0 Cfg 1 Adv-att 0

show isis [<isis-tag>] database detail


Example

n9k-5# show isis database detail

IS-IS Process: 1 LSP database VRF: default

IS-IS Level-1 Link State Database

LSPID Seq Number Checksum Lifetime A/P/O/T

IS-IS Level-2 Link State Database

LSPID Seq Number Checksum Lifetime A/P/O/T

n9k-5.00-00 * 0x00000030 0x2C9A 1131 0/0/0/3

Instance : 0x00000014

Area Address : 49.0001

NLPID : 0xCC

Router ID : 55.1.1.1

IP Address : 55.1.1.1

Hostname : n9k-5 Length : 5

Extended IS : n9k-6.00 Metric : 1



Extended IP : 10.10.111.0/24 Metric : 1 (U)



Tag : 5555


IPv6 Prefix : 0::/0 Metric : 10 (U/E)

Capability : Router-Id 55.1.1.1 Flags 0x0

Area-leader : Algorithm: 128/Dual SPT - V1 Priority: 60

Digest Offset : 0

n9k-6.00-00 0x0000001D 0xE2D1 691 0/0/0/3


Area Address : 49.0001

NLPID : 0xCC 0x8E

Router ID : 66.66.66.66

IP Address : 66.66.66.66



Interface IP Address : 10.10.111.1

IP Neighbor Address : 10.10.111.2

ADJ-SID : 18 Flags : V/L, Weight 1













Prefix-SID : 327 Flags : --


Prefix-SID : 227 Flags : --


Prefix-SID : 66 Flags : N


Prefix-SID : 666 Flags : N

IPv6 Prefix : 6::6/128 Metric : 1 (U/I)

Capability : Router-Id 66.66.66.66 Flags 0x0

SR-Range : 16000 - 23999 (8000) Flags I--

Area-leader : Algorithm: 128/Dual SPT - V1 Priority: 60

Digest Offset : 0

n9k-6.00-00 0x0000000F 0x61D8 53736 0/0/0/3


Area Address : 39.752f.0100.0014.0000.0000.0025

NLPID : 0xCC 0x8E

Router ID : 66.66.66.66

IP Address : 66.66.66.66

MT TopoId : TopoId:2 Att: 0 Ol: 0

TopoId:0 Att: 0 Ol: 0







MT-IPv6 Prefx : TopoId : 2

2100::20/124 Metric : 40 (U/I)

Digest Offset : 0

n9k-5#


When both primary and secondary Area Leaders are configured, there are two separate dynamic flooding

trees created with each of the Area Leaders as root. These trees can be viewed with the outputs below.

The final flooding topology is computed as the sum of these two trees to factor redundancy into the

algorithm.

SS1# show isis dynamic-flooding tree-1


Level 1

Dynamic flooding configured



Area Leader capable: No


L1 Algorithm: Cisco-Dual-SPT-V1/128

Priority: 40

SystemId: 0000.0002.0005



Priority: 10

SystemId: 0000.0002.0002

Level 2







Priority: 40

SystemId: 0000.0002.0005



Priority: 10

SystemId: 0000.0002.0002


SS1# show isis dynamic-flooding tree-2


Level 1







Priority: 40

SystemId: 0000.0002.0005



Priority: 10

SystemId: 0000.0002.0002

Level 2







Priority: 40

SystemId: 0000.0002.0005



Priority: 10

SystemId: 0000.0002.0002

SS1#


Interoperability

Devices that do not support dynamic flooding will continue to operate using the standard flooding

mechanism. These devices may create redundant copies of the same information but do not interfere with

the devices that are configured for dynamic flooding.

Flooding that is initiated by devices configured for dynamic flooding will remain within the flooding

topology until the IS-IS updates reach a legacy device. The legacy device will perform the legacy flooding.

Standard flooding will be bounded by nodes supporting dynamic flooding. This mechanism can help limit

the propagation of unnecessary flooding.

Figure 3.

Interoperability with legacy IS-IS device

Caveats and limitations

The initial implementation will have the following restrictions:

● No dynamic flooding support for LAN interfaces (broadcast mode). The feature is supported only on

point-to-point links.

● Centralized mode of dynamic flooding is not supported; only distributed mode of dynamic flooding

is supported.


Efficiency of flooding algorithm

In order to demonstrate the efficiency of the flooding algorithm, we look at the topology below as an

example. This is a scenario comprising 3 super-spines, 5 spines, and 30 leaf switches interconnected

together as shown in the topology below. We demonstrate the flooding topology that will be computed

with the IS-IS flooding feature.

Figure 4.

Simulated topology: flooding interfaces without the dynamic flooding feature

The flooding topology with a single tree shown below. Similarly, we can have another flooding topology

calculated with a secondary Area Leader as root (not shown) to establish redundancy.

Figure 5.

Simulated topology: flooding interfaces with dynamic flooding enabled

The original topology shown in Figure 4 includes a total of 15 links between the super-spines and spine

switches that will be used for flooding updates. In addition, there are a total of 150 links between the spine

and leaf switches, for a total of 165 links. The flooding topology attempts to greatly reduce this number, as

seen in Figure 5. Details of the improvement in flooding can be noted in Table 1 below, with further

improvements possible depending on the topology and number of ECMP paths present.


Table 1. Efficiency of the Cisco-Dual-SPT-V1 algorithm with simulated topology of 3 super spines, 5 spines, and 30 leaf

switches

Parameter Legacy flooding Dynamic flooding with dual tree (redundancy)

Dynamic flooding with single tree (no redundancy)

Number of interfaces for flooding 165 74 37

Percentage of flooding interface reduction

N/A 55.15% 77.57%

Number of copies discarded 128 37 0

Number of LSPs involved in flooding 165 74 37

Summary

The IS-IS dynamic flooding feature reduces flooding to a subset of the physical topology and addresses

the fundamental problem of link state protocol operation in highly redundant topologies. It does so without

restricting the nodes’ visibility of the topology or limiting the IGP functionality on the nodes themselves. It

mainly reduces the redundancy in flooding paths and improves the efficiency of IS-IS updates in a dense

topology, marking a clear enhancement to IGP operations in data center fabric topologies.

Printed in USA C11-743015-00 12/19

Documents

IS-IS Dynamic Flooding in Data Center Networks White Paper · protocol convergence in data center networks. The feature is being developed as per the draft draft-ietf-lsr-dynamic-flooding,