Upload
others
View
23
Download
0
Embed Size (px)
Citation preview
Cisco MDS NX-OS Configuration Limits, Release 8.x
Cisco MDS NX-OS Configuration Limits 2
Switch-Level Fibre Channel Configuration Limits for Cisco MDS 9000 Series Switches 2
Fabric-Level Fibre Channel Configuration Limits for Cisco MDS 9000 Series Switches 4
Configuration Limits for Port Line Rate Throughput on Cisco MDS 9700 40 Gbps 24-Port FCoE Modules 6
Fibre Channel Topology Combinations and Scaling 7
Fibre Channel Scale Limitations 8
Syslog and Warnings for Fibre Channel Network Scale 9
Switch-Level Configuration Limits for Fibre Channel over Ethernet 10
Fabric-Level Configuration Limits for Fibre Channel over IP 11
Fabric-Level Configuration Limits for Fibre Channel over Ethernet (FCoE) on MDS 9700 Series 11
Cisco MDS 9000 I/O Accelerator Configuration Limits 12
Fibre Channel Flow Configuration Limits 13
Revised: February 26, 2021
Cisco MDS NX-OS Configuration LimitsThis document discusses the configuration and scalability limits for various SAN switching parameters in CiscoMDSNX-OS Release8.x:
Change History
DescriptionDate
Cisco MDS 9718 switches support 8000 FLOGIs per switch.July 1, 2020
The following scale values were updated:May 1, 2020
• CiscoMDS 9710 switches support 8000 FLOGIs per switch.
• Cisco MDS 9396T switches support 4000 FLOGIs perswitch.
• Smart Zones support a maximum of 250 members per zone.
Switch-Level Fibre Channel Configuration Limits for Cisco MDS 9000Series SwitchesSystem messages are generated whenever the configuration limits are exceeded. For more information, see the Cisco MDS 9000Series and Nexus 7000 Series NX-OS System Messages Reference document.
The following table lists the switch-level configuration limits for Cisco MDS 9000 Series switches:
Table 1: Switch-Level Configuration Limits
MDS9396T
MDS9148T
MDS 9132TMDS 9396SMDS 9148SMDS 9250iMDS 9700Series
ParameterFeatureName
256Footnote.256Footnote.256Footnote.256Footnote.256Footnote.256Footnote.256Footnote.FLOGIs orFDISCs perport
Fabric Logins
None (referto chassislimits)
None (referto chassislimits)
None (referto chassislimits)
None (referto chassislimits)
None (referto chassislimits)
None (referto chassislimits)
10002
20003FLOGIs orFDISCs permodule
20006
40007200020001000100040040004
80005FLOGIs orFDISCs perswitch
888888105NPV switchesper NPIV coreswitch
N-PortVirtualization(NPV)
2
MDS9396T
MDS9148T
MDS 9132TMDS 9396SMDS 9148SMDS 9250iMDS 9700Series
ParameterFeatureName
96 ISLs,each with15 VSANs,for a totalnumber of1440 portVSANinstances
48 ISLs,each with29 VSANs,for a totalnumber of1392 portVSANinstances
32 ISLs, eachwith 29VSANs, for atotal numberof 928 portVSANinstances
96 ISLs, eachwith 15VSANs, for atotal numberof 1440 portVSANinstances
48 ISLs, eachwith 29VSANs, for atotal numberof 1392 portVSANinstances
40 ISLs,each with29 VSANs,for a totalnumber of1160 portVSANinstances
Up to 200ISLs, eachISL with 16VSANs, for atotal numberof 3200-portVSANinstances
ISL instancesper switch 8
Inter-SwitchLinks (ISLs)
964832964840256Fibre Channelport channels
Port channels(PC)9
16161616161616Members ineach FibreChannel portchannel
NoneNoneNoneNoneNone8128Ethernet portchannels
NoneNoneNoneNoneNone816Members ineach Ethernetport channel
16161616161616SimultaneousSSH sessionsper switch
SSH 10
20,00020,00020,000NoneNoneNone20,00011
40,00012Active InitiatorTarget LUNs(ITLs) orInitiator TargetNamespace ID(ITNs) persamplinginterval perswitch
SANAnalytics andSANTelemetryStreaming
20,00020,00020,000NoneNoneNoneDS-X9648-1536K9:20,000
Active ITLs orITNs persamplinginterval permodule
222NoneNoneNone2Telemetryreceivers perswitch
1 This is the recommended limit. FLOGIs or FDISCs are not rejected until 512 logins have been accepted on the port.
3
2 Supported in Cisco MDS 9706 and 9710 switches.3 Supported in Cisco MDS 9718 switches.4 Supported in Cisco MDS 9706 switches.
Supported in Cisco MDS 9710 switches in Cisco MDS NX-OS Release 8.4(1a) and earlier releases.5 Supported in Cisco MDS 9710 switches with Cisco MDS 9700 Series Supervisor-4 Module (DS-X97-SF4-K9) from CiscoMDS NX-OS Release 8.4(2) and later releases.
Supported in CiscoMDS 9718 switches with CiscoMDS 9700 Series Supervisor-1Module (DS-X97-SF1E-K9) or CiscoMDS9700 Series Supervisor-4 Module (DS-X97-SF4-K9) from Cisco MDS NX-OS Release 8.4(2a) and later releases.
6 Supported in Cisco MDS NX-OS Release 8.4(1a) and earlier releases.7 Supported from Cisco MDS NX-OS Release 8.4(2) and later releases.8 ISLs can be an individual FC link, VFC link, FC port channel, or VFC port channel. One port channel interface with up to 16member links is considered as one ISL instance.
9 A system message appears in the user’s session when the maximum supported limit of port channels and member ports isexceeded.
10 A system message appears on the user’s session when the maximum supported number of SSH sessions is exceeded.11 Supported in Cisco MDS NX-OS Release 8.3(1) and earlier releases.12 Supported from Cisco MDS NX-OS Release 8.3(2) and later releases.
When there are more than 40,000 ITLs or ITNs distributed across multiple modules in multilayer directors or 20,000 ITLs orITNs in fabric switches, a random set of ITL or ITN data is reported. The ITL or ITN data may change frequently and becomesunpredictable. Hence, we recommend that you limit the number of ITLs or ITNs in a switch to less than the maximum limit orquery only view types. For example, a single query configured can display up to the maximum active ITLs or ITNs. If thereare two queries configured, the queries together can display up to the maximum active ITLs or ITNs.
Fabric-Level Fibre Channel Configuration Limits for Cisco MDS 9000 SeriesSwitchesThe following table lists the configuration limits for Cisco MDS 9000 fabrics.
Table 2: Fabric-Level Configuration Limits
MDS Mixed Fabric
(mix of MDS 9700, 9500, 9300,9200, 9100)
MDS 9700-only Fabric
(without MDS 9500, 9300, 9200,and 9100)
ParameterFeature Name
321380VSANs per physical fabricVSANs
1000015
13000Footnote.
20000`14FCNS entries in fabricLogins and aliases
800020000Device alias entries in fabric
4
MDS Mixed Fabric
(mix of MDS 9700, 9500, 9300,9200, 9100)
MDS 9700-only Fabric
(without MDS 9500, 9300, 9200,and 9100)
ParameterFeature Name
8080Domains per physical fabricDomains and hops
77Switch hops from server to storage
10001000Active zone setsZones and Smart Zones
800018
10400Footnote.
16000 17Active zones
1600021
20800 Footnote.
32000 20Active zone members19
2 MB
3.8 MBFootnote.
3.8 MB 22Active zone database size
3232Active IVR zone setsInter-VSAN Routing(IVR)
2000Footnote.2000Footnote.Active IVR zones
40004000Active IVR zone members
1616Active IVR service groups
8080CFS peersFootnote.CFS
100100CFS static peers over IPFootnote.
13 In a mixed fabric comprising CiscoMDS 9700 and 9500 director class switches and CiscoMDS 9148 and 9148S fabric switches,the fabric can have all the switches in the first 32 VSANs. The remaining VSANs include only those switches that supportmore than 32 VSANs.
14 An error message appears in your session if the number of FCNS entries exceeds 20000.15 For platforms other than Cisco MDS 9700, a warning appears in your session if the number of FCNS entries exceeds 10000.16 This value applies to Cisco MDS 9500 with Supervisor 2A. Thus, this limit applies to a fabric comprising of these switches
and Cisco MDS 9700.17 An error message and syslog appear in your session if the total number of zones exceeds 16000.18 For platforms other than Cisco MDS 9700, a warning appears in your session if the number of zones exceeds 8000. However,
further configurations are not blocked.19 Without Smart Zones and for optimal hardware resource usage, configure 1 initiator and 1 target device per zone. The
recommended maximum limit is 50 members per zone.
From Cisco MDS NX-OS Release 8.4(2), Smart Zones support a maximum of 250 members per zone only when the zonemembers are configured as initiators or targets.
20 An error message and syslog appear in your session if the total number of the unique zone members exceeds 32000.21 For platforms other than Cisco MDS 9700, a warning appears in your session if the number of unique zone members exceeds
16000.
5
22 An error message appears on your session if the zone database size exceeds 3.8 MB.23 2000 IVR zones are available across all VSANs. However, a single VSAN must not have more than 1000 IVR zones.24 A system message appears in your session when the supported configuration limit of CFS peers is exceeded.
If there are any switches in the fabric running Cisco MDS NX-OS 6.2(5) or earlier releases, the previous 2-MB zone databaselimit prevails and the new zone scale enhancements are not available. We recommend that the 2-MB zone database limit doesnot exceed, unless all the switches in the fabric run on Cisco MDS NX-OS Release 6.2(7) or later.
Note
Example: Checking the Size of the Zone DatabaseThe following example shows how to check the size of the zone database in VSAN 310:
(config)# show zone status vsan 310 | Inc "Db Size" P 1
Full Zoning Database :Db Size: 1040524 Bytes
--Active Zoning Database :
Db Size: 962156 Bytesc
Now, Add 962156 with 1040524 = 2002680
2002680 / (1024 * 1024) = 1.9 MB.
This database size does not include the pending changes in a session.Note
Configuration Limits for Port Line Rate Throughput on Cisco MDS 9700 40Gbps 24-Port FCoE ModulesFor ports to operate at full-line rate on Cisco MDS 9706, MDS 9710, and MDS 9718 switches, the switch should have a specificconfiguration. The following table lists the maximum number of line rate ports for given Cisco MDS 9700 switch configurations.
Table 3: Line Rate Limitation for Cisco MDS 9700 Series Switches Using Cisco MDS 9700 40 Gbps 24-Port FCoE Module
Maximum No. of Ports/SpeedFront Panel FCoE Bandwidth/SlotNo. of Fabric Cards
16 ports/40 Gbps660 Gbps3
20 ports/40 Gbps880 Gbps4
24 ports/40 Gbps1100 Gbps5
6
If the number of ports that you configure exceeds the maximum limit for the number of ports, it results in all ports operatingin oversubscribed mode.
Note
Fibre Channel Topology Combinations and ScalingFrom Cisco MDS NX0-OS Release 6.2(7), the following configurable features that support scale enhancements are supported:
• Fibre Channel Name Server (FCNS) bulk notification
• Coalesce switch Registered State Change Notification (SW-RSCN)
For more information about the FCNS and SW-RSCN features, see the Cisco MDS 9000 Series NX-OS Fabric Configuration Guideand the Cisco MDS 9000 Family Command Reference documents.
In multidimensional scale configurations, supervisors with 1-GB memory may experience the system manager hap-reseterror.
Note
The following table provides the configuration limits with and without the FCNS and SW-RSCN optimizations.
Table 4: Fibre Channel Topology Combination and Scaling
Scale OptimizationsConfiguration LimitsCoalesceSW-RSCN
FCNS BulkNotification
Scale Topology
EnabledMDS NX-OS Release 6.2(7) and laterOnOnCisco MDS NX-OS Release6.2(9) and later on Cisco MDS9700 Series-only Fabric FLOGI per module: 1000
FLOGI node: 4000
FCNS: 20000
Zone: 16000
Zone members: 32000
Disabled (Disabled bydefault in CiscoMDSNX-OS Release6.2(7))
MDS NX-OS Release 6.2(5) and earlierOffOff
FLOGI per module: 500
FLOGI node: 2500
FCNS: 10000
Zone: 8000
Zone members: 16000
7
Scale OptimizationsConfiguration LimitsCoalesceSW-RSCN
FCNS BulkNotification
Scale Topology
Enabled or disabledFLOGI per module: 400
FLOGI node: 2000
FCNS: 10000, 13000 26
Zone: 8000, 10400 27
Zone members: 16000, 20800 28
On or offOn or offCisco MDS NX-OS Release6.2(9) and later onMDSMixedFabric25 (Cisco MDS 9700Series, 9500, 9250i, 9222i, and9148)
25 The scale enhancements introduced in Cisco MDS NX-OS 6.2(7) and later are available on Cisco MDS 9700 Series Switches(only networks with CiscoMDSNX-OSRelease 6.2(7) and later, running on all the switches and the configurable optimizations):FCNS bulk notification and RSCN coalesce enabled. These enhancements are not available in mixed fabrics regardless ofwhether FCNS and RSCN are enabled. In Cisco MDS NX-OS Release 6.2(9) and later, FCNS bulk notification is enabled bydefault. (To disable FCNS bulk notification, use the fcns no-bulk-notify command.) Coalesce SW-RSCN is disabled by default.
26 This increase in scale is applicable only to the Cisco MDS 9500 Series Supervisor-2A Module.27 This increase in scale is applicable only to the Cisco MDS 9500 Series Supervisor-2A Module.28 This increase in scale is applicable only to the Cisco MDS 9500 Series Supervisor-2A Module.
Fibre Channel Scale LimitationsThe various Fibre Channel scale limitations are described in this section.
Limitation 1
From Cisco MDS NX-OS Release 6.2(7), the fcns bulk-notify and rscn coalesce swrscn vsan commands are available on all CiscoMDS switches. We recommend that you use these commands only on Cisco MDS 9700 Series switches with 48-port 16-Gbps FibreChannel switching module because the higher configuration limits are currently not supported on any other MDS platform.
The FCNS bulk notification and Coalesce SW-RSCN features are supported only in NX-OS Release 6.2(7) and later releases.Enable the Coalesce SW-RSCN only if all the switches in a fabric are Cisco MDS switches that are running Cisco MDSNX-OS Release 6.2(7) or later.
Note
Limitation 2
The maximum zone database size is increased from 2 to 3.8 MB in Cisco MDS Release 6.2(7) and later for all MDS 9700 Seriesswitches to enable zone scale enhancements. The new limit of 16000 zones is supported only on the Cisco MDS 9700 only fabric.Fabrics with Cisco MDS 9500, 9200, or 9100 Series switches continue to have the 2-MB zone database limit, supporting only up to8000 zones.
Limitation 3
In a three-node serial topology, traffic imbalance may occur if the number of port channels that are configured between the switchesare the same. We recommend that you have a single port channel between two switches with any number of member ports. If youconfigure more than one port channel, ensure that the count of port channels between the switches varies.
8
Limitation 4
Themaximum latency (round-trip time) and the packet-drops that are supported on FCIP links are 250 ms and 0.05 percent receptively.
The maximum limit is the same regardless whether latency and packet drop conditions exist together or only one of theconditions exists.
Note
Syslog and Warnings for Fibre Channel Network ScaleThe following table lists the syslogs and warnings for the Fibre Channel network.
Table 5: Syslogs and Warnings for the Fibre Channel Network
Cisco MDS 9500, 9200, and 9100Cisco MDS 9700 SeriesSyslog and Warnings
8000 zones – Hard limit: Syslog warning that states nomore zones can be configured:
“Maximum configurable zonelimit of 8,000 reached.Creation of any more zonesis not supported.”
10400 zones – Hard limit for Cisco MDS 9500 SeriesSupervisor-2A Module: Syslog warning that states nomore zones can be configured:
“Maximum configurable zonelimit of 10,400 reached.Creation of any more zonesis not supported.”
16000 zones – Hard limit: Syslog warning thatstates no more zones can be configured:“Maximum configurable zonelimit of 16,000 reached.Creation of any more zonesis not supported.”
Zone limit
9
Cisco MDS 9500, 9200, and 9100Cisco MDS 9700 SeriesSyslog and Warnings
10000 FCNS entries – Soft limit: Syslog warning aboutvalidated limit:
“Number of Name-Server entries hasreached the maximum validated limitof 10,000. Any more entries couldpotentially destabilize the fabric.”
13000 FCNS entries – Soft limit for Cisco MDS 9500Series Supervisor-2A Module: Syslog warning aboutvalidated limit:
“Number of Name-Server entries hasreached the maximum validated limit of 13,000.
Any more entries could potentiallydestabilize the fabric.”
20000 FCNS entries – Hard limit: Syslog warning:
“Maximum Name-Server entrylimit of 20,000 reached.No more entries are supported.”
20000 FCNS entries – Hard limit: Syslogwarning stating that no more name serverentries are supported:
“Maximum Name-Server entrylimit of 20,000 reached.No more entries are supported.”
FCNS limit
Switch-Level Configuration Limits for Fibre Channel over EthernetSystem messages are generated whenever the configuration limits are exceeded. For more information, see the Cisco MDS 9000Series and Nexus 7000 Series NX-OS System Messages Reference document.
The following table lists the switch-level configuration limits for Fibre Channel over Ethernet (FCoE) on Cisco MDS.
Table 6: Switch-Level Configuration Limits for FCoE
MDS 9710 with 48-Port 10-Gigabit FCoEFeature Name
256FLOGIs per Port
1000FLOGIs per Module
4000FLOGIs per Switch
80VSAN
80VSAN-VLAN Mapping
128 virtual fibre channel port channel and a maximum number of 16members in one virtual fibre channel port channel
Virtual Fibre Channel Port Channel and Member Ports
10
Fabric-Level Configuration Limits for Fibre Channel over IPThe following table lists the fabric-level configuration limits for Fibre Channel over IP (FCIP):
Table 7: Fabric-Level Configuration Limits for FCIP on Cisco MDS 9250i Multiservice Fabric Switch
Cisco MDS 9250i Multiservice Fabric SwitchFeature Name
250-ms round trip and 0.05% packet dropMaximum latency (round-trip time) and packet drops supported on FCIP links
Table 8: Fabric-Level Configuration Limits for FCIP on Cisco MDS 9700 Series Multilayer Directors with Cisco MDS 24/10-Port SAN Extension Module
Cisco MDS 9700 Series Multilayer Directors with Cisco MDS24/10-Port SAN Extension Module
Feature Name
250-ms round trip and 0.05% packet dropMaximum latency (round-trip time) and packet dropssupported on FCIP links
Fabric-Level Configuration Limits for Fibre Channel over Ethernet (FCoE)on MDS 9700 SeriesThe following table lists the fabric-level configuration limits for the Fibre Channel over Ethernet (FCoE) on MDS 9700 Series.
Table 9: Fabric-Level Configuration Limits for FCoE on MDS 9700 Series Switches
MDS 9700 Network with 48-Port 10-Gigabit Fibre Channel overEthernet Module
Feature Name
16000Zones
32000Zone members
1000Zone sets
3.8 MB 29Zone database size
20000Number of FCNS entries in a network
12000Device alias
29 An error message appears in your console if the zone database size exceeds 3.8 MB.
11
• A warning appears in your console if the number of zones exceeds 8000. However, further configurations are not blocked.
• An error message appears in your console if the total number of the unique zone members exceed 16000.
• From Cisco MDS NX-OS Release 6.2(7), a zone's database size is increased from 2 to 3.8 MB, to provide zone scaleenhancements. However, if there are any switches in the fabric that are running Cisco MDS NXOS Release 6.2(5) orearlier, the previous 2-MB zone database limit prevails and the new zone scale enhancements are not available. Werecommend that the 2-MB zone database limit does not exceed, unless all the switches in the fabric run Release 6.2(7)or later.
• An error message appears in your console if the number of FCNS entries exceeds 10000.
Note
Example: Checking the Size of the Zone DatabaseThe following example shows how to check the size of the zone database in VSAN 310:
(config)# show zone status vsan 310 | Inc "Db Size" P 1
Full Zoning Database :Db Size: 1040524 Bytes
--Active Zoning Database :
Db Size: 962156 Bytesc
Now, Add 962156 with 1040524 = 2002680
2002680 / (1024 * 1024) = 1.9 MB.
This database size does not include the pending changes in a session.Note
Cisco MDS 9000 I/O Accelerator Configuration LimitsThe following table lists the Cisco MDS 9000 I/O Accelerator configurations and the corresponding limits.
Table 10: Cisco I/O Accelerator Configuration Limits
MSM-18/4 or SSN-16 Module on MDS 9222i and MDS 9500Modular Chassis and MDS 9250i Fabric Switch, 24/10-Port SANExtension Module on MDS 9710 Chassis
Parameter
4Number of switches in a cluster
16Number of clusters per switch
34Number of switches in a SAN fabric for FC-Redirect
128Number of hosts per target
12
MSM-18/4 or SSN-16 Module on MDS 9222i and MDS 9500Modular Chassis and MDS 9250i Fabric Switch, 24/10-Port SANExtension Module on MDS 9710 Chassis
Parameter
128Number of concurrent flows per Cisco MDS 9000 I/OAccelerator service engine
128 - Release 4.2(1) on MDS 9222i and MDS 9500
512 - Release 4.2(7) or later on MDS 9222i and MDS 9500
512 - Release 6.2(5) or later on MDS 9250i
512 - Release 8.2(1) or later on MDS 9710
Number of flows per Cisco MDS 9000 I/O Acceleratorservice engine (hard limit)
64 - Release 4.2(1) on MDS 9222i/MDS 9500
256 - Release 4.2(7) or later on MDS 9222i/MDS 9500
256 - Release 6.2(5) or later on MDS 9250i
If initiators or targets participating in Cisco MDS 9000 I/OAccelerator are present on MDS 9250i switches, the limit is203 for tape and 160 for disk.
Note
256 - Release 8.2(1) or later on MDS 9710
Number of flows per Cisco MDS 9000 I/O Acceleratorservice engine (soft limit)
1024 - Release 4.2(7d)
1248 - Release 5.2(6b) or later
Number of flows in a cluster
Fibre Channel Flow Configuration LimitsThe following table lists the Fibre Channel flow configurations limits.
Table 11: Fibre Channel Flow Configuration Limits
Flow Statements per ModuleAggregate Flow and Flow Statistics LimitCisco MDS Device
10241000Generation 1Modules
20482000Generation 2Modules
512512Generation 3Modules
512512Generation 4Modules
512512Generation 5Modules
13
Europe HeadquartersAsia Pacific HeadquartersAmericas HeadquartersCiscoSystemsInternationalBVAmsterdam,TheNetherlands
CiscoSystems(USA)Pte.Ltd.Singapore
Cisco Systems, Inc.San Jose, CA 95134-1706USA
Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on theCisco Website at www.cisco.com/go/offices.