VCE VxBlock and Vblock Systems 740 Expansion Guide...• Cisco Nexus 3064-T Switch (Ethernet expansion only) ... Cisco Nexus 1000V Switch for VMware vSphere Compatibility Information

Dell EMCVxBlock™ and Vblock® Systems 740Expansion Guide

Document revision 5.5

March 2017

DELL EMC CONFIDENTIALFor Internal Use Only

Revision history

Date Document revision Description of changes

March 2017 5.5 Update to include Cisco Nexus 93180YC-EX Switch

November 2016 5.4 Update to include VMAX 250F and VMAX 250FX all flash arrayinformation

September 2016 5.3 Update to include the Cisco MDS 9396S Switch and Cisco MDS9706 Multilayer Director

January 2016 5.2 Update to include support for VMware NSX

August 2015 5.1 Update to include VxBlock System 740

March 2015 5.0 Initial release

Revision history | 2DELL EMC CONFIDENTIAL

For Internal Use Only

Contents

Introduction.................................................................................................................................................4

System capacities.......................................................................................................................................5

Related software and hardware information............................................................................................8

Expanding compute capacity.................................................................................................................... 9Expanding compute capacity with additional Cisco UCS domains........................................................ 9

Requirements.................................................................................................................................10Adding Cisco UCS domains...........................................................................................................12Components...................................................................................................................................15

Licensed Cisco UCS 5108 Server Chassis.......................................................................................... 17Requirements.................................................................................................................................18Components...................................................................................................................................20

Expanding compute capacity on an unlicensed Cisco UCS 5108 Server Chassis.............................. 21Requirements.................................................................................................................................21Components...................................................................................................................................22

Expanding compute capacity with additional Cisco UCS 5108 Server Chassis...................................24Requirements.................................................................................................................................25Components...................................................................................................................................26

Expanding compute capacity with additional cabinets for new Cisco UCS 5108 Server Chassis....... 28Requirements.................................................................................................................................28Components...................................................................................................................................29

Expanding compute capacity to support VMware NSX........................................................................29Requirements.................................................................................................................................30

Expanding storage capacity.................................................................................................................... 31Expanding storage capacity with available drive slots......................................................................... 31

Requirements.................................................................................................................................31Components...................................................................................................................................31

Expanding network capacity................................................................................................................... 33Expanding network capacity with Fibre Channel bandwidth................................................................ 33

Requirements.................................................................................................................................33Components...................................................................................................................................37

Expanding network capacity with Ethernet bandwidth enhancement.................................................. 38Requirements.................................................................................................................................38Components...................................................................................................................................39

Expanding network capacity with Ethernet bandwidth enhancement - customer uplinks.................... 40Requirements.................................................................................................................................40Components...................................................................................................................................41

Expanding network capacity with four to eight link upgrade.................................................................41Requirements.................................................................................................................................41Components...................................................................................................................................43

Expanding network capacity to support VMware NSX......................................................................... 43Requirements.................................................................................................................................44

3 | ContentsDELL EMC CONFIDENTIAL


IntroductionThe guide provides information needed to expand a Converged System within the parameters ofcompute, storage, and network guidelines.

This guide does not describe how to perform the expansions. In this document, the VxBlock System 740and the Vblock System 740 are referred to as Converged Systems.

The intended audience for this guide is vArchitects, sales engineers, field consultants, and advancedservices specialists.

Refer to the following documents for more information:

• Dell EMC VxBlock and Vblock Systems 740 Architecture Overview

• Dell EMC VxBlock and Vblock Systems 740 Logical Build Guide

• Dell EMC Converged Systems Physical Planning Guide

• Dell EMC Converged Systems Site Survey

• Glossary

Introduction | 4DELL EMC CONFIDENTIAL


https://docs.vce.com/bundle/O_GLOSSARY/page/GUID-E5B5E1DA-8455-460A-8391-12F5783BE2A3.html

System capacitiesThe following information provides system capacities for compute, storage, and network components forthe Converged System.

Compute

The following table provides the components that can be added to a Converged System depending onthe available capacity and limitations:

Compute Storage Infrastructure

Cisco UCS 5108 Blade ServerChassis with Cisco UCS22xxXP fabric extender IOMuplinks

Disk drives in available drive slotsConverged System cabinets tocontain additional computecomponents.

Cisco UCS B-Series Blades

Storage

The following rules apply to all available models of VMAX3 Hybrid:

• Maximum of six DAEs in a single engine cabinet

• Maximum of four DAEs in a dual engine cabinet

• Maximum drive count:

— 2.5 inch DAE: 120

— 3.5 inch DAE: 60

• Maximum 16 Gbps FC front-end points: 32 per engine

The following rules apply to the VMAX All Flash 250F and 250FX:

• Maximum of four DAEs per system bay

• Maximum drive count: 25 per 2.5 inch DAE

The following rules apply to the VMAX All Flash 250F and 250FX single cabinet Vblock configuration:

• Maximum of two DAEs per system bay


The following rules apply to the VMAX All Flash 450F and 450FX and the VMAX All Flash 850F and850FX:

• Maximum of four DAEs per system bay


5 | System capacitiesDELL EMC CONFIDENTIAL


The following support limitations apply to the Vblock 740 with VMAX 100K:

• Maximum of two engines

• Maximum of 1440 drives (all 2.5 inch)

• Maximum of 64 front end ports (16 Gbps FC)


• Maximum of four engines




• Maximum of eight engines



The following support limitations apply to the Vblock 740 with VMAX All Flash 250F and 250FX:

• Maximum of two VBrick


• Maximum of 64 front end ports

The following support limitations apply to the Vblock 740 with VMAX All Flash 250F and 250FX singlecabinet Vblock configuration:

• Maximum of one VBrick



The following support limitations apply to the VxBlock 740 with VMAX All Flash 450F and 450FX:

• Maximum of four vBricks



The following support limitations apply to the VxBlock 740 with VMAX All Flash 850F and 850FX:

• Maximum of eight vBricks



System capacities | 6DELL EMC CONFIDENTIAL


Network

The following components support FC and Ethernet bandwidth expansion:

• Cisco UCS 6296UP Fabric Interconnects

• Cisco UCS 6248UP Fabric Interconnects (FC bandwidth expansion only)

• Cisco UCS 2208XP Fabric Extenders (Ethernet expansion only)

• Cisco Nexus 5548UP Switches (Ethernet expansion only)

• Cisco Nexus 5596UP Switches (Ethernet expansion only)

• Cisco Nexus 93180YC-EX or Cisco Nexus 9396PX Switches (Ethernet expansion only)

• Cisco Nexus 3064-T Switch (Ethernet expansion only)

• Cisco MDS 9148S Multilayer Fabric Switches (FC expansion only)

• Cisco MDS 9706 Multilayer Director Switches (FC expansion only)

The following models support network expansion:

• Converged System with VMAX 400K (supports SAN fabric expansion)

• Converged System with VMAX 200K

• Converged System with VMAX 100K

• Converged System with VMAX 250F/250FX



7 | System capacitiesDELL EMC CONFIDENTIAL


Related software and hardware informationHardware and software information links that support the expansion of a Converged System are provided.

ComputeThe following table provides links for related documentation:

Reference Link

VMware vSphere Configuration Maximums www.vmware.com

Cisco UCS Site Preparation Guide

www.cisco.com

Cisco Unified Compute System - Hardware and Software Interoperability forUCSM Managed Servers

Cisco Nexus 1000V Switch for VMware vSphere Compatibility Information

Regulatory Compliance and Safety Information for the Cisco UCS

Installing the Cisco UCS 5108 Server Chassis

Cisco UCS Hardware and Software Interoperability for UCSM ManagedServers in Release x.x.x

Dell EMC expansion blade pack SKUs reference the same blade pack SKUs used for ordering newConverged Systems.

Network

Refer to the following documents for more information:

• Dell EMC VxBlock and Vblock Systems 740 Architecture Overview

• Dell EMC VxBlock and Vblock Systems 740 Logical Build Guide

• Dell EMC Converged Systems Physical Planning Guide

• Dell EMC Converged Systems Site Survey

• Dell EMC VxBlock Systems for VMware NSX 6.2 Architecture Overview

• Dell EMC VxBlock Systems for VMware NSX 6.2 Logical Build Guide

Related information

Standard Hardware Expansion Services

Related software and hardware information | 8DELL EMC CONFIDENTIAL


https://www.emc.com/collateral/data-sheet/hardware-expansion-services.pdf

Expanding compute capacity

Expanding with additional Cisco UCS domainsSpecific conditions and guidelines must be met before expanding compute capacity with additional CiscoUCS domains.

The Cisco UCS domain consists of a pair of Cisco UCS 62xxUP fabric interconnects that provideconnectivity for fabric A and B.

To expand compute capacity, add a Cisco UCS domain by installing an additional pair of FIs:

• Cisco UCS 6248UP Fabric Interconnects include 32 base ports, 12 of which are licensed. Toexpand to the full 48 ports, add the Cisco UCS-FI-E16UP expansion module with 16 ports, 8 ofwhich are licensed.

• Cisco UCS 6296UP Fabric Interconnects include 48 base ports, 18 of which are licensed. Toexpand to the full 96 ports, add 3 Cisco UCS-FI-E16UP expansion modules, each with 16 ports, 8of which are licensed.

You can add Cisco UCS-LIC-10GE port licenses to both Cisco UCS 62xxUP fabric interconnects.

Conditions

The original architectural design of the Converged System and the following physical and logicalconfigurations limit a compute expansion:

• Verify that the model type and hardware configuration of the FIs in the domain are the same.

• All FI server ports, Ethernet uplink ports, and FC uplink ports must be licensed on each FI.

• Additional port licenses may be required for the Cisco MDS 9148S Multilayer Fabric Switch. Portlicenses are purchased in increments of 12.

• Connectivity from the FI to the LAN switch requires twinax or optical cabling. Fibre cables musthave a pair of small, form-fit, pluggable (SFP) devices to connect the LAN switch ports to theCisco UCS FI LAN uplink ports.

• Connectivity from the FIs to the SAN switches requires optical cabling. For each optical cable,order an appropriate number of SFPs for each end of the cables.

• Connectivity from the FIs to the chassis requires twinax or optical cabling. Each fibre cablerequires a pair of appropriate SFP+ modules.

• The maximum expansion limit is four domains.

• Each domain supports a maximum of 16 chassis. Each domain expansion adds up to 16additional chassis. Refer to System capacities for more information on maximums and limits.

• If there are insufficient ports available on the FI to connect additional chassis, add anotherdomain.

• Each new Cisco UCS domain requires 8 or 16 additional Ethernet-uplink ports on the LAN switchon the Converged System.

9 | Expanding compute capacityDELL EMC CONFIDENTIAL


• Each new Cisco UCS domain requires 4, 8, or 16 additional FC uplink ports on the SAN switch onthe Converged System.

Requirements

Expanding compute capacity includes requirements for the Cisco UCS 5108 Blade Server Chassis, LANor SAN switch, and related equipment.

Cisco UCS 5108 Blade Server Chassis

The following table provides the maximum number of chassis and supported blades:

Component Converged System

Cisco UCS 5108 Blade Server Chassis 64 in up to four Cisco UCS domains

Cisco B-Series Blades Half-width = 512

Full-width = 256

Double-height = 128 (physical limit)

The total number of chassis supported per domain depends on the following:

• Model of the FI

• Number of FC and Ethernet uplinks

• FC or Ethernet uplink expansion options

• Chassis IOM flex link model, based on the type of fabric extenders and number of FI connections.

You cannot select both the 16 FC uplinks and 16 Ethernet uplink expansion options within the sameCisco UCS domain.

Cisco UCS Flex link/FCuplinks/Ethernetuplinks

Maxchassisperdomain

SANswitch

Cisco Nexus support

FI IOM 5548UP 5596UP 93180YC-EX/9396PX

6248UP 2204XP2/4/8 16 Y Y Y Y

4/4/8 8 Y Y Y Y

6296UP

2204XP 4/8/8 16 Y N Y N*

2208XP

8/8/8 8 Y N Y N*

8/8/16 8 Y N Y N*

8/16/8 8 Y N Y N*

* There are 48 10 Gb ports on the Cisco Nexus 93180YC-EX Switch or Cisco Nexus 9396PX Switch.

SAN switches

The following table provides the maximum number of domains supported per SAN switch based on thenumber of FC uplink ports per domain:

Expanding compute capacity | 10DELL EMC CONFIDENTIAL


The count is double for a pair of SAN switches. One SAN switch is required per fabric.

Domains FI-SAN switchuplink count perdomain

FI-SAN switchuplink count perConverged System

Total SAN switchports required perfabric

Cisco MDS switch support

1 4 4 16 Y

1 8 8 20 Y

1 16 16 28 Y

2 4 8 20 Y

2 8 16 28 Y

2 16 32 44 Y

3 4 12 24 Y

3 8 24 36 Y

3 16 48 60 N/Y

4 4 16 28 Y

4 8 32 44 Y

4 16 64 76 N /Y

LAN switches

The following table provides the maximum number of domains supported per LAN switch, based on thenumber of Ethernet uplink ports per domain and on the LAN switch:

Domains FI-LAN switchuplink ports perdomain/system

Ports Total LANswitch portsper fabric

Cisco Nexus support

Coreuplink

vPClink

Mgmtuplink

IP backup 5548UP/9396PX*/93180YC-EX**

5596UP

1 8/8 8 2 2 4 24 Y Y

1 16/16 8 2 2 4 32 Y Y

2 8/16 8 2 2 4 32 Y Y

2 16/32 8 2 2 4 48 N Y

3 8/24 8 2 2 4 40 Y Y

3 16/48 8 2 2 4 64 N Y

4 8/32 8 2 2 4 48 Y Y

4 16/64 8 2 2 4 78 N Y

*There are 48 10 Gb ports on the Cisco Nexus 9396PX Switch.

**There are 48 10/25 Gb ports on the Cisco Nexus 93180YC-EX Switch.



Guidelines for adding Cisco UCS domains

Each Cisco UCS domain requires 8 or 16 Ethernet ports per LAN switch and 4, 8, or 16 FC uplink portsper SAN switch.

About this task

Refer to the tables in System capacities and Requirements to determine maximums and supportedconfigurations for each expansion. Use caution when spreading FI PC ports across the SAN switches.

Before you begin

Before adding domains, document and record the following:

• Number and type of fabric interconnects in the Converged System and expansion modules perFI.

• Number of port licenses installed for each FI versus the number used.

• Number of chassis blade slots available for expansion.

• Whether all installed chassis without installed blades have Cisco UCS activation licenses.

• Whether all installed chassis without installed blades are cabled to the FI.

• Number of ports and port licenses available on each SAN switch.

• Number of 10 Gb ports available on each LAN switch. Do not include 40 Gb ports or 100 Gbports.

For each blade type, multiply the number of blades by the number of required chassis slots to determinethe total number of required chassis blade slots. The following requirements apply:

• Half-width blades = one blade slot

• Full-width blades = two blade slots

• Double-height, full-width blades = four blade slots

The maximum number of chassis per domain cannot exceed the system configuration maximum. Refer tothe table in System capacities for additional information. Based on the total number of blade slotsrequired for the expansion, determine if the compute expansion can be completed with the existingnumber of available slots in the current domain. If the number of blade slots is not sufficient, determine ifthe compute expansion can be completed by adding the chassis into the existing domain.



Procedure

1 Refer to the tables in the Requirements section to document and record the number of additionaldomains based on the number of domains required for expansion. For each Cisco UCS domain,document and record the information as shown in the following example:

Domain FI type FEX IOM IOMflex linkmodelperfabric

Chassisperdomain

FIserverportsperfabric

FI FCuplinks

FIEthernetuplinks

Port usage/licensecount perfabric

2 6296UP 2208XP 8 8 64 16 8 88

2 Refer to the Cisco UCS IOM flex link chassis table to determine if the chassis requirements perdomain can be met with the selected FI, FEX, and flex link model. If the chassis requirementcannot be met, do not proceed with the expansion of the domain. You can modify the flex linkmodel or reduce the number of FC or Ethernet uplink ports.

3 Refer to the SAN switches table to determine if there are enough SAN ports available on eachSAN switch, based on the number of available SAN switch ports per fabric, the number of FI FCuplinks per fabric field, and the information in the previous table. If the requirement for theadditional ports exceeds the capacity of each of the SAN switches, do not proceed with theexpansion of the domain. You can reduce the number of FI FC uplink ports to a smallerconfiguration.

Port counts are calculated per fabric. If 16 FC ports are required on SAN switch A, 16 ports arealso required on SAN switch B.

4 Refer to the LAN switches table to determine if the number of selected ports can be met with theavailable number of LAN ports on the LAN switch. This is based on the number of available LANswitch ports per fabric and the number of FI Ethernet Uplinks per fabric field. If the requirementfor the additional ports exceeds the capacity of each of the LAN switches, consider adding anexpansion module to the LAN switch. If the requirement cannot be met with the addition of theLAN switch expansion module, do not proceed with the expansion of the domain. You can reducethe number of FI Ethernet uplink ports to a smaller valid configuration.

Port counts are calculated per fabric. If 16 Ethernet ports are required on LAN switch A, 16 portsare also required on LAN switch B.



5 After remedial actions in the previous steps are complete and configurations are valid, determinethe number of FI port licenses required for the new domain. The number of FI port licensesrequired can be determined by:

— The type of Cisco UCS 62xxUP fabric interconnect and the number of installed expansionmodules determine the number of available port licenses.

— The Cisco UCS 6248UP Fabric Interconnect ships with 12 port licenses and the Cisco UCS6296UP Fabric Interconnect ships with 18 port licenses. Each FI expansion module provides8 additional licenses. The number of port licenses required per FI can be determined usingthe Cisco UCS IOM flex link chassis table. The value in the field is for a single FI.

— Use the following formula to calculate the number of licenses required per FI:

((Flex link value x number of chassis) + number of Ethernet uplinks + number of FCuplinks)) = Total number of port licenses required per FI

— Use the following formula to calculate the number of licenses required per domain:

(Total number of port licenses required per FI x 2) = Total number of port licensesrequired per domain

— Based on the FI model and the number of expansion modules installed, document thenumber of installed licenses. Each FI expansion module comes with 8 port licenses. TheCisco UCS 6248UP Fabric Interconnect ships with 12 base licenses and the Cisco UCS6296UP Fabric Interconnect ships with 16 base licenses. Use the following formula tocalculate the number of port activation licenses to order per FI:

((Number of FI expansion modules * 8) + (FI model base licenses (12 | 18)) = Number ofavailable FI licenses

Number of available FI licenses - total number of port licenses required per domain =Number of FI licenses to order

6 After steps 3 and 4 are complete and configurations are valid, determine the number of requiredCisco MDS port licenses per fabric required for the domain.

Cisco MDS SAN licenses are ordered in increments of 12.

— Refer to the total domain FI SAN uplink count field in the Cisco UCS IOM flex link chassistable to determine the number of SAN switch licenses required per fabric. The value in thefield is for a single fabric.

— Use the following formula to calculate the number of required Cisco MDS port licenses:(Number of available licenses per SAN switch - total domain FI-SAN switch uplinkcount field) = number of SAN switch licenses required per fabric

— Use the following formula to calculate the number of licenses required per Cisco UCSdomain: (Total number of port licenses required per SAN switch x 2) = total number ofSAN switch required per Cisco UCS domain



7 After steps 3 and 4 are complete and configurations are valid, refer to Cisco UCS IOM flex linkchassis table to determine the total number of 8 Gb FC SFPs to order per Cisco UCS FI fabric forSAN switch connectivity.

— Use the following formula : (Number of FI FC uplinks (4, 8, 16) per fabric) x 2 FIs)x 2 SANswitches = total number of 8 Gb FC SFPs required for the addition of the domain

This provides SFPs for both FIs and SAN switches

8 After steps 3 and 4 are complete and configurations are valid, determine the total number oftwinax cables or 10 Gb MM SFPs to order per FI fabric for LAN switch connectivity. The longestactive twinax cable should extend from the SAN switch to the FIs for the first two domains. Morethan two domains per Converged System requires the use of 10 Gb MM SM fibre. To determinethe number of active twinax cable required or the number of 10 Gb MM SFPs required refer to theCisco UCS IOM flex link chassis table and use the following formula:

— (Number of FI Ethernet uplinks (4, 8, 16) per fabric x 2 FIs) = total number of twinaxcables required for the addition of the domain, or

— (Number of FI Ethernet uplinks (8, 16) per fabric x 2 FIs) x 2 LAN switches = Totalnumber of 10 Gb MM SFPs required for the addition of the domain

Components

Partial SKUs are provided as a reference to order components for additional Cisco UCS domains.

Equipment

To perform a compute expansion with an additional domain, order the following equipment:

• Pair of Cisco UCS 62xxUP fabric interconnects.

• Cisco UCS 62xxUP fabric interconnect expansion modules and port licenses, based onrequirements.

• Cisco MDS 9148S Multilayer Fabric Director SAN port licenses (ordered in increments of 12),based on requirements.

• Cisco MDS 9396S 16G Multilayer Fabric Director SAN port licenses (ordered in increments of12), based on requirements.

• A Cisco MDS 9706 Multilayer Director deployed as the SAN switch does not requireSAN port licenses.

• Cisco UCS 5108 Blade Server Chassis, based on requirements.

• Pair of Cisco UCS Fabric Extender 22xxXP IOM modules, based on requirements.

• Cisco UCS Fabric Extender 22xxXP IOM cabling to FIs using either twinax or optical cables,depending on distance.

— Cisco UCS 5108 Blade Server Chassis Fabric Extender 2204XP IOM supports 2 or 4 flex linkmodel.



— Cisco UCS 5108 Blade Server Chassis Fabric Extender 2208XP supports an a 4 or 8 flex linkmodel.

• Each FI requires cabling to the LAN switch using either twinax or optical cables. The selection oftwinax or optical cables depends on distance.

— Each optical cable requires a pair of SFPs.

— 8 or 16 uplink ports configurations are supported.

• Each FI requires cabling to the SAN switch using optical cables.

— Each optical cable requires a pair of SFPs.

— Supported configurations include 4, 8, or 16 FC uplink ports.

• Boot LUNs

Orderable components

The following table provides the first eight characters of SKUs for orderable components:

Description SKU Quantity

Cisco UCS 6200 Series fabric interconnect,one port 1/10GE/FC-port license

NSTK-FIPT-xx-x N/A

8 Gbps FC SW SFP+, LC Refer to the SAN SKU table N/A

10GBase-SR SFP module AGGE-SFPP-xx-xxx N/A

Active twinax cable assembly, 10m Refer to the Cabling SKU table. N/A

SAN port licenses Refer to the SAN SKU table. N/A

Cisco UCS 6248UP 1RU FabricInterconnect/No PSU/32 UP/12p LIC

COMP-FIC3-0x-xxx

SKU includes two Cisco UCS 6248UP FabricInterconnects

1

Power Cord Jumper, C13-C14 Connectors,2 Meters

N/A 2

Cisco UCS 6248UP fan module N/A 2

Cisco UCS 6248UP power supply/100-240VAC

N/A 2

Cisco UCS 6248UP chassis accessory kit N/A 1

Cisco UCS Manager N/A 1

Cisco UCS Series 6200 Series expansionmodule blanks

N/A N/A

Cisco UCS support 24X7X4OS 6248UPFabric Interconnect

SUPP-FIC3-0x-xxx

SKU is for one year - increase quantity by two foreach year of additional maintenance

2

Cisco UCS 6296UP 2RU FabricInterconnect/No PSU/48 UP/18p LIC

COMP-FIC4-xx-xxx 2



Description SKU Quantity

Power Cord Jumper, C13-C14 Connectors,2 Meters

N/A 2

Cisco UCS 6296UP power supply/100-240VAC

N/A 2

Cisco UCS 6296UP CAK N/A 1

Cisco UCS Manager N/A 1

Cisco UCS Series 6200 Series expansionmodule blanks

N/A 2

Cisco UCS support 24X7X4OS 6248UPFabric Interconnect

SUPP-FIC4-0x-xxx


2

Cisco UCS 6200 16-port expansionmodule/16 UP/8p LIC

COMP-16UP-xx-xxx 1

Cisco UCS support 24X7X4OS 16PTexpansion module 6200 Series

SUPH-16UP-xx-xxx


6

Expanding on a licensed Cisco UCS 5108 Blade ServerChassisSpecific conditions and guidelines must be met before expanding compute blade pack capacity on anexisting, licensed Cisco UCS 5108 Blade Server Chassis.

Conditions


• Type of Cisco UCS FI

• Type of Cisco UCS FEX IOM

— Type of Cisco UCS 22xxXP IOM is supported with 10 Gb connectivity

• Type of Cisco UCS IOM flex link

• Number of available FI ports to support additional IOM links or chassis expansion

The following table provides required slots for the chassis:

Blade server Required slot

Half-width Half-width

Full-width Full-width

Double-height, full-width Full-width, two adjoining, starting at the bottom of the chassis



Related information

Expanding with an additional Cisco UCS 5108 Blade Server Chassis (see page 24)

Requirements

Expanding compute capacity on a licensed Cisco UCS 5108 Blade Server Chassis includes requirementsfor compute, RCM, power and cooling, virtual switch, management, and licensing and support.

Compute

The following compute requirements apply:

• Determine the maximum number of blades allowed without exceeding the architectural limit.

— Determine the blade type (half-width, full-width, or double-height, full-width).

— Determine the blade quantity.

The number of available slots in the licensed chassis limits the number of bladesthat can be added to the Converged System.

• Determine the CPU, memory, and mezzanine adapter configuration for each new bladecombination. Memory speed is often reduced with fully populated DIMMs per channel.

• Verify that the requested blade server combination complies with the minimum hardware,software, and firmware requirements.

• Determine available space for VMware vSphere ESXi boot LUNs on the appropriate array foreach new blade server.

RCM

The following RCM requirements apply:

• Verify that an existing RCM supports the new blade server hardware, software, and firmwarecombination.

• Verify that the Converged System configuration or currently deployed RCM meets therequirements for the new blade server combination.

• Upgrade the Converged System to an RCM that supports the new blade servers beforedeployment.

Power and cooling

The following power and cooling requirements apply:

• Determine the new blade distribution on each chassis.

• Determine power requirements using the Cisco UCS power calculator or plan for maximum poweruse.



• Determine individual thermal requirements for each new blade combination or plan for maximumthermal requirements.

• Determine total power and thermal requirements for each new blade for each existing chassis.

• Calculate potential power consumption on a per chassis basis and validate that sufficient powerexists at the following levels:

— PDU overall capacity

— Branch circuit (circuit breaker)

— Phase or line

Power and thermal requirements hazard:

• Additional power and thermal requirements must not exceed chassis limitations,available power or cooling limitations for the chassis, or limitations of the cabinetlocation.

• Do not proceed with the expansion until the configuration satisfies the power andthermal requirements for each chassis.

Virtual switch

The following virtual switch requirements apply:

• If the Cisco Nexus 1000V Series Switch is deployed, determine if adding new blades exceedsany of the following default configuration maximums:

— 128 VMware vSphere ESXi hosts per VSM

— 2048 active VLANs

— 2048 port profiles

• No additional licenses are required for the Cisco Nexus 1000V Series Switch Essentials Edition.

• An additional Cisco Nexus 1000V license is required for each CPU socket on each additionalblade server for the Cisco Nexus 1000V Series Switch Advanced Edition.

• If a VMware VDS is deployed, determine if adding new blade servers exceeds defaultconfiguration maximums.

Management

The following management requirements apply:

• Determine if the Cisco UCS management subnet range supports new Cisco UCS Blade LinuxKVM IP addresses.

• If the VMware vCenter Server manages multiple Converged Systems, verify that adding newblades does not exceed configuration maximums.



Licensing and support

The following licensing and support requirements apply:

• Determine hardware and software maintenance requirements for the new blades.

• Determine if this is a bare metal or VMware installation for the new blades. If this is a VMwareinstallation, VMware vSphere ESXi and PowerPath licenses are required.

Related information


Components

Dell EMC supports primarily virtualized and bare metal blade pack configurations along with variety ofother configurations. This is relevant for new system build/orders, not when ordering expansion bladepacks.

Blade packs

Order appropriate licenses with virtualized blade packs separately.

Both virtualized and bare metal Dell EMC blade packs offer Cisco TPM. Some countries do not permitCisco TPM.

Order a customized blade pack configuration without Cisco TPM for the following destination countries:

• People's Republic of China

• Republic of Belarus

• Republic of Cuba

• Republic of Kazakhstan

• Islamic Republic of Iran

• Democratic People's Republic of Korea

• Syrian Arab Republic

• Republic of Sudan

• Russian Federation

Blade pack support

Order Dell EMC blade pack support SKUs, available in yearly increments, for each blade pack.

Order the correct quantity to provide coverage for the required time period. For example, for three yearsof support services for two blade packs (same blade server type), order six support SKUs.




Standard blade pack support SKUs offer 24x7x4hr on-site installation support for blade packs, however,this option is not available in all regions. Order other support delivery options, such as 8x5xNBD, throughcustom configurations.

Professional Services support

You can purchase blade expansion services for existing Converged Systems. A single blade expansionservice includes installation and logical configuration for up to four blade packs or eight blade servers.Purchase additional blade pack installations to increase the quantity of the blade expansion services foreach additional group of four blade packs. Refer to the Standard Hardware Expansion Services DataSheet for solutions for expanding the capacity of existing Converged Systems.

If the blade server expansion is a virtualized installation, obtain additional licenses for VMware vSphereESXi, Cisco Nexus1000V, and PowerPath.

Permutations are based on geography.

Related information


Expanding on an unlicensed Cisco UCS 5108 Blade ServerChassisSpecific conditions and guidelines must be met before expanding compute blade pack capacity on anexisting, unlicensed Cisco UCS 5108 Blade Server Chassis.

Conditions




• Type of Cisco UCS FEX IOM flex link


Related information

Expanding with an additional Cisco UCS 5108 Blade Server Chassis (see page 24)

Requirements

Determine cabling, licensing, and load balancing requirements and recommendations to expand computecapacity on an unlicensed Cisco UCS 5108 Blade Server Chassis.




The following requirements apply:

• Determine the distance between the unlicensed chassis and the associated Cisco UCS FIs.

• Determine the number of available, unused licenses on each Cisco UCS FI.

• Determine the type of Cisco UCS FEX IOM flex link model installed in each chassis and the typeof flex link model deployed with the IOM module.

— For the Cisco UCS FEX IOM flex link model installed in the chassis, fabric A and fabric Bmust be the same.

— Cisco UCS 2204XP Fabric Extender IOM module supports a 2 or 4-link, flex link model.

— Cisco UCS 2208XP Fabric Extender IOM module supports an 8-link, flex link model.

• Verify that each Cisco UCS FEX IOM port is cabled and enabled to a Cisco UCS FI to add eachunlicensed chassis to the Converged System.

— If needed, use either twinax or fiber optic cable with MM, SFP+ modules.

— If Cisco UCS FEX IOM with FET optics is deployed, use additional FET transceivers and MMfibre optic cables.

• If IOM modules are not cabled, verify that the Cisco UCS FI has contiguous Ethernet portsavailable based on the IOM module to be deployed with the selected flex link model.

To prevent unbalanced conditions when installing an additional chassis, match the new IOM modules withthe existing IOM modules.

For example, a chassis installed with Cisco UCS 2204XP Fabric Extender IOM modules has half thebackplane ports that a chassis has with Cisco UCS 2208XP Fabric Extender IOM modules installed. TheCisco UCS 2208XP Fabric Extender IOM provides twice the bandwidth to servers installed in that chassisbased on the increased number of backplane ports.

Related information

Expanding on a licensed Cisco UCS 5108 Blade Server Chassis (see page 17)

Components

Partial SKUs are provided as a reference to order components for an unlicensed Cisco UCS 5108 BladeServer Chassis.

Port licenses

The following table provides the first few characters of SKUs for the FI single port licenses for each CiscoUCS FEX:

SKU Description Notes

Cisco UCS 2204XP Fabric Extender IOM pair with a two-link, flex link model

NSTK-FIPT-xx-x Cisco UCS 62xxUP fabric interconnect oneport 1/10GE/FC-port license

Requires four Cisco UCS FI port licenses (twoper FI)




Cisco UCS 2204XP Fabric Extender IOM pair with a four-link, flex link model


Requires eight Cisco UCS FI port licenses (fourper FI)

Cisco UCS 2208XP Fabric Extender IOM pair with an eight-link, flex link model


Requires 16 Cisco UCS FI port licenses (two perFI)

If the unlicensed chassis is not connected to the Cisco UCS FI, determine the required connection cablesand lengths based on the following parameters:

• IO module type

• Number of active links from the IO module to the FIs

• Distance from the IO module to the FIs

The following table provides the number of required twinax or FET cables for each fabric extender model:

Component Cables

Cisco UCS 2204XP Fabric Extender IOM pair

2-link, flex link model 4 twinax cables or 4 MM fiber optic cables with 8 MM SFP+

4-link, flex link model 8 twinax cables or MM fiber optic cables with 16 MM SFP+

FET 2-link, flex link model 4 MM fiber optic cables, with 8 FET SFPs (if not populated)

FET pair 4-link, flex link model 8 MM fiber optic cables, with 16 FET SFPs (if not populated)

Cisco UCS 2208XP Fabric Extender IOM pair

8-link, flex link model 16 twinax cables or MM fiber optic cables with 32 MM SFP+

FET pair 8-link, flex link model 16 MM fiber optic cables, with 32 FET SFPs (if not populated)

The following table provides the first few characters of Dell EMC SKUs for the twinax cables to connectthe Cisco UCS 22xxXP IO modules to the Cisco UCS 62xxUP fabric interconnects:


CABL-CU1M-xx-xx Passive twinax cable assembly, 1 meter Single cable with optics assembly

YMC00CFPxxxxxxxx* Passive twinax cable assembly, 2 meters Single cable with optics assembly

CABL-CU3M-xx-xx Passive twinax cable assembly, 3 meters Single cable with optics assembly

CABL-CU5M-xx-xx Passive twinax cable assembly, 5 meters Single cable with optics assembly

CABL-ACU7-xx-xx Active twinax cable assembly, 7 meters Single cable with optics assembly

CABL-AC10-xx-xx Active twinax cable assembly, 10 meters Single cable with optics assembly

*Denotes an auto-generated (AG) SKU as opposed to a Dell EMC SKU.

Twinax cables provide connectivity for both IOM modules in a chassis.

Refer to the Cisco UCS 2200 Series Fabric Extenders Data Sheet for different cabling specifications andlengths supported on each IOM module.




Refer to Standard Hardware Expansion Services Data Sheet for solutions for expanding the capacity ofexisting Converged Systems. Blade expansion services are available for purchase. Single chassisupgrade/installation service may be used to complete the installation and upgrade of an existing,unlicensed chassis.

The following table provides the first few characters of Dell EMC SKUs for Professional Services support:

SKU Description

PSVC-HWEX--xx-x For hardware expansion project management, one SKU is required for eachexpanded Converged System

PSVC-HECH--xx-x Single chassis upgrade (installation)

Related information


www.cisco.com


Expanding with an additional Cisco UCS 5108 Blade ServerChassisSpecific conditions and guidelines must be met before expanding compute blade pack capacity with anadditional Cisco UCS 5108 Blade Server Chassis into existing cabinets.

Conditions




— Type of Cisco UCS 22xxXP IOM is supported with 10 Gb connectivity

• Type of Cisco UCS IOM flex link

• Number of available FI ports to support additional IOM links or chassis expansion.

• Six RUs must be available in each cabinet for additional chassis.

• Availability of contiguous FI ports for IOM connectivity depending on the IOM module and theassociated flex link model.



http://www.cisco.com/


• When configuring links between the FEX and an FI in the fabric port channel mode, the amount ofvirtual interface namespace on the adapter depends on where the FEX uplinks connect to the FIports.

— There are six sets of eight contiguous ports, managed by a single chip, inside the Cisco UCS62xxUP fabric interconnect.

— Connect all of the uplinks from an IOM to a set of ports managed by a single chip. This allowsthe Cisco UCS Manager to maximize the number of virtual interfaces used in service profilesdeployed on the chassis blades.

— If uplink connections from an IOM are distributed across ports managed by separate chips,the virtual interface count is decreased.

For example, when Cisco UCS 2208XP Fabric Extender IOM ports are split between Cisco UCS62xxUP fabric interconnect IOM chipsets, the number of available virtual interfaces is based onthe chipset with the fewest number of connected IOM ports. To calculate the number of uservirtual interfaces (vNICs and vHBAs) that are available to a slot when all IOM ports are connectedto the same chipset, use the following formula: (63*n) - 2. Where n is the number ofacknowledged links between the IOM and the FI. In this case, 502 VIFs are available to eachblade slot, however, the mezzanine adapter in each blade also limits the number of supportedVIFs.

Depending on the deployed Cisco UCS code version, the number of supported VIFs per bladeserver varies. With Cisco UCS 2.1 and later, approximately 116 VIFs are supported per bladeserver. In addition, the operating system that is deployed on the blade has its own VIF limits. Thelowest common denominator between the Cisco UCS 2208XP Fabric Extender IOM adapter, themezzanine adapter, and the host OS, determines the number of supported virtual interfaces.

• Verify that the additional environmental, power, and thermal requirement for each chassis doesnot exceed the existing cabinet limitations. If cabinet requirements are not met, do not proceedwith the expansion. Refer to the Converged Systems Physical Planning Guide for additionalinformation.

Related information


Expanding on an unlicensed Cisco UCS 5108 Blade Server Chassis (see page 21)

Requirements

Determine requirements for the number of chassis, the oversubscription value, port reservations, andfabric interconnect combinations to expand compute capacity with additional Cisco UCS 5108 BladeServer Chassis.


• A maximum of four chassis are supported in a new cabinet.

• When adding chassis to a Cisco UCS domain, follow configuration guidelines to maintain anacceptable LAN and SAN uplink oversubscription value of less than 8:1.



• Although there may be available ports on the Cisco UCS 62xxUP fabric interconnect, some portsare reserved for the FC or Ethernet oversubscription feature.

— Cisco UCS 6248UP Fabric Interconnect has four ports reserved for the FC enhancementfeature.

— Cisco UCS 6296UP Fabric Interconnect has 16 ports reserved for the FC enhancement andEthernet performance feature.

Refer to System capacities for the recommended maximum chassis values per flex link model.

Components

Partial SKUs are provided to order components for an additional Cisco UCS 5108 Blade Server Chassis.

The following components are orderable:

• Chassis combined with appropriate number of FI port licenses for additional IOM ports connectingto the FIs.

• The number of port licenses depends on the type of Cisco UCS IOM and flex link model.

The following table provides partial Dell EMC SKUs:


Cisco UCS 5108 Blade Server Chassis with Cisco UCS 2204XP Fabric Extender IOM pair with a two-link, flexlink model

COMP-CHS2-xx-xx Single chassis with Cisco UCS 2204XP Single chassis: 8x fans, 2x Cisco UCS2204XP IOMs, 4 platinum PS, and rail kit.

SFTW-FIPT-xx-xx Cisco UCS 62xxUP fabric interconnectunified port license

Single additional port license for Cisco UCS62xxUP fabric interconnect. Requires 4licenses for 2x Cisco UCS 2204XP IOMs,connecting with 2 links each.

Cisco UCS 5108 Blade Server Chassis with Cisco UCS 2204XP Fabric Extender IOM pair with a four-link,flex link model

COMP-CHS2-xx-xxx Single chassis with Cisco UCS 2204XP Single chassis: 8x fans, 2x Cisco UCS2204XP IOMs, 4 platinum PS, and rail kit.

SFTW-FIPT-xx-xxx Cisco UCS 62xxUP fabric interconnectunified port license

Single additional port license for Cisco UCS62xxUP fabric interconnect. Requires 8licenses for 2x Cisco UCS 2204XP IOMs,connecting with 4 links each.

Cisco UCS 5108 Blade Server Chassis with Cisco UCS 2208XP Fabric Extender IOM pair with an eight-link,flex link model

COMP-CHS3-xx-xxx Single chassis with Cisco UCS 2208XP Single chassis: 8x fans, 2x Cisco UCS2204XP IOMs, 4 platinum PS, and rail kit.


Single additional port license for Cisco UCS62xxUP fabric interconnect. Requires 16licenses for a 2x Cisco UCS 2208XP IOMs,connecting with 8 links each.




Cisco UCS 5108 Blade Server Chassis with Cisco UCS 2204XP Fabric Extender IOM pair with a two-link,FET flex link model

COMP-CHS4-xx-xxx Single chassis with Cisco UCS2204XP-FET

Single chassis with: 8x fans, 2x Cisco UCS2204XP IOMs and 16 FET transceivers, 4platinum PS, and rail kit.



Cisco UCS 5108 Blade Server Chassis with Cisco UCS 2204XP Fabric Extender IOM pair with a four-link,FET flex link model

COMP-CHS4-xx-xxx Single chassis with Cisco UCS 2204XPFET

Single chassis: 8x fans, 2x Cisco UCS2204XP IOMs and 16 FET transceivers, 4platinum PS, and rail kit.



Cisco UCS 5108 Server Chassis with Cisco UCS 2208XP Fabric Extender IOM pair with an eight-link, FETflex link

COMP-CHS5-xx-xxx Single chassis with Cisco UCS 2208XPFET

Single chassis with: 8x fans, 2x Cisco UCS2204XP IOMs and 32 FET transceivers, fourplatinum PS, and rail kit.



Support maintenance



SUCP-CHSX-xx-xxx Annual support on any chassis Provides support for one year for one additional chassis.




PSVC-HWEX-xx-xxx Hardware Expansion Service Hardware Expansion Project Management.

PSVC-HECH--xx-xxx Hardware Expansion Service -single chassis

Expansion of a chassis - Installation.



Expanding with additional cabinets for new Cisco UCS 5108Blade Server ChassisSpecific conditions and guidelines must be met before expanding compute blade pack capacity withadditional cabinets for a new Cisco UCS 5108 Blade Server Chassis.

Conditions


• Type of Cisco UCS 62xxUP fabric interconnect

• Type of Cisco UCS 22xxXP fabric extender IOM

• Type of Cisco UCS 22xxXP fabric extender IOM flex link


Refer to the Dell EMC Converged Systems Physical Planning Guide for additional information.

Related information


Requirements

Determine configuration requirements to expand compute capacity with additional cabinets for new CiscoUCS 5108 Blade Server Chassis.


• Six RUs per chassis.

• 4 x c19-c20 power connections per chassis.

• Ethernet cables for PDU management.

• Ethernet ports for PDU management on AMP switches.

• A maximum of four chassis is supported in a new cabinet.

• Chassis installation starts at the bottom of the cabinet with each chassis mounted as low aspossible in the cabinet.

The power calculation depends on the type of power supply that is installed.

Related information


www.cisco.comExpanding compute capacity | 28


http://www.cisco.com/

Components

Partial SKUs are provided as a reference to order components for additional cabinets for new Cisco UCS5108 Blade Server Chassis.

Compute

The following table provides partial SKUs for compute cabinets:


INFR-CHSY-xx-xxx Chassis package A One per cabinet

INFR-CSBX-xx-xxx Base cabinet compute storage One per cabinet

RACK-VBLK-xx-xxx Converged Systems cabinet sides One per cabinet

INFR-POUF-xx-xxx 33-inch vertical power strip 30 A, 208V, 6 IEC C-13 and 2 IEC C-19receptacles, master

One per cabinet

SPAR-POUF-xx-xxx 33-inch vertical power strip 30 A, 208V, 6 IEC C-13 and 2 IEC C-19receptacles, slave

4 per cabinet

Power distribution units

Order the appropriate number of PDUs required to support 4 chassis per cabinet.


SKU Description

INFR-POWR-xx-xxx Generic PDU for Converged Systems


Professional Services requires a custom Statement of Work (SOW) to complete a cabinet expansion. Thefollowing table provides partial Dell EMC SKUs:


PSVC-HWEX-xx-xxx Hardware Expansion Service Hardware Expansion ProjectManagement

PSVC-HECH-xx-xxx Hardware Expansion Service - singlechassis

Expansion of a single chassis -Installation

Expanding compute capacity to support VMware NSXVMware NSX is only available on VxBlock Systems. VMware NSX uses Cisco UCS B-Series BladeServers for the edge cluster configuration in a VMware NSX deployment. The number of Cisco UCS B-Series Blade Servers depends on the number of edge service gateways (ESGs) deployed in a singleVMware NSX domain. The Cisco UCS B-Series Blade Servers must be the M4 model servers.

Related information

Expanding on a licensed Cisco UCS 5108 Blade Server Chassis (see page 17)29 | Expanding compute capacity


Requirements

Expanding compute capacity to support VMware NSX includes requirements for compute, virtual switch,licensing and support.

Compute

The following table provides requirements to add edge service gateways (ESG):

Number of ESGs Number of required Cisco UCS B-Series M4 BladeServers

Description

4 4 Each blade server uses VXLANoffload capable network adapters(such as VIC 1340 and 1380) toimprove CPU performance.

6 5

8 6

There may be limitations where bandwidth is half the link speed of the cards.

Virtual Switch

The following virtual switch requirements apply:

• VxBlock Systems with VMware NSX require VMware VDS.

• VMware NSX does not support the Cisco Nexus 1000V Series Switch and should not have theVEMs installed on the VMware vSphere ESXi hosts.

License and support

Each CPU socket on each additional blade server requires an additional VMware NSX license.



Expanding storage capacityAll hardware expansion activities for VMAX3 arrays require an engagement and must be done throughthe VMAXSizer tool. This engagement ensures the integrity of back-end and cache configurations on thearray. All hardware modifications to VMAX3 arrays require updates to the array configuration (BIN) file,which is performed by Dell EMC personnel only.

Expanding storage capacity with available drive slotsExpanding storage capacity on existing disk array enclosures (DAEs) with available drive slots includessome conditions, requirements, and component guidelines.

Conditions

The original, architectural design of the Converged System places constraints on storage expansionsincluding the number of drives supported.

Requirements

Use this topic to expand storage capacity with available drive slots that involve disk drive, disk group, andstorage pool requirements.

Disk drives

When expanding a Converged System, consider the following drive count and usage requirements:

• Determine the number of drives by a form factor of 2.5 inch or 3.5 inch, drive technology (EFD,SAS, NL-SAS), size (for example, 300 GB), and speed (for example, 10,000 RPM).

• Determine the number of DAEs by form factor.

• Determine the number of empty drive slots by form factor.

To determine the number of drives required for expansion, consider desired capacity and IOPS.

Disk groups, thin pools, and Storage Resource Pools

Adding capacity to an existing VMAX3 can be done only through existing disk groups, thin pools, andStorage Resource Pools (SRPs). Adding disk groups, thin pools, or SRPs requires a Request PriceQuotation (RPQ).

When adding drives to an existing disk group, match the existing drive technology and RAID protection.This expansion activity automatically (through Dell EMC configuration activities) expands the poolassociated with the disk group as well as the capacity of the associated SRP.

Components

Expanding storage capacity with available drive slots involves ordering guidelines for different disk formfactors.

31 | Expanding storage capacityDELL EMC CONFIDENTIAL


Form factors

• 2.5 inch form factor drives

— EFD for all flash array - 960 GB, 1.92 TB, 3.84 TB, 7.68 TB (250F/250FX only), 15.36 TB(250F/250FX only)

— EFD - 200 GB, 400 GB, 800 GB, 960 GB, 1.6 TB, 1.92 TB

— 10K SAS - 300 GB, 600 GB, 1.2 TB

— 15K SAS - 300 GB

• 3.5 inch form factor drives

— EFD - 200 GB, 400 GB, 800 GB, 1.6 TB

— 7.2K NL-SAS - 2 TB, 4 TB

— 10K SAS - 300 GB, 600 GB, 1.2 TB

— 15K SAS - 300 GB

Dell EMC hardware maintenance is a separate SKU that must be purchased with the drives.

Expanding storage capacity | 32DELL EMC CONFIDENTIAL


Expanding network capacity

Expanding with FC bandwidthSpecific conditions and guidelines must be met before expanding network capacity with FC bandwidth.

Cisco MDS Switches provide FC connectivity between the Cisco UCS hosts and the storage array. TheCisco MDS 9148S and Cisco MDS 9396S 16G Multilayer Fabric Switches are the only fixed, FC switchavailable on a Converged System and provides no IOM expansion for additional FC ports.

The Cisco MDS 9706 Multilayer Director is a modular switch with additional expansions using the IOMline cards.

The Dell EMC standard configurations for a Cisco MDS 9148S Multilayer Fabric Switch support theconnection of up to 40 ports from FI switches and an VMAX3 storage array in both unified andsegregated networking configurations.

Conditions

The original architectural design of the Converged System and the following physical and logicalconfigurations limit a network expansion:

• Cisco UCS 62xxUP fabric interconnects support FC expansions with 10 Gb connectivity.

• The Cisco MDS 9148S Multilayer Fabric Switch, Cisco MDS 9396S 16G Multilayer Fabric Switch,and Cisco MDS 9706 Multilayer Director support FC expansions.

Requirements

Use this topic to expand network capacity with Fibre Channel (FC) bandwidth requirements for compute,storage, storage area networks, fabric interconnects, and cabling.

Before expanding network capacity, complete the following:

• Connectivity chart for Cisco UCS 62xxUP fabric interconnect to Cisco MDS 9148S MultilayerFabric Switch, Cisco MDS 9396S Multilayer Switch, or Cisco MDS 9706 Multilayer DirectorSwitch.

• Inventory of used and unused ports on each of the fabric interconnects.

• Inventory of configuration backups.

Compute

Determine the impact of host access (if clusters will be rebalanced or new ones added).

The base connectivity for the Cisco UCS 62xxUP fabric interconnects is four links to the Cisco MDS9148S Multilayer Fabric Switch, Cisco MDS 9396S 16G Multilayer Fabric Switch, or Cisco MDS 9706Multilayer Director Switch providing 32 Gbps (4x8Gbps) of FC bandwidth. The number of links can beexpanded to 8 or 16.

33 | Expanding network capacityDELL EMC CONFIDENTIAL


Available port-based licensing options on the Cisco UCS FIs allow additional networking port capacity onan individual system, increasing bandwidth incrementally.

• The Cisco UCS 6248 Fabric Interconnect has pre-installed licenses for the first twelve unifiedports enabled in Cisco UCS Manager. Expansion modules come with eight licenses that can beused on the expansion module or the base module.

• The Cisco UCS 6296 Fabric Interconnect has pre-installed licenses for the first eighteen unifiedports enabled in Cisco UCS Manager. Expansion modules come with eight licenses that can beused on the expansion module or the base module.

Determine the additional port licenses required for the addition of FC ports expansion on the Cisco UCS62xxUP fiber interconnects. To use additional fixed ports, purchase and install licenses for those ports.

Additional ports used for the FI expansion may also require additional port licenses on the Cisco MDS9148S Multilayer Fabric Switch.

Storage

Determine the impact of the storage resources expansion for adding VMAX3 arrays to the ConvergedSystems.

The Converged System base configuration ships with the following base storage array configuration,which determines the number of FC ports used on the Cisco MDS switch.

VMAX3 models 100K 200K 400K

Engines 1-2 2-4 2-8

Maximum number ofdrives

1440 2880 5760

Maximum number ofFA ports

32 64 128

The Converged System optionally ships with the following all flash array configurations.

Model Number of V-Bricks Number of drives

250F/250FX 1 - 2 100

250F/250FX single cabinet Vblockconfiguration

1 50

450F/450FX 1 - 4 960

850F/850FX 1 - 8 1,920

Model FA ports

250F/250FX 32

250F/250FX single cabinet Vblock configuration 16

450F/450FX 48

850F/850FX 96

Expanding network capacity | 34DELL EMC CONFIDENTIAL


SAN

The Cisco MDS 9706 Multilayer Director Switch and Cisco MDS 9396S Multilayer Switch support only a48 port FC port module that provides 48 line-rate 16Gbps FC ports.

Achieving line-rate on each module requires three fabric modules.

The following tables show the IOM expansion for the Cisco MDS 9706 Multilayer Director Switch andCisco MDS 9396S Multilayer Switch based on the FI connections and Cisco MDS ports used by the arraybased on the engine counts on the array:

VMAX3 engine counts (4 Subscriber Line Interface Cards (SLICs) per engine, 8 FCports)

1 2 3 4 5 6 7 8

FC ports 8 16 24 32 40 48 56 64

FI connections VMAX3 engine counts (4 SLICs per engine, 8 FC ports)

1 2 3 4 5 6 7 8

4 1 1 1 1 1 2 2 2

8 1 1 1 1 2 2 2 2

16 1 1 1 2 2 2 2 2

24 1 1 2 2 2 2 2 2

32 1 2 2 2 2 2 2 3

40 2 2 2 2 2 2 3 3

48 2 2 2 2 2 3 3 3

56 2 2 2 2 3 3 3 3

64 2 2 2 3 3 3 3 3

The following tables provide engine counts for 8 SLICs per engine and 16 FC ports:

VMAX3 engine counts (8 SLICs per engine, 16 FC ports)

1 2 3 4 5 6 7 8

FC ports 16 32 48 64 80 96 112 128


1 2 3 4 5 6 7 8

4 1 1 2 2 2 3 3 3

8 1 1 2 2 2 3 3 3

16 1 2 2 2 3 3 3 4

24 1 2 2 2 3 3 3 4

32 2 2 2 3 3 3 4 4

40 2 2 2 3 3 3 4 4




1 2 3 4 5 6 7 8

48 2 2 3 3 3 4 4 4

56 2 2 3 3 3 4 4 4

64 2 3 3 3 4 4 4 4 *

* One FI connection set must have only 14 ports used per fabric.

The following table shows the maximum total FC ports connectivity on a Converged System for the CiscoMDS 9148S Multilayer Fabric Switch with available ports for VMAX3 and FI:

Number ofdomains

Management * Domain1 Domain 2 Domain 3 Domain 4 Portsavailablefor array

Total FI FCports

1 2 8 - - - 32 8

1 2 16 - - - 24 16

2 2 8 - - 8 24 16

2 2 8 - - 16 16 24

2 2 16 - - 16 8 32

3 2 8 8 - 8 16 24

3 2 8 16 - 8 8 32

4 2 8 8 8 8 8 32

* AMP-2 connections may require more than two per switch.

The Converged System base model ships with a 12 port base license. Determine the requirements forany additional port licenses for the Cisco MDS 9148S Multilayer Fabric Switch based on the previous FCports connectivity table. Purchase additional port licenses in increments of 12. A 12 port On-Demand PortActivation license enables 24, 36, or 48 licensed ports on the Cisco MDS 9158S Multilayer Fabric Switch.

For example, to expand to a four Cisco UCS domain, 1-engine VMAX3 array with eight FI connectionsper domain, plus two management ports requires a total of 42 FC ports on the Cisco MDS 9148SMultilayer Fabric Switch and a total of four, 12 port expansion licenses enabled on the Cisco MDS 9148SMultilayer Fabric Switch.

Determine if the expansion on the Cisco MDS 9148S Multilayer Fabric Switch, Cisco MDS 9396SMultilayer Switch, or Cisco MDS 9706 Multilayer Director Switches requires procurement of more of anyof the following:

• SFPs

• Port licenses on the Cisco MDS 9148S Switch

• 48 port expansion modules for the Cisco MDS 9706 Switch and Cisco MDS 9396S Switch

• Fibre Channel cables



The following table provides the first few characters of the SKUs for the expansion:

Dell EMC SKU Part number Description

YME0000Exxxxxxxx MDS-9148S-POD Connectrix 9148S 16G FC 12P UpgLic=MA

YME0000Cxxxxxxxx 16G-FC-SFP Cisco MDS 16GB FC SW Optic

YME0000Cxxxxxxxx 48PT-16G-FC Cisco MDS 9000 48-Port 16-GbpsFC Module

Cisco UCS 62xxUP fabric interconnects

Determine whether the expansion on the Cisco UCS 62xxUP fabric interconnect requires more of any ofthe following:

• SFPs

• Port licenses

• Sixteen port expansion modules

• FC cables

In some cases, these devices may be available.

Additional partial quantities may be required to successfully implement the FC bandwidth expansion.

Cabling

Each installation requires sixteen additional FC cables. Cable lengths must be calculated for eachinstallation location.

Professional Services

• Install and connect hardware (FC cables)

• Configure hardware (network switch port channels, zoning hosts to new SP ports).

• Change the cluster configuration or create a new configuration to accommodate new storageports.

To implement network expansion, follow all blocked rules. In some cases, adequate free port licenses orSFPs may already be available. Partial quantities may be required to successfully implement FCbandwidth.

Components

Use the provided, partial SKUs to order components for expanding network capacity with Fibre Channel(FC) bandwidth.



The following table provides the first few characters of the SKUs:

SKU Description

AGGC-8GFC-xx-xxx Eight Gbps FC SFP (same part for all devices)

MDS-9148-xxx Cisco MDS 9148 Multilayer Fabric Switch eight port license

SFTW-FIPT-xx-xxx Cisco UCS Series fabric interconnect one port 1/10GE/FC-port license

Expanding with Ethernet bandwidth enhancementSpecific conditions and guidelines for links and ports must be met before expanding network capacity withEthernet bandwidth.

Ethernet bandwidth enhancement is not available on the Cisco UCS 6248UP Fabric Interconnect. TheEthernet bandwidth expansion provides:

• Each Cisco UCS 6296UP Fabric Interconnect base uplink configuration provides 80 Gbps ofconnectivity to the LAN network

• Double the base uplink configuration bandwidth between the fabric interconnects (FIs) and theCisco Nexus 5596UP Switches, Cisco Nexus 93180YC-EX Switch, or Cisco Nexus 9396PXSwitch.

• Eight additional 10 Gbps uplinks for each FI

— Four of the eight links connect to the first Cisco Nexus 5596UP Switch, Cisco Nexus93180YC-EX Switch, or Cisco Nexus 9396PX A Switch

— Four of the eight links connect to the second Cisco Nexus 5596UP Switch, Cisco Nexus93180YC-EX Switch, or Cisco Nexus 9396PX A Switch

Conditions


• Eight contiguous available Ethernet ports on each Cisco UCS 6296UP Fabric Interconnect

• Eight contiguous available Ethernet ports on each Cisco Nexus 5596UP Switch, Cisco Nexus93180YC-EX Switch, or Cisco Nexus 9396PX Switch

Requirements

Use this topic to determine connectivity and port usage requirements before expanding network capacitywith Ethernet Bandwidth enhancement.

The following requirements apply for expanding network capacity:

• Complete the Cisco UCS 6296UP Fabric Interconnect to Cisco Nexus 5596UP Switch, CiscoNexus 93180YC-EX Switch, or Cisco Nexus 9396PX Switch connectivity chart.

• Inventory used and unused ports on each of Cisco UCS 6296UP Fabric Interconnect and CiscoNexus 5596UP Switch, Cisco Nexus 93180YC-EX Switch, or Cisco Nexus 9396PX Switch.



• Verify usage for each Cisco UCS 6296UP Fabric Interconnect license.

• Determine the distance between each Cisco UCS 6296UP Fabric Interconnect and the CiscoNexus 5596UP Switch, Cisco Nexus 93180YC-EX Switch, or Cisco Nexus 9396PX Switch todetermine the type of required cables and small, form-factor pluggable (SFP) devices.



• Install and connect hardware.

• Configure hardware (network switch and FI port channels).

Components

Partial SKUs are provided as a reference to order components for Ethernet bandwidth.

Fabric interconnect licenses

You can order up to 16 additional Cisco UCS FI licenses. The following table provides partial Dell EMCSKUs:


SFTW-FIPT-xx-xxx Cisco UCS 62xxUP FI, unifiedport license

Single additional port license for Cisco UCS 62xxUP fabricinterconnects. Requires 4 licenses for two Cisco UCS2204XP Fabric Extender IOMs, connecting with 2 linkseach.

Cables

To provide connectivity between the Cisco UCS 6296UP Fabric Interconnects and the Cisco Nexus5596UP Switch, Cisco Nexus 93180YC-EX, or Cisco Nexus 9396PX Switch, order 16 additional twinaxcables or 32 MM SFPs and MM fibre cables.

The following table provides partial Dell EMC SKUs for the single twinax cables with an optics assembly:

SKU Description

CABL-CU1M-xx-xxx Passive 1 meter twinax cable assembly

YMC00CFPxxxxxxxx* Passive 2 meter twinax cable assembly



CABL-ACU7-xx-xxx Active 7 meter twinax cable assembly

CABL-AC10-xx-xxx Active 10 meter twinax cable assembly

* Denotes an SKU auto-generated by Cisco, Dell EMC, or VMware.



Expanding network capacity with Ethernet bandwidthenhancement - customer uplinksExpanding network capacity with Ethernet bandwidth enhancement with customer uplinks involvesguidelines and conditions for port availability and cabling.

The Ethernet bandwidth performance enhancement feature with customer uplinks provides:

• 80 Gbps of Ethernet bandwidth to the customer network using 10 GbE.

• 160 Gbps of Ethernet bandwidth to the customer network using 40 GbE with the Cisco Nexus93180YC-EX or Cisco Nexus 9396PX switches.

• Double the base uplink configuration bandwidth between the Cisco Nexus 5596UP Switches,Cisco Nexus 93180YC-EX Switches, or Cisco Nexus 9396PX Switches and the customernetwork.

• Four additional 10G uplinks for each Cisco Nexus switch.

• Two additional 40 GbE uplinks for each Cisco Nexus 93180YC-EX Switch or Cisco Nexus9396PX Switch.

• Two additional 100 GbE uplinks for each Cisco Nexus 93180YC-EX Switch.

Conditions

The original design parameters limit an expansion to a specific Converged System.

The following conditions apply:

• Each Cisco Nexus 5596UP Switch, Cisco Nexus 93180YC-EX Switch, or Cisco Nexus 9396PXSwitch must have eight contiguous available 10GbE Ethernet ports.

• Each Cisco Nexus 93180YC-EX Switch or Cisco Nexus 9396PX Switch QSFP+ module musthave four contiguous available 40GbE Ethernet portsor 100 GbE Ethernet ports.

• A Cisco Nexus 5596UP Switch may require an additional 16UP port module for additionalconnectivity.

• A Cisco Nexus 93180YC-EX Switch or Cisco Nexus 9396PX Switch may require additional 40GbE or 100 GbE QSFP+ ports for increasing number of available 40 GbE or 100 GbE links.

Optionally, instead of fiber cables and QSFP+ modules, use twinax cabling betweenthe Cisco Nexus 93180YC-EX Switch or Cisco Nexus 9396PX Switches and the coredevice.

The following topics describe orderable components and requirements to consider when planning anetwork expansion.

Requirements

Use this topic to determine inventories, cabling distances, and cable type requirements before expandingnetwork capacity with Ethernet bandwidth enhancement with customer uplinks.



The following requirements apply for network expansion:

• Complete the Cisco Nexus 5596UP, Cisco Nexus 93180YC-EX, or Cisco Nexus 9396PX Switchconnectivity chart.

• Inventory the ports (used and unused) on each Cisco Nexus 5596UP, Cisco Nexus 93180YC-EX,or Cisco Nexus 9396PX Switch.

• Determine the distance between each Cisco Nexus 5596UP, Cisco Nexus 93180YC-EX, or CiscoNexus 9396PX Switch and the customer uplink switches.

• Determine the physical fibre optic connection type MTP/MPO (SR4) or LC connectors todetermine correct QSFP+ modules for the Cisco Nexus 93180YC-EX or Cisco Nexus 9396PXonly.

• Determine the type of required cables and SFP or QSFP+ devices.



• Install and connect hardware.

• Configure network components.

Components

To provide connectivity for the Cisco Nexus 5596UP, Cisco Nexus 93180YC-EX Switch, or Cisco Nexus9396PX Switches and network switches, you can order up to sixteen additional cables.

Expanding with the four-to-eight link upgradeSpecific conditions and guidelines must be met before expanding network capacity with the four-to-eightlink performance upgrade feature.

Conditions

The Cisco UCS IOM four-to-eight link performance upgrade feature increases the number of connectionsfrom each Cisco UCS chassis from four 10 Gbps links (per IO module) to eight 10 Gbps links.


• Increasing the number of IOM links between the Cisco 2208XP Fabric Extender IOM and thefabric interconnects is supported on Cisco UCS 62xxUP fabric interconnects.

Requirements

Use this topic to determine connectivity, cable types, cabling distances, and port requirements beforeexpanding network capacity with a Cisco UCS four to eight link upgrade.

Cisco UCS fabric port channels are part of the Converged System design. Dell EMC has qualified fabricport channels with two, four, or eight links.



The following requirements apply for network expansion:

• Complete a Cisco UCS 62xxUP fabric interconnect to Cisco UCS Fabric Extender 2208XP IOMpoint-to-point connectivity chart before upgrading each Cisco UCS domain.

• Document used ports for the four-to-eight link upgrade on both Cisco UCS 62xxUP fabricinterconnects and the Cisco UCS 2208XP Fabric Extender IOM modules.

• The number of links connected on the IOM for fabric A and fabric B must be consistent for eachCisco UCS 5108 Server Chassis.

• Verify existing Cisco UCS 62xxUP fabric interconnect license usage and availability.

• Determine the type of required cables and small, form-factor, pluggable (SFP) devices.

• Determine the distance between each Cisco UCS 62xxUP fabric interconnect and the associatedCisco UCS Fabric Extender 2208XP Fabric Extender IOM module to determine the type of cablesand SFP devices for new connections.

• Determine the number of Cisco UCS 2208XP Fabric Extender IOM modules that use four linksand will be upgraded to eight.

• Determine the number of available Unified ports on Cisco UCS 62xxUP fabric interconnects.

— If there are insufficient ports, and a module slot is available, add a Unified expansion module.

— Verify that the Unified port expansion module provides the required number of additionalports.

You may need to upgrade the Cisco UCS 6296UP Fabric Interconnect withadditional ports to increase the number of used ports on the Cisco UCS 2208XPFabric Extender IOM module from four to eight.

• When developing the Cisco UCS 62xxUP fabric interconnect port map for the addition of theCisco UCS IOM Link Upgrade, Dell EMC recommends that all eight IOM ports, when configuredwith fabric port channel mode, be connected to the same chipset on the Cisco UCS 62xxUPfabric interconnect for maximum virtual interface (VIF) availability. VIFs include vNICs andvHBAs.

— Recabling the location of each IOM port on the Cisco 62xxUP fabric interconnects may berequired. Only one FI should be updated at a time to avoid an outage. While each FI issequentially upgraded, data traffic proceeds with the other FI.

— If ports from the same IOM module are distributed across different chipsets on the Cisco UCS62xxUP fabric interconnect, there are fewer VIFs available for the service profiles deployedon the chassis blades.

• When Cisco UCS 2208XP Fabric Extender IOM ports are split between Cisco UCS 62xxUP fabricinterconnect IOM chipsets, the number of available VIFs is based on the chipset with the fewestnumber of connected IOM ports. If the IOM ports are split between different application-specificintegrated circuits (ASICs), there will be a significant reduction in the number of VIFs availableper server. This can be exceeded in a normal configuration.

• To calculate the number of VIFs available to a slot when all IOM port are connected to the samechipset, use the following formula: (63*n) - 2. n is the number of acknowledged links between theIOM and the FI.



• In this case, 502 VIFs are available to each blade slot, however the mezzanine adapter in eachblade will also have a limit on the number of VIFs supported.

• Depending on the deployed Cisco UCS code version, the number of supported VIFs per blademay vary. With Cisco UCS Manager Release 2.1 and later, approximately 116 VIFs aresupported per blade.

• The OS being deployed on the blade has its own VIF limits. The lowest common denominatorbetween the Cisco UCS 2208XP Fabric Extender IOM adapter, the mezzanine adapter and thehost OS determines the number of supported VIFs.

The following table provides the maximum number of chassis that are configurable, based on the switchtype and IOM connectivity model:

Fabric interconnect IOM connections per chassis perfabric

Chassis maximum

Cisco UCS 6248UP 2 16

4 8

8 4

Cisco UCS 6296UP 2 Not supported

4 16

8 8

Components

To expand network capacity with the four to eight link upgrade, you can order additional fabricinterconnect licenses and cables.

Order any of the following additional components:

• Cisco UCS fabric interconnect licenses as required for all IOM ports to be connected to the CiscoUCS 62xxUP fabric interconnect.

• Twinax cables between the Cisco UCS 62xxUP fabric interconnects and the Cisco UCS FabricExtender 2208XP IOMs, or multi-mode (MM) small, form-factor, pluggable (SFP) devices and MMfibre cable (as required for distance).

Expanding network capacity to support VMware NSXVMware NSX uses standard port-channel configuration guidelines to share virtual infrastructure coretraffic with the transport VXLAN (east-west) and Disjoint Layer 2 bridge (optional) traffic.

VMware NSX supports the Cisco Nexus 9300 series switches but does not support the CiscoNexus 5500 series switches.

You should configure a Disjoint Layer 2 non-vPC port channel to support the external (north-south) edgetraffic.

VMware NSX only applies to VxBlock Systems.



Related information

Expanding with Ethernet bandwidth enhancement (see page 38)

Requirements

Use this table to determine the cabling requirements for your system configuration.

The edge service gateways (ESGs) are configured as a Disjoint Layer 2 port channel directly connectedto the Cisco Nexus 9300 series switches for all edge traffic. The number of SFP+ cables connected to theCisco UCS 6200 fabric interconnects and Cisco Nexus 9300 series switches depends on the number ofESGs deployed.

The following table provides the number of required cables for the number of NSX ESGs:

Number of VMware NSX ESGs Number of SFP+ cables required

4 8

6 12

8 16



The information in this publication is provided "as is." Dell Inc. makes no representations or warranties of any kind withrespect to the information in this publication, and specifically disclaims implied warranties of merchantability or fitnessfor a particular purpose.

Use, copying, and distribution of any software described in this publication requires an applicable software license.

Copyright © 2015-2017 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks aretrademarks of Dell Inc. or its subsidiaries. Other trademarks may be the property of their respective owners. Publishedin the USA in March 2017.

Dell EMC believes the information in this document is accurate as of its publication date. The information is subject tochange without notice.

45 | CopyrightDELL EMC CONFIDENTIAL


Documents

VCE VxBlock and Vblock Systems 740 Expansion Guide...• Cisco Nexus 3064-T Switch (Ethernet expansion only) ... Cisco Nexus 1000V Switch for VMware vSphere Compatibility Information