Brocade BCFA 250 Preparing BCFA Certified Professionals for the 16Gbps BCA Exam

8/18/2019 Brocade BCFA 250 Preparing BCFA Certified Professionals for the 16Gbps BCA Exam

1/130

FA 250

evision 0312

Preparing for the 16 Gbps BCFA Ex


2/130

FA 250

evision 0312



3/130

FA 250

evision 0312



4/130

FA 250

evision 0312



5/130

FA 250

evision 0312



6/130

FA 250

evision 0312



7/130

FC-0 and FC-1 levels specify physical and data link functions needed to physically send

data from one port to another.

FC-0 level specifications include information about feeds and speeds.FC-1 level contains specifications for 2, 4, 8 Gbps 8b/10b encoding, ordered set and

link control communication functions. 10 and 16 Gbps communication uses 64b/66b

encoding.

FC-2 level specifies content and structure of information along with how to control and

manage information delivery . This layer contains basic rules needed for sending data

across the network. This includes: (1) how to divide the data into frames, (2) how much

data should be sent at one time before sending more (flow control), and (3) where the

frame should go. It also includes Classes of Services, which define different

implementations that can be selected depending on the application.

FC-3 level defines advanced features such as striping (to transmit one data unit across

multiple links) and multicast (to transmit a single transmission to multiple destinations)

and hunt group (mapping multiple ports to a single node). While the FC-2 level concerns

itself with the definition of functions with a single port, the FC-3 level deals with

functions that span multiple ports.

FC-4 level provides mapping of Fibre Channel capabilities to pre-existing protocols, such

as IP, SCSI, or ATM.

FA 250

evision 0312



8/130

For a switch port that goes through port initialization, it arrives at an ending status of

F_Port, FL_Port or E_Port.

Footnote1: Loop ports are not supported on Condor3 ASICs.Footnote 2: EX and VEX_Ports allow communication between devices in independent

fabrics without having to merge the fabrics. This is done through the use of FC-FC

Routing. To learn more about FC-FC routing and fabric extension solutions please refer

to CFP300 or RE300 courses.

FA 250

evision 0312



9/130

When a node attaches to the fabric, it must receive a unique 24-bit address. The

network address is a three-byte address based upon the Domain ID, the Area ID and, if a

loop device, its AL_PA. This address is the source address and is used for routing data

thru the fabric from one device to another.

Footnote 1: There are other variations of addressing that are beyond the scope of this

course.

Footnote 2: XX will be 00 for Fabric OS switches.

Fabric-attached devices use an address format of “DD AA 00”. This is the address of

any Fabric-attached device that has logged into the fabric as point-to-point.

Public Loop attached devices use an address format of “DD AA PP”. The “DD AA” bytes

of the address come from the fabric login process and the “PP” byte is assigned duringFC_AL initialization.

NPIV attached devices use an address format of “DD AA PP”. The “DD AA” bytes of the

address come from the fabric login process and the “PP” byte is assigned during login

process. More information on NPIV at the end of this module.

FA 250

evision 0312



10/130

Shared Area IDs will use the Node Address to allow 384 ports to be addressed in a

single domain.

Footnote 1: The FC8-48 blade does not use shared areas when installed into a DCX-4S

since the total port count in the domain would not exceed 256.

FA 250

1evision 0312



11/130

Footnote1: The flow chart outlines the Fabric Initialization process supported by Fabric OS, but not allBrocade hardware, such as the Condor3, support loop.

A Universal Port (U_Port) is the initial state of a port. (State 1)

Is something connected (sending a light/electrical signal) to the port? If yes, continue. (Transition 1)

U_Port starts mode detection process by transmitting at least 12 LIP(F7) Primitive Sequences. (Transition2)

• If at least 3 consecutive LIP Primitive Sequences are received, then the port enters OPEN_INIT stateand attempts FC-AL loop initialization. (State 2)

• If LIP Primitive Sequences are not received, the U_Port attempts OLD_PORT initialization by takingthe link down then transmitting NOS primitives. If Link Initialization Protocol fails after 1 retry or LIPreceived after 1 second, go to FC-AL initialization. (Transition 2)

• When operating in the FL_Port mode, a U_Port will try the loop initialization procedure three times.

If these fail, the port will be marked as faulty. To ensure N_Port, reinitialize the port and the switchport will cut the laser forcing a loss of signal state for at least 20 μs. Then the switch port will bringback the laser and issue NOSs. (Transition 2)

If the attached device is not loop it continues into the G_Port stage. A device can be a switch, target(usually storage), or an initiator (usually a host). (State 3)

If the attached device is a target or initiator it changes its port state from G_Port to an F_Port. (State 5).

If the attached device is a switch then it changes its port state from a G_Port to an E_Port. (State 4).

F_Ports and E_Ports continue to login to the fabric which is explained later in this module.

FA 250

1evision 0312



12/130

Every switch has reserved 24-bit called ‘Well-Known Addresses’. The services residing at

these addresses provide a service to either nodes or management applications in the

fabric. Below are other servers not listed in the table:

FFFFF6 Clock Synchronization Server: Clock synchronization over Fibre Channel is

attained through a Clock Synchronization Server that contains a reference clock. The

server synchronizes client’s clocks to the reference clock on a periodic basis, using

either primitive signals or ELS frames.

FFFFF7 Security Server: The security-key distribution service offers a mechanism for the

secure distribution of secret encryption keys.

FFFFF8 Alias Server: The Alias Server manages the registration and deregistration of

alias IDs for both hunt groups and multicast groups. The Alias Server is not involved in

the routing of frames for any group.FFFFFB Time Server: The Time Server sends to the member switches in the fabric the

time on either the principal switch or the primary FCS switch.

FFFFFD Fabric Controller: The fabric controller provides state change notifications to

registered nodes when a change in the fabric topology occurs.

FFFFFF Broadcast Server: When a frame is transmitted to this address, the frame is

broadcast to all operational N and NL_Ports.

FA 250

1evision 0312



13/130

FA 250

1evision 0312



14/130

Provides a more robust credit recovery for lost buffer credits or frames on long distance

E_Ports on between two non-Condor3 ASIC switches or backbones. If a lost buffer credit

or frame is detected, the switch performs a Link Reset (LR) to recover the lost credit.

This is done by sending an LR to the target switch which sends back an LRR (Link ResetResponse). Because this happens on an E_Port, the link does not reset, only the BB

(frame and credit loss) counters at both ends of the link are reset.

16 Gbps – QSFP

and cable

FA 250

1evision 0312



15/130

Footnote 1: Hardware detection in the Condor3 ASIC

Footnote 2: If one side is a Condor2 it uses port-level credit recovery.

FA 250

1evision 0312



16/130

This is the case where credits are simply not returned/permanently lost. This is different

from a latency bottleneck where credits may eventually return slowly.

Footnote 1: ASICs hold a frame for 500ms if it cannot forward the frame. If the frame is

discarded the ASIC should return the credit. The 600ms time is past the time the switch

has to return the credit by about 100ms.

portCfgCreditRecovery

Enables or disables credit recovery on a port.

portcfgcreditrecovery --enable [

slot ]port

portcfgcreditrecovery --disable [slot ]port

Use this command to enable or disable credit recovery on a port. The credit recovery

feature enables credits or frames to be recovered. Only ports configured as long

distance ports can utilize the credit recovery feature. The default credit recovery

configuration is enabled.

FA 250

1evision 0312



17/130

FA 250

1evision 0312



18/130

For additional information on Brocade transceiver optics visit our web site:

http://www.brocade.com/products/all/transceivers/product-details/transceiver-

modules/index.page.

SFPs (Small Form Pluggable) come in a variety of speeds with a number of different

standards in use for different applications. The SFP/SFP+ standards define the physical

specifications for the transceiver.

In addition to the physical specifications the SFP+ standard is used for transceivers that

are capable of 10 Gbps or greater transmission rates.

Footnote 1: QSFPs (Quad SFPs) are a new transceiver design that allows for four

separate data paths through the transceiver and across an industry standard cable.

Currently these are only being used in the 16 Gbps Brocade Backbones which are

covered in the following module.

16 Gbps – QSFP

and cable

FA 250

1evision 0312


http://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/index.page


19/130

Switch1:admin> sfpshow -all

============

Slot 3/port 0:

============

QSFP No. 0 Channel No:0

Identifier: 13 QSFP+

Connector: 12 MPO Parallel OpticTransceiver: 0000000000000000 16_Gbps idEncoding: 5 64B66B

Baud Rate: 140

Length 9u: 0

Length 9u: 25 Length 50u: 0

Length 62.5u: 0

Length Cu: 0 Vendor Name: Brocade

Vendor OUI: 00:15:1e

Vendor PN: 57-0000090-01Vendor Rev: A

Wavelength: 850

Options: 00000fde

Max Case Temp: 70

Device Tech: 0x00Serial NO: HTA111111002103

Date Code: 110317

DD Type: 0x8

Enh Options 0x0

Status/Ctrl: 0x0Alarm flags [0,1] = 0x0, 0x0

Warn Flags [0,1] = 0x0, 0x0Alarm Warn

low high low high

Temperature: 29 Centigrade -5 85 0 80

Current: 6.748 mAmps 0.500 10.000 1.0009.500

Voltage: 3271.9 mVolts 2970.0 3630.0 3134.9

3465.0

RX Power: -6.9 dBm 44.6 uW2187.8uW 112.2 uW1737.8 uW

State transitions: 2

FA 250

1evision 0312



20/130

The table above offers a side-by-side comparison of the different types of Brocade Fibre

Channel optics available. For each optic the supported speeds, cable types, and

maximum distance are listed. To get a full list of compatible optics please visit the

Brocade website and download the Brocade Compatibility Matrix?

Footnote 1: The distance shown in this chart represents the maximum distance, with

high quality cabling, that is available at the maximum line speed of the transceiver.

Distances can actually be increased beyond what is listed by reducing transmission

speed. For a full list of supported distances, speeds, and cable types for any transceiver

please visit our web site:

http://www.brocade.com/products/all/transceivers/product-details/transceiver-

modules/features.page.

Footnote 2: The 4x16 Gbps QSFP is currently only used on the DCX8510-4 and DCX

8510-8 Backbone switches for ICL links between chassis. These switches will be

covered in greater detail in the next module.

FA 250

2evision 0312


http://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.pagehttp://www.brocade.com/products/all/transceivers/product-details/transceiver-modules/features.page


21/130

Brocade HBAs, CNAs, and Fabric Adapters are design to support a wide range of

networking environments. They can be broken into three main categories: Fibre Channel,

Ethernet, and converged. The media used will depend on the application that the

adapter is being used for. For example, if you wish to use a Fabric Adapter in an Ethernetor converged environment you will install Ethernet style SR or LR transceivers.

Converged networking is a new data center technology that combines the Fibre Channel

protocol with Ethernet frames for transport. For more information on converged

technologies please see the following Brocade University courses:

• http://www.brocade.com/education/product-training/index.page

– WBT: FCoE 101, VCS 101, VDX 110

– ILT/VCT: FCoE 200, CEF 200

Fibre Channel Ethernet Converged

HBA

CNA

Fabric Adapter

FA 250

2evision 0312


http://www.brocade.com/education/product-training/index.pagehttp://www.brocade.com/education/product-training/index.pagehttp://www.brocade.com/education/product-training/index.pagehttp://www.brocade.com/education/product-training/index.page


22/130

Additional Condor3 Features

• 768 Gbps of bandwidth per ASIC, 420 million frames switched per second

• Layer 2 latency of 800ns without FEC (F_Port to F_Port)

• Layer 2 latency of 1.2µs with FEC (E_Port to E_Port)• Up to 5,000 Km distance at 2 Gbps

• Integrated in-flight encryption and compression

• Auto-detection of fill word on links (IDLE vs ARB)

Forward Error Correction FEC)

Forward Error Correction (FEC) can only be enabled on E_Ports. FEC is a system oferror control for data transmissions, whereby the sender adds error-correcting code(ECC) to its transmission. This allows the receiver to detect and correct errors withoutthe need to ask the sender for additional data. The Condor3 FEC implementation can

enable corrections of up to 11 error bits in every 2,112 bit transmission. FEC can beenabled or disabled using the portcfgfec command.

Footnote 1: FL_Ports are not supported on the Condor3 ASIC.

Footnote 2: 2 Gbps speeds requires the use of an 8 Gbps Brocade branded SFP. 1 Gbpsspeeds are not supported.

Footnote 3: The number of user accessible buffer credits vary depending on the type ofplatform the ASIC is installed in. Use the portbuffershow command to see availablecredits.

FA 250

2evision 0312



23/130

The Brocade switches, directors, backbones and blades are discussed later.

Footnote 1: On the 16-port and 32-port blades, two of these port groups (2 x 8 = 16 total

per ASIC) are used for external ports and the other three port groups are used forinternal ports. On the 48-port blade, three of these port groups (3 x 8 = 24 total per

ASIC) are used for external ports and the other two port groups are used for internal

ports. This will be discussed later in the course. The FC8-64 uses four Condor2 ASICs

with two port groups per ASIC dedicated to user ports.

Footnote 2: The number of user accessible buffer credits vary depending on the type of

platform the ASIC is installed in and how it is being used.

Condor2 ASICs are used in the Brocade 5100, DCX, DCX-4S, 8000, Brocade EncryptionSwitch, FC8-16, FC8-32, FC8-48, and FC8-64.

Condor3 ASICs are used in the Brocade 6510, CR16-8, CR16-4, FC16-32, FC16-48.

GoldenEye2 ASICs are used in the Brocade 300, 5300, and 7800.

FA 250

2evision 0312



24/130

The two rack kit options for the Brocade 6510 use rails that are slimmer than standard rails toaccommodate the slightly wider chassis. Be sure to use one of these kits. Do not use standardrails to install the Brocade 6510 in a rack, they will not fit with the switch.

Footnote 1: Power supplies are available in front-to-back or back-to-front airflow options.

Footnote 2: The Brocade 6510 comes with 24 licensed ports. Additional ports are available in12 port increments (36 and 48 ports).

Footnote 3: Supports 2/4/8/10/16 Gbps speeds with the following optics:

• 16 Gbps optics: 4/8/16 Gbps

• 8 Gbps optics: 2/4/8 Gbps

• 10 Gbps FC optics: 10 Gbps

– Support for 10 Gbps FC is by using a 10 Gbps FC optic. 10 Gbps Ethernet optics do notwork.

Footnote 4: The Condor3 has a total of 8192 buffer credits, 7712 are available to the user inthe Brocade 6510.

Footnote 5: Brocade 6510 Integrated Routing support requires an Integrated Routing license.Once the license is installed, you can configure EX_Ports on a per-port basis as needed. Formore information on Integrated Routing refer to CFP 300 Brocade Certified Fabric Professional(BCFP) 16 Gbps training course.

Supports Virtual Fabrics with up to 4 Logical Switches and data-in-flight encryption andcompression.

For more detailed information about installation techniques and compatible components,please visit the Brocade website and download this products Hardware Reference Manual

FA 250

2evision 0312



25/130

To learn more about Brocade Fabric Adapters visit our Adapter’s page on the Brocade

website http://www.brocade.com/products/all/adapters/index.page.

Brocade AnyIO Technology

Brocade AnyIO technology is a unique capability that allows a single Brocade 1860

Fabric Adapter to support either native 16 Gbps Fibre Channel or 10 GbE on a port-by-

port, user selectable basis. Brocade AnyIO technology enables a dual-port adapter to

run 16 Gbps Fibre Channel on one port, and 10 GbE on the other port. In addition, the

Brocade 1860 can run TCP/IP, Fibre Channel over Ethernet (FCoE), and iSCSI

simultaneously on the same 10 GbE port.

A Brocade 1860 adapter port can be configured in any of the following modes:

• HBA mode Appears as a 16 Gbps Fibre Channel HBA to the operating system

(OS).

• NIC mode Appears as a 10 GbE NIC to the OS. It supports 10 GbE with DCB,

iSCSI, and TCP/IP simultaneously.

• CNA mode Appears as two independent devices—a 16 Gbps Fibre Channel HBA

and a 10 GbE NIC to the OS. It supports 10 GbE with DCB, FCoE, iSCSI, and TCP/IP

simultaneously.

Footnote1: HCM can also be used to manage, test and troubleshoot Brocade 4/8G

HBA’s and CNA’s.

FA 250

2evision 0312


http://www.brocade.com/products/all/adapters/index.pagehttp://www.brocade.com/products/all/adapters/index.page


26/130

Fabric Adapter Key Features

• Brocade AnyIO technology: 16/8/4/2 Gbps Fibre Channel and 10 GbE DCB for

TCP/IP, FCoE, and iSCSI

• Single- and dual-port models

• Line-rate 16 Gbps Fibre Channel, 1600 MB/sec throughput per port (3200

MB/sec full duplex)

• Line-rate 10 GbE performance with stateless networking offloads for the highestlevels of performance and CPU efficiency

• Jumbo frame support, up to 9600 bytes

• Over 500,000 IOPS per port for storage (Fibre Channel/FCoE/iSCSI)

• Brocade Server Application Optimization (SAO): Application-aware Quality of

Service QoS) and N_Port Trunking

• Brocade Virtual Machine Optimized Ports (VMOPs): Offload the hypervisor of

network packet classification and sorting tasks

• Brocade virtual Fabric Link (vFLink) I/O Virtualization (IOV): Up to eight virtual

adapters with flexible bandwidth allocations

• Single-Root I/O Virtualization: Extends Brocade vFLink with up to 255 Virtual

Functions (VFs)

• Fully integrated virtual switching with support for Edge Virtual Bridging (EVB)

802.1Qbg, including Virtual Ethernet Bridging (VEB) and Virtual Ethernet Port

Aggregator (VEPA)

• Boot-from-Storage Area Network (SAN), including Fabric-based Boot LUN

Discovery for unmatched simplicity

• Dynamic Fabric Provisioning (DFP) virtualizes host WWNs to simplify pre-

provisioning and eliminate time-consuming fabric reconfigurations when replacingadapters

• Pre-boot eXecution Environment (PXE) for booting over a network connection

• PCI-Express (PCIe) Gen2 (2.0), x8 lanes, with INTx and MSI-X Support

• Low-profile design for industry-standard 1U rack servers (high-profile bracket also

Included)

• Multiple media options, including Short Wave-Length (SWL) and Long Wave-

Length (LWL) optics for Fibre Channel; and Short-Reach (SR), Long-Reach (LR),

and active Twinax copper for 10 GbE

The Brocade 1860 is available in the following models:

• Brocade 1860-1F: Single port, one 16 Gbps Fibre Channel SWL SFP+ included

• Brocade 1860-2F: Dual port, two 16 Gbps Fibre Channel SWL SFP+ included

• Brocade 1860-1P: Single port, one 10 GbE SR SFP+ included

• Brocade 1860-2P: Dual port, two 10 GbE SR SFP+ included

• Brocade 1860-1C: Single port, no media included

• Brocade 1860-2C: Dual port, no media included

FA 250

2evision 0312



27/130

The DCX 8510-8 is intended for large enterprise fabric core.

Switchtype – 120.X (8 port blade slots)

Footnote 1: You can achieve 512 user ports by using FC8-64 blades; these will onlyoperate at 8 Gbps. Using FC16-48 blades you can operate at 16 Gbps with 384 ports.

Footnote 2: Blades are mutually exclusive for the FOS v7.0.0 release. You can install one

or the other, but not both at the same time.

For more detailed information about installation techniques and compatible

components, please visit the Brocade website and download this products Hardware

Reference Manual

FA 250

2evision 0312



28/130

Field Replaceable Units (FRUs) accessible from the non-port side of the chassis are:

• Power supplies

– One power supply is required to run the base system with two required for N+1

redundancy

– Power requirements may vary based on the number and types of blades

installed, see the Release Notes for the version of Fabric OS you are running

– The Brocade Power and Bandwidth Calculator can be used to determine overall

power usage for a given configuration:

http://www.brocade.com/sites/dotcom/data-center-best-practices/

competitive-information/power.page

• Blowers

– Two functional blowers are required to cool the DCX

– If a blower failure occurs the remaining blowers will operate at a higher RPM to

ensure chassis cooling requirements are met

• WWN cards

– Stores FRU serial number, run time hours, OEM specific information, and

event/error logs, licenses, IP addresses, and world wide names

FA 250

2evision 0312



29/130

The chart above shows a brief comparison between the different director chassis. Core

ASICs are mentioned here for completeness and will be discussed in much greater

detail later in this course.

The bandwidth numbers shown are uni-directional. Brocade marketing numbers are

usually shown as bi-directional, which would be twice what is shown in the presentation.

The DCX 8510 family provides increased total chassis bandwidth:

• 8.2 Tbps (384x 16 Gbps + 2 Tbps ICL)


Note: These numbers are calculated using local switching. The total chassis bandwidth

if using the backplane would be:

(8 x 512 Gbps slot bandwidth = 4.096 Tbps) + 2 Tbps ICL = 6.096 Tbps for DCX 8510-8


Footnote 1: The maximum number of ports in the DCX 8510-8 is based on using either

FC16-48 blades (384 ports max) or FC8-64 port blades (512 ports max). There is no

FC16-64 blade available at this time.

FA 250

2evision 0312



30/130

FA 250

3evision 0312



31/130

Footnote 1: The weight of a configured DCX-4S or DCX 8510-4 chassis varies depending

on which blade options are installed. The weight referenced above reflects a fully

populated chassis. Consult the hardware reference manual for the specific chassis you

are configuring for individual blade chassis and blade weights.

Weights for the DCX-4S is based on 256-port configuration with four FC8-64 port blades.

Weight for the DCX 8510-4 is based on 192-port configuration with four FC16-48 port

blades including two CP blades, 2 core switch blades. two blowers, two power supplies,

and two cable management finger assemblies.

Footnote 2: Heat output varies depending on installed options. The heat output

numbers shown above are based on a full configuration (all power supplies, all blowers,

and all CP, CR, and 48-port blades). Consult the hardware reference manual for the

specific chassis you are configuring for more detailed information.Note: The following links you to the Brocade Power Calculator web page:

http://www.brocade.com/data-center-best-practices/competitive-

information/power.page

For more detailed information about installation techniques and compatible

components, please visit the Brocade website and download these products Hardware

Reference Manual.

FA 250

3evision 0312


http://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.pagehttp://www.brocade.com/data-center-best-practices/competitive-information/power.page


32/130

The DCX 8510-4 is intended for midsize enterprise fabric core or large enterprise edge.

Footnote 1: Up to 256 user ports at 8 Gbps.

Footnote 2: Blades are mutually exclusive for the Fabric OS v7.0.0 release. You caninstall one or the other, but not both at the same time.

Switchtype – 121.X (4 port blade slots)

FA 250

3evision 0312



33/130

If using 220 VAC, power supplies generate 2000 watts each. If using 110 VAC, power

supplies generate 1000 watts each.

There are a number of FRUs that are compatible between the DCX and DCX-4S chassis:

• Control Processor blade

• Power supply

• Blower

• Port blades

• SFPs

The following components are NOT compatible between the two chassis:

• Core Router blades– The Core Router blades are designed to support four slots in the DCX-4S versus

eight slots in the DCX

• WWN cards

– The WWN cards in the DCX-4S have a different form factor than the DCX WWN

cards

FA 250

3evision 0312



34/130

The chart above shows a brief comparison between the different director chassis. Core

ASICs are mentioned here for completeness and will be discussed in much greater

detail later in this course.

The bandwidth numbers shown are uni-directional. Brocade marketing numbers are

usually shown as bi-directional, which would be twice what is shown in the presentation.

The DCX 8510 family provides increased total chassis bandwidth:



Note: These numbers are calculated using local switching. The total chassis bandwidth

if using the backplane would be:



Footnote 1: The maximum number of ports in the DCX 8510-8 is based on using either FC16-48

blades (192 ports max) or FC8-64 port blades (256 ports max). There is no FC16-64 blade

available at this time.

FA 250

3evision 0312



35/130

The CR16-8 is the new core routing blade used in the DCX 8510-8. It handles ICL

connections between chassis as well as routing between slots within the same chassis.

Slots for up to 16 Brocade branded QSFPs offer up to 64 16 Gbps FC ports in total. New

ICLs use an optical cable, eliminating previous distance limitations and allowing up tosix chassis to be connected together.

FA 250

3evision 0312



36/130

The CR16-4 is the new core routing blade used in the DCX 8510-8. It handles ICL

connections between chassis as well as routing between slots within the same chassis.

Slots for up to 8 Brocade branded QSFPs offer up to 32 16 Gbps FC ports in total. New

ICLs use an optical cable, eliminating previous distance limitations and allowing up tosix chassis to be connected together.

FA 250

3evision 0312



37/130

Control processing and core routing blades are at the heart of the Brocade director

switches. The core processing blades contain the CPUs for the chassis and run the

Brocade Fabric OS software. The core routing blades handle the routing of Fibre

Channel frames between different blades installed in the chassis.

The 4-slot and 8-slot chassis utilize separate routing and processing blades for

increased redundancy.

FA 250

3evision 0312



38/130

16 Gbps FC ports support E, F, EX, Diagnostic, and Mirror ports. No FL port support.

Front end ports can operate at 2, 4, 8, 10, and 16 Gbps

•

10 Gbps ports can be configured as E_Ports only• Support for 10 Gbps ports is limited to first 8 ports on each blade with FOS v7.0.0

Diagnostic port for Condor3-based ports:

• Support electrical and optical loopback (16 Gbps Brocade branded optics only),

cable length detection and spinfab-like cable saturation tests (enough to saturate

a 16 Gbps link).

10 Gbps FC using 10GE license

• Existing slot-based 10GbE FCIP license (introduced in FOS v6.3 for FX8-24 blades)

is extended to enable Condor3 FC ports running at 10 Gbps rate. These ports needto work with DWDM equipment with plain transponder cards).

• When applied to a 16 Gbps blade, license allows all ports to be enabled as 10

Gbps FC.

FA 250

3evision 0312



39/130

Footnote 1: 16 Gbps tri-mode SFPs operate at 4, 8, or 16 Gbps, 8 Gbps tri-mode SFPsoperate at 2,4 or 8 Gbps. 4 Gbps tri-mod SFPs operate at 1, 2 or 4 Gbps. Mini SFPs(mSFPs) operate at 2, 4, or 8 Gbps and are used with FC8-64.

Footnote 2: The backend ports connecting to the core run at 4 Gbps while the front endports run at 8 Gbps. If the initiator and target are on the same ASIC, the frame wouldnot go through the core and local switching would be used. If the frame has to passthrough the core ASICs than the oversubscription numbers above become important.The front end ports operate at 8 Gbps and the back end ports, the ones that connect tothe core ASICs, are limited to 4 Gbps .

Footnote 3: Each Condor2 ASIC has 1420 user buffer-to-buffer (BB) credits, and eachfront-end port is allocated 8 credits. The 16 and 32 port blades each have 16 front-endports per ASIC, (8 credits x 16 ports = 128 credits) which leaves 1292 credits availableper Condor2 ASIC.

Footnote 4: Each Condor2 ASIC has 1420 user buffer-to-buffer (BB) credits, and eachfront-end port is allocated 8 credits. The 48-port blade has 24 front-end ports per ASIC,(8 credits x 24 ports = 192 credits) which leaves 1228 credits available per Condor2ASIC. Use the portbuffershow command to see available credits.

Footnote 5: Long distance configurations are not supported on the FC8-64. The FC8-64

blade uses mini-SFPs (mSFP) for increased port density. The mSFP has a smaller formfactor and takes up less space than a standard sized SFP; they are not interchangeable.The FC8-64, also, does not support FICON or Extended Fabrics.

FA 250

3evision 0312



40/130

Port blades are named based on their speed and number of user ports available. For

example the FC8-32 blade operates at 8 Gbps and contains 32 Fibre Channel ports, the

FC4-16 blade operates at 4 Gbps and contains 16 Fibre Channel ports.

Although the 8 Gbps blades are supported the 48000 utilizes a 4 Gbps core. Devices

will connect to the ports at 8 Gbps but any Fibre Channel frames that pass through the

CP blades will be limited to the 4 Gbps available.

The available Brocade port blades will be discussed in more detail later in this course.

FA 250

4evision 0312



41/130

GREEN AMBER CONDITION

OFF OFF (Cable is not present) OR (Local end is not ready) OR

(Far end is not ready)

OFF ON N/A

ON OFF (Cable is present) AND (Local end is ready) AND

(Far end is ready)

ON ON Blinking) (Cable is present) AND (Local end is ready) AND

(Far end is ready) AND (Attention is required)

FA 250

4evision 0312



42/130

FA 250

4evision 0312



43/130

DCX Backbones have special Inter-Chassis Links ports for connecting two backbone

chassis using special ICL cables. Latest generation ICL cables are now optical. This

enables greater distance of 50 meters for ICL cables.

Improved Inter-Chassis Link (ICL) connectivity up to:

• 2 Tbps (32x 64 Gbps) for DCX 8510-8

• 1 Tbps (16x 64 Gbps) for DCX 8510-4

The DCX 8510-8/DCX 8510-4 offers an ICL POD licensing option, with a single ICL POD

license enabling sixteen 64 Gbps QSFPs (8 per core). A second ICL POD license can be

applied to the DCX 8510-8 to enable the remaining sixteen 64 Gbps QSFPs. (A single ICL

POD license enables the full ICL capability on a DCX 8510-4.) The existing ICL licensing

infrastructure for DCX/DCX-4S will be used to support the ICL POD licensing.

DCX/DCX-4S 8 Gbps ICL licensing does not change with FOS v7.0.0 – retain support for

existing 8-link and 16-link licenses.

FA 250

4evision 0312



44/130

Preparing for the 16 Gbps B

4evision 0312

FA 250


45/130

Each QSFP optical cable bundles four 16 Gbps channels; each channel is an independent

FC port. On the CR16-8 each QSFP channel terminates on a different ASIC. Since the CR16-

4 only has two ASICs two channels from each QSFP terminate on a single ASIC in separate

trunk groups.

Footnote :

Eight link trunks can only be formed between two DCX 8510-8s or two DCX

8510-4s. Trunks between a DCX 8510-8 and an 8510-4 are limited to four links (ICL ports 0-

3, 4-7, etc.). This is due to the different number of ASICs on the CR16-8 and CR16-4 blades

and how the QSFP ports are aligned on those ASICs.


4evision 0312

FA 250


46/130

In the above trunkshow output we can see the trunks that form the ICL connections to a

neighboring backbone. Since each ICL QSFP port has four ports in separate trunk groups

there is a difference of four between each of the ports in an ICL trunk group. This is a result

of one port per QSFP being a member of the same trunk group.


4evision 0312

FA 250


47/130

FA 250

4evision 0312



48/130

ICL licensing for the DCX 8510 is a Ports-on-Demand (POD) implementation with a new

part number

The display strings in licenseshow CLI and GUI output has changed for DCX families

running Fabric OS v 7.0.0

Pre-Fabic OS v7.0.0:

DCX:admin> licenseshow

X3ffNTZM9CNmM4SKFMYTGS4WmCRCgAZZBJDTB:

Inter Chassis Link (16 link) license

Fabric OS v7.0.0:

DCX:admin> licenseshow

X3ffNTZM9CNmM4SKFMYTGS4WmCRCgAZZBJDTB:

Inter Chassis Link (2nd POD) license

FA 250

4evision 0312



49/130

The following table lists the switch type assigned to each switch or blade and is

displayed in the switchshow command.

Below is a slotshow -m output from a DCX 8510-4 chassis with FC16-32, FC16-48

and CR16-4 blades.Slot BladeType ID Model Name Status

--------------------------------------------------

1 SW BLADE 96 FC16-48 ENABLED


3 CORE BLADE 99 CR16-4 ENABLED

4 CP BLADE 50 CP8 ENABLED





Below is a slotshow -m output on a DCX 8510-8 chassis with FC16-32, FC16-48

and CR16-8 blades.Slot Blade Type ID Model Name Status

--------------------------------------------------

1 UNKNOWN VACANT

2 UNKNOWN VACANT


4 UNKNOWN VACANT





9 UNKNOWN VACANT

10 UNKNOWN VACANT


12 UNKNOWN VACANT

Brocade Switch/Blade Switch Type ASIC

FC16-48 (48 Port 16 Gbps blade) 96 Condor3

FC16-32 (32 port 16 Gbps blade) 97 Condor3

FC8-64 (64 port 8 Gbps blade) 77 Condor2

CR16-8 (16 Gbps core for DCX 8510-8) 98 Condor3

CR16-4 (16 Gbps core for DCX 8510-4) 99 Condor3

CP-8 50 Condor2

Brocade 6510 (16 Gbps FC switch) 109.1 Condor3

DCX 8510-4 121.3 Condor3

DCX 8510-8 120.1 Condor3

FA 250

4evision 0312



50/130

The port blades in DCX 8510-8 are assigned port numbers from 0-383 and 768-895.

The ICL ports are assigned the following numbers:

On Slot 5:• 384 through 415

• 1152 through 1183

On Slot 8:

• 416 through 447

• 1184 through 1215

FA 250

5evision 0312



51/130

FA 250

5evision 0312



52/130

FA 250

5evision 0312



53/130

When a new switch has arrived for installation into a fabric, it is suggested to use a

serial cable to configure the switch with an IP address. After the IP address is

configured, the serial connection to the switch may be dropped and an SSH, telnet, or

Web Tools session may be used for further switch configuration because of itsconvenience and speed.

To configure the connection in a B-Series environment

• Bits per second: 9600

• Data bits: 8

• Parity: None

• Stop bits: 1

• Flow control: None

Installation steps

1. Insert the serial cable provided to an RS-232 serial port on the workstation. FOS

switches use a straight-through cable.

2. Verify the switch has power and is past the POST stage.

3. Enter the ipaddrset command to set the IP address, subnet mask, and default

gateway.

FA 250

5evision 0312



54/130

IP addresses are assigned to the management interface of a switch, director, or

backbone and used to remotely manage the switch through telnet or SSH. A switch (e.g.

B300, B5100) only has a single management interface and only uses a single IP

address.

Backbones require three IP addresses: one for chassis/switch management and one for

each CP blade. The IP addresses used for the CP blades always connect to the blades

they are assigned, the chassis management IP always connect to the active CP.

Additionally, the DCX backbones have two Ethernet management interfaces on each CP

(eth0 and eth3). These interfaces use port bonding to create a logical interface (bond0)

to which the IP address of the CP is assigned. Ethernet bonding provides link (physical)

layer redundancy using an active/standby model. By default all traffic is transmitted

over the active interface, eth0. If eth0 experiences a link failure (e.g. cable unplugged)then eth3 becomes active.

For more information on port bonding refer to the Fabric OS Command Reference

Manual and the ifmodeshow command.

FA 250

5evision 0312



55/130

You can use the IPFilter policies to block any of the IP based Mangement interfaces

Footnote 1: The SCP(Secure Copy) protocol is a network protocol, based on the BSD RCP

protocol, which supports file transfers between hosts on a network. SCP uses SecureShell (SSH) for data transfer and utilizes the same mechanisms for authentication,

thereby ensuring the authenticity and confidentiality of the data in transit. Usually used

for uploading configuration files from the switch. USBs can also be used to accomplish

more secure file transfers.

Footnote 2: Secure Socket Layer (SSL) is part of base Fabric OS. SSL works by using a

key to encrypt data transferred over an SSL connection. By convention, URLs that

require an SSL connection start with “https:” instead of “http:” All Brocade supported

Internet browsers support SSL.

Configuration of the SSL protocol involves obtaining, installing, and configuring PKIcertificates:

• Public Key Infrastructure (PKI) is a system of public key encryption using digital certificates

from a Certificate Authority (CA) and other registration authority to verify and authenticate

the validity of each party involved in an electronic transaction.

• The CA works as part of a Public Key Infrastructure (PKI) and therefore checks with a

registration authority (RA) to verify digital certificate requestor information. Once RA

verifies information CA can issue a certificate. The information that the RA verifies

depends on the CA, but includes items such as owners public key; certificate expiration

date; owners name and other public key owner information.

FA 250

5evision 0312



56/130

Setting the domain ID

1. Connect to the switch and log in on an account assigned to the admin role.

2. Enter the switchDisable command to disable the switch.3. Enter the configure command.

4. Enter y after the Fabric Parameters prompt.

Fabric parameters (yes, y, no, n): [no] y

5. Enter a unique domain ID at the Domain prompt. Use a domain ID value from 1

through 239 for normal operating mode (FCSW-compatible).

Domain: (1..239) [1] 3

6. Respond to the remaining prompts, or press Ctrl-D to accept the other settings andexit.

7. Enter the switchEnable command to re-enable the switch.

FA 250

5evision 0312



57/130

Footnote1: Date parameters defined:

• mm is the month, valid values are 01-12 - mm is minutes, valid values are 00-59• dd is the date, valid values are 01-31 - yy is the year, valid values are 00-99• hh is the hour, valid values are 00-23

SW1:admin> dateTue May 16 15:00:57 UTC 2006

SW1:admin> tsclockserverLOCL

SW1:admin> tsclockserver 128.118.25.3Updating Clock Server configuration...done.

SW1:admin> tsclockserver128.118.25.3

SW1:admin> date "0516073406"External Time Synchronization in place. Cannot execute thiscommand.

SW1:admin> tsclockserver LOCLUpdating Clock Server configuration...done.

SW1:admin> tsclockserverLOCL

SW1:admin> date "0516073406"Tue May 16 07:34:00 UTC 2006

FA 250

5evision 0312



58/130

Default timeout on switches is 10 minutes. When changing the timeout value you can

use the login command to restart the login session and use the new timeout value.

FA 250

5evision 0312



59/130

sw1:admin> licenseshow

SRScQbceeyTSTdRI:

Fabric Watch license

cQQ9edcRzedRVAfw:

Extended Fabric license

Second Ports on Demand license - additional 8 port upgradelicense

FFQ4FNRPQrNNMrN79BJWfPSLgETXHfmYB7fZM:

Fabric Watch license

Performance Monitor license

Trunking licenseFICON_CUP license

First Ports on Demand license - additional 8 port upgradelicense

Integrated Routing license

Adaptive Networking license

8 Gig FC license

Unknown30 license

One feature per

license key

Multiple features

per license key

Unknown license results when a license

from a previous OS is no longer valid

Example: Web Tools is no longer a

licensed feature

FA 250

5evision 0312



60/130

Synopsis:

fabricprincipal --help|-h

fabricprincipal [--show|-q]

fabricprincipal --enable [-priority|-p priority] [-force|-f]

fabricprincipal --disable

fabricprincipal [-f] mode

Description:

Use this command to set principal switch selection mode for a switch and to set

priorities for principal switch selection. The implementation of the fabricprincipal

command is based solely on mechanisms specified in the Fibre Channel standards.

These mechanisms provide a preference for a switch requesting to be the principal

switch in a fabric, but they do not provide an absolute guarantee that a switchrequesting to be the principal switch is granted this status.

When dealing with large fabrics, the selection of the principal switch is less

deterministic. In these cases, to help ensure that the desired switch is selected as the

principal switch, a small cluster of switches should be interconnected first, followed by

additional switches to enlarge the fabric.

FA 250

6evision 0312



61/130

SW1:admin> motd --set "Access by unauthorized personnel is

prohibited.“

SW1:admin> motd --show

Access by unauthorized personnel is prohibited.

SW1:admin> bannerset "You have successfully logged into the

switch."

SW1:FID128:admin> bannershow

You have successfully logged into the switch.

login as: admin

Access by unauthorized personnel is prohibited.

[email protected]'s password:

You have successfully logged into the switch.

------------------------------------------------------------

SW1:admin> exit

FA 250

6evision 0312



62/130

Footnote 1: All printable punctuation characters except colon ":" are allowed.

Footnote 2: The minimum password length may be set from 8 to 40 characters in length.

The password length is the total number of lowercase, uppercase, digits, andpunctuation characters. The total number of these characters may not exceed 40. Keep

this in mind as you specify the minimum number of each type of character required.

Footnote 3: The password history policy is not enforced when an administrator sets a

password for another user, but the password set by the administrator is recorded in the

user's password history.

SW1:admin> passwdcfg --set -lowercase 3 -uppercase 1 -digits

2 -punctuation 2 -minlength 10 -history 3

SW1:admin> passwd

Changing password for adminEnter old password:

Enter new password:

Password must be between 10 and 40 characters long.Enter new password:

Insufficient number of upper case letters

FA 250

6evision 0312



63/130

FA 250

6evision 0312



64/130

Footnote 1: Fabric OS v5.3 and later loads default IP Filter policies for both IPv4 and

IPv6.

Footnote 2: Policy rules will be discussed in more detail later.Footnote 3: There are also two default policies, one for IPv4 and another for IPv6 and

some implicit policies that enable communication like syslog, from the switch out.

FA 250

6evision 0312



65/130

Use the portCfgOctetSpeedCombo command to set the combination speed for the first

port

octet to a setting that supports 10 Gbps operations. Valid settings for 10 Gbpsoperations

include:

• 2—autonegotiated or fixed port speeds of 10 Gbps, 8 Gbps,4 Gbps, and 2 Gbps

• 3—autonegotiated or fixed port speeds of 16 Gbps and 10 Gbps

Use the portCfgSpeed command to set the port speed on each port you want to operate

at 10

Gbps.

FA 250

6evision 0312



66/130

A new element called Error Ports has been introduced in Fabric OS v7.0.0 to monitor

ports which are segmented or disabled for reasons such as security violations or fabric

watch port fencing.

The unit for the default and user defined thresholds Marginal, Faulty, Error ports and

Missing SFPs has been changed to a percentage of the current number of physical ports

present in the switch at a given time rather than the absolute number of physical ports.

Port-based components in switchstatuspolicy can move either to a Down or

Marginal state based on the thresholds calculated from the percentage of number of

physical FC ports present in the logical switch.

The total number of physical ports used for threshold calculation excludes FCoE ports,

VE ports, and internal ports

FA 250

6evision 0312



67/130

FA 250

6evision 0312



68/130

Footnote 1: If this switch was purchased from an OEM vendor there may be upgrade

requirements different from those listed here. Contact your vendor for additional

information. Always read the new firmware release notes to review any open or resolved

FOS issues. To review new feature sets, review the FOS Administrator’s Guide.

Footnote 2: In other words, upgrading a switch from Fabric OS v6.3.0 to v7.0.0 is a two-

step process—first upgrade to v6.4.0, and then upgrade to v7.0.0. If you are running a

pre-Fabric OS v6.2.0 version you must upgrade to v6.2.0, then to v6.3.0,then to v6.4.0,

and finally to v7.0.0.

Footnote 3: Ensure that all serial consoles (both CPs for directors) and any open network

connection sessions, such as Telnet, are logged and included with any trouble reports.

FA 250

6evision 0312



69/130

Footnote1: Ensure there is an active FTP service running on a network server where the

firmware package has been unzipped. Alternatively you can use SCP or the USB option

under this command. You can also choose to upgrade the switch with GUI based

applications by using Brocade Network Advisor or Web Tools.

FA 250

6evision 0312



70/130

Attempting to mount an unsupported USB device will result in the following failure:

dcx1:admin> usbstorage -e

Fail to enable USB storage device. Error:Device is not found

FA 250

7evision 0312



71/130

FA 250

7evision 0312



72/130

The FC-SW-2 standard for Storage Area Networks (SANs) uses an algorithm called Fabric

Shortest Path First (FSPF). FSPF is a link state path selection protocol and directs traffic

along the shortest path between the source and destination, based upon the link cost,

and makes it possible to detect link failures, determine shortest route for traffic, updatethe routing table, provide fixed routing paths within a fabric, and maintains correct

ordering of frames. FSPF keeps track of the state of the links on all switches in the

Fabric and associates a cost with each link. The protocol computes paths from a switch

to all the other switches in the fabric by adding the cost of all links traversed by the

path, and chooses the path that minimizes the costs. This collection of the link states

(including costs) of all the switches in the fabric constitutes the topology database (or

link state database). Once established, FSPF programs the hardware routing tables for

all active ports on the switch. FSPF is not involved in frame switching.

There are two types of primary routing protocols in intranet networks, Distance Vectorand Link State:

• Distance Vector is based on hop count. This is the number of switches you traverse

through to get from the source domain (switch) to the destination domain (switch).

• Link State is based on a metric value based on a cost. The cost could be based on

bandwidth.

FA 250

7evision 0312



73/130

FSPF makes minimal use of the ISLs bandwidth, leaving virtually all of it available for

traffic. In a stable fabric, a Brocade switch will transmit 64 bytes every 20 seconds in

each direction. FSPF frames have the highest priority in the fabric. This guarantees that

a control frame is not delayed by user data and that FSPF routing decisions occur very

quickly during convergence.

FSPF guarantees a routing loop free topology at all times. Why is this important? It is

essential for a fabric to include many physical loops, because without loops therewould be no multiple path between switches, and therefore no redundancy. Without

multiple paths, if a link goes down part of the fabric becomes isolated. FSPF ensures

that the topology is loop free and that the frame will never be forwarded over the same

ISL more than once.

Brocade recommends no more than 7 hops between two switches. This limit is not

required or enforced by FSPF. Its purpose is to ensure that a frame will never be

delivered to a destination after E_D_TOV has expired.

FSPF calculates paths based on the destination domain ID. The fabric protocol must

complete domain ID assignments before routing can begin. ISLs provide the physical

pathway when the Source ID (SID) address has a frame destined to a port on a remote

switch Destination ID (DID). When an ISL is attached or removed from a switch, the

FSPF updates the route tables to reflect the addition or deletion of the new routes.

As each host transmits a frame to the switch, the switch reads the SID and DID in the

frame header. If the domain ID of the destination address is the same as the switch

(intra-switch communications), the frame buffer is copied to the destination port and a

credit R_RDY is sent to the host. The switch only needs to read word zero and word one

of the Fibre Channel frame to perform what is known as cut-through routing. A frame

may begin to emerge from the output port before it has been entirely received by the

input port. The entire frame does not need to be buffered in the switch.

If the destination domain ID is different than the source domain ID, then the switch

consults the FSPF route table to identify which local E_Port provides the Fabric

Shortest Path First to the remote domain. If IOD is not set, frames are held for 650 ms

before being sent down a new route.

FA 250

7evision 0312



74/130

Fabric Shortest Path First (FSPF) calculates paths based on the destination domain ID.

The Fabric Protocol must complete domain ID assignments before routing can begin.

ISLs provide the physical pathway for routing frames from a Source ID (SID) to aDestination ID (DID) on a different domain (switch). When an ISL goes online or offline

FSPF will update the routing tables to reflect the change.

As each host transmits a frame to the switch, the switch will read the SID and DID in

the frame header. If the domain ID of the destination address is the same as the switch,

intra-switch communication, the frame buffer is copied to the destination port and an

R_RDY is sent to the host.

Since the SID and DID are in the first two words of the frame Brocade switches perform

cut-through routing. The first two words of an incoming frame are read, if the DID is

another port on the local domain the frame input is immediately transferred to the DIDport. The entire frame does not need to be buffered in the switch; a frame may begin to

emerge from the output port before it has been entirely received by the input port.

Between Domain 1 and Domain 3 in the figure above there are three paths: port 2 andport 5, each with a cost of 500, and port 6 with a cost of 1000. Only the lowest costroutes are in the routing table. In the figure above Domain 1 ports 2 and 5 would be in

the routing table, port 6 would not.

The routing table can be viewed using the urouteshow command. Static routes can beassigned using the urouteconfig command.

FA 250

7evision 0312



75/130

Footnote 1: Different OEMs may use different default settings. Please check with your

switch vendor for settings.

2 Gbps ASIC routing is handled by the Fabric Shortest Path First (FSPF) protocol and

uses only Port-based routing.

FA 250

7evision 0312



76/130

DLS is a standard feature in Fibre Channel to share multiple available routes to a

destination domain. If multiple routes exist in the routing table, FSPF will dynamically

load share according to the ratio of bandwidth available on the routes.

Exchange-based routing depends on DLS for dynamic routing path selection. When

using Exchange-based routing, DLS is enabled by default and cannot be disabled. In

other words, you cannot enable or disable DLS when Exchange-based routing is in

effect. When Port-based routing is in force, you can enable DLS to optimize routing. DLS

recomputates load sharing when a switch boots, an E_Port/EX_port goes offline or

online, or when an Nx_Port comes online or goes offline. DLS is unidirectional, meaning

that it must be set at both ends of the link to be effective in both directions.

In a stable fabric, frames are always delivered in order, even when the traffic between

switches is shared among multiple paths. However, when topology changes occur in thefabric (for example, if an E_Port goes down), traffic is rerouted around the failure, and

some frames could be delivered out of order. Most destination devices tolerate out-of-

order delivery, but some do not.

FA 250

7evision 0312



77/130

The choice of the routing path is based only on the incoming port and the destination

domain. To optimize port-based routing, Dynamic Load Sharing (DLS) round-robins the

input ports across the available output ports within a domain.

FA 250

7evision 0312



78/130

Exchange-based routing is also known as Dynamic Path Selection (DPS).

DLS cannot be disabled if exhanged-based routing is enabled.

The choice of routing path is based on the Source ID (SID), Destination ID (DID), andFibre Channel originator exchange ID (OXID), optimizing path utilization for the best

performance. Thus, every exchange can take a different path through the fabric.

Exchange-based routing requires the use of the Dynamic Load Sharing (DLS) feature;

when this policy is in effect, you cannot disable the DLS feature.

Exchange-based routing is also known as Dynamic Path Selection (DPS). DPS is where

exchanges or communication between end-devices in a fabric are assigned to egress

ports in ratios proportional to the potential bandwidth of the ISL or trunk group. When

there are multiple paths to a destination, the input traffic will be distributed across the

different paths in proportion to the bandwidth available on each of the paths. Thisimproves utilization of the available paths, thus reducing possible congestion on the

paths. Every time there is a change in the network (which changes the available paths),

the input traffic can be redistributed across the available paths. This is a very easy and

non-disruptive process when the exchange-based routing policy is engaged.

FA 250

7evision 0312



79/130

Data traffic Virtual Channels (VCs) are collapsed to optimize performance over long

distances using the portcfglongdistance command, as shown in the diagram

below.

Information about switch characteristics and capacity in terms of buffers per port group,

port speed, and distances supported is contained in the Fabric OS Administrator's Guide

and the appropriate Hardware Reference manual specific to the switch you are

configuring.

VC 2 only

VC2

VC3

VC4

VC5

VC2

VC3

VC4

VC5

FA 250

7evision 0312



80/130

The maximum Extended Fabric distance depends on the version of Brocade switch ASIC

installed in the switch.

Extended Fabric distance levels (L0, LE, LD, LS) :• Cannot be set or removed by configure or configdefault

• Can be cleared by portcfgdefault

• Saved in a switch configuration file (configupload ) as portcfg parameter

Use the portcfglongdistance command to support long distance links and to

allocate sufficient numbers of full size frame buffers on a particular port. Changes made

by this command are persistent across switch reboots and power cycles. This command

supports the following long-distance link modes:

Static Mode (L0) - L0 is the normal (default) mode for a port. It configures the port as aregular port. A total of 20 frame buffers are reserved for data traffic, regardless of the

port’s operating speed; therefore, the maximum supported link distance is up to 10 km

at 1 Gbps, up to 5 km at 2 Gbps, up to 2 km at 4 Gbps, and up to 1 km at 8 Gbps.

FA 250

8evision 0312



81/130

The routing database determines how frames are routed from input port to output port

when going to the next destination. Fabric Shortest Path First (FSPF) puts available

equal cost routes in the routing data base. One output port in the trunk group is put into

the routing data base. When a communication between two end devices in a fabric isassigned a route through a trunk, the ASIC of the assigned trunk group port will be the

same ASIC as all ports in the trunk group. This ASIC will multiplex frames across ISLs in

the trunk group and maintain in-order delivery. The ASIC will send a frame down each

link to determine the links latency. These individual link latency calculations will be used

to maintain in-order delivery.

If some ports in a trunk group have QoS enabled and some ports have QoS disabled, the

two different trunks are formed: one with QoS enabled and one with QoS disabled.

FA 250

8evision 0312



82/130

Footnote 1: Automatically creates ISL trunks using from 2 to 8 ISLs when the switches

are connected and all trunking requirements are met.

Footnote 2: Must use the first 8 ports of the switch or blade

Trunk Monitoring

To monitor E_Port (ISL) and F_Port trunks, you can set monitors only on the master port

of the trunk. If the master changes, the monitor automatically moves to the new master

port. If a monitor is installed on a port that later becomes a slave port when a trunk

comes up, the monitor automatically moves to the master port of the trunk.

For masterless trunking, if the master port goes offline, the new master acquires all the

configurations and bottleneck history of the old master and continues with bottleneck

detection on the trunk.

FA 250

8evision 0312



83/130

Dynamic Path Selection (DPS) is exchange-based routing where exchanges or

communications between end devices in a fabric are assigned to egress ports in ratios

proportional to the potential bandwidth of the ISL or trunk group.

When there are multiple routes to a destination, the input traffic will be distributed

across the different routes in proportion to the bandwidth available on each of the

routes. This improves utilization of the available routes, thus reducing possible

congestion on the routes. Every time there is a change in the network (which changes

the available routes), the input traffic can be redistributed across the available routes.

This is a very easy and non-disruptive process when the Exchange-based Routing Policy

is engaged.

Exchanges in the example depicted on this slide are allocated based on the primary

criteria: link cost and secondary criteria: potential bandwidth. The potential bandwidthallocation depicted in this example yields flow allocations of 3:1.

FA 250

8evision 0312



84/130

When the Trunk Master is disabled another pre-determined port takes over the role

without fabric disruption.

FA 250

8evision 0312



85/130

Footnote 1:

Up to eight ports can be grouped together in one trunk group to create high performance

128 Gbps ISL trunks between switches.Trunk links can be 2 Gbps, 4 Gbps, 8 Gbps, 10 Gbps, or 16 Gbps depending on the

Brocade platform, or individual port speed settings.

The maximum number of ports per trunk and trunks per switch depends on the Brocade

platform.

There must be a direct connection between participating switches.

The port ISL R_RDY mode must be disabled (using the portCfgIslMode command).

Trunks operate best when the cable length of each trunked link is roughly equal to theothers in the trunk. For optimal performance, no more than 30 meters difference is

recommended. Trunks are compatible with both short wavelength (SWL) and long

wavelength (LWL) fiber optic cables and transceivers.

The compatibility matrix can be found at: http://www.brocade.com/products-

solutions/technology-architecture/compatibility/index.page

FA 250

8evision 0312



86/130

Light in a vacuum travels much faster, but in optical cable the rate is about 5 ns/meter.

5ns/meter multiplied by 30 meters is 150 ns. The difference in cable lengths between

the ISLs in a trunk determines the deskew value. This is needed for timing purposes so

that delivery of frames across the trunk can be ensured. The shortest ISL is selected asthe base and is assigned a deskew value of 150 nsec. The deskew values are expressed

(shown in all command displays) by dividing the time value by 10. Example: A deskew

value of 150 nanoseconds is shown as 15 (150/10).

The first ISL in the trunk to initialize is selected as the trunk master. The length of the

cable is not a consideration when selecting the master. The deskew values for the other

ISLs in the trunk will be calculated from the base ISL and will have a higher value. Each

switch connected by the ISL will have a deskew value since each has a separate

transmit line to the other. Due to the signal quality/optical media, cables that are

identified as the same length may have a different deskew value. For example, onecable may have a deskew value of 16 and a cable of the same length may calculate to

be 17. This is not a problem since deskew is a true measurement of its transmission

capabilities.

FA 250

8evision 0312



87/130

FA 250

8evision 0312



88/130

FA 250

8evision 0312



89/130

Footnote 1: A “regular” or “normal” zone enables you to partition your fabric into logical

groups of devices that can access each other. Zoning also supports special zones called

LSAN, QoS, and TI zones. Unless otherwise specified, all references to zones in this

module refer to normal zones.

FA 250

8evision 0312



90/130

Member:

• Alias is given a name, e.g. “Server_1”, “Disk_Array_2”.

• Physical Fabric port number or area number.

• Node World Wide Name - Obtained using nsshow or switchshow.• Port World Wide Name – Obtained using nsshow or portloginshow.

• 64 characters maximum: A-Z, a-z, 0-9 and the “_” are allowed.

Zone:

• Is given a name, e.g. “Red_Zone”.

• Contains two or more members and uses a “;” as a separator.

• The same member can be in multiple zones.

• Zone definition is persistent; it remains until deleted or changed by anadministrator.

Configuration:

• Is given a name, e.g. “Production_Cfg ”.

• Is one or more zones.

• Configuration may be disabled or one configuration may be in effect from anyswitch in the fabric.

• An administrator selects which configuration is currently enabled.

• A configuration is saved when enabled and then distributed to the remainingswitches in the fabric where it is enabled and saved.

FA 250

9evision 0312



91/130

Footnote1: Zone and configuration names are also limited to 64 characters maximum.

Zone aliases simplify repetitive entry of zone objects, such as PWWNs. For example, thename “Eng” could be used as an alias for PWWN: 10:00:00:80:33:3f:aa:11. An alias is a

name assigned to a device or group of devices. By creating an alias, you can assign a

familiar name to a device, or you can group multiple devices into a single name. This can

simplify cumbersome entries and it allows an intuitive naming structure such as using

NT_Storage to define all NT storage ports in the fabric.

Alias objects only appear in the defined configuration since they are used to assign a

meaningful name to a device or group of devices

Zone objects identified by “Domain, Index” are specified as a pair of decimal numberswhere “Domain” is the Domain ID of the switch and “Index” is the index number for the

port on that switch.

Zone objects identified by World Wide Name (WWN) are specified as a 16 digit

hexadecimal number separated by colons, for example 10:00:00:90:69:00:00:8a. When

a node name is used to specify a zone object, all ports on that device are in the zone.

When a port name is used to specify a zone object, only that single port is in the zone.

FA 250

9evision 0312



92/130

A zone configuration is a group of zones that is enforced whenever that zone

configuration is enabled. A zone can be included in more than one zone configuration.

To define a zone configuration, specify the list of zones to be included and assign a zoneconfiguration name. Zoning may be disabled at any time. When a zone configuration is

in effect, all zones that are members of that configuration are in effect.

• Defined configuration: The complete set of all zone objects that have been defined

in the fabric.

• Effective configuration: A single zone configuration that is currently in effect. The

effective configuration is built when an administrator enables a specified zone

configuration. This configuration is “compiled” by checking for undefined zone

names, or zone alias names, or other issues.• Saved configuration: A copy of the defined configuration plus the name of the

effective configuration which is saved in flash memory by the cfgsave command.

There may be differences between the saved configuration and the defined

configuration if the system administrator has modified any of the zone definitions

and has not saved them.

FA 250

9evision 0312



93/130

The default zone feature can enable or disable device access within a fabric. Default

zones are based on the FC-GS standard.

The defzone -–

allaccess is the default because it matches how zoning workedprior to Fabric OS v5.1.0.

The defzone command configures a default zone configuration and displays the

current configuration. The command has no optional parameters, and takes one of

three required arguments:

--allaccess: Enables all device-to-device access within the fabric. This is the

default behavior in Fabric OS v5.1, and matches the default behavior in a non-

zoned fabric.

--noaccess: Create a default zone that disables all device-to-device access within

the fabric.--show: Display the current default zone.

Names beginning with d__efault__ are reserved for default zoning use (note: two

underscore characters are used in each instance.)

Note: The setting of the defzone command is stored in the zoning transaction buffer.

Normally, a cfgsave is used to commit the zoning transaction to the entire fabric. Acfgenable or cfgdisable will do the commit since each command does an implied

cfgsave. Because the setting is stored in the zoning transaction buffer, a

cfgtransabort could be used to abort the

defzone command.

FA 250

9evision 0312



94/130

A normal zone must be in effect granting access between devices before a TI zone will

be effective. TI zones will only appear in the defined zoning configuration, not in the

effective zoning configuration and can only be created using D,I (domain,index) notation.

TI zones must include E_Ports and F_Ports in order to create a complete, dedicated,end-to-end route from initiator to target and ports can only be members of a single TI

zone.

FA 250

9evision 0312


8/18/2019 Brocade BCFA 250 Preparing BCFA Ce

Documents

Brocade BCFA 250 Preparing BCFA Certified Professionals for the 16Gbps BCA Exam