FibreChannel to Gigabit Ethernet IP Gateway …...100MB FibreChannel uses the same physical layer interface and encoding as Gigabit Ethernet (TBI). The only difference is that the

FibreChannel to Gigabit Ethernet IP Gateway Application Guide

C-WARE SOFTWARE TOOLSET, VERSION 2.4

CSTAFC2G-UG/DRev 01

Copyright © 2004 Motorola, Inc. All rights reserved. No part of this documentation may be reproduced in any form or by any means or used to make any derivative work (such as translation, transformation, or adaptation) without written permission from Motorola.

Motorola reserves the right to revise this documentation and to make changes in content from time to time without obligation on the part of Motorola to provide notification of such revision or change.

Motorola provides this documentation without warranty, term, or condition of any kind, either implied or expressed, including, but not limited to, the implied warranties, terms or conditions of merchantability, satisfactory quality, and fitness for a particular purpose. Motorola may make improvements or changes in the product(s) and/or the program(s) described in this documentation at any time.

C-3e, C-5, C-5e C-Port, and C-Ware are all trademarks of C-Port, a Motorola Company. Motorola andthe stylized Motorola logo are registered in the US Patent & Trademark Office. All other product or service names are the property of their respective owners.

MOTOROLA GENERAL BUSINESS INFORMATION

CSTAFC2G-UG/D

Rev 01

CONTENTS

About This GuideGuide Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Using PDF Documents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Guide Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Revision History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10References to CST Pathnames . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Related Product Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

CHAPTER 1 FibreChannel to Gigabit Ethernet IP Gateway Application GuideOverview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Prerequisite Reading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13System Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Application Feature Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

\Feature Overview and Standards Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Application Components Used . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Application Control and Data Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Resource Utilization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

XPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15CP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

CPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17RxSDP Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17FibreChannel Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Basic Rx Frame Processing Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Link State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Credit Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

CPRC - Transmit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Basic Transmit Frame Building Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22Active Link State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23RxBit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

SDP Receive Bit (RxBit) Processor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

CSTAFC2G-UG/D REV 01

4 CONTENTS

8b10b Decoder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23Loss of Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Loss of Synchronization Interrupt . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Ordered Set Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

Frame Detection/Delineation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Detection of Primitive Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Detection of Primitive Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Primitive Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Frame Forwarding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

RxBit Processor State Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26RxSync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30RxByte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

RxByte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30SOF Type Indication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Frame/R_RDY Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Extract R_CTL Routing_Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Extract CS_CTL Class Specific Control (FC-PH-2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Extract S_ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Extract TYPE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Extract F_CTL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Extract SEQ_ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31Extract DF_CTL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Extract SEQ_CNT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Extract OX_ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Extract RX_ID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Initiate TLU lookups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32FibreChannel CRC-32 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33EOF Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

SDP - Transmit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33TxByte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34TxBit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34TxBit Processor Microcode State Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348b/10b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40TxByte . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

TLU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41BMU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42QMU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

CSTAFC2G-UG/D REV 01 MOTOROLA GENERAL BUSINESS INFORMATION

CONTENTS 5

FP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43Host Processor Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

Supplied Application Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Public Headers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44XPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44CPRC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45SDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46FDP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47Binaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48Simulation Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

Design Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Network Header . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49Layout of FC-IP Frame . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Physical Port Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

MOTOROLA GENERAL BUSINESS INFORMATION CSTAFC2G-UG/D REV 01

6 CONTENTS



CSTAFC2G-UG/D

Rev 01

ABOUT THIS GUIDE

Guide Overview This document describes the design and features of the C-Ware FibreChannel to Gigabit Ethernet IP Gateway application (application identifier gbeFc).

This guide is intended for users of the C-Ware Software Toolset (CST) who want to build any application provided in the CST or who want to develop new C-Ware-based applications targeted to a C-Port network processor device.

This guide contains one chapter that covers the following major topics:

• Overview

• System Configuration

• Application Feature Overview

• Application Control and Data Flow

• Resource Utilization

• Supplied Application Files

• Design Issues


8 ABOUT THIS GUIDE

Using PDF Documents Electronic documents are provided as PDF files. Open and view them using the Adobe® Acrobat® Reader application, version 3.0 or later. If necessary, download the Acrobat Reader from the Adobe Systems, Inc. web site:

http://www.adobe.com/prodindex/acrobat/readstep.html

PDF files offer several ways for moving among the document’s pages, as follows:

• To move quickly from section to section within the document, use the Acrobat bookmarks that appear on the left side of the Acrobat Reader window. The bookmarks provide an expandable ‘outline’ view of the document’s contents. To display the document’s Acrobat bookmarks, press the ‘Display both bookmarks and page’ button on the Acrobat Reader tool bar.

• To move to the referenced page of an entry in the document’s Contents or Index, click on the entry itself, each of which is “hot linked.”

• To follow a cross-reference to a heading, figure, or table, click the blue text.

• To move to the beginning or end of the document, to move page by page within the document, or to navigate among the pages you displayed by clicking on hyperlinks, use the Acrobat Reader navigation buttons shown in this figure:

Beginning of document End of document

Next pagePrevious page

Previous or next hyperlink


www.adobe.com/prodindex/acrobat/readstep.html

Guide Conventions 9

Table 1 summarizes how to navigate within an electronic document.

Guide Conventions The following visual elements are used throughout this guide, where applicable:

This icon and text designates information of special note.

Warning: This icon and text indicate a potentially dangerous procedure. Instructions contained in the warnings must be followed.

Warning: This icon and text indicate a procedure where the reader must take precautions regarding laser light.

This icon and text indicate the possibility of electrostatic discharge (ESD) in a procedure that requires the reader to take the proper ESD precautions.

Table 1 Navigating Within a PDF Document

TO NAVIGATE THIS WAY CLICK THIS

Move from section to section within the document.

A bookmark on the left side of the Acrobat Reader window

Move to an entry in the document’s Contents or Index.

The entry itself

Follow a cross-reference (highlighted in blue text).

The cross-reference text

Move page by page. The appropriate Acrobat Reader navigation buttons

Move to the beginning or end of the document.

The appropriate Acrobat Reader navigation buttons

Move backward or forward among a series of hyperlinks you have selected.

The appropriate Acrobat Reader navigation buttons


10 ABOUT THIS GUIDE

Revision History Table 2 provides details about changes made for each revision of this guide.

Table 2 Build System Conventions Guide Revision History

REVISION DATE DATE CHANGES

01 9/2001 New document.


References to CST Pathnames 11

References to CST Pathnames

You typically install the C-Ware Software Toolset (CST) on your development workstation in a directory path suggested by the installation procedure, such as:

• C:\C-Port\Cstx.y\ (on Windows 2000/XP)

• /usr/yourlogin/C-Port/Cstx.y/ (on Sun SPARC Solaris and Linux)

or:

/usr/cport/C-Port/Cstx.y/

or:

/opt/C-Port/Cstx.y/

where ‘x’ is a major version number and ‘y’ is a minor (or intermediate) version number.

You typically install each CST version under some directory path ...\C-Port\Cstx.y\. However, the user can install the CST in any directory on the development workstation. The user can also install more than one CST version on the same workstation.

Therefore, to refer to installed CST directories, we use pathnames that are relative to the ...\C-Port\Cstx.y\ path, which is the “root” of a given CST installation.

For example, the apps\gbeSwitch\ directory path refers to the location of the Gigabit Ethernet Switch application that is installed as part of the CST. The full path of this directory on a Windows 2000/XP system might be C:\C-Port\Cst2.1\apps\gbeSwitch\, so this convention is convenience for shortening the pathname.

Other top-level directories that are installed as part of the CST include bin\, diags\, Documentation\, services\, and so on. These directories are described in the C-Ware Software Toolset Getting Started Guide document, which is part of the CST documentation set.


12 ABOUT THIS GUIDE

Related Product Documentation

Table 3 lists the documentation for the C-Ware library of reference applications.

Table 3 C-Ware Application Library Documentation Set

DOCUMENT NAME PURPOSE DOCUMENT ID

AAL-5 Fabric Port SAR to Gigabit Ethernet Switch Application Guide

Describes the key characteristics of the gbeOc12SarFp applications.

CSTAA5F2G-UG

AAL-5 SAR to Gigabit Ethernet Switch Application Guide

Describes the key characteristics of the gbeOc12Sar application. CSTAA52G-UG

FibreChannel to Gigabit Ethernet IP Gateway Application Guide

Describes the key characteristics of the gbeFc application. CSTAFC2G-UG

Frame Relay to ATM to 10/100 Ethernet Switch Router Application Guide

Describes the key characteristics of the switchRouter application. CSTAFRAE-UG

Gigabit Ethernet Switch Application Guide Describes the key characteristics of the gbeSwitch application. CSTAGBE-UG

Multi-PHY Switch Application Guide Describes the key characteristics of the mphySwitch application. CSTAMPHYS-UG

Packet Over SONET Switch Application Guide Describes the key characteristics of the posOc48Sc application. CSTAPOS-UG

Packet Over SONET to Ethernet Switch Application Guide

Describes the key characteristics of the enetOc3Switch application.

CSTAPOS2E-UG

Packet Over SONET to Gigabit Ethernet Switch Application Guide

Describes the key characteristics of the posGbeSwitch application.

CSTAPOS2G-UG

Voice Over IP to Voice Over ATM Media Gateway Application Guide

Describes the key characteristics of the voIpToVoAtmSwitch application.

CSTAVOIP-UG

Fabric Processor Configuration Component Guide

Describes the key characteristics of the fabrics application component.

CSTCFPC-UG

GMII Gigabit Ethernet Autonegotiation Component Guide

Describes the key characteristics of the gmiiAutoNeg application component.

CSTCGEAN-UG

ICMP Support Component Guide Describes the key characteristics of the ip application component. CSTCICMP-UG

MPC750 SBC Host Stack Support Component Guide

Describes the key characteristics of the stackSupport application component.

CSTCMHSS-UG

PHY Configuration Component Guide Describes the key characteristics of the phy application component.

CSTCPHYC-UG

QMU Configuration and RC Support Component Guide

Describes the key characteristics of the queueUtils application component.

CSTCQRCS-UG

SONET Monitoring Component Guide Describes the key characteristics of the sonet application component.

CSTCSMC-UG



CSTAFC2G-UG/D

Rev 01
Chapter 1
FIBRECHANNEL TO GIGABIT ETHERNET IP GATEWAY APPLICATION GUIDE

Overview This document is a functional and design specification for the gbeFc application in the CST.

This document goes into detail about the following topics:

• System Configuration

• Application Feature Overview

• Application Control and Data Flow

• Resource Utilization

• Supplied Application Files

• Design Issues

Prerequisite Reading Readers of this document are assumed to have read or be familiar with the topics in the following documents in the CST:

• C-Ware Software Toolset Getting Started Guide — How to get started with the CST.

• Build System Conventions — Description of how the build system works.

System Configuration This application runs on the following modules as a part of the C-Ware Development System (CDS):

• C-5 Switch Module

This application runs with the following Physical Interface Modules (PIMs):

• 2 x Gigabit Ethernet (TBI)


14 CHAPTER 1: FIBRECHANNEL TO GIGABIT ETHERNET IP GATEWAY APPLICATION GUIDE

The gbeFc application runs on the same PIM as the 2 X TBI Gigabit Ethernet application. 100MB FibreChannel uses the same physical layer interface and encoding as Gigabit Ethernet (TBI). The only difference is that the clock frequency of the interface when running 100MB FibreChannel is 106.25MHz, rather than 125MHz.

Application Feature Overview

The gbeFc application is a SAN (100MB FibreChannel) to LAN (Gigabit Ethernet) gateway, using IP address translation to provide mapping between FibreChannel addresses and Ethernet addresses. Traffic routing is bidirectional.

This design is based upon the C-Ware Gigabit Ethernet Switch application (gbeSwitch) application in the CST. This application uses the same physical layer 0 as the gbeSwitch application. Beyond that, most of the FibreChannel implementation is different from the Gigabit Ethernet application.

Because this is an IP routing gateway, the Ethernet 802.1D bridging, VLAN tagging, and multicast features are not supported.

\Feature Overview andStandards Support

This application supports the following features:

• The Gateway FibreChannel Port is configured as a fabric F_Port and N_Port.

• Only an “in order” fabric is supported.

• Service Class 3 is supported. (Service Classes 1, 2 and 4 are not supported.)

• Only FibreChannel to IP data frames are forwarded.

• Routing occurs only between FibreChannel and Ethernet ports. (No FibreChannel to FibreChannel or Ethernet to Ethernet routing is supported.)

• Instrumentation for performance analysis: probe points throughout the CPRC code that are used in combination with the C-Ware Integrated Performance Analyzer (CWIPA) to collect detailed performance data during simulation of the application

This application does not include the following:

• Arbitrated Loop support

• Host-side components (that is, all tables are built and initialized in the Executive Processor (XP) code)


Application Control and Data Flow 15

Application ComponentsUsed

The CST provides a number of application components that are used across applications. This application uses the following application components provided in the CST:

• phy (TBI for Gigabit Ethernet interface)

• ip (ICMP support for Ethernet)

• queueUtils

See the documentation in the apps\components\<componentName>\doc\ directory for the documentation on the software components that this application uses.

Application Control and Data Flow

The FibreChannel to Gigabit Ethernet data flow is fairly simple. It is governed by static Table Lookup Unit (TLU) tables defined in the Executive Processor (XP) initialization program. All data is routed between the FibreChannel port and the Gigabit Ethernet port; no switching from Gigabit Ethernet to Gigabit Ethernet or FibreChannel to FibreChannel occurs. See Figure 1 on page 16.

Resource Utilization To support this application’s features, the processors within the C-5 NP are performing a variety of tasks. The sections below enumerate what functionality is taking place and how the different processors are being used.

XPRC The XPRC is used for the following functions in this application:

• Initialization of all FibreChannel services

• Configuration and enabling FibreChannel and Gigabit Ethernet Ports

• FC Link State Control

• C-5 configuration (QMU, BMU, TLU)

CP This application uses the CPs for the following:

• Support for 1 dual-cluster Gigabit Ethernet (TBI) interface

• Support for 1 dual-cluster 100MB FibreChannel interface.

Because the interfaces for this application are “dual-cluster”, 8 CPs (2 clusters) are used for each interface.



Figure 1 FibreChannel to Gigabit Ethernet Control and Data Flow

PHY

Payload Bus

Ring Bus

Global Bus

TLU

Table Storageand Statistics

(SRAM)

XP

ExternalHostCPU

PCI

Fabric

FP

Queue Storage(SRAM)

QMU BMU

SDRAM

Processor Boundary

C-5 NP

Cluster

CP1CP0 CP2 CP3 CP12 CP13 CP14 CP15CP4 CP5 CP6 CP7 CP8 CP9 CP10 CP11

➊ This is the FibreChannel to Gigabit Ethernet data flow.

➋ This is the Gigabit Ethernet to FibreChannel data flow.

➌ This is IP address lookup and response for FibreChannel RX.

➍ This is the enqueue and dequeue from FibreChannel to Gigabit Ethernet, respectively.

➎ This is the IP address lookup and response for Gigabit Ethernet RX.

➏ This is the enqueue and dequeue from Gigabit Ethernet to FibreChannel, respectively.

➐ This is the control flow for the FibreChannel port state machine.

➌

FibreChannel RX FibreChannel TX Gigabit RX Gigabit TX

➊➊➋

➋➍

➍➎

➌

➎➏

➏

➐

➐


Resource Utilization 17

CPRCThe CPRC is used for the following in this application:

• FC-4 level functions

• Table lookup processing

• Forwarding decision/enqueuing

• Handling of Link Services

• Credit management

The CPRC FibreChannel code operates at the FC-2 and the FC-4 layers. The CPRC program uses the C-Ware CPIs for the programming interface. The C-Ware CPIs provide a high-level abstraction of the C-5 NP’s function calls for ease of programming the C-5 NP.

On the receive side, the CPRC performs the following functions:

• Processing of lookup results from the TLU

• Initiating lookups to TLU

• Calculating forwarding information

• Queuing to the output port

• Handling Link Response frames

• Handling Basic Link Services

• Handling Extended Link Services

• Performing credit management

• R_RDY processing

RxSDP InterfaceThe CPRC interfaces to the RxByte processor via a set of special-purpose registers called Extract Space registers. The Extract Space registers are used to communicate information like frame status fields and receive results from the RxByte processor to the CPRC, so that CPRC can process and forward the frame. Table 4 shows the format of Extract Space for the Fibre Channel port and Table 5 shows the format of Extract Space for the GbE port.



Table 4 Fibre Channel Extract Space Field Descriptions

Table 5 GbE Extract Space Field Descriptions

FIELD NAME SIZE OFFSET DESCRIPTION

frameStatus 1 0 Status code of FC frame processing (valid when entire frame has been parsed)

headerStatus 1 1 Status code of IP header processing

pad 2 2 Reserved

pad2 4 4 Reserved

ipLength 2 8 IP total length field

pad3 2 10 Reserved

pad4 2 12 Reserved

pid 2 14

pad5 4 16 Reserved

pad6 4 20 Reserved

type_fCtl 4 24

pad7 4 28 Reserved


hdrStatus 1 0 Status code of MAC and IP header processing (valid when header parsing is complete)

frameStatus 1 1 Status code of MAC frame processing (valid when entire frame has been parsed)

rxPath 1 2 Indicates which RX processing CPRC should apply

protocol 1 3 Derived from MAC protocol (IP, Pause, ARP, etc.)

macType 2 4 Not used in this application

frameLen 2 6 Length of frame in bytes

unused2 4 8 Reserved

priority 1 12 Not used in this application

badFrameCount 1 13 Count of drops while SDP waits for scope

vlanId 2 14 Not used in this application

macDaHi 4 16 Four MSBs of MAC destination address

macDaLo 2 20 Two LSBs of MAC destination address



The CPRC and RxByte processor also share a set of registers to do table lookup processing. The RxByte processor initiates lookups using the Ring Bus register requests for the destination MAC address for the first frame of each Sequence. (There is also a set of Ring Bus registers in which lookup responses are returned for processing by the CPRC program.) For subsequent frames of the Sequence the CPRC performs the TLU lookup of the forwarding information based on the Destination MAC Address.

Access to the lookup response request and response registers are by tag. The tag uniquely identifies all lookups on a frame by frame basis. The tag assigned is the for the destination MAC address.


macSaHi 4 24 Four MSBs of MAC source address

macSaLo 2 28 Two LSBs of MAC source address


ipControl 4 32 Not used in this application

version 1 36 IP version (should be 4)

IHL 1 37 IP header length

TOS 1 38 IP Type of Service field

pad 1 39 Reserved

length 2 40 IP total length

indentifier 2 42 IP identifier field

flags 1 44 IP flags

pad1 1 45 Reserved

fragOffset 2 46 IP fragmentation offset

TTL 1 48 IP time to live

protocol 1 49 IP protocol field

checksum 2 50 IP checksum

sourceAddr 4 52 IP source address

destAddr 4 56 IP destination address

pad2 4 60 Reserved




The RxByte processor signals the CPRC that EOF was detected and frame reception is complete via the RxCTLX_STATUS registers.

FibreChannel Frame

Figure 2 FibreChannel Frame

Basic Rx Frame Processing Example1 The Frame bytes are forwarded through the large FIFO to the RxByte Processor for

header extraction.

2 Header fields are written by the RxByte processor to Extract Space - CPRC reads Extract Space.

3 A table lookup is done in the TLU for the Destination MAC Address.

4 Data is DMA’d from DMEM to the BMU.

5 The CPRC calculates the route via an output port from the TLU lookup.

6 The CPRC creates a descriptor and queues it to the CPRC associated with the output port via the QMU.

Link StateOne CPRC in each receive cluster implements the link state machine as defined in Clause 16.5 of FC-PH r4.3. This link state machine implements the link initialization protocol and the link reset protocol. See Figure 3 on page 21. State transitions typically occur on the receipt of a NOS/OLS/LR/LRR Primitive Sequence. State transitions can also occur as the result of an internal C-5 request. For example, the XP may instruct a Port to go off line.

The CPRC in the cluster responsible for the link state machine is the one associated with the RxSDP with the lowest ID in the cluster that implements the receive function for one FC Port. This will usually be CP0 and CP8.

If a NOS/OLS/LR/LRR is received on a RxSDP other than CP0 or CP8, the receiving CPRC forwards the Primitive Signal to CP0 or CP8 by a descriptor queued to the QMU.

(4 Bytes)SOF

(24 Bytes)Header

FramePayload

(4 Bytes)CRC

(4 Bytes)EOF

NetworkHeader

Data Field = (0 - 2112 Bytes)



Figure 3 Link State Diagram

OL2

LR2

AC

LF1

LF2

OLS15 ms

Link Init/Offline

Output State:LRR

Output State: OLS

Output State: OLS

Output State: IDLE

Output State: NOS

Output State: OLS

LOS received

LR3

LR1

Link Reset

Output State: LR

Link Fail



If a state transition requires a Primitive Sequence to be transmitted, the CPRC creates a descriptor and enqueues it via the QMU to the Link Services queue for the matching CPRC of the TxSDP. For example, if the Primitive Signal that caused the state change was received on CP0, the descriptor will be enqueued to the CPRC for CP4.

Credit ManagementCurrently, only minimal R_RDY handling is in place. This reference application assumes infinite credit.

CPRC - TransmitThe CPRC program uses the C-Ware CPIs for the programming interface. The TxBit and TxByte Sequencers are resources used by the TxCPRC via the Merge Space register.

TxCPRC functions are:

• Exchange and Sequence management

• IP encapsulation into FC Frames

• Generation of BLS and ELS requests and replies

• Mapping IP addresses to FC addresses

Basic Transmit Frame Building Example1 Packet enqueued from Gateway port receiver.

2 TxCPRC dequeues packet for processing.

3 Merge Network header and FC header with Payload at TxByte processor.

4 Calculate CRC and append to frame.

5 Forward frame to TxBit processor and wrap with SOF/EOF.

6 Forward Frame and delimiters to link.

Active Link StateA port in the Active State can transmit and receive frames and primitive signals. When no frames are active on the link, the receiver must always receive transmission characters to maintain the link in an active state. The transmitting port is responsible for these characters.



SDP The SDP does the following in this application:

RxBit(The RxBit and TxBit microcode are in the common components for physical interfaces (apps/compnents/phy) area. See the documentation there for more information.)

RxBit does the following in this application:

• Processing ordered sets including sync detection and frame delineation.

• Signal Link-level State to RxCPRC

SDP Receive Bit (RxBit) ProcessorThe RxBit processor is responsible for processing Ordered Sets in the FibreChannel stream from the physical interface. The RxBit processor operates only at the FC-1 level.

RxBit initialization does the following two operations:

• Point the CREGS address to CREGS0. CREGS0 is used for storing Link State changes.

• Sync on K28.5 character before starting transmission word detection.

Aggregation note: The base RxBit processor in each SDP runs in lock step with one exception. When a SOFx3 has been detected, only the processor with the token will pass the data stream to the RxByte processor. The others are waiting for the next K28.5 character.

8b10b Decoder The decoding/encoding function for FibreChannel and Gigabit Ethernet are identical. The output of the decoder on the RxSDP generates 8 data bits and a ninth bit for identifying the byte as a control character (K) or a data character (D).

Loss of SynchronizationThere is a difference between Gigabit Ethernet implementation and FibreChannel for Synchronization-Acquired/Loss-of-synchronization State transitions. Should the character stream for receive data violate the rules for bit transitions described in section 12.1.3.1 of FC-PH2-4.3, a loss-of-synchronization procedure will begin.

The SDP 8b/10b functional unit detects loss-of-synchronization and performs a port state change in microcode should this event occur. A mode bit in the C-5 NP will enable a special FibreChannel state sequencer for the TBI configuration.



Unidentified or unsupported K characters are ignored by the microcode. In some cases, if a transmission word is defective, the sequencer code writes LOS signal to CREGS0 and goes back to looking for the next valid K28.5 character. Currently, the XP will transition the FibreChannel transmitter into OLS on this event and for unsupported transmission words.

Loss of Synchronization InterruptThe RxBit processor is also capable of generating an interrupt when a Loss-Of-Sync (LOS) or a running Disparity error is detected in hardware. An interrupt handler can be created to deal with such an error. This is accomplished with an event bit assigned to the SONET engine (bit 30). Currently, the handler code is not complete.

The following error conditions will trigger the LOS interrupt:

Code violation.Invalid Special Code alignment condition.Invalid Beginning Running Disparity condition.

The LOS interrupt is invisible to the sequencer microcode.

Ordered Set Processing

Frame Detection/DelineationThe RxBit processor is programmed to detect FibreChannel Start of Frame (SOFi3, SOFn3) delimiter. After the SOFx pattern is detected, FC headers, Payload, CRC, and frame delimiters are forwarded to the SDPs large FIFO. At some point the RxBit processor will detect an EOFx frame delimiter in the inbound character stream.

Table 6 Start-of-Frame and End-of-Frame Ordered Sets

ORDERED SET TC1 TC2 TC3 TC4

SOFi3 K28.5 D21.5 D22.2 D22.2

SOFn3 K28.5 D21.5 D22.1 D22.1

EOFn- K28.5 D21.4 D21.6 D21.6

EOFn+ K28.5 D21.5 D21.6 D21.6

EOFt- K28.5 D21.4 D21.3 D21.3

EOFt+ K28.5 D21.5 D21.3 D21.3

EOFa- K28.5 D21.4 D21.7 D21.7

EOFa+ K28.5 D21.5 D21.7 D21.7



Detection of Primitive SignalsThe RxBit processor detects IDLE and R_RDY Primitive Signals. IDLE Primitive Signals are discarded. An R_RDY Primitive Signal is forwarded to the RxByte processor to signal that BB_CREDIT has been received.

The RxBit sequencer does not count IDLEs between Frames or IDLEs between R_RDYs and other signals. The specification states that 6 primitives separate Frames (IDLES and R_RDY). See FC_PH4.3 Section 16.3.2.

Detection of Primitive SequencesOrdered Sets must be received at least three at a time without intervening data to be considered valid Ordered Sets. The Primitive Sequences will be constant and continuous until the link state changes.

After detection in the RxBit processor, a Primitive Sequence will signal the CPRC via the RxCTL_BITSEQX register for handling. The CPRC must maintain Link State and generate a response by firing back the appropriate Primitive Sequence from the matching transmit port.

Primitive Sequences are never forwarded to the RxByte processor.

Primitive Sequences

Table 7 Primitive Signal Ordered Sets


IDLE K28.5 D21.4 D21.5 D21.5

R_RDY K28.5 D21.4 D10.2 D10.2

Table 8 List of Primitive Sequences

PRIMITIVE SEQUENCE DESCRIPTION

Not_Operational (NOS) The transmitting port has detected a link failure or is offline and is transmitting the NOS sequence and is waiting for the OLS Primitive Sequence to be received.

Offline (OLS) The link is signaled offline. No further frame forwarding will occur.

Link_Reset (LR) The transmitting port has started the Link Reset protocol or recovering from a link timeout.

Link_Reset_Response (LRR) Port acknowledgement of the Link Reset State.



The RxBit processor also validates that the control characters are received at the appropriate time such that no primitive characters are received between the start and end of frame.

A table of all of the Ordered Sets that the RxBit processor must decode is shown in Table 9.

Additionally, the RxBit processor writes Link State to its CREG register and passes an R_RDY signal to the Rxbyte processor.

Frame ForwardingThe RxBit Processor will complete loading the frame into the FIFO and pass the Rxbit token to switch to the next SDP for the next frame. The next frame is then passed to CP1 and so on until the token arrives back at CP0.

RxBit Processor State DiagramsThe following series of diagrams can be used in conjunction with the actual microcode to better understand the application’s design.

Table 9 Ordered Sets Requiring Decoding


NOS K28.5 D21.1 D10.4 D21.2

OLS K28.5 D21.2 D31.5 D5.2

LR K28.5 D9.2 D31.5 D9.2

LRR K28.5 D21.1 D31.5 D9.2



Figure 4 RxBit Processor Microcode State Diagram 1 of 3









RxSyncThe RxSync processor is not used in this application.

RxByteThe RxByte processor does the following in this application:

• Extract frame header and pass to RxCPRC for processing

• Load Extract Space

• Post Ring bus commands

• Transfer payload to DMEM

• CRC accumulation

• Signal error conditions

RxByte The RxByte Processor is responsible for the FC-2 processing of the FibreChannel stream from the FC-1 level.

SOF Type IndicationStart-of-Frame indication is not used in this application. F_CTL is used to indicate frame sequence type.

Frame/R_RDY Detection The entire FibreChannel header is always saved off to Extract Space. If the frame is the first of sequence, the Network header is saved to Extract Space and part of the SNAP header is also saved (the 2 byte PID).

Extract R_CTL Routing_Control The RxByte processor processes frames of the following RC type:

• 0000 = FC-4 Device_Data frame (only TYPE = FC-IP frames processed)

• 0010 = Extended Link_Data frame

• 1000 = Basic Link_Data frame

• 1100 = Link_Control frame

The RxByte processor processes frames of the following IC type:



• 0010 = Unsolicited Control for Extended Link Data

• 0011 = Solicited Control for Extended Link Data

• 0100 = Unsolicited Data for FC-4 Device_Data frame with TYPE = FC-IP

(If RC is Basic Link Data or Link Control, then IC bits are interpreted as command bits.)

If there is a match on the above RC & IC fields, the frame will be forwarded to the CPRC for further processing. If there is not a match on the above RC & IC fields and the Class of Service is class 3 (SOFx3), the frame is discarded and the CPRC is signaled that a class 3 frame has been discarded.

The R_CTL field is loaded into the RxSDPX_Ext0 Extract Space Register. Note that “X” = 0 or 1, depending on whether data scope 0 or 1 is in effect.

Extract CS_CTL Class Specific Control (FC-PH-2)Word 1 < 31:24> of the Frame_Header is defined as the Class Specific Control (CS_CTL) field. The CS_CTL field for Class 3 is used for QOS.

The CS_CTL field is loaded into the RxSDPX_Ext2 Extract Space Register.

Extract S_IDThe S_ID field is loaded into the RxSDPX_Ext3 Extract Space Register.

Extract TYPEIf the RC & IC bits indicate FC-4 Device Data & Unsolicited Data and the FIRST_FRAME_SEQUENCE flag is set, then the DO_TLU_LOOKUP1 flag is set. If the RC & IC bits indicate FC-4 Device Data & Unsolicited Data and the FIRST_FRAME_SEQUENCE flag is clear, then look up the SSB.

The TYPE field is loaded into the RxSDPX_Ext4 Extract Space Register.

Extract F_CTLThe F_CTL field is loaded into the RxSDPX_Ext5 Extract Space Register.

Extract SEQ_IDThe SEQ_ID field is loaded into the RxSDPX_Ext6 Extract Space Register.



Extract DF_CTLIf the FIRST_FRAME_SEQUENCE flag is set, the RxByte processor verifies that bit 21 of DF_CTL (Network Header present) is set. If bit 21 is not set and the Class of Service is class 3, the frame is discarded and the CPRC is signaled that a class 3 frame has been discarded.

The DF_CTL field is loaded into the RxSDPX_Ext7 Extract Space Register.

Extract SEQ_CNTThe SEQ_CNT field is loaded into the RxSDPX_Ext8 Extract Space Register.

Extract OX_IDThe OX_ID field is loaded into the RxSDPX_Ext9 Extract Space Register.

Extract RX_IDThe RX_ID field is loaded into the RxSDPX_Ext10 Extract Space Register.

Initiate TLU lookups If the DO_TLU_LOOKUP1 flag is set, the RxByte processor performs lookups on the destination IP address to the TLU. The CPRC is free to evaluate the resulting lookup concurrently while frames are received. The RxByte processor extracts the Destination MAC Address High from Word 0 of the Network Header and loads it into the RxSDPX_Ext11 Extract Space Register. The RxByte processor extracts the Destination MAC Address Low from Word 1 of the Network Header and loads it into the RxSDPX_Ext12 Extract Space Register. The RxByte processor extracts the Source MAC Address High from Word 2 of the Network Header and loads it into the RxSDPX_Ext11 Extract Space Register. The RxByte processor extracts the Source MAC Address Low from Word 3 of the Network Header and loads it into the RxSDPX_Ext12 Extract Space Register. The RxByte processor then performs a TLU lookup on bits 00 to 47 of the Destination MAC Address.

If the DO_MAC_LOOKUP1 flag is set, the RxByte processor will also initiate a TLU lookup for the login state of the frame originator by issuing a TLU command to perform a Port login state lookup based on the Port_ID indicated by the S_ID in the frame header. The RxByte processor then clears the DO_MAC_LOOKUP1 flag.

If the DO_MAC_LOOKUP2 flag is set, then the CPRC must initiate the TLU lookup, since the CPRC maintains the Sequence Status Block which contains the Destination MAC Address. The RxByte processor signals the CPRC to initiate the TLU lookup by setting the RxStatusX_L0 bit in the RxStatusX register, which raises an event for the CPRC. The RxByte processor then clears the DO_MAC_LOOKUP2 flag.



FibreChannel CRC-32The RxByte processor’s calculates FibreChannel CRC on received frames and verifies that the CRC on the frame is valid. CRC is accumulated starting with the first byte of the FC-Header to the last byte of the CRC. SOFs and EOFs are not part of the calculation. If the CRCs do not match the microcode will signal a CRC error to the CPRC.

EOF ProcessingWhen the RxByte processor transfers the EOF to DMEM it sets the Avail bit (31) in the RxCTLX_STATUS register to return ownership of the data scope to the CPRC. The setting of the Avail bit will cause an Event to be indicated to the CPRC.

SDP - Transmit The TxSDP is a series of microsequencers that performs byte-level transmission of the outgoing FibreChannel stream. The TxSDP performs the following FibreChannel Transmit functions:

• Assembling IP header and Payload in a mapped FC frame

• Maintaining FibreChannel Link State

• Forwarding Primitive Signals and Sequences

• CRC generation for FC frames

Table 10 Receive Status Reported in the RxSDPX_Ext31 Register

BIT RECEIVE STATUS

31 Forward IP frame

30 Forward IP frame and generate ACK

29 Link control frame received

28 BLS received

27 ELS received

26 Class 3 frame discarded

25 Class 2 frame discarded

24 Class 3 ELS discarded

23 Class 2 ELS discarded

22 CRC error



TxByteTxByte does the following in this application:

• Merge Ethernet header with payload and pass to Txbit Processor

• CRC calculation

TxBitTxBit does the following in this application:

• Signal Link-level protocol status to N-Port

• Pass frames with delimiters

• Link Initialization procedure

• Determine the correct Running Disparity for the EOF frame delimiter

• Handle FIFO underrun condition

The TxBit processor polls the LinkStateCmd register (CREGS0) to check for the next state. The contents of LinkStateCmd register is a branch address for the TxBit Sequencer. Branch addresses originate from the XP. The TxBit processor Transmits Idle primitives to maintain link signal quality. At least six Primitive Signals must be transmitted between frames.

TxBit Processor Microcode State DiagramsThe following series of diagrams can be used in conjunction with the actual microcode to better understand the application’s design.



Figure 7 TxBit Processor Microcode State Diagram 1 of 5















8b/10bConverts the 8-bit data, primitives and delimiters into ten 10-bit encoded Transmission Characters. It is assumed that the encoder will generate the correct running parity for all transmitted characters.

TxByteThe TxByte processor read the PDU fields from Merge Space and encapsulates them in the frame. The TxByte Processor constructs the FC Frame by assembling the frame with SOF, Header, IP Payload, CRC, and EOF. The TxByte processor receives information about outgoing packets through merge space. Table 11 describes how merge space is used between TxByte and the CPRC for the Fibre Channel port and Table 12 on page 41 shows merge space for the GbE port.

Table 11 Fibre Channel Merge Space Field Descriptions


txAlgorithm 1 0 Indicates how SDP should treat outgoing packet

datalinkHeaderSize 1 1 Not used in this application

pad 2 2 Reserved

macDestHi 4 4 Four MSBs of MAC destination address

macDestLo 2 8 Two LSBs of MAC destination address







Table 12 GbE Merge Space Field Descriptions

TLU The only table defined by the FibreChannel application is the IP routing table. This table is either a VP Trie-Data table (C-5 NP) or a LPM table (C-5e NP). The key is the destination address of an incoming packet. A lookup into this table returns the queue number of the Gigabit Ethernet transmit port.

• For SAN to LAN routing, the Key for the forwarding table is the Network portion of the Destination IP Address of the LAN port.

A variable prefix match algorithm is used to implement SAN-LAN IP forwarding table in the TLU. A lookup will be initiated by receiving a valid FibreChannel Sequence containing a IP destination address found in the IP header.

• For LAN to SAN routing, the Key for the forwarding table is the Network portion of the Destination IP Address of the SAN port, which is contained in the IP header of the Ethernet frame.


pauseTime 2 0 Amount of time to silence port

txAlgorithm 1 2 Indicates how SDP should treat outgoing packet

datalinkHeaderSize 1 3 Not used in this application

macDaHi 4 4 Four MSBs of MAC destination address

vlanTag 2 8 Not used in this application

macDaLo 2 10 Two LSBs of MAC destination address






A variable prefix match algorithm is used to implement LAN-SAN IP forwarding table in the TLU. A lookup will be initiated by receiving a valid Network Packet containing a IP destination address. Table 13 shows the format of the IP forwarding table.

Table 13 IP Forwarding Table Field Descriptions

The Gigabit Ethernet portion of the application defines all the tables that the standalone Gigabit Ethernet application uses, though not all of them are used.

BMU This application uses 16 buffer pools, one for each CP. The CPs that are allocated to do processing for Gigabit Ethernet allocate buffer pools that contain buffers that are 2k bytes each. The reason is that Ethernet frames be up to 1518 bytes, and buffer sizes are only available in powers of two.

For the same reason, the CPs that are allocated for FibreChannel allocated 4k bytes for the buffers in it’s pool.

QMU The application allocates 16 queues per Channel Processor (CP). However, only two queues are used; one for FibreChannel transmit, and one for Gigabit Ethernet transmit. That is, the routing tables are set up so that any IP lookup performed by the Gigabit Ethernet receiver returns the queue number corresponding to FibreChannel transmit. Similarly, any IP lookup done by the FibreChannel receiver returns the queue number corresponding to Gigabit Ethernet transmit. Thus, only one best-effort class of service is supported.


maskBits 1 0 Number of significant bits in the key

queueId 1 1 Queue ID of the egress interface

type 1 2 IP route type (remote, local, etc.)

ifIndex 1 3 Egress interface number

appData1 4 4 Application specific data (e.g. MAC destination address)

appData2 4 8 Same as above

fabricId 1 12 Not used in this application

pad1 1 13 Reserved

pad2 2 14 Reserved



When PDUs are forwarded from one interface to another, a descriptor is associated with the PDU that describes its contents. Table 14 shows the format of the IP forwarding descriptor.

Table 14 IP PDU Forwarding Descriptor

FP This application uses the following Fabric Port configuration:

Host Processor Interaction In this application, there is no host processor support - besides what is included in the C-5 Device Driver.


bufHandle 4 0 Buffer handle of PDU being forwarded

length 2 4 Length of PDU

VNID_client 2 6 Upper 12 MSBs contain VNID not used by this application. Lower 4 MSBs contain the transmit client: IP, MAC Pause, ICMP, etc.

appData1 4 8 Application specific data. In this application the data could be the MAC destination address or the outgoing VPI/VCI.





Supplied Application Files Below is a list of the files that are a part of this application and a brief description of their contents.

Public Headers Files that are used as processor independent, public headers are as follows:

XPRC Files that run on the XPRC are as follows:

FILE NAME LOCATION DESCRIPTION

fcDefsIf.h .../chip/np/inc/ FibreChannel definitions.

fcIf.h .../chip/np/inc/ Public interface to FibreChannel support.

fcStatsIf.h .../chip/np/inc/ FibreChannel statistics definitions.

gbeStatsIf.h .../chip/np/inc/ Gigabit Ethernet statistics definitions.

ipChecksumIf.h .../chip/np/inc/ Public interface to routines to support IP checksums.

ipHeadersIf.h .../chip/np/inc/ Definitions for IP header for both FibreChannel and Gigabit Ethernet.

ipIf.h .../chip/np/inc/ Public interface for IP.

rcSdpApiIf.h .../chip/np/inc/ Interface definitions between the CPRC and SDP.


fcXp.h .../chip/np/xprc/inc/ Contains definitions for the XPRC FibreChannel state machines.

fcXp.c .../chip/np/xprc/src/ Implementation of XPRC port state machine for FibreChannel.

xpMain.c .../chip/np/xprc/src/ Main program and entry point for this application.

xpMainInit.c .../chip/np/xprc/src/ Main initialization program and entry point for this application.

tle_writes.h .../chip/np/xprc/c5/inc/

Offline table building transactions for application supporting C-5.

tle_restore.h .../chip/np/xprc/cxe/inc/

Offline table building transactions and support for C-5e.


Supplied Application Files 45

CPRC Files that run on the CPRC are as follows:


queueCpInline.h .../chip/np/cprc/c5/inc/

Queueing support for application that runs on C-5.

queueCpInline.h .../chip/np/cprc/cxe/inc/

Queueing support for application that runs on C-5e.

anCp.h .../chip/np/cprc/inc/ Interface for autonegotiation support for Gigabit Ethernet on the CPRC.

fcCp.h .../chip/np/cprc/inc/ Interface for FibreChannel on the CPRC.

ipCp.h .../chip/np/cprc/inc/ Interface for IP on the CPRC.

macCp.h .../chip/np/cprc/inc/ Interface for the Ethernet MAC on the CPRC.

fcCpMainRx.c .../chip/np/cprc/src/ Main program for FibreChannel receive cluster.

fcCpMainTx.c .../chip/np/cprc/src/ Main program for FibreChannel transmit cluster.

fcIrqCp.c .../chip/np/cprc/src/ Interrupt handling support for FibreChannel clusters.

fcRxCp.c .../chip/np/cprc/src/ Receive processing for FibreChannel.

fcTxCp.c .../chip/np/cprc/src/ Transmit processing for FibreChannel.

gbeAnCp.c .../chip/np/cprc/src/ Gigabit Ethernet Autonegotiation support on CPRC.

gbeCpMainRx.c .../chip/np/cprc/src/ Main program for Gigabit Ethernet receive cluster.

gbeCpMainTx.c .../chip/np/cprc/src/ Main program for Gigabit Ethernet transmit cluster.

gbeMacControlCp.c .../chip/np/cprc/src/ MAC control support for 802.1X.

gbeMacInitCp.c .../chip/np/cprc/src/ MAC intiaizliation for Gigabit Ethernet clusters.

gbeMacIrqCpHandler.c

.../chip/np/cprc/src/ Interrupt handling for Gigabit Ethernet clusters.

gbeMacPauseCp.c .../chip/np/cprc/src/ MAC PAUSE support for Gigabit Ethernet clusters.

gbeMacRxCp.c .../chip/np/cprc/src/ Gigabit Ethernet receive processing.

gbeMacTxCp.c .../chip/np/cprc/src/ Gigabit Ethernet transmit processing.



SDP File that run on the SDP are as follows:

Note that some files that run on both the SDP and FDP come from the common application components area and aren’t located with the rest of the files for this application.


enetDefs.h .../chip/np/sdp/inc/ Ethernet definitions for micro-code parsing.

enetParseSdp.h .../chip/np/sdp/inc/ Ethernet parsing routines for micro-code parsing in RxByte.

fcIpParseSdp.h .../chip/np/sdp/inc/ FibreChannel IP parsing routines for micro-code in RxByte.

fcNetHdrSdp.h .../chip/np/sdp/inc/ FibreChannel Network header parsing routines in RxByte.

fcParseSdp.h .../chip/np/sdp/inc/ Main FibreChannel parsing routines in RxByte.

fcPktHdrSdp.h .../chip/np/sdp/inc/ FibreChannel header parsing routines in RxByte.

fcStreamPduSdp.h .../chip/np/sdp/inc/ FibreChannel payload micro-code parsing in RxByte.

fcTxSdp.h .../chip/np/sdp/inc/ FibreChannel transmit micro-code support in TxByte.

gbeIpParseSdp.h .../chip/np/sdp/inc/ Gigabit Ethernet IP parsing micro-code support for RxByte.

fcRxBit.c .../chip/np/sdp/src/ FibreChannel RxBit micro-code.

fcRxByte.c .../chip/np/sdp/src/ FibreChannel RxByte micro-code main program.

fcTxBit.c .../chip/np/sdp/src/ FibreChannel TxBit micro-code.

fcTxByte.c .../chip/np/sdp/src/ FibreChannel TxByte micro-code program.

gbeRxByte.c .../chip/np/sdp/src/ Gigabit Ethernet RxByte main program.

gbeTxByte.c .../chip/np/sdp/src/ Gigabit Ethernet TxByte main program.


Supplied Application Files 47

FDP There are no files for the FDP in this application. The files that are used to support the FP come from the application components area as they are common to all applications.

Host Files that run on the host or offline for this application are as follows:


offline.c .../offline/c5 Offline table build used to support C-5.

tables.c .../offline/c5 Application-defined table creation routines used to support C-5.

tables.h .../offline/c5 Definitions for table support for C-5.

offline.c .../offline/cxe Offline table build used to support C-5e.

tables.c .../offline/cxe Application-defined table creation routines used to support C-5e.

tables.h .../offline/cxe Definitions for table support for C-5e.



Binaries Binary files used for this application are as follows:


gbeFc.dsc .../run Package descriptor file for this application that describes what MIPS and micro-sequencer images run on which processors for this application.

gbeFc.pkg .../run/bin/<$variant> Package file for this application.

gbe.sdp .../run/bin/<$VARIANT> Gigabit Ethernet SDP images for this application.

fc.sdp .../run/bin/<$VARIANT> FibreChannel SDP images for this application.

gbeFcXp.dcp .../run/bin/<$VARIANT> XPRC main MIPS ELF file for this application.

gbeFcXp.map .../run/bin/<$VARIANT> XPRC main memory map for this application.

gbeFcXpInit.dcp .../run/bin/<$VARIANT> XPRC init MIPS ELF file for this application.

gbeFcXpInit.map .../run/bin/<$VARIANT> XPRC init memory map for this application.

fcRx.dcp .../run/bin/<$VARIANT> FibreChannel CPRC receive MIPS ELF file for this application.

fcRx.map .../run/bin/<$VARIANT> FibreChannel CPRC receive memory map for this application.

fcTx.dcp .../run/bin/<$VARIANT> FibreChannel CPRC transmit MIPS ELF file for this application.

fcTx.map .../run/bin/<$VARIANT> FibreChannel CPRC transmit memory map for this application.

gbeRx.dcp .../run/bin/<$VARIANT> Gigabit Ethernet CPRC receive MIPS ELF file for this application.

gbeRx.map .../run/bin/<$VARIANT> Gigabit Ethernet CPRC receive memory map for this application.

gbeTx.dcp .../run/bin/<$VARIANT> Gigabit Ethernet CPRC transmit MIPS ELF file for this application.

gbeTx.map .../run/bin/<$VARIANT> Gigabit Ethernet CPRC transmit memory map for this application.


Design Issues 49

Simulation Files Simulation file needed for this application.

Design Issues This section describes the design considerations that were made for the FibreChannel to Gigabit Ethernet application based on the needs of this type of application.

Network Header The FC Network Header is required for this application. An FC Network Header carries source and destination Port WWNs. A port WWN consists of a 60-bit Network Address and an upper 4-bit Network Address Authority (NAA) field. The 4-bit NAA field is used to distinguish between the various name registration authorities used to define the Network Address.

Both the Source and Destination 4-bit NAA identifiers are set to binary ‘0001’ indicating that an IEEE 48-bit MAC address is contained in the lower 48 bits of the network address fields. The high order 12 bits in the network address fields are set to 0x0000. The NAA field value equal to binary ‘0001’ allows FC networks to be bridged with LANs.


config .../run/ Software simulator configuration file used for this application.

sim-c5i.in .../run/ Software simulator input file used for this application for C-5 NP.

sim-c5ii.in .../run/ Software simulator input file used for this application for C-5e NP.

(4 Bytes)SOF

(24 Bytes)FC

Header

FCSequencePayload

(4 Bytes)CRC

(4 Bytes)EOF

(16 Bytes)NetworkHeader

(4 Bytes)SOF

(24 Bytes)FC

Header

Additional FCSequencePayload

(4 Bytes)CRC

(4 Bytes)EOF



Layout of FC-IP Frame Figure 12 Layout of FC-IP Frame

Physical PortConfiguration

For the C-5 NP to adapt to a FibreChannel application, the CPs must be configured in TBI mode. Ten-bit-interface is described in detail in the ANSI Technical Report TR/X3.18-1997 FibreChannel - 10-bit Interface, and 802.3-1998, Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specification (Clause 36.1.5 Inter-sub-layer interfaces, p.924).

TBI requires channel aggregation as follows:

The C-5 NP has four clusters, each with four CPs. FibreChannel requires that one cluster be dedicated to the FibreChannel receive path and a second cluster to the transmit path. In theory the C-5 NP can have two FibreChannel ports across four aggregated clusters. Note that in this configuration, 4 frames can be processed concurrently in the TxSDPs.

LLC/SNAPHeader

(8 Bytes)

IP Header(20 Bytes)

IP DataOptional IP

Header(40 Bytes Max)

DSAP SSAP OUICTRL

OUI PID

IP Packet Data

LLC/SNAP Header

D_NAA(4 - Bits)

Network Destination Address (Low Order Bits)(32 Bits)

S_NAA(4 - Bits)

Network Source Address (Low Order Bits)(32 Bits)

Network Destination Address (High-Order Bits)(28 Bits)

Network Source Address (High-Order Bits)(28 Bits)

FC Header


Design Issues 51

Within a FibreChannel cluster, CP_0 acts as the base RxCP for the cluster. The pin configuration for the N_Port, both transmit and receive, will map as in Table 15.

Table 15 Pin Configuration for FibreChannel

PIN PURPOSE FC-RX FC-TX PIN PURPOSE FC-RX FC-TX

CP0_0 outclk nc TCLK CP4_0 outclk nc nc

CP0_1 inclk nc nc CP4_1 inclk nc RCLK

CP0_2 data nc TXD_0 CP4_2 data nc RXD_0





PIN PURPOSE FC-RX FC-TX PIN PURPOSE FC-RX FC-TX

CP1_0 outclk nc nc CP5_0 outclk nc nc

CP1_1 inclk nc nc CP5_1 inclk nc RCLKN






CP2_0-6 N/C N/C CP6_0-6 N/C N/C

CP3_0-6 N/C N/C CP7_0-6 N/C N/C

TRANSMIT CLUSTER 0 RECEIVE CLUSTER 1


