7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
1/34
Using FPGAs to
Design Gigabit
Serial Backplanes
April 17, 2002
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
2/34
A1 - 2
Outline
System Design Trends
Serial Backplanes Architectures
Building Serial Backplanes with FPGAs
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
3/34
A1 - 3
Key System Design Trends
Need for .
Easy Scalability of Performance
Extremely High AvailabilityFlexible Architecture
Use of Standards
Rapid Time to Market
Reduces Costs
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
4/34
A1 - 4
Merging Communications & Computers
Communications Systems have serial backplanes
Computers moving to serial backplanes
Internet
Ethernet Switches
Application servers
Database servers
1U Web servers
Bladed Server, PICMG 2.16
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
5/34
A1 - 5
Serial Standards
Serial Standards Fiber Channel 1 Gbps
Ethernet 1 Gbps
InfiniBand2.5 Gbps
Parallel Standards Going Serial PCI => 3GIO
RapidIO=> Serial RapidIO
ATA => Serial ATA
Channel Bonded Serial Standards XAUI 10 Gigabit Ethernet (4 x 3.125 Gbps) 3GIO
3GIO
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
6/34
A1 - 6RJG
Serial Connectivity = Higher
Bandwidth & Fewer Pins
50x higher bandwidth with less than 1/3 of the pins
1 2 3 4 5
50
Example - PCI: 32-bit x 33MHz = 1 Gbps, Shared among 5 clients 250 total pins
Virtex-II Pro System: 2.5 Gbps x 4 x 5 = 50 Gbps 80 total pins
Each Client has 2.5 Gbps guaranteed to every other Client
1 2 3 4 5
4
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
7/34
A1 - 7
Serial Link Rates
155M
3.125G
1.25G
622M
10G
40G
20062000 2002 20041998
Li
nkRates(bitspersec.)
Mainstream Deployment
OC-192
Infiniband
GigE
OC-48
OC-3
OC-12
10GE
Rapid IO
3GIO
FC
FC-2x
FC-10xSxI-5
OC-768
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
8/34
A1 - 8
Serial Topologies
Full MeshSwitched
PICMG 2.16PICMG 2.2
PICMG 3.x
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
9/34
A1 - 9
Building Serial Backplanes
with FPGAs
Virtex-II
Gigabit Ethernet
Phy x2
Gigabit Ethernet
MAC x2
Switched
PICMG 2.16
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
10/34
A1 - 10
Virtex-II ProPlatform FPGA
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
11/34
A1 - 11
Virtex-II Pro Rocket I/O
Technology
Up to sixteen multi-gigabit serial transceiversSupport 622 Mbps to 3.125 Gbps full duplex operationFlexible PCS/PMA feature set
RX+RX-
TXDATA
RXDATA
Channel Bonding
and
Clock Correction
TX Clock Generator
RX Clock Recovery
40 156.3 MHzREFCLK
Deserializer
Comma Det.
Serializer
ReceiveBuffer
TransmitBuffer
FIFO
8B / 10B
Encode
8B / 10B
Decode
Elastic
Buffer
20X Multipl ier
Physical Coding Sublayer Physical Media Attachment
CRC
CRC
Transmitter
ReceiverLoop-back
32 / 16 / 8 bi ts
32 / 16 / 8 bi tsTX+TX-
(PCS) (PMA)
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
12/34
A1 - 12
Xilinx Rocket I/O Serial Backplane
Interface TechnologyRocket I/O Serial Backplane Interface (RSBI)Technology
Simple solution for serial backplane applications
Consists of two parts
The RSBI/ Aurora link protocol
An RSBI Reference Design (IP core) implementingthat protocol
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
13/34
A1 - 13
RSBI/ Aurora Protocol
An Efficient, Lightweight Protocol to MoveData Point to Point across Serial Link
Used to transport higher-level protocols
Standard protocols like Ethernet
Proprietary, customer-defined, protocols
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
14/34
A1 - 14
RSBI Core
Implements RSBI/ Aurora Protocol
Provides a simple interface for transferring packetsacrossA single gigabit serial link
Up to 16 bonded links
Minimal Use of Resources (~350 slices)
Frequency and Technology Independent Specifications and Source Code available fromXilinx
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
15/34
A1 - 15
RSBI/ Aurora Features
Uses 8B/10B and includes CRC protection
Transmitter wraps user data in Rocket I/O Auroraprotocol
Handles packet formation Sends error notification to remote core
Receiver recovers frames and realigns data Strips packet/frame wrappers
Aligns data on double word boundary Monitors error conditions
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
16/34
A1 - 16
RSBI User Interface
RSBI Rocket I/O Block
VeryHigh
Speed
Data
32
32
32
32
COMPLEX
CONTROL
SIMPLE
CONTROLUSER
User interface created so no knowledge of Rocket I/O block is necessary
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
17/34
A1 - 17
All lanes must have passed comma detect beforechannel bonding starts
Uses same technique as 10 Gigabit Ethernet and
Serial RapidIO Sends semi-random IDLE pattern
RSBI Channel Bonding
Virtual data
channel
Channel Bonding LogicSynchronization of individual channelsinto a single large data channel
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
18/34
A1 - 18
Allows maximum frame size of 8KUnlimited bytes
32-bit internal interface
Other frame sizes and internal interface widthsplanned in future releases
RSBI/ Aurora
Implementation Parameters
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
19/34
A1 - 19
Errors can be either in Rocket I/O block or link(s)
Five sources of link failure detected: Link physical error threshold overflow
Channel misalignment
FIFO overflow Receiver or transmitter
Transmitter sending an illegal K character Upon detection of link failure, core resets itself and forceslink partner to reset
Error Detection in Hardware
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
20/34
A1 - 20
RSBI Transmission
All data frames have a SOP/SOF and EOP/EOF toindicate the start and end of packet
User must send at least 2 DWORDs (8 bytes)
Core will add pad data to allow CRC functionality
User can not transmit data in increments less than inputwidth
Data can now be transmit ted in any increment
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
21/34
A1 - 21
Strips off SOP/SOF* and EOP/EOF**
from frame
Strips off pad data if necessary
Signals any errors detected
Checks CRC for correctness
RSBI Reception
*SOP/F = Starting of Packet/Frame
**EOP/F = End of Packet/Frame
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
22/34
A1 - 22
Serial Backplane: Full Mesh
Rocket I/O Transceivers
2VP50
RSBI layerX 15
Design Approach Supporting
16 slot, 3.125 Gbps per linkFull Mesh Serial Backplane
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
23/34
A1 - 23
Implementing RSBI in
Virtex-II Pro Platform FPGA
Implementation in Virtex-II Pro FPGA
Features Up to 40 Gbps
32-bit user interface Low overhead
Utilization 350 Slices
AutoNegotiation &
Synchronization
Framer
Framer
AutoNegotiation &
Synchronization
Framer
Framer
User
App
RSBI Core
Rocket I/O
InterfaceUser
Interface
RSBI
Link
Rocket I/O
Transceiver
FPGA EditorView of 2VP7
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
24/34
A1 - 24
Serial Backplane:Private Connection
2VP50
X 15
2VP2
X 4
4 Transceivers
Channel Bonded
for Private boardto board connection
Parallel
Connectionon line card
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
25/34
A1 - 25
Sample Serial Backplane
Approaches
XAUI
Switch
XAUI
Switch
44
Backplane
Rocket I/O Transceivers
XAUI Core
Switched
2VP20
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
26/34
A1 - 26
Implementing XAUI in
Virtex-II Pro FPGA
Implementation in V-II Pro
Uses 4 Rocket I/O Transceivers
Channel bonding 8B/10B encoding
Utilization 800 Slices
XAUI CoreXAUI Floorplan
2VP7
Rocket I/O
Transceivers
Transmitter
Receiver
State
Machine
Deskew
TXD, TXC
RXD, RXCSync
Sync
Sync
Sync
XAUI_TX_L0
XAUI_RX_L0
XAUI_TX_L1
XAUI_RX_L1
XAUI_TX_L2
XAUI_RX_L2
XAUI_TX_L3
XAUI_RX_L3
Management Ports
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
27/34
A1 - 27
Signal Integrity Design Resources
http://www.xilinx.com/signalintegrity
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
28/34
A1 - 28
Power Filtering Network
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
29/34
A1 - 29
Power Filtering Network Components
CapacitorsC = 0.22uF
Package: 0603 SMT
Dielectric: X7R
Tolerance: 10%WVDC: 10V
Spacing: Within 1cm of pin
Ferrite BeadsMurata BLM11A102S
Voltage Regulator
Linear Technology LT1963
All parts specified in PCB Design RequirementsVoltage Regulator must use circuit specified by manufacturer
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
30/34
A1 - 30
VTTX, VTRX
Top of board
Capacitors
Ferrite Bead
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
31/34
A1 - 31
AVCCAUXTX, AVCCAUXRX
Bottom of board
Ferrite Bead
Capacitor
FPGA Bypass Caps
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
32/34
A1 - 32
Reference Clock
Must use EPSON EG-2121CA 2.5V LVPECL output oscillator
Use according to manufacture specifications
Resistor network for clock input:
R1 510 ohm
R2 130 ohm
R3 25 ohm
R4 100 ohm
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
33/34
A1 - 33
Implementation Resources
Physical (PCB) Design Information
Signal Integrity Central on Xilinx Web Site(http://www.xilinx.com/signalintegrity)
Rocket I/O User Guide
Spice Suite RSBI Design Information (http://www.xilinx.com/rsbi)
Specification, Implementation notes
Source code (Verilog)
Test bench
XAUI information (http://www.xilinx.com/connectivity) In Networking/DatapathProducts (10 Gigabit Ethernet)
7/28/2019 Using FPGAs to Design Gigabit Serial Backplanes_Xilinx
34/34
Thank You