Upload
dinhthuan
View
226
Download
2
Embed Size (px)
Citation preview
Copyright © 2009. Avnet, Inc. All rights reserved.
11
Attention
The material contained in this presentation is the property of Avnet Electronics Marketing. Use of this material in it’s whole or in part is restricted to Avnet’s X-Fest program and Avnet employees. Any use by non-Avnet employees outside of the X-Fest program is prohibited.
For additional information, please contact Jim Beneke at Avnet ([email protected]).
Designing with the Spartan®-6 Gigabit Transceivers and the PCIe®
Endpoint Block
Copyright © 2009. Avnet, Inc. All rights reserved.
2Objectives
Introduce engineers to the Xilinx® Spartan-6 RocketIO™ GTP transceiver and integrated block for PCIe
Provide an overview of the GTP and PCIe Endpoint Block design and verification tools
At the end of the presentation, engineers will learn– Basic understanding of the serial communications– The basic architecture of the Spartan-6 GTP and PCIe Endpoint Block– What Xilinx tools to use to design with the Spartan-6 GTP and PCIe
Endpoint Block
Copyright © 2009. Avnet, Inc. All rights reserved.
3Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
4Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
5Spartan-6 Platform Family
Spartan-6 is the 3rd generation of Advanced Silicon Modular Block (ASMBL™) architecture– Virtex®-4, 1st generation– Virtex-5, 2nd generation
Spartan-6 sub-families– LX : High-performance logic and parallel IO
– LXT: Logic-oriented with serial capabilities
Users can choose the best mix of resources to optimize cost and performance
LXPlatformPlatform
LXTPlatformPlatform
Copyright © 2009. Avnet, Inc. All rights reserved.
6Spartan-6 LXT Platform Family Embedded Features
High-speed serial GTP transceivers– Up to 3.125Gbps serial data rate– Depending on the package, LXT
devices support 2-8 GTP transceivers
PCI Express® endpoint block– One PCI Express Endpoint Block– PCI Express Gen1 at 2.5Gbps
Memory Controller– Up to 4 memory controllers– Support for DDR, DDR2, DDR3, and
Mobile DDR memories– DDR2 and DDR3 at 800Mbps– DDR and Mobile DDR at 400Mbps– Higher performance and lower power
than a soft memory controller
DDRDDRDDR2DDR2DDR3DDR3LPDDRLPDDR
Spartan-6
MCB
Copyright © 2009. Avnet, Inc. All rights reserved.
7Spartan-6 LXT Platform Family
LX100T LX150TLX45TLX25T101K 148K44K24K
15,822 23,0386,8223,750126K 184K54K30K268 268116524.8K 4.8K2.1K93612/6 12/68/44/2180 1805838
1 1114 4228 842
190,4190,2296,4 296,4296,4250,2296,4 296,4296,4
High Speed External Memory Ports
Logic CellsSlices
FPGA Flip-Flops18k BRAM Blocks
BRAM KbitsClock Managers (DCM/PLL)
DSP48A MultipliersHardened PCI Express Lanes
SerDes LanesPackage SizeCSG324 15x15mm
CSG484 19x19mmFG676 27x27mm
x,y x = SelectIO, y = GTP Transceivers
LX75T75K
11,66293K1723.1K12/6132
148
FG484 23x23mm
FG900 31x31mm
268,4292,4348,8 376,8
498,8396,8540,8
Copyright © 2009. Avnet, Inc. All rights reserved.
8Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
9Serial Signaling
Serial signaling is the preferred choice in all markets such as telecom, datacom, computing, and storage – Supports very high multi-gigabit data rates– Avoids clock/data skew by using embedded clock – Reduces EMI & power consumption– It is the only IO technology that meets today's high-speed requirements– It is faster than parallel signaling
• Parallel I/O data rates are inherently limited due to unavoidable skew between clock lines and multiple data lines
Multi-GbpsSerDesUserLogic
ParallelData SerDes User
Logic
ParallelData
Device A Device B
Copyright © 2009. Avnet, Inc. All rights reserved.
10Clock and Data Recovery (CDR)
A PLL is used to generate the receive sampling clock (RCLK)– PLL first locks to input reference clock (frequency locking)– PLL then looks for transitions on RX serial input to lock to the
frequency and phase of the incoming data stream (phase locking)
RXPD
SERDES
2.5Gbps
PLL
QRXNRXDATA
RCLK
ReferenceClock @125MHz
RXP/N
RCLK
RXDATA
Copyright © 2009. Avnet, Inc. All rights reserved.
11Byte Alignment
Byte alignment is necessary to convert the serial data stream toparallel data
Receiver looks for and aligns to a pre-defined alignment pattern
101 2A 3
TransmitOrder
00 0 0 1 01 0 1 0 0 0 1 1
Pre-definedAlignment
Pattern
A 2 3
Copyright © 2009. Avnet, Inc. All rights reserved.
12Misalignment
If the byte alignment pattern is not unique, misalignment can happen
Special encoding such as 8B/10B™ is often used to ensure the alignment pattern does not occur in regular data sequence
A 1 02 1 3
TransmitOrder
00 0 0 101 0 0 0 0 1 1
Pre-definedAlignment
Pattern
A 8 4
IntendedSequence
1
Copyright © 2009. Avnet, Inc. All rights reserved.
138B/10B Data Encoding
8B/10B takes an 8-bit input and encodes it to a 10-bit output256 data patterns are defined (data 0x00 – 0xFF)12 special characters (K-characters) are defined for control– K28.1 and K28.5 K-characters are commonly used for alignment
(Comma Characters)
Current RD –abcdei fghj
Current RD +abcdei fghj
BitsHGF EDCBA
Special CodeNam e
001111 0100 110000 1011000 11100K28.0
001111 1001 110000 0110001 11100K28.1
001111 0101 110000 1010010 11100K28.2
001111 0011 110000 1100011 11100K28.3
001111 0010 110000 1101100 11100K28.4
001111 1010 110000 0101101 11100K28.5
001111 0110 110000 1001110 11100K28.6
001111 1000 110000 0111111 11100K28.7
111010 1000 000101 0111111 10111K23.7
110110 1000 001001 0111111 11011K27.7
101110 1000 010001 0111111 11101K29.7
011110 1000 100001 0111111 11110K30.7
How to read the8B/10B code
name:
101 11100
K28.5
Copyright © 2009. Avnet, Inc. All rights reserved.
148B/10B Characteristics
General 8B/10B characteristics– No more than 5 consecutive 1’s or 0’s in an encoded data or control
character• Helps CDR PLL not to drift from the center of the eye
K-Character– K character is independent from data character
• Byte alignment logic will not incorrectly align to data character – Each data/K character is encoded into a Plus or Minus symbol
• Transmitter will send either Plus or Minus symbol to maintain a DC balanced line (this concept is referred to as Running Disparity)
• DC balanced is needed to avoid Inter-Symbol Interference (ISI)• Plus and Minus symbols are complement of each other for easier
symbol detection at the receiver
Copyright © 2009. Avnet, Inc. All rights reserved.
15Byte Alignment with Comma
0 0 1 1 1 1 1 0 1 01
TransmitOrder
Pre-definedAlignment
Pattern
D1.
7
K28
.5
D31
.7
D30
.2
0 1 0 1 0
K28
.5
D31
.7
D30
.2
Use K28.5 as Comma-Character– Since K-characters are unique, miss-alignment is not possible
Other methods of non-8B/10B byte alignment are also possible– SONET aligns to a periodically repeated A1A2 sequence
Copyright © 2009. Avnet, Inc. All rights reserved.
16Data Enable Indication
To avoid long streaks of 1’s and 0’s, data needs to be sent continuously– When there are no data available, a pre-defined idle sequence is
transmitted until data is available– Idle sequence needs to be stripped out by the receiver before passing the
data to the user logic
Idle
K28.5
D21.4
Data
Data
Data
Data
Data
Data
K28.5
D21.4
K28.5
D21.4
Data
Data
Pre-definedIdle
SequenceData Data Data Idle Idle Data
EN
DATA[15:0]
16 b
it
The EN signal is called RXCHARISK[0:3] for Spartan-6 GTP transceiver
Copyright © 2009. Avnet, Inc. All rights reserved.
17Review Questions
What hard IP cores are integrated into the Spartan-6 LXT family?– GTP transceivers, PCIe Block, and Memory Controllers
How is byte alignment implemented in serial data transfer?– Using an alignment character
Why is data encoding such as 8B/10B needed in serial signaling?– 8B/10B provides sufficient number of transitions in the serial data– It also provides unique alignment characters
Copyright © 2009. Avnet, Inc. All rights reserved.
18Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
19Spartan-6 RocketIO GTP Transceivers
The Spartan-6 RocketIO GTP transceiver is a highly configurable and power-efficient transceiver– Available in Spartan-6 LXT family– Current Mode Logic (CML) serial drivers/buffers with configurable
termination and voltage swing– Programmable TX pre-emphasis and RX equalization for optimized
signal integrity– Line rates from 614Mbps to 3.125Gbps (-2 speed grade devices
support up to 2.7Gbps)• 614Mbps to 810Mbps• 1.22Gbps to 1.62Gbps• 2.45Gbps to 3.125Gbps
– Built-in PCS features, such as 8B/10B encoding, comma alignment, channel bonding, and clock correction
– PRBS pattern generation and checker
Copyright © 2009. Avnet, Inc. All rights reserved.
20Spartan-6 GTP Migration from Virtex-5
Virtex-5GTX6.5G
Virtex-5GTP
3.75G
65nm
Virtex-6GTX6.5G
Virtex-6GTH10G+
45nm 40nm
Virtex-6 GTH is a New Design
Virtex-6 GTX is a port from
Virtex-5
Spartan-6 GTP is a port from
Virtex-5
Spartan-6GTP
3.125G
Copyright © 2009. Avnet, Inc. All rights reserved.
21Spartan-6 GTP Tile
A single GTP_DUAL tile consists of two GTP transceivers. The tile has 2 PLLs that can be configured in one of two modes of operations– The TX and RX sides of
each transceiver have one independent PLL,for a total of two active PLLs for each GTP_DUAL tile
– The TX and RX sides of both transceivers share one PLL and the second PLL should be powered down for power savings
TXClock
Divider
RXClock
Divider
TXPMA
TX PCS
RXPMARX
PCS
PLL0..
TXClock
Divider
RXClock
Divider
TXPMA
TX PCS
RXPMARX
PCS
PLL1..
REF
CLK
Dis
trib
utio
n
GTP0
GTP1
Copyright © 2009. Avnet, Inc. All rights reserved.
22Spartan-6 GTP PLL
The PLL has a nominal operating range of 1.2GHz to 1.62GHz– The TX and RX clock dividers can divide the PLL clock by 1, 2, or 4
PLL Clock = PLL CLKIN * N1 * N2 / MLine Rate = PLL Clock * 2 / (TX or RX Clock Divider)
TX/RX Clock Divider Supported Line Rate1 2.45Gbps – 3.25Gbps2 1.22Gbps – 1.62Gbps4 0.61Gbps – 0.81Gbps
PLLCLKIN Phase
FrequencyDetector
ChargePump
LoopFilter VCO
LockIndicator
/ N2 / N1
/ MPLL
CLKOUT
PLLLOCKED
Copyright © 2009. Avnet, Inc. All rights reserved.
23Spartan-6 GTP Transmitter Overview
PISO
FPGATXI/F
TXPCS
TXPMA
Pola
rity
Con
trol
PhaseAdjustFIFO
TX P
re-e
mp
PRBSGenerator
PCIeBeacon
SATAOOB
8B/10BEncoder
TX PIPEControl
PLL
PCS ParallelClock Domain
PMA ParallelClock Domain
GTP transmitter features– 614 Mbps to 3.125 Gbps
operation– TX Pre-emphasis– PRBS Pattern generator – 1, 2 or 4-byte fabric interface– Internal data path is 1-byte
wide
Physical Media Attachment (PMA)– Differential drivers– Programmable pre-emphasis
Physical Coding Sub-layer (PCS)– Support for multiple coding standards– Channel bonding and clock correction
Copyright © 2009. Avnet, Inc. All rights reserved.
24Spartan-6 GTP Receiver Overview
GTP transmitter features– 614 Mbps to 3.125 Gbps
operation– RX linear equalization– PRBS Pattern checker – 1, 2 or 4-byte fabric interface– Internal data path is 1-byte
wide
Physical Media Attachment (PMA)– Differential receiver input– Linear Equalization– Clock and Data Recovery (CDR)
Physical Coding Sub-layer (PCS)– Support for multiple coding standards– Channel bonding and clock correction
Loss of Sync
SIPO
FPGARX
Interface
RX
CD
R
RXPCS
RXPMA
CommaDetect
&Align
RX Status Control
RXEQ
PLL
RX
Pola
rity
8B/1
0BD
ecod
er
Elas
tic B
uffe
rPRBS
Checker
RXOOB
PMA ParallelClock Domain
PCS ParallelClock Domain
Copyright © 2009. Avnet, Inc. All rights reserved.
25Spartan-6 GTP PRBS Pattern Generator/Checker
Pseudo-random bit sequences (PRBS) are commonly used to test the signal integrity of high-speed links– These sequences appear random but have specific properties that can be
used to measure the quality of a link– The Spartan-6 GTP transceiver pattern generator block can generate
several industry-standard PRBS patterns listed below
PRBS-15 is often used for jitter measurement 215 – 1 bits1+ x14 + x15PRBS-15
PRBS-31 is often used for non-8B/10B encoding scheme. A recommended PRBS test pattern for 10 Gigabit Ethernet.
231 – 1 bits1+ x28 + x31PRBS-31
PRBS-23 is often used for non-8B/10B encoding scheme. One of the recommended test patterns in the SONET specification.
223 – 1 bits1+ x18 + x23PRBS-23
27 – 1 bits
Length of Sequence
Used to test channels with 8B/10B coding.1+ x6+ x7PRBS-7
DescriptionsPolynomialName
Copyright © 2009. Avnet, Inc. All rights reserved.
26Spartan-6 GTP PRBS Pattern Generator/Checker
In addition to PRBS patterns, the GTP transceiver supports 20 UI (or 16 UI) and 2 UI square wave test patterns and PCIe compliant pattern generation
Symbol K28.5 D21.5 K28.5 D10.2
Disparity 0 1 1 0
Pattern 0011111010 1010101010 1100000101 0101010101
PCI Express Compliance Pattern
PCI Express 20 UI Square Wave
20 UI
Copyright © 2009. Avnet, Inc. All rights reserved.
27Spartan-6 GTP Transceiver Protocol Support
Data RatesProtocols Supported2.5 Gb/sPCI Express (Rev 1.0a)
1.62 Gb/s, 2.7 Gb/sDisplayPort™
1.25 Gb/sEPON
622 Mb/s, 1.25 Gb/s, 2.5 Gb/sGPON
1.25 Gb/sGigabit Ethernet
768 Mb/s, 1536 Mb/s, 3072Mb/sOBSAI RP3 (V4.0)
614.4 Mb/s, 1228.8 Mb/s,2457.6 Mb/s, 3072.0 Mb/sCPRI (V3.0)
3.125 Gb/sXAUI 802.3ae D5p0
1.5 Gb/sSATA Gen 1
Lanes Supported1, 4
4
1, 2, 4
1
1
1
1
1
1
1.25 Gb/s, 2.5 Gb/s, 3.125Gb/sSerial RapidIO
614 Mb/s - 3.2 Gb/sAurora
1, 2, 4
1, 2, 4
Copyright © 2009. Avnet, Inc. All rights reserved.
28Spartan-6 GTP Clocking
PLL0
CLK00GCLK00
PLLCLK00CLKINEAST0
CLK10GCLK10
PLLCLK10CLKINWEST0
PLL1
CLK01GCLK01
PLLCLK01CLKINEAST1
CLK11GCLK11
PLLCLK11CLKINWEST1
REFSELDYPLL0[2:0]
REFSELDYPLL1[2:0]
01234567
01234567
CLK00 and CLK01 are default clock sources forPLL0 and PLL1 respectively.
For each PLL within a Spartan-6 GTP tile, seven clock sources can be used as the reference clock input
– Two reference clock pin pairs from external pins (GTP dedicated clock inputs)
• CLK00, CLK10 for PLL0• CLK01, CLK11 for PLL1
– One reference clock pin pair from the east or west GTP tile
• CLKINEAST0 or CLKINWEST0 for PLL0• CLKINEAST1 or CLKINWEST1 for PLL1
– Two reference clock pin pairs from the PLL of the FPGA
• PLLCLK00, PLLCLK10 for PLL0• PLLCLK01, PLLCLK11 for PLL1
– Two reference clock signals from the FPGA global clock inputs (for testing only)
• GCLK00, GCLK10 for PLL0• GCLK01, GCLK11 for PLL1
Copyright © 2009. Avnet, Inc. All rights reserved.
29GTP Reference Clock Oscillator
A high-quality crystal oscillator is essential for good performance– When using one of the recommended oscillators, the manufacturer’s
power supply design guide must be followed – Spartan-6 device characterization is based on the same recommended
oscillators
When considering alternate clock sources, the alternate oscillators must meet or exceed the specifications of the recommended oscillators
Depending on the application and its performance goals, it is possible to stray from the clock source specifications, but in that case the specified performance of the GTP is not guaranteed
0.1uF
LVDS ClockSource
Z0
Z00.1uF
IBUFDS
Copyright © 2009. Avnet, Inc. All rights reserved.
30Reference Clock Source Example
The Maxim® MAX3674 programmable clock source meets or exceeds the specifications of the Xilinx recommended oscillators– Dual programmable differential LVPECL outputs– Output frequency range of 21.25MHz to 1360MHz– Crystal input frequency range of 15MHz to 20MHz– Total period jitter of 18ps (typical)
Copyright © 2009. Avnet, Inc. All rights reserved.
31Reference Clock Source Example
The Texas Instruments® CDCM61002 programmable clock source meets or exceeds the specifications of the Xilinx recommended oscillators– Two programmable differential LVPECL/LVDS outputs– Output frequency range of 43.75MHz to 683.264MHz– Crystal input frequency range of 21.875MHz to 28.47MHz– Total period jitter of 27ps (typical)
Copyright © 2009. Avnet, Inc. All rights reserved.
32Application Example - 10G Packet Processing
UTOPIA
Level 2
Core
UTOPIA
Level 2
Core
Packet
Processor
Packet
Processor XAUIXAUI
GTPGTP
MicroBlazeMicroBlaze
UL2
Memory
Controller
Memory
Controller
DDR3 SDRAMDDR3
SDRAM
DSL Chipset
DSL Chipset 10G
MAC
10G
MAC
GTPGTP
GTPGTP
GTPGTP
GTP features used– 8B10B encoder/decoder– Channel bonding– Clock correction– XAUI
4 x 3.125 Gbps
Bac
kpla
ne
Hard memory controller
MicroBlaze soft processor
National DS64BR401
(Use this device for traces >
14”)
Copyright © 2009. Avnet, Inc. All rights reserved.
33Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
34Spartan-6 GTP Configuration Overview
There are two ways to configure a Spartan-6 GTP– Static configuration - The transceiver is configured using a
combination of GTP port tie-offs and attribute settings at design time to support a specific protocol
– Dynamic configuration - The transceiver is configured by driving the GTP ports and using the Dynamic Reconfiguration Port (DRP) to modify the run-time configuration of the GTP
GTP configuration can be complex because of the large number of possible settings– Xilinx provides a RocketIO wizard to help manage the configuration
process of the Spartan-6 GTP– The wizard is highly recommended for any design that utilizes the
Spartan-6 RocketIO GTP
Copyright © 2009. Avnet, Inc. All rights reserved.
35RocketIO GTP Coregen Wizard
Click on Spartan-6 FPGA RocketIO GTP Transceiver Wizard.
Click on Spartan-6 FPGA RocketIO GTP Transceiver Wizard.
Click on Customize to configure the Spartan-6 GTP(s).
Click on Customize to configure the Spartan-6 GTP(s).
Select the number of GTP tiles used in the design.Select the number of GTP tiles used in the design.
For each GTP in a tile, select the GTP reference clock source.For each GTP in a tile, select the GTP reference clock source.
Follow the wizard to customize the remainder of the GTP parameters.Follow the wizard to customize the remainder of the GTP parameters.
1
2
3
4
5
The RocketIO Wizard can be used to create a design for a pre-defined (Gigabit Ethernet, CPRI, XAUI, etc.) or a custom protocol– Each Coregen wrapper includes example design, test bench; and both
implementation and simulation scripts
Copyright © 2009. Avnet, Inc. All rights reserved.
36Review Questions
What are the min and max line rates of the Spartan-6 GTP transceiver?– 614Mbps and 3.125Gbps
For each PLL within a Spartan-6 GTP tile, how many clock sources can be used as the reference clock input and what are they?– 7 clocks: 2 differential clocks from the FPGA pins, 1 clock from the East
or West GTP tile, 2 from the internal PLL, and 2 from the FPGA global clock pins
What Xilinx tool can you use to control and monitor the operation of the Spartan-6 GTP in real-time?– Serial IO Toolkit
Copyright © 2009. Avnet, Inc. All rights reserved.
37Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
38PCI Express Standard
The PCI Express standard is a next-generation evolution of the older PCI and PCI-X™ parallel bus standards– It is a high-performance, general-purpose interconnect architecture,
designed for a wide range of computing and communications platforms– It is a packet-based, point-to-point serial interface that is backward
compatible with PCI and PCI-X configurations, device drivers, and application software
– The effective bandwidth is 80% of the raw bandwidth due to 8B/10B encoding
Link
x1x2x4x8
Effective Bandwidth PerDirection
2Gbps4Gbps8Gbps16Gbps
Raw Bandwidth PerDirection2.5Gbps5Gbps10Gbps20Gbps
PCI Express Bandwidth
x12x16x32
24Gbps32Gbps64Gbps
30Gbps40Gbps80Gbps
Copyright © 2009. Avnet, Inc. All rights reserved.
39PCI Express Use Model
Root Complex– A Root Complex (RC) denotes
the root of an I/O hierarchy that connects the CPU and memory subsystem to the I/O
– Comparable to the PCI North Bridge
Switch– Logical assembly of multiple
virtual PCIe-PCIe bridge devices
– Comparable to the PCI South Bridge
Endpoint– Previously known as the
Peripheral
Root Complex MemoryPCIe to
PCIBridge
CPU
Switch PCIeEndpoint
PCIeEndpoint
LegacyPCIe
Endpoint
PCIeEndpoint
UpstreamPort
DownstreamPort
RootPort
End-Point
At a minimum the Host Bridge, Memory Arbiter, and Root Port are required to create a Root Complex
Copyright © 2009. Avnet, Inc. All rights reserved.
40PCIe v1.1
PCI Express v1.1 (Gen 1)– 2.5Gbps per lane
PCI Express specifications– Obtained from PCI-SIG® (PCI Special Interest Group)
http://www.pcisig.com/specifications/pciexpress/specifications– Base Specification for protocol– Card Electromechanical Specification (CEM) for electrical characteristics
PCIe 1.1 specifications defines 100 Ohms differential trace impedance
Copyright © 2009. Avnet, Inc. All rights reserved.
41Xilinx PCIe Solutions
Soft IP (Alliance Partner)Gen 1 / Gen 21, 2, 4, and 8Virtex-5 FXT/TXT
Soft IP (Alliance Partner)Gen 12 and 4Spartan-6
Integrated BlockGen 11, 2, 4, and 8Virtex-5 LXT/SXT/FXT/TXT
Integrated BlockGen 1 / Gen 21, 2, 4, and 8Virtex-6
Gen 1Gen 1
Gen 2
Gen 1Gen 1 / Gen 2
Integrated Block1Spartan-6
Soft IP (Alliance Partner)4 and 8Virtex-6
Soft IP (DO-DI-PCIE-PIPE)Soft IP (DO-DI-PCIEXP)
IP
1Spartan-31, 4, and 8Virtex-4 FX
LanesFPGA Device
Alliance program members: Northwest Logic® and PLDA®
Copyright © 2009. Avnet, Inc. All rights reserved.
42Xilinx Spartan-6 PCI Express IP Roadmap
End-point Wrapper– x1 Gen 1– Delivered through Coregen™– Simulation and implementation– Verilog only top level– Full documentation
Beta 2 (March 09) ISE® 11.3 (Sept 09)
End-point Wrapper– x1 Gen 1– All package/device combinations– Delivered through Coregen– Source code RTL wrapper– Simulation and implementation– Verilog/VHDL– Full documentation
Alliance Partner– x4 Gen 1
Copyright © 2009. Avnet, Inc. All rights reserved.
43Avnet® Spartan-6 LX150T PCIe x4 Board
Board Features– Spartan-6 LX150T FPGA (Xilinx®)– PCIe x1 and x4 support– SFP & SATA Connectors (Tyco®)– 128MB DDR3 SDRAM (Micron®)– 32MB Parallel Flash (Spansion®)– 10/100/1000 Ethernet PHY (NSC®)– USB® 2.0 PHY (NXP®)– USB Bridge (Silicon Labs®)– LVDS Clock Generator (TI®)– Temp Sensor & RTC (Maxim®)– Platform Flash (Xilinx®)– LCD Panel Interface– Dual FMC LPC Slots– SD Card Slot
$995 Target ResaleAvailable ~Q4 2009
FMC LPC Slot
Communication Ports
Miscellaneous I/O
Memory Interfaces
GTP Interfaces
Clock Sources
PCI-Express x4
SFP Connector
SATA Connector
10/100/1000 PHY
USB-RS232 Bridge
USB 2.0 PHY
Push and DIPSwitches
User LEDs
ALI Connector
128MB DDR3 SDRAM
SD Card Connector
32MB Parallel Flash
ProgrammableLVDS Clock Source
LVTTL OSC@ 100 MHz
0.75V, 1.2V, 1.5V,1.8V, 2.5V, and 3.3V
Regulators
Power Supply
JTAG Port
Con
nect
or
Spartan-6LX150TFGG676
RS232 Port
Temp Sensor andReal-Time Clock
Platform Flash
FMC LPC Slot
Con
nect
or
Copyright © 2009. Avnet, Inc. All rights reserved.
44Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
45Xilinx Spartan-6 PCI Express Endpoint Features
The Spartan-6 PCIe Endpoint block contains the functionality defined in the specifications maintained by the PCI-SIG– Available in all Spartan-6 LXT devices– Compliant with PCI Express base specification, revision 1.1 (2.5Gbps)– Compliant PCI-SIG Endpoint requires minimal fabric logic (< 100 LUTs)– RocketIO GTP transceivers can implement a fully compliant PCIe PHY– Cut-through mode on transmit path for reduced latency– Maximum payload size of 128/256/512 bytes supported– Up to 6 x 32 bit or 3 x 64 bit BARs– BARs configurable for memory or I/O
Copyright © 2009. Avnet, Inc. All rights reserved.
46Spartan-6 PCI Express Endpoint Block Diagram
TransactionLayer
Module
Data LinkLayer
Module
PhysicalLayer
Module
Configuration and Capabilities Module
PCIe Block
Block RAM Interface
TransactionLayer Interface
Clock andReset
Interface
ConfigurationManagement
Interface
RocketIOGTP
DebugPort
Copyright © 2009. Avnet, Inc. All rights reserved.
47Spartan-6 PCIe Clocking
The Integrated PCIe Block core requires a 125 or 250MHz system clock input– The clock frequency used must match the clock frequency selection in
the CORE Generator GUI– In a typical PCI Express solution, the PCI Express reference clock is a
Spread Spectrum Clock (SSC), provided at 100MHz
PCIe100MHzClock
Spartan-6PCIe Block
ExternalPLL
125 or 250MHzCLKP
CLKN
Copyright © 2009. Avnet, Inc. All rights reserved.
48Spartan-6 PCIe Clocking Examples
REFCLK@ 100MHz
PCIe LinkPCIe Switch orRoot Complex
Device
Spartan-6EndpointDevice
PCIe ClockOSC External PLL
CLK @125/
250MHz
100MHz
Embedded System Using 125/250MHzSystem Clock
100MHz PCI ExpressClock (SSC)
PCIeLink
Spartan-6EndpointDevice
PCIe EdgeConnector
CLK @ 125/250MHz
Open System Add-In Card Using125/250MHz System Clock
External PLL
Copyright © 2009. Avnet, Inc. All rights reserved.
49PCIe Clocking on Avnet S6LX150T Board
PCIe_RefClk_100_P
PCIe_RefClk_100_N
ICS874003-05
321
ON
3.3V
OFF F_SEL0F_SEL1F_SEL2
nQA0
QA0
PBSwitch MR
CLKnCLK
nQB0
QB0MGTREFCLK0P
MGTREFCLK0N
MGTREFCLK1P
MGTREFCLK1N
The QA0/nQA0 and QB0/nQB0 outputs can be set to 125MHz or 250MHz using the F_SEL[2:0] inputs of the ICS874003-05 device.
Copyright © 2009. Avnet, Inc. All rights reserved.
50Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
51Designing with Spartan-6 PCIe Endpoint Block
Typical design flow uses the Core Generator – The Core Generator wizard configures the required blocks such as GTP,
BRAM, Clock, and Reset– It outputs a Programmed Input Output (PIO) example design
The Core Generator example simulation design consists of the following discrete parts
The example design has been tested and verified with Mentor Graphics®ModelSim™, NC-Sim®, and Synopsys® VCS™ simulators– Aldec® simulator can also be used (see the Appendix for more information)– ISIM™ support (possibly in 11.4)
Example Design Simulation/Design Files
PIOA completer application
for PCI Express
Design Files - PIO example designTestbench - The Root Port Bus Functional Model (BFM), a test benchthat generates, consumes, and checks PCI Express bus traffic
Copyright © 2009. Avnet, Inc. All rights reserved.
52Xilinx Spartan-6 PCI Express Endpoint Configuration
The Transceiver, Memory, Clock and the Reset interfaces are automatically connected in the CORE Generator wrappers– These interfaces are not visible outside of the wrappers– User application must be implemented in the FPGA fabric and
interfaced to the PCIe block using the Transaction Layer Interface
Automatically connected by the CORE Generator
Optional connection
User connection
TransactionLayer
Data LinkLayer
PhysicalLayer
Configuration and Capabilities ModulePCIeBlock
Block RAM Interface
TransactionLayer Interface
TransceiverInterface
Clock andReset
Interface
ConfigurationManagement
Interface
DebugPort
Copyright © 2009. Avnet, Inc. All rights reserved.
53PCI Express Wizard Endpoint Block Wrapper Output
TransactionLayer
Data LinkLayer
PhysicalLayer
Configuration and Capabilities Module PCIeBlock
UserInterface
SerialInterface
ConfigurationManagement
Interface
BlockRAM
BlockRAM
ClockModule
ResetModule
GTPTile
PCIe Endpoint Block Wrapper
DebugPort
Copyright © 2009. Avnet, Inc. All rights reserved.
54Core Generator Deliverables
Parameterized hard IP core (RTL wrapper source code)
Programmed Input Output (PIO) example design
Customer simulation demonstration test bench– Verilog HDL simulation flow (VHDL is scheduled for 11.4)
Customer implementation demonstration– Example UCF– Complete implementation scripts delivered for PIO design
Copyright © 2009. Avnet, Inc. All rights reserved.
55Review Questions
What is the minimum fabric logic needed to implement a PCIe x1 Endpoint using the Spartan-6 integrated PCIe block?– 100 LUTs
In addition to the PCIe hard block, what other blocks are automatically configured by the PCIe Coregen Wizard?– GTP transceiver, BRAM, Clock, and Reset
What is the procedure for assigning GTP pins to the PCIe lane?– Use Coregen to generate a UCF
Copyright © 2009. Avnet, Inc. All rights reserved.
56Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
57Xilinx Spartan-6 PCIe PCI-SIG Compliance Testing
Spartan-6 PCIe Endpoint block has passed the following tests– PCI-SIG compliance test – 3 SIG Gold suites (Electrical, Configuration and Protocol)– Interoperability– x1 Endpoint configuration
Reference board used for PCI-SIG compliance– Xilinx SP605™ evaluation board
Copyright © 2009. Avnet, Inc. All rights reserved.
58Agenda
Spartan-6 LXT Platform FamilyIntroduction to Serial Communications
Introduction to Spartan-6 GTP TransceiversDesigning with Spartan-6 GTP Transceivers
Xilinx PCIe solutionsIntroduction to Spartan-6 PCIe Endpoint BlockDesigning with Spartan-6 PCIe Endpoint BlockPCIe Compliance Testing
Wrap-Up
Copyright © 2009. Avnet, Inc. All rights reserved.
59Key Takeaways
The Spartan-6 RocketIO GTP transceiver is a highly configurable and power-efficient transceiver – Xilinx Coregen provides a RocketIO wizard to help manage the
configuration process of the Spartan-6 GTP transceiver – The Wizard can be used to create a design for a pre-defined (Ethernet,
XAUI, etc.) or a custom protocol
Spartan-6 integrated PCIe block is fully compliant with the PCI Express Base 1.1 Specification– Compliant PCI-SIG Endpoint requires minimal fabric logic (< 100 LUTs)– RocketIO GTP transceiver implement a fully compliant PCIe PHY– Xilinx Coregen can be used to generate a PIO example design
• Complete implementation script and UCF delivered for PIO design• Verilog HDL simulation support (VHDL is scheduled for 11.4)
Copyright © 2009. Avnet, Inc. All rights reserved.
60Spartan-6 PCIe Demo
S6 PCIe Demo Description– GTP transceiver running @ 2.5Gbps (PCIe Gen 1) – Memory Controller Block in DDR3 mode – Packet based DMA Design – Uses Xilinx SP605 evaluation kit
Other Equipment used – Software Application / GUI / Device Drivers – PC with Windows XP® / Vista® OS
Copyright © 2009. Avnet, Inc. All rights reserved.
61Closing Comments
Please go to the following URL to download the X-Fest 2009 course presentations
http://em.avnet.com/xfsupport2010
Visit the X-Fest 2009 forum for latest discussions on various courseshttp://community.em.avnet.com/t5/XFEST-2009/ct-p/XFEST_2009
Please Visit the Demo AreaThank You
Appendix
Copyright © 2009. Avnet, Inc. All rights reserved.
63Agenda
Serial communications definitions and terminologiesSpartan-6 PCIe Block overviewDebugging PCIe Designs
Copyright © 2009. Avnet, Inc. All rights reserved.
64Eye Diagram
An eye diagram corresponds to a superposition of a substantial number of bits of a particular stream of data after this data has propagated through the designed subsystem– One of the most accepted standards for qualifying design performance– Vertical thickness of the lines indicate the magnitude of the AC voltage
noise– Horizontal thickness of the lines indicate jitter
Copyright © 2009. Avnet, Inc. All rights reserved.
65
Every serial communications standard has its own set of eye maskparameters– Defined as part of the electrical specifications for each standard– Usually an eye mask for signal at output of the transmitter and input to the
receiver is defined– Eye Mask is the size of the opening in the center of the eye diagram– Eye Mask indicates the amount of voltage and timing margin available to
sample the signal
Eye Mask
Eye mask without violations
Copyright © 2009. Avnet, Inc. All rights reserved.
66Unit Interval (UI)
The Unit Interval (UI) is the unit of time corresponding to one bit period (the time it takes to send one bit)– The number of 1s and 0s must be equal to yield DC balance
0 1
Bit time or Unit Interval (UI)
# of Bits without transition (Run Length)
Copyright © 2009. Avnet, Inc. All rights reserved.
67
Jitter– Delay between the actual and expected transition of a signal
Jitter Tolerance– The peak-to-peak amplitude of sinusoidal jitter applied to the input signal
that causes an acceptable loss of information at the output
Differential Signaling– A signaling scheme which uses two complementary (P and N) signals to
transmit data– Offers improved data rates with reduced signal swing
Bit Error Rate– The number of errors detected at a receiver in a given length of time
Miscellaneous Terms and Definitions
Copyright © 2009. Avnet, Inc. All rights reserved.
68Pre-emphasis
Pre-emphasis is the boosting of the magnitude of high frequency spectral components in the transmitted signal to improve the eyemask of the received signal
Copyright © 2009. Avnet, Inc. All rights reserved.
69Agenda
Serial communications definitions and terminologiesSpartan-6 PCIe Block overviewDebugging PCIe Designs
Copyright © 2009. Avnet, Inc. All rights reserved.
70PCI Express Configuration Space
The PCI Express configuration space consists of three primary sections– Legacy PCI v3.0 type 0 configuration header– Legacy extended capability items
• PCIe capability item• Power management• Message signaled interrupt
– PCIe extended capabilities• Device serial number• Virtual channel
DataAddress7 015 823 1631 24
000hVendor IDDevice ID004hCommandStatus008h
Cache LnLat TimerHeaderBIST 00ChBase Address Register 0 010h
Rev IDClass Code
Copyright © 2009. Avnet, Inc. All rights reserved.
71PCI Express Configuration Space
PCI Express configuration space consists of 4096 bytes– First 256 bytes match the PCI configuration space
Some configuration data is simply read by the Root Complex– Vendor and Device ID– Memory blocks and size requirements– Certain capabilities of the subsystem such as the device type and
power management
Some configuration data is assigned by the Root Complex– Base Address Register (BAR)
• Specific address for each requested memory block• Six maximum BARs per subsystem
– Transaction ID consisting of Bus and Device Number• Consistent with the physical slot
Copyright © 2009. Avnet, Inc. All rights reserved.
72PCI Express Communication Methodology
The communication between two PCIe devices is referred to as a transaction– Transactions are packet-based
Each PCIe device can be a Requester and/or a Completer– Requester initiates a transaction– Completer responds to a request– Both the Root Complex and the Endpoint device can function as a
Completer as well as a Requester
Root Complex
Switch PCIeEndpoint
LegacyPCIe
Endpoint
Packet BasedTransactions
Packet BasedTransactions
Copyright © 2009. Avnet, Inc. All rights reserved.
73PCI Express Protocol Support
The PCI Express protocol supports four types of transactions– Memory (read and write)– I/O (read and write)– Configuration (read and write)– Message (communication information outside of the Memory, I/O, and
Configuration spaces such as interrupt signaling, error signaling, etc.)
Transactions are divided into three categories– Posted transactions– Non-posted transactions– Completion transactions
Copyright © 2009. Avnet, Inc. All rights reserved.
74Posted and Non-Posted Transactions
Memory writes and message transactions are posted transactions– The requester sends a packet, but the receiver does not return a
completion
Non-posted transactions (memory reads, I/O reads and writes, and configuration reads and writes) require a response and are implemented as split transactions– Completion packets can be directed to the correct originator
because each packet has a unique identifier
Copyright © 2009. Avnet, Inc. All rights reserved.
75Transaction Layer Module
The Transaction Layer Module (TLM) is the upper layer in the architecture– This module takes Transaction Layer Packets (TLPs) presented by user
logic at the Transaction Layer interface and schedules them for transmission over the link
– The Transaction Layer module also advises the user application when TLPs are received
– TLPs can both make requests and complete requests from another device
Transaction LayerModule
TransactionLayer Interface
Data Link LayerModule
Block RAM Interface4 bytes, 62.5MHzInterface
Copyright © 2009. Avnet, Inc. All rights reserved.
76Transaction Layer Module - Packet
A Transaction Layer Packet (TLP) is composed of a header, data payload (for most packets), and optional end-to-end CRC (ECRC)– Packets must be formed by the user in accordance with the PCI
Express specification– Packets must be decoded by the user properly
Header
32-bitMachine 3 DWORDS 1 DWORD for address
64-bitMachine 4 DWORDS 2 DWORD for address
Payload
ReadRequest 0 No Payload
Write or ReadCompletion
Max Payload Size(MPS) or less
512 bytes in Spartan-6 PCIeBlock
ECRC(Digest)
Included 1 DWORD Passed by TL to UserTrimmed 0 Still exists, but trimmed by TL
Criteria Size Notes
Copyright © 2009. Avnet, Inc. All rights reserved.
77Data Link Layer Module
The Data Link Layer Module (DLLM) resides between the Transaction Layer and the Physical Layer modules– Its primary responsibility is to provide a reliable mechanism for the
exchange of TLPs between two components on a link– Data Link Layer provides data exchange (TLPs), error detection and
recovery, initialization services and the generation and consumption of Data Link Layer Packets (DLLPs)
– The transmission portion of the DLL accepts TLPs from the Transaction Layer and generates the appropriate TLP sequence number and LinkCRC (LCRC), then passes the packet to the Physical Layer
– Data Link Layer Module also places a copy of the packet in a retry buffer, making it available if the packet needs to be resent
TransactionLayer
Module
Data LinkLayer
Module
PhysicalLayer
Module
TransactionLayer Interface
Block RAM Interface
Copyright © 2009. Avnet, Inc. All rights reserved.
78Data Link Layer Module – Data Link Layer Packets
The DLL also generates and consumes special packets called Data Link Layer Packets (DLLPs) that do not pass to the Transaction Layer– Types of DLLPs include acknowledgment (ACK/NAK), flow control, and
power management
The reception portion of the DLL checks the integrity of received TLPs– It also orders retransmission when the received TLP is found to be
corrupted
The transmission portion controls the order of release of the different types of packets– A prioritizer is included to sort the different sources of transmission into
order of priority and schedule them for transmission according to the priority order recommended in the PCIe Base Specification (Rev 1.1)
Copyright © 2009. Avnet, Inc. All rights reserved.
79Physical Layer Module
The Physical Layer (PL) module carries out the following functions defined for the PL of a PCIe device– Packet framing and de-framing (Start of Frame and End of Frame)– Byte striping and un-striping; that is, distributing Tx packets over the
associated PL lanes and reassembling Rx packets received over the different PL lanes
– Link initialization and training– Scrambling, de-scrambling, and 8B/10B encoding and decoding of data
are also performed by the Physical Layer Module
TransactionLayer
Module
Data LinkLayer
Module
PhysicalLayer
Module
TransactionLayer Interface
Block RAM Interface
TransceiverInterface
Copyright © 2009. Avnet, Inc. All rights reserved.
80PCI Express Packet Summary
A TLP is composed of a header, data payload (for most packets), and optional end-to-end CRC (ECRC)
The transmission portion of the DLL accepts TLPs from the Transaction Layer and generates the appropriate TLP sequence number and Link CRC (LCRC)
The Physical Layer appends the Start and End to the packet
Data Payload
Presented to Transaction Layer
HeaderSequenceNumberStart EndLCRCECRC
Appended by Data Link LayerAppended by Physical Layer
Copyright © 2009. Avnet, Inc. All rights reserved.
81Configuration and Capabilities Module
The Configuration and Capabilities module provides the repository for the different registers within the Configuration Space– It implements the legacy PCI configuration header defined in both the
PCI Express Base Specification and the earlier PCI bus specifications– It also implements the extended configuration space supported by PCI
Express systems that contain • Power management• Message signaled interrupts• Error reporting
TransactionLayer
Module
Data LinkLayer
Module
PhysicalLayer
ModuleTransaction
Layer Interface
Configuration and Capabilities ModuleConfigurationManagement
Interface
Copyright © 2009. Avnet, Inc. All rights reserved.
82Block RAM Interface
The PCIe block buffers are implemented using block RAMs– The sizes of the buffers can vary based on the application’s needs
There are two options for configuring these buffers in PCIe Wizard – Minimize Block RAM usage– Maximize performance
Transaction LayerModule
Data LinkLayer
Module
TransmitBlock RAM
ReceiveBlock RAM
RetryBlock RAM
Copyright © 2009. Avnet, Inc. All rights reserved.
83Agenda
Serial communications definitions and terminologiesSpartan-6 PCIe Block overviewDebugging PCIe Designs
Copyright © 2009. Avnet, Inc. All rights reserved.
84PCIe Debug Overview
Debug using PCIe configuration space registers– Why are these registers useful– PCI Express capability structure– How to read the capability structure linked-list
De-scrambling– What it is– How it works– Why it might be useful
Debug Ports– Overview of debug options included in Spartan-6 and Virtex-6 to help
customers debug PCIe designs
Copyright © 2009. Avnet, Inc. All rights reserved.
85PCI Express Configuration Space Review
The PCIe configuration space can be divided into three sections– PCI Express extended capabilities (0x100 – 0xfff)– Legacy extended capabilities list (0x40 – 0xff)– PCI 3.0 compatible configuration space (0x00 – 0x3f)
Copyright © 2009. Avnet, Inc. All rights reserved.
86Where Does the Capability List Start?
The value contained in configuration space address 0x34 is the starting point and always points to the very first item in the linked-list structure– Note that since PCIe
requires the PCIe capability structure, this register will always contain a value
Copyright © 2009. Avnet, Inc. All rights reserved.
87Structure of the Extended Capabilities Linked-lists
There are two separate linked lists of capability structures– Legacy PCI – starts with the
capabilities pointer at 0x34– PCIe Capabilities – always
starts at 0x100– Next Ptr = 0 terminates both
lists
Each capability structure must contain two items– ID which identifies the type of
structure– Pointer to next item in the list
Size of the capability structure is implied by the ID Capabilities Pointer0x34
0x40
0x100
ID
ID Next Ptr
ID 0
ID Next Ptr
ID 0PCI Express
Extended
Capabilities
PCI Extended Capabilities
Next Ptr
ID Next Ptr
Copyright © 2009. Avnet, Inc. All rights reserved.
88PCI Express Capability Structure
Lots of useful debug info in this structure– Device control register
• Max payload, max read request
– Device status register• Errors detected
– Link control register• Read Completion
Boundary (RCB)– Link status register
• Negotiated link width• Current link speed
For exhaustive discussion see Chapter 7.8 of the PCI Express Base Specification Rev 2.0
PCI ExpressCap ID
Next CapPointerPCI Express Capabilities Register
Device Capabilities
Device Status Device Control
Link Capabilities
Link Status Link Control
Slot Capabilities
Slot Status Slot Control
Root Capabilities
Root Status Root Control
Device Capabilities 2
Device Status 2 Device Control 2
Link Capabilities 2
Link Status 2 Link Control 2
Slot Capabilities 2
Slot Status 2 Slot Control 2
071531
00h
04h
08h
0Ch
10h
14h
18h
1Ch
20h
24h
28h
2Ch
30h
34h
38h
ByteOffset
Copyright © 2009. Avnet, Inc. All rights reserved.
89Customers Common PCIe Issues
Example: Customer is seeing very low bandwidth
Possible reasons– Link partner lane width capability is less than expected
• Many x8 connectors on motherboards only route 4 lanes– Width down-trained due to SI issues on upper lanes– Gen 2 speed change did not occur at link-training due to SI– Customer is not using bus-mastering DMA
Check the link capability and link status registers of both devices on the link– Check the link capability register of both devices– Check “negotiated link width” and “current link speed” fields in the
link status register
Copyright © 2009. Avnet, Inc. All rights reserved.
90
PCI ExpressCap ID
Next CapPointerPCI Express Capabilities Register
Device Capabilities
Device Status Device Control
Link Capabilities
Link Status Link Control
Slot Capabilities
Slot Status Slot Control
Root Capabilities
Root Status Root Control
Device Capabilities 2
Device Status 2 Device Control 2
Link Capabilities 2
Link Status 2 Link Control 2
Slot Capabilities 2
Slot Status 2 Slot Control 2
071531
00h
04h
08h
0Ch
10h
14h
18h
1Ch
20h
24h
28h
2Ch
30h
34h
38h
ByteOffset
Reading the Link Capability/Status Register to Verify Negotiated Link Width and Speed
Also check the same register in the Link Partner to verify Link Partner capabilities
Link capability register (0Ch)– Where we declare our
capabilities• Supported link speed• Max link width• Etc.
Link status register (12h)– What’s currently happening
on the link• Current link speed• Negotiated link width• Slot clock configuration• Etc.
Copyright © 2009. Avnet, Inc. All rights reserved.
91
PCI ExpressCap ID
Next CapPointerPCI Express Capabilities Register
Device Capabilities
Device Status Device Control
Link Capabilities
Link Status Link Control
071531
00h
04h
08h
0Ch
10h
ByteOffset
Reading the Link Status Register to Verify Negotiated Link Width and Speed
Also check the same register in the Link Partner to verify Link Partnercapabilities
Copyright © 2009. Avnet, Inc. All rights reserved.
92Determining the Max Payload Size
Device capability register– Where we declare the
maximum payload size of which we are capable
Device control register– Where the system software
sets all of the devices to a common maximum payload size
PCI ExpressCap ID
Next CapPointerPCI Express Capabilities Register
Device Capabilities
Device Status Device Control
Link Capabilities
Link Status Link Control
Slot Capabilities
Slot Status Slot Control
Root Capabilities
Root Status Root Control
Device Capabilities 2
Device Status 2 Device Control 2
Link Capabilities 2
Link Status 2 Link Control 2
Slot Capabilities 2
Slot Status 2 Slot Control 2
071531
00h
04h
08h
0Ch
10h
14h
18h
1Ch
20h
24h
28h
2Ch
30h
34h
38h
ByteOffset
Copyright © 2009. Avnet, Inc. All rights reserved.
93Debugging Using PCIe Registers
Use PCITREE/HWDirect (NT) or LSPCI (Linux) to read registers in PCIe Endpoint– Most of these register values are outputs to user via buses on the
CFG port– Device control register = cfg_dcommand[15:0]– Device status register = cfg_dstatus[15:0]
Example: Use for ChipScope Triggers– Imagine system is blue screening– Usually, FPGA is transmitting a fatal error message– Triggering on device status bit 2 gives view of situation when
message transmitted• cfg_dstatus<2> = fatal error detected
– At this trigger point, you may want to view status of MGT ports or other PCIe block ports to see if you can determine reason for failure
RsvdZ
015 123456
Transaction PendingAUX Power Detected
Unsupported Request DetectedFatal Error Detected
Non-Fatal Error DetectedCorrectable Error Detected
Copyright © 2009. Avnet, Inc. All rights reserved.
94Using PCItree
Configuration Registers
Selected PCIe Device
Copyright © 2009. Avnet, Inc. All rights reserved.
95Using HWDirect
PCItree cannot read extended capability space
HWDirect can read extended space– Low-cost Windows
shareware program (~$38)
– www.eprotek.com
On Linux, use LSPCI with –xxxx switch:e.g.: /sbin/lspci -xxxx
Copyright © 2009. Avnet, Inc. All rights reserved.
96Debug Using Configuration Space Review
PCI Configuration Space is useful for certain debug situations
– Bandwidth issues – speed and lane width
– Error logging/reporting
– Verifying capabilities of device
– Checking if system software enabled certain capabilities
Knowing how to read the linked-list is helpful
– Especially when checking the link partner capabilities list
Configuration port outputs of the PCIe core can be used as ChipScope or link analyzer triggers
Copyright © 2009. Avnet, Inc. All rights reserved.
97Spartan-6 Debug Ports
Available to help user understand causes of errors– Fatal, non-fatal,
correctable
User knows an error is generated based on the cfg_dstatus bus– User does not know
why– Debug ports will tell
user why an error occurs
Copyright © 2009. Avnet, Inc. All rights reserved.
98Using the Spartan-6 Debug Ports
Debug ports are included in the Coregen PCIe wrapper file– S6_pcie_v1_1.v
Debug ports are not pulled up to user level
User would need to add ports to block wrapper file or add logic directly in wrapper
.DBGBADDLLPSTATUS ( dbg_bad_dllp_status ),
.DBGBADTLPLCRC ( dbg_bad_tlp_lcrc ),
.DBGBADTLPSEQNUM ( dbg_bad_tlp_seq_num ),
.DBGBADTLPSTATUS ( dbg_bad_tlp_status ),
.DBGDLPROTOCOLSTATUS ( dbg_dl_protocol_status ),
.DBGFCPROTOCOLERRSTATUS ( dbg_fc_protocol_err_status ),
.DBGMLFRMDLENGTH ( dbg_mlfrmd_length ),
.DBGMLFRMDMPS ( dbg_mlfrmd_mps ),
.DBGMLFRMDTCVC ( dbg_mlfrmd_tcvc ),
.DBGMLFRMDTLPSTATUS ( dbg_mlfrmd_tlp_status ),
.DBGMLFRMDUNRECTYPE ( dbg_mlfrmd_unrec_type ),
.DBGPOISTLPSTATUS ( dbg_poistlpstatus ),
.DBGRCVROVERFLOWSTATUS ( dbg_rcvr_overflow_status ),
.DBGREGDETECTEDCORRECTABLE ( dbg_reg_detected_correctable),
.DBGREGDETECTEDFATAL ( dbg_reg_detected_fatal ),
.DBGREGDETECTEDNONFATAL ( dbg_reg_detected_non_fatal ),
.DBGREGDETECTEDUNSUPPORTED ( dbg_reg_detected_unsupported),
.DBGRPLYROLLOVERSTATUS ( dbg_rply_rollover_status ),
.DBGRPLYTIMEOUTSTATUS ( dbg_rply_timeout_status ),
.DBGURNOBARHIT ( dbg_ur_no_bar_hit ),
.DBGURPOISCFGWR ( dbg_ur_pois_cfg_wr ),
.DBGURSTATUS ( dbg_ur_status ),
.DBGURUNSUPMSG ( dbg_ur_unsup_msg ),
wire dbg_bad_dllp_status;wire dbg_bad_tlp_lcrc;wire dbg_bad_tlp_seq_num;wire dbg_bad_tlp_status;wire dbg_dl_protocol_status;wire dbg_fc_protocol_err_status;wire dbg_mlfrmd_length;wire dbg_mlfrmd_mps;wire dbg_mlfrmd_tcvc;wire dbg_mlfrmd_tlp_status;wire dbg_mlfrmd_unrec_type;wire dbg_poistlpstatus;wire dbg_rcvr_overflow_status;wire dbg_reg_detected_correctable;wire dbg_reg_detected_fatal;wire dbg_reg_detected_non_fatal;wire dbg_reg_detected_unsupported;wire dbg_rply_rollover_status;wire dbg_rply_timeout_status;wire dbg_ur_no_bar_hit;wire dbg_ur_pois_cfg_wr;wire dbg_ur_status;wire dbg_ur_unsup_msg;
Copyright © 2009. Avnet, Inc. All rights reserved.
99Debugging Conclusion
Standard PCIe register set is helpful in debugging problems
Virtex-6 and Spartan-6 user guides contain new debug chapters
Debug ports available to assist in analyzing problems
Copyright © 2009. Avnet, Inc. All rights reserved.
100De-scrambling Design
All TLPs, DLLPs, and Idles are scrambled– Required by the PCIe specification to reduce EMI noise
De-scrambling design places a descrambler on both the TX and RX path– Descrambler taps off TX and RX data path– Produces data in legible format– Packet decoder also included to provide trigger signals for ChipScope
Allows user to view traffic at the MGTs using ChipScope Pro
Also works in simulation
The De-scrambling design will be available as an Answer Record
Copyright © 2009. Avnet, Inc. All rights reserved.
101De-scrambler Block Diagram
TransactionLayer
Module
GTP/GTXTransceivers
Data LinkLayer
Module
PhysicalLayer
Module
TXDe-scrambler
PCIe Block
Block RAM Interface
ChipScopeILA
ChipScopeILA
RXDe-scrambler
FPGA
Copyright © 2009. Avnet, Inc. All rights reserved.
102How is De-scrambling Useful?
Many customers do not have link analyzer– This is not a replacement for a link analyzer (better than nothing at all)
Verify ACKs/NAKs being returned as expected– Excessive NAKs or no ACK/NAKs at all could cause problems
Verify flow control is progressing– Sometimes, there seems to be a stall on the link and customers assume
it’s the PCIe block’s fault– Being able to verify flow control credits are being returned and the
ACK/NAK status may be insightful
Verify integrity of packet– Many cases have come up where customer believes block is
“manipulating packets”– Usually, this turns out to be a software issue– Using this design will allow customer to verify packet integrity right before
it goes into MGTs or as it comes out of MGTs
Copyright © 2009. Avnet, Inc. All rights reserved.
103Agenda
Serial communications definitions and terminologiesSpartan-6 PCIe Block overviewDebugging PCIe DesignsUsing the Aldec Simulator
Copyright © 2009. Avnet, Inc. All rights reserved.
104Table of Contentsof Contents
What ALDEC® Simulators Can Do?ALDEC® XILINX® Support TimelineALDEC® and XILINX®
Starting Active-HDL™ from ISE™Xilinx Tools in Active-HDL FlowStarting Coregen from Active-HDL
Copyright © 2009. Avnet, Inc. All rights reserved.
105What ALDEC Simulators Can Do?
Simulate Xilinx ISE designs using any combination of source languages: VHDL, Verilog, System Verilog, SystemC, EDIF, etc.Include pre-compiled Xilinx libraries (including SecureIP)Support latest Xilinx families: Spartan-6 and Virtex-6Support all popular IP Cores (PCIe, RocketIO Transceivers, SERDES, EMAC/TEMAC, etc.)Support SmartModels/SWIFTCan handle DSP co-simulation (MATLAB® and Simulink® interfaces )Simulate structural MPU models from EDKSupport all legacy Xilinx families (all versions of Virtex, Spartan, Coolrunner, XC4000, XC9500)
Copyright © 2009. Avnet, Inc. All rights reserved.
106Aldec Active-HDL 8.2™
Common-Kernel Mixed Language Simulator
Languages: VHDL, Verilog®, SystemVerilog (Design & Assertions), SystemC & EDIF
HDL Design Tools: Design Entry, Design Creation, Code2Graphics™, Block and State Diagram, Waveform Editor, stimulus generation, code auto-complete and language templates, scripting, legacy design support.
Design Flow Manager: use popular third-party tools throughout the design flow within the same FPGA environment.
Debugging: Code execution tracing, Waveform/Compare, Memory Viewer, Xtrace, Advanced Dataflow and Profiler.
Coverage: Code Coverage, Toggle & Functional Coverage.
Additional Interfaces: MATLAB® and Simulink® co-simulation interfaces, Zuken CADSTAR PCB Design interface
Assertion and Coverage(OPTION) SystemVerilog, PSL & OVA support; dedicated Assertion Viewer, assertion coverage, assertion breakpoints.
Users who need LinuxUsers who need Linux--based simulation based simulation tools or are not interested in graphical tools or are not interested in graphical entry may want to try entry may want to try RivieraRiviera--PROPROline of highline of high--performance simulators.performance simulators.
Copyright © 2009. Avnet, Inc. All rights reserved.
107
Q4 Q1 Q2 Q3 Q4 Q1 Q2
201020092008Q3
ALDEC® XILINX® Support Timeline
Aldec and Xilinx sign Cooperation Agreement
Over 30 mutual customers confirm working SecureIP
ISE Design Suite 12.1 with SecureIP sourcessupporting Aldec keys and Library Compiler support for Aldec
Second release of Aldec simulators with SecureIP forSpartan-6 and Virtex-6 (Active-HDL 8.2 and Riviera-PRO 2009.06SR1)
First Aldec simulators with SecureIP support(Active-HDL 8.1sp2 and Riviera-PRO 2009.02)
Copyright © 2009. Avnet, Inc. All rights reserved.
108ALDEC® and XILINX®
Aldec was the creator of design entry, project management and gate-level simulation tools in Xilinx Foundation Classic suites (our tools can import Foundation schematic designs and turn them into HDL designs)
Aldec tools support design flows including all versions of Xilinx tools, starting from Xilinx Alliance/Foundation Classic up to Xilinx ISE 11.2
Xilinx precompiled HDL libraries and SecureIP are currently available directly from Aldec
Xilinx library sources (including SecureIP) will compile for Aldec simulators using ‘compxlib’ in ISE Design Suite 12.1
Copyright © 2009. Avnet, Inc. All rights reserved.
109Starting Active-HDL from ISE
With very little effort, it is possible to start Active-HDL simulator from Xilinx ISE Project Navigator to run behavioral and timing simulations
The setup procedure is described in the application note “Starting Active-HDL as the Default Simulator in Xilinx ISE” available on ALDEC website (follow the link below or do the search on the title of the document)
The application note provides detailed, easy to follow instructions and the link to multimedia presentation demonstrating the use of theinterface
http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000771http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000771
Copyright © 2009. Avnet, Inc. All rights reserved.
110Xilinx Tools in Active-HDL Flow
Active-HDL is equipped with Design Flow Manager that, when configured properly, allows quick start of Xilinx or third partysynthesis tools and ISE implementation engine
Options required to start each application can be configured in user-friendly GUI windows
Simulation files are imported back to Active-HDL
Applications like Coregen, Constraints Editor, STA, ChipScope Pro can be stared from the Flow
Detailed instructions are available in the “Using Active-HDL with Xilinx ISE” application note (link listed below)
http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000640http://support.aldec.com/KnowledgeBase/Article.aspx?aid=000640
Copyright © 2009. Avnet, Inc. All rights reserved.
111
STARTING COREGEN FROM ACTIVE-HDL
Copyright © 2009. Avnet, Inc. All rights reserved.
112Flow Enabling
To access Coregen in Xilinx ISE 11.2 from Active-HDL, user has to enable Design Flow Manager in Preferences window accessible fromthe Tools menu in Active-HDL
Copyright © 2009. Avnet, Inc. All rights reserved.
113Flow Tools Configuration
After changes of installed tool versions users should update locations of synthesis and implementation tools in the Tools | Preferences window, Environment | Flows | Integrated Tools category (both version and location of each tool can be modified)
Copyright © 2009. Avnet, Inc. All rights reserved.
114Tool/Version Selection
When Active-HDL installation is up-to-date, the latest Xilinx tool versions should be visible in flow configuration
Copyright © 2009. Avnet, Inc. All rights reserved.
115Showing the Flow Manager
Once enabled, Design Flow Manager can be displayed in Active-HDL using main toolbar button:
Displayed Design Flow Manager looks like this:
Copyright © 2009. Avnet, Inc. All rights reserved.
116Flow Modifications
If quick change of tool version or default Xilinx family is needed, it is possible in Flow Configuration Settings window displayed after clicking Flow Settings button
Copyright © 2009. Avnet, Inc. All rights reserved.
117Starting Coregen
To start Coregen from the Flow, user should click Tools button in the Design Flow Manager and select CoreGen & Architecture Wizard button in the popup window (see the illustration below)
Copyright © 2009. Avnet, Inc. All rights reserved.
118Coregen Interface Window
Active-HDL provides interface window that allows starting Coregen and managing generated cores
Users should adjust basic options before clicking Run CORE Generatorbutton (see illustration below)
Copyright © 2009. Avnet, Inc. All rights reserved.
119Selecting GTP in Coregen
Once Xilinx CORE Generator window shows, user should follow the typical steps to generate Spartan-6 GTP core
Be sure to select correct chip and save log before closing this window
Copyright © 2009. Avnet, Inc. All rights reserved.
120Importing Files to Active-HDL
After Xilinx CORE Generator window closing, generated IP will show up in the Active-HDL interface window
Users should select the IP, then click Add IP Core File to transfer files needed for simulation back to Active-HDL
Copyright © 2009. Avnet, Inc. All rights reserved.
121Other Options for PCIe Simulation
HDL models of PCIe are accurate, but slow
For increased simulation speed, Bus Functional Models (BFM) of PCIe are available as Verification IP
Aldec simulators are officially supported by many respectable IPvendors, including providers of PCIe cores:
– Northwest Logic with its “PCI Express Verification Suite”
– nSys with its “PCI Express nVS” (golden standard in PCIe verification)
To get more info, visit IP Products page in the Products section of our website
http://www.aldec.com/products/ipcoreshttp://www.aldec.com/products/ipcores
Copyright © 2009. Avnet, Inc. All rights reserved.
122Simulation
No matter if you are using VHDL, Verilog or both: our simulators will be able to handle your designIf you need to get result quickly – you have speedExpect performance matching leading competitors, but in easier to use packageIf you have to investigate some design issues –you have extensive debugging capabilities Happy Simulating!Happy Simulating!