Intel® FPGA P-Tile Avalon® memory mapped IP for PCI ... › content › dam › www › programmable › ...Master, Bursting Avalon memory mapped Slave and the data movers. • Support

Intel® FPGA P-Tile Avalon® memorymapped IP for PCI Express* UserGuide

Updated for Intel® Quartus® Prime Design Suite: 20.2

IP Version: 3.0.0

SubscribeSend Feedback

UG-20237 | 2020.07.13Latest document on the web: PDF | HTML

https://www.intel.com/content/www/us/en/programmable/bin/rssdoc?name=aib1557867923977

mailto:[email protected]?subject=Feedback%20on%20Intel%20FPGA%20P-Tile%20Avalon%20memory%20mapped%20IP%20for%20PCI%20Express%20User%20Guide%20(UG-20237%202020.07.13)&body=We%20appreciate%20your%20feedback.%20In%20your%20comments,%20also%20specify%20the%20page%20number%20or%20paragraph.%20Thank%20you.

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/ug/ug_ptile_pcie_avmm.pdf

https://www.intel.com/content/www/us/en/programmable/documentation/aib1557867923977.html

Contents

1. Introduction................................................................................................................... 41.1. Overview..............................................................................................................41.2. Features...............................................................................................................51.3. Release Information...............................................................................................61.4. Device Family Support............................................................................................71.5. Performance and Resource Utilization....................................................................... 7

2. IP Architecture and Functional Description.....................................................................92.1. Top-Level Architecture............................................................................................9

2.1.1. Avalon-MM Bridge Architecture.................................................................. 102.1.2. Clock Domains.........................................................................................122.1.3. Refclk.....................................................................................................132.1.4. Reset..................................................................................................... 15

2.2. Functional Description.......................................................................................... 162.2.1. PMA/PCS................................................................................................ 162.2.2. Data Link Layer Overview..........................................................................172.2.3. Transaction Layer Overview.......................................................................192.2.4. Avalon-MM Bridge.................................................................................... 20

3. Parameters................................................................................................................... 243.1. Top-Level Settings............................................................................................... 243.2. Core Parameters..................................................................................................25

3.2.1. Base Address Registers.............................................................................263.2.2. Device Identification Registers................................................................... 273.2.3. PCI Express and PCI Capabilities Parameters............................................... 273.2.4. Configuration, Debug and Extension Options................................................32

3.3. Avalon-MM Settings..............................................................................................32

4. Interfaces..................................................................................................................... 344.1. Overview............................................................................................................ 344.2. Clocks and Resets................................................................................................35

4.2.1. Interface Clock Signals............................................................................. 354.2.2. Interface Reset Signals............................................................................. 36

4.3. Avalon-MM Interface ........................................................................................... 374.3.1. Endpoint Mode Interface (512-bit Avalon-MM Interface)................................ 384.3.2. Root Port Mode Interface (256-bit Avalon-MM Interface)............................... 55

4.4. Serial Data Interface............................................................................................ 584.5. Hard IP Status Interface....................................................................................... 584.6. Interrupt Interface...............................................................................................59

4.6.1. Legacy Interrupts.................................................................................... 604.6.2. MSI........................................................................................................604.6.3. MSI-X.....................................................................................................63

4.7. Hot Plug Interface (RP Only)..................................................................................664.8. Power Management Interface................................................................................ 674.9. Configuration Output Interface.............................................................................. 684.10. Hard IP Reconfiguration Interface.........................................................................72

4.10.1. Address Map for the User Avalon-MM Interface...........................................744.10.2. Configuration Registers Access.................................................................76

Contents

Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide Send Feedback

2


4.11. PHY Reconfiguration Interface..............................................................................78

5. Advanced Features....................................................................................................... 805.1. PCIe Port Bifurcation and PHY Channel Mapping....................................................... 80

6. Troubleshooting/Debugging......................................................................................... 816.1. Hardware............................................................................................................81

6.1.1. Debugging Link Training Issues..................................................................826.1.2. Debugging Data Transfer and Performance Issues........................................ 88

6.2. Debug Toolkit...................................................................................................... 916.2.1. Overview................................................................................................ 916.2.2. Enabling the P-Tile Debug Toolkit............................................................... 926.2.3. Launching the P-Tile Debug Toolkit............................................................. 926.2.4. Using the P-Tile Debug Toolkit....................................................................956.2.5. Enabling the P-Tile Link Inspector.............................................................1066.2.6. Using the P-Tile Link Inspector................................................................. 107

A. Configuration Space Registers.................................................................................... 110A.1. Configuration Space Registers..............................................................................110

A.1.1. Register Access Definitions...................................................................... 112A.1.2. PCIe Configuration Header Registers.........................................................112A.1.3. PCI Express Capability Structures.............................................................114A.1.4. Physical Layer 16.0 GT/s Extended Capability Structure...............................116A.1.5. MSI-X Registers..................................................................................... 116

A.2. Intel-Defined VSEC Capability Registers................................................................ 117A.2.1. Intel-Defined VSEC Capability Header (Offset 00h)..................................... 118A.2.2. Intel-Defined Vendor Specific Header (Offset 04h)...................................... 119A.2.3. Intel Marker (Offset 08h)........................................................................ 119A.2.4. JTAG Silicon ID (Offset 0x0C - 0x18)........................................................ 119A.2.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D).......................119A.2.6. General Purpose Control and Status Register (Offset 0x30)..........................120A.2.7. Uncorrectable Internal Error Status Register (Offset 0x34)...........................120A.2.8. Uncorrectable Internal Error Mask Register (Offset 0x38)............................ 121A.2.9. Correctable Internal Error Status Register (Offset 0x3C)..............................122A.2.10. Correctable Internal Error Mask Register (Offset 0x40)..............................122

B. Document Revision History.........................................................................................123B.1. Document Revision History for the Intel FPGA P-Tile Avalon memory mapped IP for

PCI Express User Guide....................................................................................123

Contents

Send Feedback Intel® FPGA P-Tile Avalon® memory mapped IP for PCI Express* User Guide

3


1. Introduction

1.1. Overview

The P-Tile Avalon® memory mapped IP for PCIe combines the functionality of previousAvalon memory-mapped (Avalon-MM) and Avalon memory-mapped with directmemory access (DMA) interfaces. The IP core using the Avalon-MM interface removesmany of the complexities associated with the PCIe protocol. It handles all of theTransaction Layer Packet (TLP) encoding and decoding, simplifying the design task. Italso includes optional Read and Write Data Mover modules facilitating the creation ofhigh-performance DMA designs. Both the Avalon-MM interface and the Read and WriteData Mover modules are implemented in soft logic. This IP Core natively supportsEndpoint and Root Port configurations with Gen3/Gen4 data rates and x4/x8/x16 linkwidths. Gen1/Gen2 data rates and x1/x2 link widths are supported via link down-training.

The P-Tile Avalon memory mapped IP for PCIe consists of:

• Modules, implemented in soft logic, that perform Avalon memory mappedfunctions. Together, these modules form an Avalon memory mapped Bridge.

• A PCIe Hard IP that implements the Transaction, Data Link, and Physical layersstack that is compliant with PCI Express Base Specification 4.0 . This stack allowsthe user application logic in the Intel FPGA to interface with another device via aPCI Express link.

This IP provides support for an Avalon memory mapped interface with DMA and isdesigned to optimize the performance of large-size data transfers. If you want toachieve maximum performance with small-size transfers, Intel recommends the use ofthe P-Tile Avalon streaming IP for PCIe.

Note: The P-Tile Avalon memory mapped IP for PCIe does not include an internal descriptorcontroller for DMA operations. This descriptor controller should be implemented in theuser application logic. The design example provided for this IP includes an example ofa descriptor controller.

Related Information

Intel FPGA P-Tile Avalon memory mapped IP for PCI Express Design Example UserGuide

UG-20237 | 2020.07.13

Send Feedback

Intel Corporation. All rights reserved. Agilex, Altera, Arria, Cyclone, Enpirion, Intel, the Intel logo, MAX, Nios,Quartus and Stratix words and logos are trademarks of Intel Corporation or its subsidiaries in the U.S. and/orother countries. Intel warrants performance of its FPGA and semiconductor products to current specifications inaccordance with Intel's standard warranty, but reserves the right to make changes to any products and servicesat any time without notice. Intel assumes no responsibility or liability arising out of the application or use of anyinformation, product, or service described herein except as expressly agreed to in writing by Intel. Intelcustomers are advised to obtain the latest version of device specifications before relying on any publishedinformation and before placing orders for products or services.*Other names and brands may be claimed as the property of others.

ISO9001:2015Registered

https://www.intel.com/content/www/us/en/programmable/documentation/uro1573085463987.html

https://www.intel.com/content/www/us/en/programmable/documentation/uro1573085463987.html


https://www.intel.com/content/www/us/en/quality/intel-iso-registrations.html



1.2. Features

The P-Tile Avalon memory mapped IP for PCI Express supports the following features:

• Configurations supported:

Table 1. Configurations Supported by the P-Tile Avalon memory mapped IP for PCIExpress

Gen3/Gen4 x16 Gen3/Gen4 x8 Gen3/Gen4 x4

Endpoint (EP) Yes Yes N/A

Root Port (RP) (1) N/A Yes

Note: Gen1/Gen2 configurations are supported via link down-training.

• Support for 256-bit and 512-bit data paths.

• 512-bit data path with 250 MHz interfaces to user logic to ease timing closure forGen3 x16.

• Support for a single function (PF0).

• High-throughput Bursting Avalon memory mapped Slave (BAS).

— Byte enables with byte granularity.

• High-throughput Bursting Avalon memory mapped Master (BAM).

• Support for up to 7 BARs, including expansion ROM BAR.

• Support for byte enables with byte granularity.

• Support for up to 64 outstanding Non-Posted requests.

• Summary of outstanding Non-Posted requests supported:

Table 2. Outstanding Non-Posted Requests Supported

Ports Active Cores Outstanding Non-Posted Requests

0 x16 64, 512 (*)

1 x8 64

2 and 3 x4 64

Note: (*) : 512 outstanding Non-Posted requests support may be available in afuture Intel Quartus Prime release.

• Data movers with high throughput for DMA support

— Move data using PCIe Memory Read and Memory Write packets.

— Bursting Avalon memory mapped Master interfaces for data path.

— Byte enables with dword granularity.

— Avalon streaming interfaces for control and status.

— DMA transfers of 1 dword to (1 MB - 1 dword) in 1 dword increments.

— All addresses are dword-aligned.

(1) These configurations may be available in a future release of Intel® Quartus® Prime.

1. Introduction

UG-20237 | 2020.07.13


5


• Bursts of up to 8 cycles (512 bytes) for the Bursting Avalon memory mappedMaster, Bursting Avalon memory mapped Slave and the data movers.

• Support for Max Payload Size values of 128, 256 and 512 bytes.

• Support for Max Read Request Size values of 128, 256 and 512 bytes.

• Available as a Platform Designer component with standard Avalon interfaces.

• MSI and MSI-X.

• Separate Refclk with Independent Spread Spectrum Clocking (SRIS).

• You cannot change the pin allocations for the P-Tile Avalon memory mapped IP forPCI Express* in the Intel Quartus Prime project. However, this IP does supportlane reversal and polarity inversion on the PCB.

• Supports Autonomous Hard IP mode.

— This mode allows the PCIe Hard IP to communicate with the Host before theFPGA configuration and entry into User mode are complete.

• Modular implementation allowing users to enable the required features for aspecific application. For example:

— Simultaneous support for DMA modules and high-throughput Avalon memorymapped Slaves and Masters.

— Avalon memory mapped Slave for easy access to the whole PCIe addressspace.

• VCS is the only simulator supported in the 20.2 release of Intel Quartus Prime.Other simulators may be supported in a future release.

Note: Throughout this User Guide, the term Avalon-MM may be used as an abbreviation forthe Avalon memory mapped interface or IP.

1.3. Release Information

Table 3. P-Tile Avalon memory mapped IP for PCI Express Release Information

Item Description

IP Version 3.0.0

Intel Quartus Prime Version 20.2

Release Date June 2020

Ordering Codes No ordering code is required

IP versions are the same as the Intel Quartus Prime Design Suite software versions upto v19.1. From Intel Quartus Prime Design Suite software version 19.2 or later, IPshave a new IP versioning scheme.

The IP version (X.Y.Z) number may change from one Intel Quartus Prime softwareversion to another. A change in:

1. Introduction

UG-20237 | 2020.07.13


6


• X indicates a major revision of the IP. If you update your Intel Quartus Primesoftware, you must regenerate the IP.

• Y indicates the IP includes new features. Regenerate your IP to include these newfeatures.

• Z indicates the IP includes minor changes. Regenerate your IP to include thesechanges.

Intel verifies that the current version of the Intel Quartus Prime Pro Edition softwarecompiles the previous version of each IP core, if this IP core was included in theprevious release. Intel reports any exceptions to this verification in the Intel IPRelease Notes or clarifies them in the Intel Quartus Prime Pro Edition IP Update tool.Intel does not verify compilation with IP core versions older than the previous release.

Related Information

P-Tile IP for PCI Express IP Core Release NotesThis document provides information about the new features and updates for eachIP release.

1.4. Device Family Support

The following terms define device support levels for Intel FPGA IP cores:

• Advance support—the IP core is available for simulation and compilation for thisdevice family. Timing models include initial engineering estimates of delays basedon early post-layout information. The timing models are subject to change assilicon testing improves the correlation between the actual silicon and the timingmodels. You can use this IP core for system architecture and resource utilizationstudies, simulation, pinout, system latency assessments, basic timing assessments(pipeline budgeting), and I/O transfer strategy (data-path width, burst depth, I/Ostandards tradeoffs).

• Preliminary support—the IP core is verified with preliminary timing models forthis device family. The IP core meets all functional requirements, but might still beundergoing timing analysis for the device family. It can be used in productiondesigns with caution.

• Final support—the IP core is verified with final timing models for this devicefamily. The IP core meets all functional and timing requirements for the devicefamily and can be used in production designs.

Table 4. Device Family Support

Device Family Support Level

Intel Stratix® 10 DX Preliminary support

Intel Agilex™ Preliminary support

Other device familiesNo supportRefer to the Intel PCI Express Solutions web page on the Intel website for support information onother device families.

1.5. Performance and Resource Utilization

The Avalon-MM variants include an Avalon-MM DMA bridge implemented in soft logic.It operates as a front end to the hardened protocol stack. The resource utilizationtable below shows results for the Simple DMA dynamically generated design example.

1. Introduction

UG-20237 | 2020.07.13


7

https://www.intel.com/content/www/us/en/programmable/documentation/fny1569358723952.html


The results are for the current version of the Intel Quartus Prime Pro Edition software.

Table 5. Resource Utilization of the Avalon-MM IP for PCI Express IP Core

DesignExample Used

LinkConfiguration

Device Family Typical ALMs M20K MemoryBlocks(2)

Logic Registers

DMA Gen3 x16, EP Intel Stratix 10DX 15956 120 42345

DMA Gen3 x16, EP Intel Agilex 17116 120 42940





RP Gen3 x4, RP Intel Stratix 10DX 61339 172 133999

RP Gen3 x4, RP Intel Agilex 62185 172 113475

RP Gen4 x4, RP Intel Stratix 10DX 61467 172 134365

RP Gen4 x4, RP Intel Agilex 61878 172 118199

(2) These results only include resources in the Avalon-MM IP partition in the design example asthe Table title indicates. They do not include resources for external blocks such as the on-chipmemory, DMA controller, and other interconnect logic.

1. Introduction

UG-20237 | 2020.07.13


8


2. IP Architecture and Functional Description

2.1. Top-Level Architecture

The P-tile Avalon-MM IP for PCI Express consists of the following major sub-blocks:

• PMA/PCS

• Four PCIe* cores (one x16 core, one x8 core and two x4 cores)

• Embedded Multi-die Interconnect Bridge (EMIB)

• Soft logic blocks in the FPGA fabric to implement the Avalon-MM Bridge, whichtranslates the PCIe TLPs from the PCIe Hard IP into standard Avalon memory-mapped reads and writes.

Figure 1. P-tile Avalon-MM IP for PCI Express top-level block diagram

PHY P-Tile

P-Tile Avalon-MM PCIe IP Top Level

FPGA Fabric

PCIe x16 Lanes

PCIe ControllersPMA Quad 3PLL A/B

PMA Quad 2PLL A/B

PMA Quad 1PLL A/B

PMA Quad 0PLL A/B

x16 PCIe PCS

Bifurcation Mux

Avalon -MM

Bridge

User Logic

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x4

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x4

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x8

DataLink

Layer

Trans- actionLayer

PHYLayer(MAC)

x16

EMIB

refclk0 refclk1

pin_perst_n

Note: Each core in the IP implements its own Data Link Layer and Transaction Layer.

The four cores in the IP can be configured to support the following topologies:

Table 6. Configuration Modes Supported by the P-tile Avalon-MM IP for PCI Express

Configuration Mode Native Hard IP ModeEndpoint(EP) / RootPort (RP)

Active Cores

Configuration Mode 0 Gen3x16 or Gen4x16 EP/RP x16

Configuration Mode 1 Gen3x8/Gen3x8 or Gen4x8/Gen4x8 EP x16, x8

Configuration Mode 2 Gen3x4/Gen3x4/Gen3x4/Gen3x4 orGen4x4/Gen4x4/Gen4x4/Gen4x4 RP x16, x8, x4_0, x4_1

UG-20237 | 2020.07.13

Send Feedback







In Configuration Mode 0, only the x16 core is active, and it operates in Gen3 x16mode or Gen4 x16 mode.

In Configuration Mode 1, the x16 core and x8 core are active, and they operate as twoGen3 x8 cores or two Gen4 x8 cores.

In Configuration Mode 2, all four cores (x16, x8, x4_0, x4_1) are active, and theyoperate as four Gen3 x4 cores or four Gen4 x4 cores.

2.1.1. Avalon-MM Bridge Architecture

The P-Tile Avalon-MM Bridge can support three modes of operation:

• Endpoint mode with Data Movers.

• Endpoint mode.

• Root Port mode.

In the first two modes, the P-Tile Avalon-MM IP functions as an Endpoint (EP). In RootPort mode, it functions as a Root Port (RP).

The Avalon-MM Bridge consists of five main modules: Read Data Mover (RDDM), WriteData Mover (WRDM), Bursting Avalon-MM Master (BAM), Bursting Avalon-MM Slave(BAS) and Control Register Access (CRA). These modules are shown in Figure 2 onpage 11 and described below. Depending on the mode of operation, differentmodules in the IP core are enabled.

Table 7. Operating Modes of the Avalon-MM BridgeIn the following table, Yes means the block is enabled for that operating mode. No means the block is notenabled for that mode.

Modes

Modules

Read DataMover

(RDDM)

Write DataMover

(WRDM)

Bursting Avalon-MMMaster (BAM)

Bursting Avalon-MMSlave (BAS)

Control RegisterAccess (CRA)Non-

BurstingMode

BurstingMode

Non-Bursting

Mode

BurstingMode

Endpointmode with

Data Movers(EP)

Yes Yes Yes No No No No

Endpointmode (EP) No No Yes No No Yes No

Root Portmode (RP) No No No Yes Yes No Yes


UG-20237 | 2020.07.13


10


Here is a block diagram of the P-Tile Avalon-MM Bridge showing the main modules:

Figure 2. P-Tile Avalon-MM Bridge Block Diagram

512 512

512 512

Avalon ConduitAvalon-ST Sink

Avalon-ST Source

Avalon-MM Master

Avalon-MM Slave

Hard

IP In

terfa

ce

Response Re-ordering

Embe

dded

/Sep

arat

edHe

ader

Adap

tor

Avalon-MM Bridge

O

I

BurstingMaster

ReadData Mover

O

I

IIOM

WriteData Mover O

IIOM

Control RegisterAccess O

IS

M

BurstingSlave

OOI

S

P-Tile Avalon-MM IP for PCIe M

S

I

O

C

250 MHz

500 MHz

P-Til

e PCle

Har

d IP

P-Til

e PCle

Avalo

n-ST

Widt

h and

Rate

Adap

tor


UG-20237 | 2020.07.13


11


• Bursting Master (BAM): This module converts memory read and write TLPsinitiated by the remote link partner and received over the PCIe link into Avalon-MM burst read and write transactions, and sends back CplD TLPs for read requestsit receives. It can also function in a non-bursting mode.

• Bursting Slave (BAS): This module converts Avalon-MM read and writetransactions initiated by the application logic into PCIe memory read and writeTLPs to be transmitted over the PCIe link. This module also processes the CplDTLPs received for the read requests it sent. It can also function in a non-burstingmode.

• Read Data Mover (RDDM): This module uses PCIe memory read TLPs and Avalon-MM write transactions to move large amounts of data from the system memory inthe PCIe space to the FPGA memory in the Avalon-MM space. It fetchesdescriptors from the system memory through one of its two Avalon-ST sinkinterfaces. These descriptors define the data transfers to be executed. The RDDMalso reports the status of these data transfers via its Avalon-ST source interface.

• Write Data Mover (WRDM): This module uses PCIe memory write TLPs andAvalon-MM read transactions to move large amounts of data from your applicationlogic in the Avalon-MM space to the system memory in the PCIe space. The WRDMalso supports immediate writes, which are enabled by a bit in the descriptors thatthe WRDM receives via one of its Avalon-ST descriptor sink interfaces. For moredetails on immediate writes, refer to Write Data Mover Avalon-ST Descriptor Sinkson page 49. Similar to the RDDM, the WRDM also has its own Avalon-ST sourceinterface to report the status of its data transfers.

• Control Register Access (CRA) Avalon-MM Slave (Root Port only): This module isused in Root Port mode only to issue accesses to the Endpoint's configurationspace registers. It supports a single transaction at a time. It converts single-cycle,32-bit Avalon-MM read and write transactions into PCIe configuration read andwrite TLPs (CfgRd0, CfgRd1, CfgWr0 and CfgWr1) to be sent over the PCIe link.This module also processes the completion TLPs (Cpl and CplD) it receives inreturn.

The Response Reordering module assembles and reorders completion TLPs receivedover the PCIe link for the Bursting Slave and the Read Data Mover. It routes thecompletions based on their tags.

No re-ordering is necessary for the completions sent to the CRA module as it onlyissues one request TLP at a time.

Endpoint applications typically need the Bursting Master to enable the host to provideinformation for the other modules.

2.1.2. Clock Domains

The P-Tile Avalon-MM IP for PCI Express has three primary clock domains:

• PHY clock domain (i.e. core_clk domain): this clock is synchronous to theSerDes parallel clock.

• EMIB/FPGA fabric interface clock domain (i.e. pld_clk domain): this clock isderived from the same reference clock (refclk0) as the one used by the SerDes.However, this clock is generated from a stand-alone core PLL.

• Application clock domain (p<n>_app_clk): this clock is an output from the P-TileIP, and it has the same frequency as pld_clk. This is a per-port signal (i.e, n =0,1,2,3).


UG-20237 | 2020.07.13


12


Figure 3. Clock Domains

User Logic

Avalon- MM Bridge

FPGA Fabric P-Tile

EMIB PCIe Hard IP

x16 core_clk

x8 core_clk

x4_0 core_clk

x4_1 core_clk

pld_clk

p<n>_app_clk

PCS

PMA

The PHY clock domain (i.e. core_clk domain) is a dynamic frequency domain. ThePHY clock frequency is dependent on the current link speed.

Table 8. PHY Clock and Application Clock Frequencies

Link Speed PHY Clock Frequency Application Clock Frequency

Gen1 125 MHz

Gen1 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP

Parameter Editor.

Gen2 250 MHz

Gen2 is supported only via link down-training andnot natively. Hence, the application clock frequencydepends on the configuration you choose in the IP

Parameter Editor.

Gen3 500 MHz 250 MHz

Gen4 1000 MHz350 MHz / 400 MHz (Intel Stratix 10 DX)

350 MHz / 400 MHz / 500 MHz (Intel Agilex)

Note: Refer to the P-Tile IP for PCI Express IP Core Release Notes for the matrix ofconfigurations supported by the P-Tile Avalon memory mapped IP for PCI Express.

2.1.3. Refclk

P-Tile has two reference clock inputs at the package level, refclk0 and refclk1.You must connect a 100 MHz reference clock source to these two inputs. Dependingon the port mode, you can drive the two refclk inputs using either a single clocksource or two independent clock sources.

In 1x16 and 4x4 modes, drive the refclk inputs with a single clock source (througha fanout buffer) as shown in the figure below.


UG-20237 | 2020.07.13


13


Figure 4. Using a Single 100 MHz Clock Source in 1x16 and 4x4 Modes

PCIe x4

(Port 3)

PHY3(x4)La

ne 15

Lane

12 PHY2(x4)La

ne 11

Lane

8 PHY1(x4)La

ne 7

Lane

4 PHY0(x4)La

ne 3

Lane

0

PCIe x8/x4

(Port 1)

Refclk distribution on the package substrate

Refclk1 Refclk0

Fanout Buffer

100MHz

PCIe x16/x8/x4

(Port 0)

PCIe x4

(Port 2)

In 2x8 mode, you can drive the refclk inputs with either a single 100 MHz clocksource as shown above, or two independent 100 MHz sources (see the figure below)depending on your system architecture. For example, if your system has each x8 portconnected to a separate CPU/Root Complex, it may be required to drive these refclkinputs using independent clock sources. In that case, it is strongly recommended thatthe refclk0 input for Port 0 (lanes 0 - 7) be always running because it feeds thereference clock for the P-Tile core PLL that controls the data transfers between the P-Tile and FPGA fabric via the EMIB. If this clock goes down, Port 0 link will go down andPort 1 will not be able to communicate with the FPGA fabric. Following are theguidelines for implementing two independent refclks in 2x8 mode:

• If the link can handle two separate reference clocks, drive the refclk0 of P-Tilewith the on-board free-running oscillator.

• If the link needs to use a common reference clock, then PERST# needs to indicatethe stability of this reference clock. If this reference clock goes down, the entire P-Tile must be reset.


UG-20237 | 2020.07.13


14


Figure 5. Using Independent 100 MHz Clock Sources in 2x8 Mode

PCIe x4

(Port 3)

PHY3(x4)La

ne 15

Lane

12 PHY2(x4)La

ne 11

Lane

8 PHY1(x4)La

ne 7

Lane

4 PHY0(x4)La

ne 3

Lane

0

PCIe x8/x4

(Port 1)

Refclk distribution on the package substrate

Refclk1 Refclk0100MHz

PCIe x16/x8/x4

(Port 0)

PCIe x4

(Port 2)

100MHz

2.1.4. Reset

There is only one PERST# (pin_perst_n) pin on P-Tile. Therefore, togglingpin_perst_n will affect the entire P-Tile. If the P-Tile x16 port is bifurcated into twox8 Endpoints, toggling pin_perst_n will affect both x8 Endpoints. To reset each portindividually, use the in-band mechanism such as Hot Reset and the Function-LevelReset (FLR). Following are the guidelines for implementing the P-Tile pin_perst_nreset signal:

• pin_perst_n is a "power good" indicator from the associated power domain (towhich P-Tile is connected). Also, it shall qualify that both the P-Tile refclk0 andrefclk1 are stable. If one of the reference clocks becomes stable later, deassertpin_perst_n after this reference clock becomes stable.

• pin_perst_n assertion is required for proper Autonomous P-Tile functionality. InAutonomous mode, P-Tile can successfully link up upon the release ofpin_perst_n regardless of the FPGA fabric configuration and will send out CRS(Configuration Retry Status) until the FPGA fabric is configured and ready.

The following is an example where a single PERST# (pin_perst_n) is driven withindependent refclk0 and refclk1. In this example, the add-in card (FPGA and Soc)is powered up first. P-Tile refclk0 is fed by the on-board free-running oscillator. P-Tile refclk1 driven by the Host becomes stable later. Hence, the PERST# isconnected to the Host.


UG-20237 | 2020.07.13


15


Figure 6. Single PERST# Connection in Bifurcated 2x8 Mode

2.2. Functional Description

2.2.1. PMA/PCS

The P-Tile Avalon-MM IP for PCI Express contains Physical Medium Attachment (PMA)and PCI Express Physical Coding Sublayer (PCIe PCS) blocks for handling the Physicallayer (PHY) packets. The PMA receives and transmits high-speed serial data on theserial lanes. The PCS acts as an interface between the PMA and the PCIe controller,and performs functions like data encoding and decoding, scrambling anddescrambling, block synchronization etc. The PCIe PCS in the P-Tile Avalon-MM IP forPCI Express is based on the PHY Interface for PCI Express (PIPE) Base Specification4.4.1.

In this IP, the PMA consists of up to four quads. Each quad contains a pair of transmitPLLs and four SerDes lanes capable of running up to 16 GT/s to perform the variousTX and RX functions.

PLLA generates the required transmit clocks for Gen1/Gen2 speeds, while PLLBgenerates the required clocks for Gen3/Gen4 speeds. For the x8 and x16 lane widths,one of the quads acts as the master PLL source to drive the clock inputs for each ofthe lanes in the other quads.

The PMA performs functions such as serialization/deserialization, clock data recovery,and analog front-end functions such as Continuous Time Linear Equalizer (CTLE),Decision Feedback Equalizer (DFE) and transmit equalization.

The transmitter consists of a 3-tap equalizer with one tap of pre-cursor, one tap ofmain cursor and one tap of post-cursor.

The receiver consists of attenuation (ATT), CTLE, Voltage gain amplifier (VGA) and a5-tap DFE blocks that are adaptive for Gen3/Gen4 speeds. RX Lane Margining issupported by the PHY. The Lane Margining supports timing margining only. Theoptional voltage margining is not supported. Timing margining capabilities/parametersare as follows:

• Maximum Timing Offset: -0.2UI to +0.2UI.

• Number of timing steps in each direction: 9.

• Independent left and right timing margining is supported.

• Independent Error Sampler is not supported (lane margining may produce logicalerrors in the data stream and cause the LTSSM to go to the Recovery state).


UG-20237 | 2020.07.13


16


The PHY layer uses a fixed 16-bit PCS-PMA interface width to output the PHY clock(core_clk). The frequency of this clock is dependent on the current link speed. Referto Table 8 on page 13 for the frequencies at various link speeds.

Related Information

PHY Interface for PCI Express (PIPE) Base Specification

2.2.2. Data Link Layer Overview

The Data Link Layer (DLL) is located between the Transaction Layer and the PhysicalLayer. It maintains packet integrity and communicates (by DLL packet transmission) atthe PCI Express link level.

The DLL implements the following functions:

• Link management through the reception and transmission of DLL Packets (DLLP),which are used for the following functions:

— Power management of DLLP reception and transmission

— To transmit and receive ACK/NAK packets

— Data integrity through the generation and checking of CRCs for TLPs andDLLPs

— TLP retransmission in case of NAK DLLP reception or replay timeout, using theretry (replay) buffer

— Management of the retry buffer

— Link retraining requests in case of error through the Link Training and StatusState Machine (LTSSM) of the Physical Layer

Figure 7. Data Link LayerTo Transaction Layer

Tx Transaction LayerPacket Description & Data Transaction Layer

Packet Generator

Retry Buffer

To Physical Layer

Tx Packets

Ack/NackPackets

RX Datapath

TX Datapath

Rx Packets

DLLPChecker

Transaction LayerPacket Checker

DLLPGenerator

Tx Arbitration

Data Link Controland Management

State Machine

Control& StatusConfiguration Space

Tx Flow Control Credit Information

Rx Flow Control Credit Information

Rx Transation LayerPacket Description & Data

PowerManagement

Function

Note:(1) The L0s (Standby) or L1 (Low Power Standby) states are not supported.


UG-20237 | 2020.07.13


17

https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/phy-interface-pci-express-sata-usb30-architectures-3.1.pdf


The DLL has the following sub-blocks:

• Data Link Control and Management State Machine—This state machine connects toboth the Physical Layer’s LTSSM state machine and the Transaction Layer. Itinitializes the link and flow control credits and reports status to the TransactionLayer.

• Power Management—This function handles the handshake to enter low powermode. Such a transition is based on register values in the Configuration Space andreceived Power Management (PM) DLLPs. For more details on the power statessupported by the P-Tile Avalon-MM IP for PCIe, refer to section Power ManagementInterface on page 67.

• Data Link Layer Packet Generator and Checker—This block is associated with theDLLP’s 16-bit CRC and maintains the integrity of transmitted packets.

• Transaction Layer Packet Generator—This block generates transmit packets,including a sequence number and a 32-bit Link CRC (LCRC). The packets are alsosent to the retry buffer for internal storage. In retry mode, the TLP generatorreceives the packets from the retry buffer and generates the CRC for the transmitpacket.

• Retry Buffer—The retry buffer stores TLPs and retransmits all unacknowledgedpackets in the case of NAK DLLP reception. In case of ACK DLLP reception, theretry buffer discards all acknowledged packets.

• ACK/NAK Packets—The ACK/NAK block handles ACK/NAK DLLPs and generates thesequence number of transmitted packets.

• Transaction Layer Packet Checker—This block checks the integrity of the receivedTLP and generates a request for transmission of an ACK/NAK DLLP.

• TX Arbitration—This block arbitrates transactions, prioritizing in the followingorder:

— Initialize FC Data Link Layer packet

— ACK/NAK DLLP (high priority)

— Update FC DLLP (high priority)

— PM DLLP

— Retry buffer TLP

— TLP

— Update FC DLLP (low priority)

— ACK/NAK FC DLLP (low priority)


UG-20237 | 2020.07.13


18


2.2.3. Transaction Layer Overview

The following figure shows the major blocks in the P-Tile Avalon-MM IP for PCI ExpressTransaction Layer:

Figure 8. P-Tile Avalon-MM IP for PCI Express Transaction Layer Block Diagram

Data Link Layer +Physical Layer

RAS

RX

CONFIG

TX

Avalon-ST RX

User Avalon-MM

Avalon-ST TX

RASCPL TimeoutLogic

Avalon-MM TX

Avalon-MM RX

Avalo

n-M

M Br

idge

The RAS (Reliability, Availability, and Serviceability) block includes a set of features tomaintain the integrity of the link.

For example: Transaction Layer inserts an optional ECRC in the transmit logic andchecks it in the receive logic to provide End-to-End data protection.

When the application logic sets the TLP Digest (TD) bit in the Header of the TLP, the P-Tile Avalon-MM IP for PCIe will append the ECRC automatically.

The TX block sends out the TLPs that it receives as-is. It also sends the informationabout non-posted TLPs to the Completion (CPL) Timeout Block for CPL timeoutdetection.

The P-Tile Avalon-MM IP for PCI Express RX block consists of two main blocks:

• Filtering block: This module checks if the TLP is good or bad and generates theassociated error message and completion. It also tracks received completions andupdates the completion timeout (CPL timeout) block.

• RX Buffer Queue: The P-Tile IP for PCIe has separate queues for posted/non-posted transactions and completions. This avoids head-of-queue blocking on thereceived TLPs and provides flexibility to extract TLPs according to the PCIeordering rules.


UG-20237 | 2020.07.13


19


Figure 9. P-Tile Avalon-MM IP for PCI Express RX Block Overview

Avalon-ST

RX Buffer Queue Filter

TLP Filtering

Config TX

MSG ERR

CFG Data

Received CPLProcessing (*)

MessageProcessingCPL

NP

P

MSG

DataLinkLayer

Trash

Avalon-ST

User Avalon-MM

RoutingAvalon-MM Avalon-STLogical

PHYLayer

Avalon-MM

Bridge

WidthandRate

Adapter

Note: The Received CPL Processing block includes the CPL tracking mechanism.

2.2.4. Avalon-MM Bridge

As described in Table 7 on page 10, the P-Tile Avalon-MM Bridge can support threemodes of operation:

• Endpoint mode with Data Movers.

• Endpoint mode.

• Root Port mode.

2.2.4.1. Endpoint Mode with Data Movers

When the Avalon-MM Bridge is used in this mode, the following modules are enabled:

• Read Data Mover (RDDM)

• Write Data Mover (WRDM)

• Bursting Master (BAM) in Non-Bursting Mode

The following figure shows how the DMA example design that you can generate usingthe Intel Quartus Prime software interfaces with the P-Tile Avalon-MM IP to performDMA operations. If you are not using the provided DMA example design, you need toimplement your custom DMA Controller and BAR Interpreter in your application logic.


UG-20237 | 2020.07.13


20


Figure 10. P-Tile Avalon-MM IP in Endpoint Mode with Data Movers Enabled

DMA Example Design P-Tile Avalon-MM IP

Qsys Interconnect

Memory

Writedata mover(512 bits)

S

S

S

SC

M

WASTOASTO

ASTIASTIASTO

M

Readdata mover(512 bits) ASTI

ASTOASTIASTIASTO

M

Bursting Master(non-bursting

mode)(512 bits)

ASTI

ASTOCM

WASTOWASTI

RASTORASTORASTI

DMA Controller

BARInterpreter

CompletionRe-ordering

TX

RX

Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512

Avalon-ST SinkASTI

ASTO Avalon-ST Source

M Avalon-MM Master

S Avalon-MM Slave

C Avalon Conduit

In this DMA example design, the BAM is used in non-bursting mode by the host toprogram the Control and Status registers of the DMA controller in the user Avalon-MMspace. The DMA controller, after being programmed, sends descriptor-fetchinginstructions to the host via the RDDM. After the fetched descriptors are processed bythe WRDM and RDDM, status and/or MSI-X messages are sent to the host via theWRDM in “Immediate” mode. In this mode, the data payload is embedded in bits[31:0] or [63:0] of the fetched descriptors that the WRDM receives (depending onwhether a one- or two-dword immediate transfer is needed respectively). For moredetails on immediate transfers, refer to Write Data Mover Avalon-ST Descriptor Sinkson page 49.

The RDDM uses PCIe memory read TLPs and Avalon-MM write transactions (which canbe bursting transactions) to move large amounts of data from the host memory inPCIe space to the local FPGA memory in Avalon-MM space. On the other hand, theWRDM uses PCIe memory write TLPs and Avalon-MM read transactions to move largeamounts of data from the FPGA memory in Avalon-MM space to the host memory inPCIe space. The Data Movers' transfers are controlled by descriptors that are providedto the Data Movers through one of their Avalon-ST sink interfaces. The Data Moversreport the transfers’ status through their Avalon-ST source interfaces.

2.2.4.2. Endpoint Mode

In this mode, the external master (in user logic) sends memory reads and writesupstream via the Bursting Slave. The following modules are enabled:

• Bursting Slave (in bursting mode)

• Bursting Master (in non-bursting mode)


UG-20237 | 2020.07.13


21


Figure 11. P-Tile Avalon-MM IP in Endpoint Mode

DMA Application Logic P-Tile Avalon-MM IP

Qsys Interconnect

S Memory

S

M Custom DMA

ControllerM

SC

MBursting Master (non-bursting

mode)(512 bits)

CM

BurstingSlave

(512 bits) ASTI

ASTO

ASTI

ASTO

S

DMA Control/

BAR AccessLogic


Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512

TX

RX

ASTI Avalon-ST Sink


M Avalon-MM Master

S Avalon-MM Slave

C Avalon Conduit

The external Avalon-MM master can be a custom DMA controller that uses theBursting Slave in the IP core to send memory reads and writes upstream. Thesememory reads and writes can be up to 512-bytes long. The reordering buffer in the IPcore reorders the Completion TLPs received over the PCIe link and sends them to theBursting Slave.

The Bursting Master provides the host with access to the registers and memory in theAvalon-MM address space of the FPGA. It converts PCIe memory reads and writes toAvalon-MM reads and writes.

Registers in the custom DMA controller can be programmed by software via theBursting Master port.

2.2.4.3. Root Port Mode

In this mode, the IP core needs to be able to process memory read and write TLPscoming from the DMA controller that resides on the Endpoint side. The followingmodules are enabled:

• Bursting Master (in bursting and non-bursting modes)

• Bursting Slave (in non-bursting mode)

• Control Register Access

Figure 12. P-Tile Avalon-MM IP in Root Port Mode

Custom DMA Application Logic P-Tile Avalon-MM IP

Qsys Interconnect

S Memory

SLocal

ProcessorM

M

SC

MBurstingMaster

(512 bits) ASTI

ASTOCM

Control RegisterAccess

(512 bits) ASTI

ASTOS

Bursting Slave(non-bursting

mode)(512 bits)

ASTI

ASTOS

S

S

BurstingSlave


Avalon-ST Interface

P-Tile PCIe

Hard IP

512

512RX

TX


ASTI Avalon-ST Sink

M Avalon-MM Master

S Avalon-MM Slave

C Avalon Conduit


UG-20237 | 2020.07.13


22


The IP core must be able to generate and process configuration reads and writes tothe Endpoint and to the Hard IP configuration registers. This is done via theConfiguration Slave. Since the DMA controller resides on the Endpoint side, its controlregisters need to be programmed by the FPGA local processor. Using the BurstingSlave (in non-bursting mode), the local processor can program the Endpoint controlregisters for DMA operations. The Endpoint can also send updates of its DMA status tothe local processor via the Bursting Master.


UG-20237 | 2020.07.13


23


3. ParametersThis chapter provides a reference for all the parameters that are configurable in theIntel Quartus Prime IP Parameter Editor for the P-Tile Avalon-MM IP for PCIe.

3.1. Top-Level Settings

Table 9. Top-Level Settings

Parameter Value Default Value Description

Hard IP Mode

Gen4x16, Interface - 512-bitGen3x16, Interface - 512-bitGen4x8, Interface - 256-bitGen3x8, Interface - 256-bitGen4x4, Interface - 128-bitGen3x4, Interface - 128-bit

Gen3x16,Interface - 512-bit

Select the lane data rate and lanewidth.

Note:

Refer to the P-Tile IP forPCI Express IP CoreRelease Notes for thematrix of configurationssupported by the P-TileAvalon memory mapped IPfor PCI Express.

Port ModeRoot PortNative Endpoint

Native Endpoint Specifies the port type.

Enable PHY Reconfiguration True/False False Enable the PHY ReconfigurationInterface.

PLD Clock Frequency

500 MHz400 MHz350 MHz250 MHz

250 MHz

Select the frequency of theApplication clock.

Note:

400 MHz / 350 MHz are forGen4 support in IntelStratix 10 DX. 500 MHz /400 MHz / 350 MHz are forGen4 support in IntelAgilex. 250 MHz is forGen3 support.

Enable SRIS Mode True/False FalseEnable the Separate ReferenceClock with Independent SpreadSpectrum Clocking (SRIS) feature.

P-Tile Sim Mode True/False False

Enabling this parameter willenable P-Tile simulations to runfaster.

Note: Do not enable this option ifyou need to run synthesis.

UG-20237 | 2020.07.13

Send Feedback







The following figure shows how to enable Root Port mode:

Figure 13. Intel P-Tile Avalon-MM Top-Level IP Parameter Editor for a Gen3x4 Hard IPin Root Port Mode

3.2. Core Parameters

Depending on which Hard IP Mode you choose in the Top-Level Settings tab, youwill see different tabs for setting the core parameters.

Figure 14. Intel P-Tile Avalon-MM Top-Level IP Parameter Editor for a x8 Hard IP ModeIf you choose a x8 mode (either Gen4 or Gen3), the PCIe0 Settings and PCIe1 Settings tabs will appear.

3. Parameters

UG-20237 | 2020.07.13


25


3.2.1. Base Address Registers

Table 10. BAR Registers

Parameter Value Description

BAR0 Type

Disabled64-bit prefetchable memory64-bit non-prefetchable memory32-bit non-prefetchable memory32-bit prefetchable memory

If you select 64-bit prefetchable memory, 2contiguous BARs are combined to form a 64-bitprefetchable BAR; you must set the highernumbered BAR to Disabled.Defining memory as prefetchable allows contiguousdata to be fetched ahead. Prefetching memory isadvantageous when the requestor may requiremore data from the same region than wasoriginally requested. If you specify that a memoryis prefetchable, it must have the following 2attributes:• Reads do not have side effects such as

changing the value of the data read.• Write merging is allowed.

BAR1 TypeDisabled32-bit non-prefetchable memory32-bit prefetchable memory

For a definition of prefetchable memory, refer tothe BAR0 Type description.

BAR2 Type


For a definition of prefetchable memory and adescription of what happens when you select the64-bit prefetchable memory option, refer to theBAR0 Type description.



BAR4 Type


For a definition of prefetchable memory and adescription of what happens when you select the64-bit prefetchable memory option, refer to theBAR0 Type description.



BARn Size 128 Bytes - 16 EBytesSpecifies the size of the address space accessibleto BARn when BARn is enabled.n = 0, 1, 2, 3, 4 or 5

Expansion ROM

Disabled4 KBytes - 12 bits8 KBytes - 13 bits16 KBytes - 14 bits32 KBytes - 15 bits64 KBytes - 16 bits128 KBytes - 17 bits256 KBytes - 18 bits512 KBytes - 19 bits1 MByte - 20 bits2 MBytes - 21 bits4 MBytes - 22 bits8 MBytes - 23 bits16 MBytes - 24 bits

Specifies the size of the expansion ROM from 4KBytes to 16 MBytes when enabled.

3. Parameters

UG-20237 | 2020.07.13


26


3.2.2. Device Identification Registers

The following table lists the default values of the Device ID registers. You can use theparameter editor to change the values of these registers.

Table 11. Device ID Registers

Register Name Range Default Value Description

Vendor ID 16 bits 0x00001172

Sets the read-only value of theVendor ID register. This parametercannot be set to 0xFFFF per thePCI Express Base Specification.

Note: Set your own Vendor ID bychanging this parameter.

Address offset: 0x000.

Device ID 16 bits 0x00000000

Sets the read-only value of theDevice ID register. This register isonly valid in the Type 0 (Endpoint)Configuration Space.Address offset: 0x000.

Revision ID 8 bits 0x00000001Sets the read-only value of theRevision ID register.Address offset: 0x008.

Class Code 24 bits 0x00FF0000

Sets the read-only value of theClass Code register.Address offset: 0x008.This parameter cannot be set to0x0 per the PCI Express BaseSpecification.

Subsystem Vendor ID 16 bits 0x00000000

Sets the read-only value of theSubsystem Vendor ID register inthe PCI Type 0 ConfigurationSpace. This parameter cannot beset to 0xFFFF per the PCI ExpressBase Specification. This value isassigned by PCI-SIG to the devicemanufacturer.Address offset: 0x02C.

Subsystem Device ID 16 bits 0x00000000

Sets the read-only value of theSubsystem Device ID register inthe PCI Type 0 ConfigurationSpace.Address offset: 0x02C.

3.2.3. PCI Express and PCI Capabilities Parameters

For each core (PCIe0/PCIe1/PCIe2/PCIe3), the PCI Express / PCI Capabilities tabcontains separate tabs for the device, PRS (Endpoint mode), MSI (Endpoint mode),ACS capabilities (Root Port mode), slot (Root Port mode), MSI-X, and legacy interruptpin register parameters.

3. Parameters

UG-20237 | 2020.07.13


27


Figure 15. PCI Express / PCI Capabilities Parameters

3.2.3.1. Device Capabilities

Table 12. Device Capabilities


Maximum payload sizessupported

128 bytes256 bytes512 bytes

512 bytes

Specifies the maximumpayload size supported. Thisparameter sets the read-only value of the maxpayload size supported fieldof the Device Capabilitiesregister.

3.2.3.2. Link Capabilities

Table 13. Link Capabilities


Link port number (RootPort only) 0 - 255 1

Sets the read-only value ofthe port number field in theLink Capabilitiesregister. This parameter isfor Root Ports only. It shouldnot be changed.

Slot clock configuration True/False True

When this parameter isTrue, it indicates that theEndpoint uses the samephysical reference clock thatthe system provides on theconnector. When it is False,the IP core uses anindependent clock regardlessof the presence of a

continued...

3. Parameters

UG-20237 | 2020.07.13


28



reference clock on theconnector. This parametersets the Slot ClockConfiguration bit (bit 12) inthe PCI Express LinkStatus register.

3.2.3.3. MSI Capabilities

Table 14. MSI Capabilities


PF0 Enable MSI True/False False

Enables MSI functionality forPF0.If this parameter is True,the Number of MSImessages requestedparameter will appearallowing you to set thenumber of MSI messages.

PF0 MSI 64-bitAddressing True/False False Enables or disables MSI 64-

bit addressing for PF0.

PF0 MSI Extended DataCapable True/False False

Enables or disables MSIextended data capability forPF0.

PF0 Number of MSImessages requested

1248

1632

1

Sets the number ofmessages that theapplication can request inthe multiple messagecapable field of the MessageControl register.

3.2.3.4. MSI-X Capabilities

Table 15. MSI-X Capabilities


Enable MSI-X (Endpointonly) True/False False

Enables the MSI-Xfunctionality.For SR-IOV: VFs and PFs arealways MSI-X capable.

MSI-X Table Size0x0 - 0x7FF (only values ofpowers of two minus 1 are

valid)0

System software reads thisfield to determine the MSI-Xtable size <n>, which isencoded as <n-1>.For example, a returnedvalue of 2047 indicates atable size of 2048. This fieldis read-only.Address offset:0x068[26:16]

MSI-X Table Offset 0x0 - 0xFFFFFFFF 0

Points to the base of theMSI-X table. The lower 3 bitsof the table BAR indicator(BIR) are set to zero bysoftware to form a 64-bit

continued...

3. Parameters

UG-20237 | 2020.07.13


29



qword-aligned offset. Thisfield is read-only after beingprogrammed.

Table BAR indicator 0x0 - 0x5 0

Specifies which one of afunction's BARs, locatedbeginning at 0x10 inConfiguration Space, is usedto map the MSI-X table intomemory space. This field isread-only after beingprogrammed.

Pending bit array (PBA)offset 0x0 - 0xFFFFFFFF 0

Used as an offset from theaddress contained in one ofthe function's Base Addressregisters to point to the baseof the MSI-X PBA. The lower3 bits of the PBA BIR are setto zero by software to forma 32-bit qword-alignedoffset. This field is read-onlyafter being programmed.

PBA BAR indicator 0x0 - 0x5 0

Specifies the function's BaseAddress register, locatedbeginning at 0x10 inConfiguration Space, thatmaps the MSI-X PBA intomemory space. This field isread-only after beingprogrammed.

3.2.3.5. Power Management

Table 16. Power Management


Enable L0s acceptablelatency

Maximum of 64 nsMaximum of 128 nsMaximum of 256 nsMaximum of 512 nsMaximum of 1 usMaximum of 2 usMaximum of 4 usNo limit

Maximum of 64 ns

This design parameterspecifies the maximumacceptable latency that theapplication layer can toleratefor any link between thedevice and the root complexto exit the L0s state. It setsthe read-only value of theEndpoint L0s acceptablelatency field of the DeviceCapabilities Register(0x084).This Endpoint does notsupport the L0s or L1 states.However, in a switchedsystem, there may be linksconnected to switches thathave L0s and L1 enabled.This parameter is set toallow system configurationsoftware to read theacceptable latencies for alldevices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).

continued...

3. Parameters

UG-20237 | 2020.07.13


30



This setting is disabled forRoot Ports.The default value of thisparameter is 64 ns. This isthe safest setting for mostdesigns.

Endpoint L1 acceptablelatency

Maximum of 1 usMaximum of 2 usMaximum of 4 usMaximum of 8 usMaximum of 16 usMaximum of 32 usMaximum of 64 usNo limit

Maximum of 1 us

This value indicates theacceptable latency that anEndpoint can withstand inthe transition from the L1state to L0 state. It is anindirect measure of theEndpoint’s internal buffering.It sets the read-only value ofthe Endpoint L1 acceptablelatency field of the DeviceCapabilities Register.This Endpoint does notsupport the L0s or L1 states.However, a switched systemmay include links connectedto switches that have L0sand L1 enabled. Thisparameter is set to allowsystem configurationsoftware to read theacceptable latencies for alldevices in the system andthe exit latency for each linkto determine which links canenable Active State PowerManagement (ASPM).This setting is disabled forRoot Ports.

3.2.3.6. Vendor Specific Extended Capability (VSEC) Registers

Table 17. VSEC Register


Vendor Specific ExtendedCapability 0/1 0 Enables the Vendor Specific

Extended Capability (VSEC).

User ID register from theVendor Specific ExtendedCapability

0 - 65534 0

Sets the read-only value ofthe 16-bit User ID registerfrom the Vendor SpecificExtended Capability. Thisparameter is only valid forEndpoints.

Drops Vendor Type0Messages 0/1 0

When this parameter is setto 1, the IP core dropsvendor Type 0 messageswhile treating them asUnsupported Requests (UR).When it is set to 0, the IPcore passes these messageson to the user logic.

Drops Vendor Type1Messages 0/1 0

When this parameter is setto 1, the IP core silentlydrops vendor Type 1messages.

continued...

3. Parameters

UG-20237 | 2020.07.13


31



When it is set to 0, the IPcore passes these messageson to the user logic.

3.2.3.7. Legacy Interrupt Pin Register

Table 18. Legacy Interrupts Parameters


Enable Legacy Interruptsfor PF0

True/False False Enable Legacy Interrupts(INTx) for PF0 of PCIe0.

Set Interrupt Pin for PF0 NO INTINTA

NO INT When Legacy Interrupts arenot enabled, the only optionavailable is NO INT.When Legacy Interrupts areenabled, the only optionavailable is INTA.

3.2.4. Configuration, Debug and Extension Options

Table 19. Configuration, Debug and Extension Options


Gen 3 Requestedequalization far-end TXpreset vector

0 - 65535 0x00000004

Specifies the Gen 3requested phase 2/3 far-endTX preset vector. Choosing avalue different from thedefault is not recommendedfor most designs.

Gen 4 Requestedequalization far-end TXpreset vector

0 - 65535 0x00000270

Specifies the Gen 4requested phase 2/3 far-endTX preset vector. Choosing avalue different from thedefault is not recommendedfor most designs.

Enable Debug Toolkit True/False False

Enable the P-Tile DebugToolkit for JTAG-basedSystem Console debugaccess.

Enable HIP dynamicreconfiguration of PCIeregisters

True/False FalseEnable the user Hard IPreconfiguration Avalon-MMinterface.

3.3. Avalon-MM Settings

Table 20. Avalon-MM Parameters


Endpoint Settings

Enable 512 PCIe ReadOutstanding Requests forRead Data Mover

True/False False When this is enabled, theRead Data Mover can issueup to 512 PCIe Readoutstanding requests.However, no completion

continued...

3. Parameters

UG-20237 | 2020.07.13


32



reordering is performedwithin the Avalon-MMBridge.When this is not enabled,the Read Data Mover canonly issue up to 64 PCIeRead outstanding requests.In this case, the Avalon-MMBridge performs completionreordering.Note: This feature is not

available in the 20.2release of IntelQuartus Prime. Only64 outstanding Readrequests can besupported in thisrelease. Support for512 outstandingRead requests maybe available in afuture release.

Address width of ReadData Mover

{10:64} 64 Address width of Read DataMover.

Address width of WriteData Mover

{10:64} 64 Address width of Write DataMover.

Export interrupt conduitinterfaces

True/False False Export internal signals tosupport generation ofLegacy Interrupts/multipleMSI/MSI-X.

Enable Bursting SlaveMode

True/False False Enable bursting Avalon-MMSlave Interface. This willenable the Endpoint mode(where the IP's BAS andBAM modules are enabled,but its Data Movers are notenabled).

Address width of BurstingMaster

{10:64} 64 Only present in Root Portmode.In Endpoint modes, thisparameter is set by thelargest BAR address width.

Root Port Settings

3. Parameters

UG-20237 | 2020.07.13


33


4. Interfaces

4.1. Overview

The P-Tile Avalon-MM IP for PCIe includes many interface types to implement differentfunctions.These include:

• High-performance bursting master (BAM) and slave (BAS) Avalon-MM interfaces totranslate between PCIe TLPs and Avalon-MM memory-mapped reads and writes

• Read and Write Data Movers to transfer large blocks of data

• Standard PCIe serial interface to transfer data over the PCIe link

• System interfaces for interrupts, clocking, reset

• Optional reconfiguration interface to dynamically change the value ofConfiguration Space registers at run-time

• Optional status interface for debug

Unless otherwise noted, all interfaces to the Application layer are synchronous to therising edge of app_clk. For Gen3 operation, this clock runs at 250 MHz. Thefrequency of this clock is exactly half the frequency of the pld_clk generated by theIP, with 0ppm difference. You enable the interfaces using the component IP ParameterEditor.

Read Data Mover (RDDM) interface: This interface transfers DMA data from the PCIesystem memory to the memory in Avalon-MM address space.

Write Data Mover (WRDM) interface: This interface transfers DMA data from thememory in Avalon-MM address space to the PCIe system memory.

Bursting Master (BAM) interface: This interface provides host access to the registersand memory in Avalon-MM address space. The Busting Master module converts PCIeMemory Reads and Writes to Avalon-MM Reads and Writes.

Bursting Slave (BAS) interface: This interface allows the user application in the FPGAto access the PCIe system memory. The Bursting Slave module converts Avalon-MMReads and Writes to PCIe Memory Reads and Writes.

Control Register Access (CRA) interface: This optional, 32-bit Avalon-MM Slaveinterface provides access to the Control and Status registers. You must enable thisinterface when you enable address mapping for any of the Avalon-MM slaves or ifinterrupts are implemented. The address bus width of this interface is fixed at 15 bits.The prefix for this interface is cra*.

The modular design of the P-Tile Avalon-MM IP for PCIe lets you enable just theinterfaces required for your application.

UG-20237 | 2020.07.13

Send Feedback







Table 21. Avalon-MM Interface Summary

Avalon-MM Type Data Bus Width Max Burst Size Byte EnableGranularity

Max OutstandingRead Request

Bursting Slave 512 bits

Bursting Mode: 8cycles

Non-Bursting Mode: 1cycle

dword/byteBursting Mode: 64

Non-Bursting Mode: 1

Bursting Master 512 bits

Bursting Mode: 8cycles

Non-Bursting Mode: 1cycle

dword/byteBursting Mode: 32

Non-Bursting Mode: 1

Read Data MoverWrite Master 512 bits 8 cycles dword N/A

Write Data MoverRead Master 512 bits 8 cycles dword 32

Control RegisterAccess 32 bits 1 cycle byte 1

Note: The number of read requests issued by the Write Data Mover's Avalon-MM ReadMaster is controlled by the assertion of waitrequest by the connected slave(s). TheRead Master can handle 128 outstanding cycles of data. You cannot set this parameterin Platform Designer. The slave needs to correctly back-pressure the master once itcannot handle the incoming requests.

Note: The 512-bit Bursting Slave interface does not support transactions where all byteenables are set to 0.

Note: All transfers of four bytes or more are done in multiples of dwords.

4.2. Clocks and Resets

4.2.1. Interface Clock Signals

Table 22. Interface Clock Signals

Name I/O Description EP/RP Clock Frequency

p<n>_app_clk(where n = 0, 1,

2, 3)O

This is the application clock generatedfrom coreclkout_hip or from thesame source as refclk. This is a

per-port signal.

EP/RPNative Gen3: 250 MHz

Native Gen4: 350 MHz / 400MHz

coreclkout_hip O

This clock drives the ApplicationLayer.

The frequency depends on the datarate and the number of lanes being

used.

EP/RP

Native Gen3: 250 MHzNative Gen4: 350 MHz / 400MHz (Intel Stratix 10 DX) /

500 MHz (Intel Agilex)

refclk[1:0] I

These are the input reference clocksfor the IP core. These clocks must be

free-running.For more details on how to connectthese clocks, refer to the section

Clock Sharing in Bifurcation Modes.

EP/RP

100 MHz ± 300 ppmWhen the Enable SRIS Modeparameter is enabled in the IP

Parameter Editor, the P-TileAvalon-MM IP can

communicate with a linkpartner whose clock domain is

not synchronized to therefclk domain of the P-Tile.

continued...

4. Interfaces

UG-20237 | 2020.07.13


35


Name I/O Description EP/RP Clock Frequency

In this mode of operation, P-Tile and its link partner canboth have their own spread

spectrum clocks.

p<n>_hip_reconfig_clk (where n = 0,

1, 2, 3)I

Clock for the hip_reconfiginterface. This is an Avalon-MM

interface. It is an optional interfacethat is enabled when the Enable HIPdynamic reconfiguration of PCIe

registers option in the PCIeConfiguration, Debug and

Extension Options tab is enabled.

EP/RP50 MHz - 125 MHz (range)100 MHz (recommended)

xcvr_reconfig_clk I

Clock for the PHY reconfigurationinterface. This is an Avalon-MM

interface. This optional interface isenabled when you turn on the EnablePHY reconfiguration option in the

Top-Level Settings tab. Thisinterface is shared among all the

cores.

EP/RP50 MHz - 125 MHz (range)100 MHz (recommended)

4.2.2. Interface Reset Signals

Table 23. Interface Reset Signals

Signal Name Direction Clock EP/RP Description

pin_perst_n Input Asynchronous

EP/RP This is an active-low input to the PCIe Hard IP, andimplements the PERST# function defined by the PCIespecification.

p0_reset_status_n Output Synchronous

EP/RP This active-low signal is held low until pin_perst_n hasbeen deasserted and the PCIe Hard IP has come out ofreset. This signal is synchronous to coreclkout_hip.When port bifurcation is used, there is one such signal foreach interface. The signals are differentiated by theprefixes pn.

p0_link_req_rst_n Output Synchronous

EP/RP This active-low signal is asserted by the PCIe Hard IPwhen it is about to go into reset.The Avalon-MM Bridge IP will reset all its PCIe-relatedregisters and queues including anything related to tags.It will also stop sending packets to the PCIe Hard IP untilthe Bus Master Enable bit is set again. The Bridge willalso ignore any packet received from the PCIe Hard IP.

p0_pld_warm_rst_rdy Input Synchronous

EP/RP This active-high signal is asserted by the user logic inresponse to p0_link_req_rst_n when it hascompleted its pre-reset tasks.Note: When not using this signal, set it to 1'b1.

ninit_done Input Asynchronous

EP/RP A "1" on this active-low signal indicates that the FPGAdevice is not yet fully configured. A "0" indicates thedevice has been configured and is in normal operatingmode.Intel recommends using the output of the Reset ReleaseIntel Fpga IP to drive this ninit_done input. For moredetails on this IP, refer to the Application Note AN891 at https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an891.pdf

4. Interfaces

UG-20237 | 2020.07.13


36

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an891.pdf

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/an/an891.pdf


4.3. Avalon-MM Interface

The figures below provide the top-level block diagrams of the P-Tile Avalon-MM IP withall interfaces while operating in Endpoint mode with Data Movers or in Endpoint mode.These interfaces are described in more details in following sections.

Figure 16. P-Tile Avalon-MM IP for PCIe in Endpoint Mode with Data Movers Top-LevelBlock Diagram

P-Tile Avalon-MM IP for PCI Express

Clocksrefclk[1:0]

coreclkout_hip

p0_wrdm_read_o

p0_wrdm_address_o[63:0]p0_wrdm_readdata_i[511:0]p0_wrdm_burst_count_o[3:0]p0_wrdm_wait_request_ip0_wrdm_readdatavalid_i

Write Data Mover Interface:Writes Data from FPGA

Memory to Host Memory

tx_n_out[15:0]rx_p_in[15:0] Serial Data

p0_rddm_write_op0_rddm_address_o[63:0]p0_rddm_write_data_o[511:0]p0_rddm_burst_count_o[3:0]p0_rddm_byte_enable_o[63:0]p0_rddm_wait_request_i

Read Data Mover Interface:Writes Data from Host

Memory to FPGA Memory

Hard IPReconfiguration(Optional)

p0_hip_reconfig_clkp0_hip_reconfig_address[20:0]

p0_hip_reconfig_readp0_hip_reconfig_readdata[7:0]

p0_hip_reconfig_writep0_hip_reconfig_writedata[7:0]

p0_hip_reconfig_waitrequest

p0_hip_reconfig_readdatavalid

p0_bam_pfnum_o[1:0]p0_bam_bar_o[2:0]

p0_bus_master_enable_o[1:0]

p0_wrdm_desc_valid_i

p0_bam_waitrequest_i

Write Data MoverNormal Descriptor

Queue

p0_pld_warm_rst_rdy_ip0_reset_status_n pin_perst_n ninit_done

Reset

xcvr_reconfig_clkxcvr_reconfig_address[25:0]

xcvr_reconfig_readxcvr_reconfig_readdata[7:0]xcvr_reconfig_readdatavalid

xcvr_reconfig_writexcvr_reconfig_writedata[7:0]

xcvr_reconfig_waitrequest

PHYReconfiguration(Optional)

p0_wrdm_desc_ready_o

p0_wrdm_desc_data_i[173:0]

Write Data MoverPriority Descriptor

Queue

p0_wrdm_prio_ready_op0_wrdm_prio_valid_ip0_wrdm_prio_data_i[173:0]

Read Data MoverPriority Descriptor

Queue

Read Data MoverNormal Descriptor

Queue

p0_rddm_prio_ready_op0_rddm_prio_valid_ip0_rddm_prio_data_i[173:0]

p0_rddm_desc_ready_op0_rddm_desc_valid_ip0_rddm_desc_data_i[173:0]

Bursting Avalon-MMMaster Interface

p0_rddm_pfnum_o[1:0]

p0_wrdm_pfnum_o[1:0]

p0_wrdm_byteenable_o[63:0]p0_wrdm_response_i[1:0]

p0_rddm_tx_valid_op0_rddm_tx_data_o[31:0]

p0_wrdm_tx_valid_op0_wrdm_tx_data_o[31:0]

p0_pld_link_req_rst_o

p0_app_clk

dummy_user_avmm_rst

tx_p_out[15:0]

rx_n_in[15:0]

p0_bam_response_i[1:0]p0_bam_address_o[BAM_ADDR_WIDTH-1:0]p0_bam_byteenable_o[63:0]p0_bam_read_op0_bam_readdata_i[511:0]p0_bam_readdatavalid_ip0_bam_write_op0_bam_writedata_o[511:0]p0_bam_burstcount_o[3:0]

p0_tl_cfg_func_o[2:0]p0_tl_cfg_add_o[4:0]p0_tl_cfg_ctl_o[15:0]

Configuration OutputInterface

4. Interfaces

UG-20237 | 2020.07.13


37


Figure 17. P-Tile Avalon-MM IP for PCIe in Endpoint Mode Top-Level Block Diagram

P-Tile Avalon-MM IP for PCI Express

Clocksrefclk[1:0]

coreclkout_hip

p0_bas_waitrequest_o

p0_bas_address_i[63:0]p0_bas_readdata_o[511:0]

p0_bas_burstcount_i[3:0]

p0_bas_read_i

p0_bas_write_i

Bursting Avalon-MMSlave Interface

tx_n_out[15:0]rx_p_in[15:0] Serial Data

Hard IPReconfiguration(Optional)

p0_hip_reconfig_clkp0_hip_reconfig_address[20:0]

p0_hip_reconfig_readp0_hip_reconfig_readdata[7:0]

p0_hip_reconfig_writep0_hip_reconfig_writedata[7:0]

p0_hip_reconfig_waitrequest

p0_hip_reconfig_readdatavalid

p0_bam_pfnum_o[1:0]p0_bam_bar_o[2:0]

p0_bus_master_enable_o[1:0]

p0_bam_waitrequest_i

p0_pld_warm_rst_rdy_ip0_reset_status_n pin_perst_n ninit_done

Reset

xcvr_reconfig_clkxcvr_reconfig_address[25:0]

xcvr_reconfig_readxcvr_reconfig_readdata[7:0]xcvr_reconfig_readdatavalid

xcvr_reconfig_writexcvr_reconfig_writedata[7:0]


PHYReconfiguration(Optional)

Bursting Avalon-MMMaster Interface

p0_bas_byteenable_i[63:0]p0_bas_pfnum_i[1:0]

p0_bas_writedata_i[511:0]

p0_pld_link_req_rst_o

p0_app_clk

dummy_user_avmm_rst

tx_p_out[15:0]

rx_n_in[15:0]

p0_bam_response_i[1:0]p0_bam_address_o[BAM_ADDR_WIDTH-1:0]p0_bam_byteenable_o[63:0]p0_bam_read_op0_bam_readdata_i[511:0]p0_bam_readdatavalid_ip0_bam_write_op0_bam_writedata_o[511:0]p0_bam_burstcount_o[3:0]

p0_bas_readdatavalid_o

p0_bas_response_o[1:0]

p0_tl_cfg_func_o[2:0]p0_tl_cfg_add_o[4:0]p0_tl_cfg_ctl_o[15:0]

Configuration OutputInterface

4.3.1. Endpoint Mode Interface (512-bit Avalon-MM Interface)

Table 24. Avalon-MM Interface Summary



Bursting Slave 512 bits 8 cycles byte 64

Bursting Master 512 bits 8 cycles byte 32

continued...

4. Interfaces

UG-20237 | 2020.07.13


38




Read Data MoverWrite Master

512 bits 8 cycles dword N/A

Write Data MoverRead Master

512 bits 8 cycles dword 128

Control RegisterAccess

32 bits 1 cycle byte 1

These interfaces are standard Avalon interfaces. For timing diagrams, refer to theAvalon Interface Specifications.

Note: The number of read requests issued by the Write Data Mover's Avalon-MM ReadMaster is controlled by the assertion of waitrequest by the connected slave(s). TheRead Master can handle 128 outstanding cycles of data. You cannot set this parameterin Platform Designer. The slave needs to correctly back-pressure the master once itcannot handle the incoming requests.

Note: The 512-bit Bursting Slave interface does not support transactions where byte enablesare set to 0.

Related Information

Avalon Interface Specifications

4.3.1.1. Bursting Avalon-MM Master and Conduit

The Bursting Avalon-MM Master module has one user-visible Avalon-MM Masterinterface.

You enable this interface by turning On the Enable Bursting Avalon-MM Masterinterface option in the Avalon-MM Settings tab of the IP Parameter Editor.

Table 25. Bursting Avalon-MM Master and Conduit

Signal Name Direction Description Platform DesignerInterface Name

bam_pfnum_o[1:0] O

Physical function number• PF0: bam_pfnum_o[1:0] = 2'b00• Others: bam_pfnum_o[1:0] =

Reserved

bam_bar_o[2:0] O

This bus contains the BAR address for aparticular TLP. This bus acts as anextension of the standard address bus.000: Memory BAR 0001: Memory BAR 1010: Memory BAR 2011: Memory BAR 3100: Memory BAR 4101: Memory BAR 5110: Reserved111: Expansion ROM BAR

bam_waitrequest_i IWhen asserted, indicates that the Avalon-MM slave is not ready to respond to arequest.

bam_master

continued...

4. Interfaces

UG-20237 | 2020.07.13


39

https://www.intel.com/content/dam/www/programmable/us/en/pdfs/literature/manual/mnl_avalon_spec.pdf



waitrequestAllowance = 8The master can still issue 8 transfers afterbam_waitrequest_i is asserted.

bam_address_o[BAM_ADDR_WIDTH-1:0]

O

The width of the Bursting Master’saddress bus is the maximum of thewidths of all the enabled BARs.For BARs narrower than the widest BAR,the address bus’ additional mostsignificant (MSB) bits are driven to 0.

bam_byteenable_o[63:0] O

Specify the valid bytes ofbam_writedata_o[511:0]. Each bitcorresponds to a byte inbam_writedata_o[511:0].For single-cycle read bursts and for allwrite bursts, all contiguous sets ofenabled bytes are supported.For multi-cycle read bursts, all bits ofbam_byteenable_o[63:0] areasserted, regardless of the First ByteEnable (BE) and Last BE fields of thecorresponding TLP.

bam_read_o O When asserted, indicates the master isrequesting a read transaction.

bam_readdata_i[511:0] I Read data bus

bam_readdatavalid_i I

Asserted by the slave to indicate that thebam_readdata_i[511:0] bus containsvalid data in response to a previous readrequest.

bam_write_o O When asserted, indicates the master isrequesting a write transaction.

bam_writedata_o[511:0] O Data signals for write transfers.

bam_burstcount_o[3:0] O The master uses these signals to indicatethe number of transfers in each burst.

bam_response_i[1:0] I

00 : OKAY - successful response for atransaction.01 : RESERVED10 : SLAVEERROR11 : DECODEERROR

4.3.1.1.1. Bursting Avalon-MM Master and Conduit in Non-Bursting Mode

In non-bursting mode, the Bursting Avalon-MM Master module has the same interfaceas in bursting mode, except for some limitations in the size of transactions asdescribed below:

4. Interfaces

UG-20237 | 2020.07.13


40


• Burst count must be 1.

• The request size ranges from 1 to 16 dwords with the following limitations:

— The address and size combination must generate a TLP that fits in one 512-bitchunk of data. For example, if the address starts at dword 15 of a 512-bittransaction, only one dword of data transfer is allowed. If the address starts atdword 0, all data transfer sizes up to 16 dwords are possible. The same ruleapplies to read completions.

— Byte enables are supported for a transfer size of one dword. For largertransfer sizes, dword enables apply.

• If non-bursting mode is enabled, sending a TLP larger than 64 bytes targeting thisinterface causes the interface to misbehave. In this case, a reset is required toallow the interface to recover.

• One outstanding read at a time. Incoming RX read/write TLPs will be delayed whilea downstream outstanding read exists.

4.3.1.2. Bursting Avalon-MM Slave and Conduit

The Bursting Avalon-MM Slave module has one user-visible Avalon-MM slave interface.

You enable this interface by turning On the Enable Bursting Avalon-MM Slaveinterface option in the IP Parameter Editor.

For more details on these interface signals, refer to the Avalon InterfaceSpecifications.

Table 26. Bursting Avalon-MM Slave and Conduit


bas_pfnum_i[1:0] I

Physical function number• PF0:

bas_pfnum_i[1:0] =2'b00

• Others:bas_pfnum_i[1:0] =Reserved

bas_waitrequest_o O

When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 0The master cannot issue anytransfer afterbas_waitrequest_o isasserted.

bas_master

bas_address_i[63:0] ISpecify the byte addressregardless of the data widthof the master.

bas_byteenable_i[63:0] I

Specify the valid bytes ofbas_writedata_i[511:0]. Each bit corresponds to abyte inbas_writedata_i[511:0].

continued...

4. Interfaces

UG-20237 | 2020.07.13


41



For single-cycle read burstsand for all write bursts, allcontiguous sets of enabledbytes are supported.For burst read transactions,thebas_byteenable_i[63:0]must be64'hFFFF_FFFF_FFFF_FFFF.

bas_read_i IWhen asserted, indicates themaster is requesting a readtransaction.

bas_readdata_o[511:0] OEnsure that disabled bytesdo not contain stale data ifSR-IOV is enabled.

bas_readdatavalid_o O

maximumPendingReadTransactions: 64The maximum number ofpending reads that theAvalon-MM slave can queueup is 64.

bas_response_o[1:0] O

These bits contain theresponse status for anytransaction happening onthe BAS interface:• 00: OKAY - Successful

response for atransaction.

• 01: RESERVED - Thisencoding is reserved.

• 10: SLAVEERROR - Errorfrom an endpoint slave.Indicates an unsuccessfultransaction.

• 11: DECODEERROR -Indicates an attemptedaccess to an undefinedlocation.

bas_write_i IWhen asserted, indicates themaster is requesting a writetransaction.

bas_writedata_i[511:0] I Data signals for writetransfers.

bas_burstcount_i[3:0] I

The master uses thesesignals to indicate thenumber of transfers in eachburst.

4.3.1.2.1. Bursting Avalon-MM Slave and Conduit in Non-Bursting Mode

The Bursting Avalon-MM Slave module supports bursting mode while operating in RootPort mode.

This module can also operate in non-bursting mode. In this mode, the moduleinterface is the same as in bursting mode except that it has limitations in the size oftransactions as described below:

4. Interfaces

UG-20237 | 2020.07.13


42


• Burst count must be 1.

• The request size ranges from 1 to 16 dwords with the following limitations:

— The address and size combination must generate a TLP that fits in one 512Bchunk of data. For example, if the address starts at dword 15 of a 512Btransaction, only one dword of data transfer is allowed. If the address starts atdword 0, all data transfer sizes up to 16 dwords are possible. The same ruleapplies to read completions.

— Byte enables are supported for a transfer size of one dword. For largertransfer sizes, dword enables apply.

• One outstanding read at a time (back-pressures the Avalon-MM Master while theoutstanding read exists).

4. Interfaces

UG-20237 | 2020.07.13


43


4.3.1.3. Data Movers

4.3.1.3.1. Read Data Mover

The Read Data Mover has four user-visible interfaces:

• One Avalon-MM Write Master with sideband signals to write data to the Avalondomain.

• Two Avalon-ST Sinks to receive descriptors. One acts as a queue for prioritydescriptors, and the other acts as a queue for normal descriptors.

• One Avalon-ST Source to report status.

Read Data Mover Avalon-MM Write Master and Conduit

This interface provides the Read data from the Host memory to the user application.The rddm_address_o value is set within the descriptor destination address.

Table 27. Read Data Mover Avalon-MM Write Master and Conduit


rddm_pfnum_o[1:0] O

Physical function number.• PF0:

rddm_pfnum_o[1:0] =2'b00

• Others:rddm_pfnum_o[1:0] =Reserved

rddm_conduit

rddm_waitrequest_i I

When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 16The master can still issue 16transfers afterrddm_waitrequest_i isasserted.

rddm_master

rddm_write_o OWhen asserted, indicates themaster is requesting a writetransaction.

rddm_address_o[63:0] OSpecify the byte addressregardless of the data widthof the master.

rddm_burstcount_o[3:0] O


rddm_byteenable_o[63:0]

O

Specify the valid bytes ofrddm_writedata_o[511:0]. Each bit corresponds to abyte inrddm_writedata_o[511:0].

rddm_writedata_o[511:0]

O Data signals for writetransfers.

4. Interfaces

UG-20237 | 2020.07.13


44


Read Data Mover Avalon-ST Descriptor Sinks

The Read Data Mover has two Avalon-ST sink interfaces to receive the descriptors thatdefine the data transfers to be executed. One of the interfaces receives descriptors fornormal data transfers, while the other receives descriptors for high-priority datatransfers.

The descriptor format for the Read Data Mover is described in the section DescriptorFormat for Data Movers.

Note: The user application is responsible for performing the scheduling between priority andnormal queues. No arbitration is performed inside the Read Data Mover.

Table 28. Read Data Mover Avalon-ST Normal Descriptor Sink Interface


rddm_desc_ready_o O When asserted, this readysignal indicates the normaldescriptor queue in the ReadData Mover is ready toaccept data. The readylatency of this interface is 3cycles.

rddm_desc

rddm_desc_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the normaldescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.

rddm_desc_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : single destination (3)

[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address

Table 29. Read Data Mover Avalon-ST Priority Descriptor Sink Interface


rddm_prio_ready_o O When asserted, this readysignal indicates the prioritydescriptor queue in the ReadData Mover is ready to

rddm_prio

continued...

(3) When the single destination bit is set, the same destination address is used for all thetransfers. If the bit is not set, the address increments for each transfer.

4. Interfaces

UG-20237 | 2020.07.13


45



accept data. The readylatency of this interface is 3cycles.

rddm_prio_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the prioritydescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.

rddm_prio_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : single destination[147] : reserved[146] : reserved[145:128]: number ofdwords to transfer up to 1MB[127:64]: destinationAvalon-MM address[63:0]: source PCIe address

The Read Data Mover internally supports two queues of descriptors. The priority queuehas absolute priority over the normal queue. Use it carefully to avoid starving thenormal queue.

If the Read Data Mover receives a descriptor on the priority interface while processinga descriptor from the normal queue, it switches to processing descriptors from thepriority queue as soon as it has completed the current descriptor. The Read DataMover resumes processing the descriptors from the normal queue once the priorityqueue is empty. Do not use the same descriptor ID simultaneously in the two queuesas there would be no way to distinguish them on the Status Avalon-ST sourceinterface.

The Read Data Mover handles one descriptor at a time. When a descriptor has beenprocessed (the memory command has been issued to the PCIe link), the Read DataMover will read the next descriptor from the priority or normal descriptor interface.

Note: There is no buffer to store descriptors inside the Read Data Mover. In Intel's DMAdesign example, the buffer is located in the external DMA controller and supports upto 128 descriptors.

Software should only send new descriptors when the Read Data Mover has processedall previously sent descriptors. The P-Tile Avalon-MM IP indicates the completion of theRead Data Mover's data processing by performing an immediate write to the systemmemory using its Write Data Mover. For more details, refer to the Read DMA Examplesection in the P-tile Avalon Memory Mapped (Avalon-MM) IP for PCI Express DesignExample User Guide (see the link in the Related Information below).

Related Information

Read DMA Example

4. Interfaces

UG-20237 | 2020.07.13


46

https://www.intel.com/content/www/us/en/programmable/documentation/uro1573085463987.html#isy1573171719773


Read Data Mover Status Avalon-ST Source

Table 30. Read Data Mover Status -ST Source


rddm_tx_data_o[31:0] O [31:16]: reserved[15]: error[14:12]: application specific[11:9] : reserved[8] : priority[7:0]: descriptor ID

rddm_tx

rddm_tx_valid_o O Valid status signal

This interface does not have a ready input. The application logic must always be readyto receive status information for any descriptor that it has sent to the Read DataMover.

The Read Data Mover copies over the application specific bits in the rddm_tx_data_obus from the corresponding descriptor. A set priority bit indicates that the descriptor isfrom the priority descriptor sink.

A status word is output on this interface when the processing of a descriptor hascompleted, including the reception of all completions for all memory read requests.

4.3.1.3.2. Write Data Mover

The Write Data Mover has four user visible interfaces:

• One Avalon-MM Read Master with sideband signals to read data from the Avalondomain.

• Two Avalon-ST Sinks to receive descriptors. One Sink acts as a queue for prioritydescriptors, and the other acts as a queue for normal descriptors.

• One Avalon-ST Source to report status

Write Data Mover Avalon-MM Read Master and Conduit

This interface reads data from the Avalon-MM Read Master interface and writes it tothe Host memory.

The wrdm_address_o value is set within the descriptor source address.

Table 31. Write Data Mover Avalon-MM Read Master and Conduit


wrdm_pfnum_o[1:0] O Physical function number.

continued...

4. Interfaces

UG-20237 | 2020.07.13


47



• PF0:wrdm_pfnum_o[1:0] =2'b00

• Others:wrdm_pfnum_o[1:0] =Reserved

wrdm_waitrequest_i I

When asserted, indicatesthat the Avalon-MM slave isnot ready to respond to arequest.waitrequestAllowance = 4The master can still issue 4transfers afterwrdm_waitrequest_i isasserted.

wrdm_master

wrdm_read_o OWhen asserted, indicates themaster is requesting a readtransaction.

wrdm_address_o[63:0] OSpecify the byte addressregardless of the data widthof the master.

wrdm_burstcount_o[3:0] O


wrdm_byteenable_o[63:0]

O

Specify the valid bytes ofwrdm_writedata_o[511:0]. Each bit corresponds to abyte inwrdm_writedata_o[511:0].

wrdm_readdatavalid_i I

Asserted by the slave toindicate that thewrdm_readdata_i[511:0]signals contain valid data inresponse to a previous readrequest.

wrdm_readdata_i[511:0] I Data signals for readtransfers.

wrdm_response_i[1:0] I

The response signals areoptional signals that carrythe response status.

Note:

Because the signalsare shared, aninterface cannotissue or accept awrite response and aread response in thesame clock cycle.

The following encodings areavailable:

continued...

4. Interfaces

UG-20237 | 2020.07.13


48



• 00: OKAY - Successfulresponse for atransaction.

• 01: RESERVED -Encoding is reserved.

• 10: SLAVEERROR - Errorfrom an endpoint slave.Indicates an unsuccessfultransaction.

• 11: DECODEERROR -Indicates an attemptedaccess to an undefinedlocation.

For read responses:• One response is sent

with each readdata. Aread burst length of Nresults in N responses. Itis not valid to producefewer responses, even inthe event of an error. Itis valid for the responsesignal values to bedifferent for eachreaddata in the burst.

• The interface must haveread control signals.Pipeline support ispossible with thereaddatavalid signal.

• On a read error, thecorresponding readdatais a "don't care".

Write Data Mover Avalon-ST Descriptor Sinks

The Write Data Mover has two Avalon-ST sink interfaces to receive the descriptors thatdefine the data transfers to be executed. One of the interfaces receives descriptors fornormal data transfers, while the other receives descriptors for high-priority datatransfers.

The descriptor format for the Write Data Mover is described in the section DescriptorFormats for Data Movers.

Note: The user application is responsible for performing the scheduling between priority andnormal queues. No arbitration is performed inside the Write Data Mover.

Table 32. Write Data Mover Avalon-ST Normal Descriptor Sink Interface


wrdm_desc_ready_o O When asserted, this readysignal indicates the normaldescriptor queue in theWrite Data Mover is ready to

wrdm_desc

continued...

4. Interfaces

UG-20237 | 2020.07.13


49



accept data. The readylatency of this interface is 3cycles.

wrdm_desc_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the normaldescriptor queue. On eachcycle where this signal isactive, the queue samplesthe data.

wrdm_desc_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : reserved[147] : single source (4)

[146] : immediate (5)

[145:128]: number ofdwords to transfer up to 1MB[127:64]: destination PCIeaddress[63:0]: source Avalon-MMaddress / immediate data

Table 33. Write Data Mover Avalon-ST Priority Descriptor Sink Interface


wrdm_prio_ready_o O When asserted, this readysignal indicates the prioritydescriptor queue in theWrite Data Mover is ready toaccept data. The readylatency of this interface is 3cycles. wrdm_prio

wrdm_prio_valid_i I When asserted, this signalqualifies valid data on anycycle where data is beingtransferred to the prioritydescriptor queue. On each

continued...

(4) When the single source bit is set, the same source address is used for all the transfers. If thebit is not set, the address increments for each transfer. Note that in single source mode, thePCIe address and Avalon-MM address must be 64-byte aligned.

(5) When set, the immediate bit indicates immediate writes. Immediate writes of one or twodwords are supported. For immediate transfers, bits [31:0] or [63:0] contain the payload forone- or two-dword transfers respectively. The two-dword immediate writes cannot cross a 4kboundary.

4. Interfaces

UG-20237 | 2020.07.13


50



cycle where this signal isactive, the queue samplesthe data.

wrdm_prio_data_i[173:0]

I [173:160]: reserved. Shouldbe tied to 0.[159:152]: descriptor ID[151:149] : applicationspecific[148] : reserved[147] : single source[146] : immediate[145:128]: number ofdwords to transfer up to 1MB[127:64]: destination PCIeaddress[63:0]: source Avalon-MMaddress / immediate data

The Write Data Mover internally supports two queues of descriptors. The priorityqueue has absolute priority over the normal queue, so it should be used carefully toavoid starving the normal queue.

If the Write Data Mover receives a descriptor on the priority interface while processinga descriptor from the normal queue, it switches to processing descriptors from thepriority queue after it has completed processing the current descriptor. The Write DataMover resumes processing descriptors from the normal queue once the priority queueis empty. Do not use the same descriptor ID simultaneously in the two queues asthere would be no way to distinguish them on the Status Avalon-ST source interface.

The Write Data Mover handles one descriptor at a time. When a descriptor has beenprocessed, the Write Data Mover will read the next descriptor from the priority ornormal descriptor interface.

Note: There is no buffer to store descriptors inside the Write Data Mover. In Intel's DMAdesign example, the buffer is located in the external DMA controller and supports upto 128 descriptors.

Software should only send new descriptors when the Write Data Mover has processedall previously sent descriptors. The Write Data Mover indicates the completion of theits data processing by performing an immediate write to the system memory using thelast descriptor in the descriptor table. For more details, refer to the Write DMAExample section in the P-tile Avalon Memory Mapped (Avalon-MM) IP for PCI ExpressDesign Example User Guide (see the link in the Related Information below).

Related Information

Write DMA Example

4. Interfaces

UG-20237 | 2020.07.13


51

https://www.intel.com/content/www/us/en/programmable/documentation/uro1573085463987.html#uie1573171813455


Write Data Mover Status Avalon-ST Source

Table 34. Write Data Mover Status Avalon-ST Source


wrdm_tx_data_o[31:0] O [31:16]: reserved[15]: error[14:12] : application specific[11:9] : reserved[8] : priority bit[7:0]: descriptor ID

wrdm_tx

wrdm_tx_valid_o O Valid status signal

This interface does not have a ready input. The application logic must always be readyto receive status information for any descriptor that it has sent to the Write DataMover.

The ready latency does not matter because there is no ready input.

The Write Data Mover copies over the application specific bits in the wrdm_tx_data_obus from the corresponding descriptor. A set priority bit indicates that the descriptorwas from the priority descriptor sink.

4.3.1.3.3. Descriptor Format for Data Movers

The Read and Write Data Movers uses descriptors to transfer data. The descriptorformat is fixed and specified below:

Table 35. Descriptor Format for Data Movers

Signals Description (forrddm_desc_data_i orwrdm_desc_data_i)

Read Data Mover Write Data Mover

[173:160]: reserved N/A N/A

[159:152]: descriptor ID ID of the descriptor ID of the descriptor

[151:149]: application-specific Application-specific bits.Example of an Intel application isprovided below.

Application-specific bits.Example of an Intel application isprovided below.

[148]: single destination When the single destination bit is set,the same destination address is usedfor all the transfers. If the bit is notset, the address increments for eachtransfer.

N/A

[147]: single source N/A When the single source bit is set, thesame source address is used for all thetransfers. If the bit is not set, theaddress increments for each transfer.Note that in single source mode, thePCIe address and Avalon-MM addressmust be 64-byte aligned.

[146]: immediate N/A When set, the immediate bit indicatesimmediate writes. Immediate writes ofone or two dwords are supported.

continued...

4. Interfaces

UG-20237 | 2020.07.13


52


Signals Description (forrddm_desc_data_i orwrdm_desc_data_i)

Read Data Mover Write Data Mover

For immediate transfers, bits [31:0] or[63:0] contain the payload for one- ortwo-dword transfers respectively. Thetwo-dword immediate writes cannotcross a 4k boundary.This can be used for MSI/MSI-X forexample.

[145:128]: transfer size Number of dwords to transfer (up to 1MB).

Number of dwords to transfer (up to 1MB).

[127:64]: destination address Avalon-MM address PCIe Address

[63:0]: source address PCIe Address Avalon-MM address

Application-Specific Bits

Three application-specific bits (bits [151:149] ) from the Write Data Mover and ReadData Mover Status Avalon-ST Source interfaces control when interrupts are generated.

Table 36. Encodings for Application-Specific Bits

Bit [151] Bit [150] Bit [149] Action

0 1 1 Interrupt always

0 1 0 Interrupt if error

0 0 1 No interrupt

0 0 0 No interrupt and drop statusword

The External DMA Controller makes the decision whether to drop the status word andwhether to generate an interrupt as soon as it receives the status word from the DataMover. When the generation of an interrupt is requested, and the corresponding RI orWI register does enable interrupts, the DMA Controller generates the interrupt. It doesso by queuing an immediate write to the Write Data Mover's descriptor queue(specified in the corresponding interrupt control register) using the MSI address andmessage data provided in that register.

4.3.1.3.4. Avalon-MM DMA Operations

Avalon-MM DMA operations are used to transfer large blocks of data. The P-TileAvalon-MM IP for PCIe can support DMA operations with an external descriptorcontroller implemented in the user application.

To interface to the DMA logic included in the P-Tile Avalon-MM IP for PCIe, the customDMA descriptor controller must implement the following functions:

• It must provide the descriptors to the Read Data Mover and Write Data Mover inthe P-Tile IP.

• It must process the status that the DMA Avalon-MM Read and Write mastersprovide.

The following figure shows the Avalon-MM DMA Bridge when a custom externaldescriptor controller drives the Read and Write Data Movers.

4. Interfaces

UG-20237 | 2020.07.13


53


Figure 18. Avalon-MM DMA Bridge Block Diagram with Externally InstantiatedDescriptor Controller

Avalon-MM (bam_*) 512 Bits

Read Data MoverWrite Master

Read Data Mover Descriptor Sinks

Read Data MoverStatus Source

Read Data Mover

Avalon-ST (rddm_desc, rddm_prio)

174 Bits

Avalon-ST (rddm_tx)32 Bits

Avalon-MM (rddm_master)512 Bits

Write Data MoverDescriptor Sinks

Write Data MoverStatus Source

Write Data MoverRead Master

Write Data Mover

Avalon-ST (wrdm_desc, wrdm_prio)

174 Bits

Avalon-ST (wrdm_tx)32 Bits

Avalon-MM (wrdm_master)512 Bits

Non-Bursting Avalon-MM MastersMWr, Mrd, CpID

PCIeTX

PCIeRX

PCIeHard IP

X16

X16

P-Tile Avalon-MM IP for PCI Express IP Core

Intel FPGA

Avalon-MM DMA withexternal Descriptor

ControllerAvalon-MM DMA Bridge (Soft Logic)

Custom Descriptor Controller

(Implemented inFPGA Fabric)

MRd, CpID

MWr

Hard

IP In

terfa

ce

This configuration includes the PCIe Read DMA and Write DMA Data Movers. Thecustom DMA descriptor controller must connect to the following Data Mover interfaces:

• PCIe Read Descriptor Sinks: These are two 174-bit, Avalon-ST sink interfaces (fornormal and priority descriptors). The custom DMA descriptor controller drives readdescriptor table entries on this bus. For more details on this interface, refer to Read Data Mover Avalon-ST Descriptor Sinks on page 45.

• PCIe Write Descriptor Sinks: These are two 174-bit, Avalon-ST sink interfaces (fornormal and priority descriptors). The custom DMA descriptor controller driveswrite descriptor table entries on this bus. For more details on this interface, referto Write Data Mover Avalon-ST Descriptor Sinks on page 49.

• PCIe Read Data Mover Status Source: The Read Data Mover reports status to thecustom DMA descriptor controller on this interface. For more details on thisinterface, refer to Read Data Mover Status Avalon-ST Source on page 47.

• PCIe Write Data Mover Status Source: The Write Data Mover reports status to thecustom DMA descriptor controller on this interface. For more details on thisinterface, refer to Write Data Mover Status Avalon-ST Source on page 52.

4. Interfaces

UG-20237 | 2020.07.13


54


4.3.2. Root Port Mode Interface (256-bit Avalon-MM Interface)

In Gen3 x4 and Gen4 x4 Root Port modes, the IP core uses the 256-bit Avalon-MMbridge instead of the 512-bit Avalon-MM bridge for the performance purpose. In RootPort mode, DMA functionalities are not available. The table below shows the interfacesfor the P-tile 256-bit Avalon-MM bridge.

Table 37. Summary of Interfaces for the 256-bit Avalon-MM Bridge

Interface Name Data Width Burst Count Width Byte Enable Width Wait Request

Bursting Master 256 5 32 Yes

Non-Bursting Slave(optional)

32 N/A 4 Yes

Bursting Slave(optional)

256 5 32 Yes

Control RegisterAccess (CRA)

32 N/A 4 Yes

4.3.2.1. High Performance Avalon-MM Slave (HPTXS) Interface

The High Performance Avalon-MM Slave has a 256-bit-wide data bus. It supports up to16-cycle bursts with dword granularity byte enable on the first and last cycles of awrite burst and for single-cycle read bursts. It also supports optional address mappingwhen the address bus is less than 64-bit wide.

This interface is optional. You enable it by turning On the Enable Bursting Slaveoption in the GUI.

Table 38. High Performance Avalon-MM Slave (HPTXS) Interface


hptxs_address_i[hptxs_address_width_hwtcl-1:0]

I Byte address. Bits [4:0] areassumed to be zeros.

hptxs_slave

hptxs_byteenable_i[31:0]

I Specifies the valid bytes fora write command.

hptxs_read_i I When asserted, specifies aTX Avalon-MM slave readrequest.

hptxs_readdata_o[255:0]

O This bus contains the readcompletion data.

hptxs_write_i I When asserted, specifies aTX Avalon-MM slave writerequest.

hptxs_writedata_i[255:0]

I This bus contains theAvalon-MM data for a writecommand.

continued...

4. Interfaces

UG-20237 | 2020.07.13


55



hptxs_waitrequest_o O When asserted, indicatesthat the Avalon-MM slaveport is not ready to respondto a read or write request.

hptxs_readdatavalid_o O When asserted, indicatesthat the read data is valid.

hptxs_burstcount_i[4:0]

I When asserted, the value onthe response signal is a validwrite response.Writeresponsevalid isonly asserted one clock cycleor more after the writecommand is accepted.There is at least a one clockcycle latency from commandacceptance to the assertionof writeresponsevalid.

4.3.2.2. High Performance Avalon-MM Master (HPRXM) Interface

The bursting Avalon-MM master is always enabled in Root Port mode and is notassociated with any BAR. Packets targeting addresses outside of the range of the baseand limit registers are forwarded to the host via the HPRXM master. The burstingAvalon-MM master has a 256-bit-wide data bus and supports up to 16-cycle burstswith dword granularity byte enable on the first and last cycles of a write burst and onsingle-cycle read bursts. Byte granularity access is supported for single-cycle one-dword or smaller transactions.

Table 39. High Performance Avalon-MM Master (HPRXM) Interface


rxm_write_o O Asserted by the core torequest a write to anAvalon-MM slave.

hprxm_master

rxm_address_o[avmm_addr_width_hwtcl-1:0]

O The address of the Avalon-MM slave being accessed.

rxm_writedata_o[255:0] O This bus contains the RXdata being written to theslave.

rxm_byteenable_o[31:0] O These bits specify the validbytes for the write data.

rxm_burstcount_o[4:0] O The burst count, measuredin qwords, of the RX write orread request. The maximumamount of data in a burst is512 bytes.

rxm_waitrequest_i I When asserted by theexternal Avalon-MM slave,this signal indicates that theslave is not ready for thenext read or write request.

continued...

4. Interfaces

UG-20237 | 2020.07.13


56



rxm_read_o O Asserted by the core torequest a read.

rxm_readdata_i[255:0] I Read data returned from theAvalon-MM slave in responseto a read request. This datais sent to the IP corethrough the TX interface.

rxm_readdatavalid_i I Asserted by the systeminterconnect fabric toindicate that the read data isvalid.

4.3.2.3. 32-Bit Control Register Access (CRA) Slave (Root Port only)

The CRA interface provides access to the control and status registers of the Avalon-MMbridge. This interface has the following properties:

• 32-bit data bus

• Supports a single transaction at a time

• Supports single-cycle transactions (no bursting)

Note: When the Avalon-MM Hard IP for PCIe IP Core is in Root Port mode, and theapplication logic issues a CfgWr or CfgRd via the CRA interface, it needs to fill the Tagfield in the TLP Header with the value 0x10 to ensure that the correspondingCompletion gets routed to the CRA interface correctly. If the application logic sets theTag field to some other value, the Avalon-MM Hard IP for PCIe IP Core does notoverwrite that value with the correct value.

Table 40. Avalon-MM CRA Slave Interface


cra_read_i I Read enable.

cra

cra_write_i I Write request.

cra_address_i[14:0] I

cra_writedata_i[31:0] I Write data. The currentversion of the CRA slaveinterface is read-only.Including this signal as apart of the Avalon-MMinterface makes futureenhancements possible.

cra_readdata_o[31:0] O Read data.

cra_byteenable_i[3:0] I Byte enable.

cra_waitrequest_o O Wait request to hold offadditional requests.

cra_chipselect_i I Chip select signal to thisslave.

cra_irq_o O Interrupt request. A portrequest for an Avalon-MMinterrupt.

4. Interfaces

UG-20237 | 2020.07.13


57


4.4. Serial Data Interface

The P-Tile Avalon-MM IP for PCIe natively supports 4, 8, or 16 PCIe lanes. Each laneincludes a TX differential pair and an RX differential pair. Data is striped across allavailable lanes.

Table 41. Serial Data Interface

Signal Name Direction Description

tx_p_out[<b>-1:0],tx_n_out[<b>-1:0]

O Transmit serial data outputs using theHigh Speed Differential I/O standard.

rx_p_in[<b>-1:0],rx_n_in[<b>-1:0]

I Receive serial data inputs using theHigh Speed Differential I/O standard.

Note: The value of the variable b depends on which configuration is active (1x16, 2x8 or4x4).

• For 1x16, b = 16.

• For 2x8, b = 8.

• For 4x4, b = 4.

4.5. Hard IP Status Interface

This interface includes the signals that are useful for debugging, such as the linkstatus signal, LTSSM state outputs, etc. These signals are available when the optionalPower Management interface is enabled.

Table 42. Hard IP Status Interface

Signal Name Direction Description Clock Domain EP/RP

link_up_o O When asserted, this signal indicates thelink is up.

coreclkout_hip

EP/RP

dl_up_o O When asserted, this signal indicates theData Link (DL) Layer is active.

coreclkout_hip

EP/RP

ltssm_state_o[5:0]

O Indicates the LTSSM state:• 6'h00: S_DETECT_QUIET• 6'h01: S_DETECT_ACT• 6'h02: S_POLL_ACTIVE• 6'h03: S_POLL_COMPLIANCE• 6'h04: S_POLL_CONFIG• 6'h05: S_PRE_DETECT_QUIET• 6'h06: S_DETECT_WAIT• 6'h07: S_CFG_LINKWD_START• 6'h08: S_CFG_LINKWD_ACCEPT• 6'h09: S_CFG_LANENUM_WAIT• 6'h0A: S_CFG_LANENUM_ACCEPT• 6'h0B: S_CFG_COMPLETE• 6'h0C: S_CFG_IDLE• 6'h0D: S_RCVRY_LOCK• 6'h0E: S_RCVRY_SPEED• 6'h0F: S_RCVRY_RCVRCFG

coreclkout_hip

EP/RP

continued...

4. Interfaces

UG-20237 | 2020.07.13


58



• 6'h10: S_RCVRY_IDLE• 6'h11: S_L0• 6'h12: S_L0S• 6'h13: S_L123_SEND_EIDLE• 6'h14: S_L1_IDLE• 6'h15: S_L2_IDLE• 6'h16: S_L2_WAKE• 6'h17: S_DISABLED_ENTRY• 6'h18: S_DISABLED_IDLE• 6'h19: S_DISABLED• 6'h1A: S_LPBK_ENTRY• 6'h1B: S_LPBK_ACTIVE• 6'h1C: S_LPBK_EXIT• 6'h1D: S_LPBK_EXIT_TIMEOUT• 6'h1E: S_HOT_RESET_ENTRY• 6'h1F: S_HOT_RESET• 6'h20: S_RCVRY_EQ0• 6'h21: S_RCVRY_EQ1• 6'h22: S_RCVRY_EQ2• 6'h23: S_RCVRY_EQ3

surprise_down_err_o

O When active, indicates that a surpriselink down event is occurring.

coreclkout_hip

RP

4.6. Interrupt Interface

The P-Tile Avalon-MM IP for PCI Express supports Message Signaled Interrupts (MSI),MSI-X interrupts, and legacy interrupts. MSI and legacy interrupts are mutuallyexclusive.

Legacy interrupts, MSI, and MSI-X interrupts are all controlled and generatedexternally to the Avalon-MM IP to ensure total flexibility of allocating interruptresources based on the user’s application needs.

To support domain-isolation, legacy interrupt messages, MSI, and MSI-X TLPs need tobe sent with the appropriate source IDs.

The following figure shows an example integrating an external interrupt controller withthe P-Tile Avalon-MM IP. The interrupt controller takes interrupt requests from theexternal DMA controller as well as those from the user application.

4. Interfaces

UG-20237 | 2020.07.13


59


Figure 19. Example of an Interrupt Controller Integrated with an Endpoint P-TileAvalon-MM IP for PCI Express IP Core

SM

S

BurstingSlave

P-Tile Avalon-MM IP for PCIe

User’s Interrupt Controller

M

tl_cfg_ctl_o[15:0]

intx_req_i[3:0]

BAR, PF & VF Numbers

PF & VF Numbers

Avalon-MM master for MSI/MSI-X

generation

Interrupt to VFm, nMapping

tl_cfg_add_o[4:0]tl_cfg_func_o[2:0]

BurstingMaster

Interrupt to MSI/MSI-XMapping

MSI-X TableDMA ControllerInterrupts

Interrupts fromUser’s Application

Interrupt Status Registers

4.6.1. Legacy Interrupts

If legacy interrupts are enabled at IP configuration time, the user’s interrupt controllergenerates legacy interrupts by asserting the intx_req_i input signal which causesthe PCIe Hard IP to send the corresponding interrupt message. Use of legacyinterrupts to signal the completion of DMA transfers is not recommended as theirordering with respect to the DMA traffic is not guaranteed.

4.6.2. MSI

If MSI or MSI-X are enabled at IP configuration time, the external interrupt controllercan generate MSI/MSI-X transactions by issuing memory writes to the Bursting Slaveor using the immediate write feature of the Write Data Mover, especially if signalingthe completion of a DMA transfer by the Write Data Mover. The interrupt controllergets the address and data information to generate the MSI/MSI-X messages from theMSI or MSI-X capability registers in the Transaction Layer in the P-Tile IP.

MSI interrupts are signaled on the PCI Express link using a single dword Memory WriteTLP. The user application issues an MSI request (MWr) through the Avalon-ST interfaceand updates the configuration space register using the MSI interface.

For more details on the MSI Capability Structure, refer to Figure 54 on page 114.

The Mask Bits register and Pending Bits register are 32 bits in length each, with eachpotential interrupt message having its own mask bit and pending bit. If bit[0] of theMask Bits register is set, interrupt message 0 is masked. When an interrupt messageis masked, the MSI for that vector cannot be sent. If software clears the mask bit andthe corresponding pending bit is set, the function must send the MSI request at thattime.

4. Interfaces

UG-20237 | 2020.07.13


60


You should obtain the necessary MSI information (such as the message address anddata) from the configuration output interface (tl_cfg_*) to create the MWr TLP inthe format shown below to be sent via the Avalon-ST interface.

Figure 20. Creating a MWr TLP for an MSI Request

MSI Message Data

Message Control Next CapabilityPointer

MSI Capability Structure

MSI (Memory Write) Transaction

Capability ID(05h) DW0

DW1

DW2

DW3Message Data

Byte 16

Byte 12

Byte 8

Byte 4

Byte 0

MSI Message Address [31:0]

Message Address [63:32]

Message Address [31:0]

1631 15 8 7 0

MSI Message Address [63:32]

Tag

+3+2+1+0

Last DW

Length

First DW1 1 1 10 0 0 0

0 0 0 0 0 0 0 0 0 10 0 0 0 00 1 1

7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0 7 6 5 4 3 2 1 0

Requester ID

Fmt Type TC TH

TD

EP ATAt

trAttr00

0000h Data

Header

0 0

R RR

Table 43. MSI Pending Bits Interface


msi_pnd_func_i[2:0] IFunction number select for thePending Bits register in the MSIcapability structure.

coreclkout_hip

EP

msi_pnd_addr_i[1:0] I

Byte select for Pending BitsRegister in the MSI CapabilityStructure. For example ifmsi_pnd_addr_i[1:0] = 00,bits [7:0] of the Pending Bitsregister will be updated withmsi_pnd_byte_i[7:0]. Ifmsi_pnd_addr_i[1:0] = 01,bits [15:8] of the Pending Bitsregister will be updated withmsi_pnd_byte_i[7:0].

coreclkout_hip

EP

msi_pnd_byte_i[7:0] I Indicate that function has apending associated message.

coreclkout_hip

EP

4. Interfaces

UG-20237 | 2020.07.13


61


The following figure shows the timings of msi_pnd_* signals in three scenarios. Thefirst scenario shows the case when the MSI pending bits register is not used. Thesecond scenario shows the case when only physical function 0 is enabled and the MSIpending bits register is used. The last scenario shows the case when four physicalfunctions are enabled and the MSI pending bits register is used.

Figure 21. Example Timing Diagrams for msi_pnd* Signals

0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3

B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3

0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1 0x2 0x3 0x0 0x1

0x10x0

0x2 0x3

0x0 0x1

B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3 B0 B1 B2 B3

0x0

0x0

0x0

0x0

msi_pnd_byte_i[7:0]

msi_pnd_func_i[2:0]

coreclkout_hip

coreclkout_hip

msi_pnd_addr_i[1:0]

msi_pnd_byte_i[7:0]

msi_pnd_func_i[2:0]

msi_pnd_addr_i[1:0]

coreclkout_hip

msi_pnd_byte_i[7:0]

msi_pnd_func_i[2:0]

msi_pnd_addr_i[1:0]

There are 32 possible MSI messages. The number of messages requested by aparticular component does not necessarily correspond to the number of messagesallocated. For example, in the following figure, the Endpoint requests eight MSIs but isonly allocated two. In this case, you must design the Application Layer to use only twoallocated messages.

Figure 22. MSI Request Example

Endpoint

8 Requested2 Allocated

Root Complex

CPU

Interrupt Register

RootPort

InterruptBlock

The following table describes three example implementations. The first exampleallocates all 32 MSI messages. The second and third examples only allocate 4interrupts.

4. Interfaces

UG-20237 | 2020.07.13


62


Table 44. MSI Messages Requested, Allocated, and Mapped

MSI Allocated

32 4 4

System Error 31 3 3

Hot Plug and Power Management Event 30 2 3

Application Layer 29:0 1:0 2:0

MSI interrupts generated for Hot Plug, Power Management Events, and System Errorsalways use Traffic Class 0. MSI interrupts generated by the Application Layer can useany Traffic Class. For example, a DMA that generates an MSI at the end of atransmission can use the same traffic control as was used to transfer data.

The following figure illustrates a possible implementation of the Interrupt HandlerModule with a per vector enable bit in the Application Layer. Alternatively, theApplication Layer could implement a global interrupt enable instead of this per vectorMSI.

Figure 23. Example Implementation of the Interrupt Handler Block

Interrupt Enable 0

Interrupt Request 0

msi_req0

Interrupt Enable 1

Interrupt Request 1

msi_req1

app_int_i

Arbitration & TLP Generator

MSI info (from tl_cfg_ctl* /tl_cfg_addr* / tl_cfg_func*) & Master Enable

Avalon-MMsingle-dword MWR TLPs

msi_pnd_*

Vector 1

Vector 0

IRQGeneration

App Layer

IRQGeneration

App Layer

R/W

R/W

4.6.3. MSI-X

The P-Tile Avalon-MM IP provides a Configuration Intercept Interface. User soft logiccan monitor this interface to get MSI-X Enable and MSI-X function mask relatedinformation. User application logic needs to implement the MSI-X tables for all PFs andVFs at the memory space pointed to by the BARs as a part of your Application Layer.

For more details on the MSI-X related information that you can obtain from theConfiguration Intercept Interface, refer to the MSI-X Registers section in the Registerschapter.

4. Interfaces

UG-20237 | 2020.07.13


63


MSI-X is an optional feature that allows the user application to support large amountof vectors with independent message data and address for each vector.

When MSI-X is supported, you need to specify the size and the location (BARs andoffsets) of the MSI-X table and PBA. MSI-X can support up to 2048 vectors perfunction versus 32 vectors per function for MSI.

A function is allowed to send MSI-X messages when MSI-X is enabled and the functionis not masked. The application uses the Configuration Output Interface (address 0x0Cbit[5:4]) or Configuration Intercept Interface to access this information.

When the application needs to generate an MSI-X, it will use the contents of the MSI-XTable (Address and Data) and generate a Memory Write through the Avalon-STinterface.

You can enable MSI-X interrupts by turning on the Enable MSI-X option under thePCI Express/PCI Capabilities tab in the parameter editor. If you turn on theEnable MSI-X option, you should implement the MSI-X table structures at thememory space pointed to by the BARs as a part of your Application Layer.

The MSI-X Capability Structure contains information about the MSI-X Table and PBAStructure. For example, it contains pointers to the bases of the MSI-X Table and PBAStructure, expressed as offsets from the addresses in the function's BARs. TheMessage Control register within the MSI-X Capability Structure also contains the MSI-XEnable bit, the Function Mask bit, and the size of the MSI-X Table. For a picture of theMSI-X Capability Structure, refer to Figure 56 on page 115.

MSI-X interrupts are standard Memory Writes, therefore Memory Write ordering rulesapply.

Example:

Table 45. MSI-X Configuration

MSI-X Vector MSI-X Upper Address MSI-X Lower Address MSI-X Data

0 0x00000001 0xAAAA0000 0x00000001

1 0x00000001 0xBBBB0000 0x00000002

2 0x00000001 0xCCCC0000 0x00000003

Table 46. PBA Table

PBA Table PBA Entries

Offset 0 0x0

If the application needs to generate an MSI-X interrupt (vector 1), it will read the MSI-X Table information, generate a MWR TLP through the Avalon-ST interface and assertthe corresponding PBA bits (bit[1]) in a similar fashion as for MSI generation.

The generated TLP will be sent to address 0x00000001_BBBB0000 and the data willbe 0x00000002. When the MSI-X has been sent, the application can clear theassociated PBA bits.

4. Interfaces

UG-20237 | 2020.07.13


64


4.6.3.1. Implementing MSI-X Interrupts

Section 6.8.2 of the PCI Local Bus Specification describes the MSI-X capability andtable structures. The MSI-X capability structure points to the MSI-X Table structureand MSI-X Pending Bit Array (PBA) registers. The BIOS sets up the starting addressoffsets and BAR associated with the pointer to the starting address of the MSI-X Tableand PBA registers.

1. Host software sets up the MSI-X interrupts in the Application Layer by completingthe following steps:

a. Host software reads the Message Control register at 0x050 register todetermine the MSI-X Table size. The number of table entries is the <valueread> + 1.

The maximum table size is 2048 entries. Each 16-byte entry is divided in 4fields as shown in the figure below. For multi-function variants, BAR4 accessesthe MSI-X table. For all other variants, any BAR can access the MSI-X table.The base address of the MSI-X table must be aligned to a 4 KB boundary.

b. The host sets up the MSI-X table. It programs MSI-X address, data, andmasks bits for each entry as shown in the figure below.

Figure 24. Format of MSI-X Table

Vector ControlVector ControlVector Control

Vector Control

Message DataMessage DataMessage Data

Message Data

DWORD 3 DWORD 2Message Upper AddressMessage Upper AddressMessage Upper Address

Message Upper Address

DWORD 1Message AddressMessage AddressMessage Address

Message Address

DWORD 0 Host Byte AddressesEntry 0Entry 1Entry 2

Entry (N - 1)

BaseBase + 1 × 16Base + 2 × 16

Base + (N - 1) × 16

c. The host calculates the address of the <nth> entry using the followingformula:

nth_address = base address[BAR] + 16<n>

2. When Application Layer has an interrupt, it drives an interrupt request to the IRQSource module.

3. The IRQ Processor reads the entry in the MSI-X table.

a. If the interrupt is masked by the Vector_Control field of the MSI-X table,the interrupt remains in the pending state.

b. If the interrupt is not masked, IRQ Processor sends Memory Write Request tothe TX slave interface. It uses the address and data from the MSI-X table. IfMessage Upper Address = 0, the IRQ Processor creates a three-dwordheader. If the Message Upper Address > 0, it creates a 4-dword header.

4. The host interrupt service routine detects the TLP as an interrupt and services it.

Related Information

• Floor and ceiling functions

• PCI Local Bus Specification, Rev. 3.0

4. Interfaces

UG-20237 | 2020.07.13


65

http://en.wikipedia.org/wiki/Floor_and_ceiling_functions

http://www.pcisig.com/home


4.7. Hot Plug Interface (RP Only)

Note: The Hot Plug interface is not supported in the 20.2 release of Intel Quartus Prime. Itmay be supported in a future release.

Hot Plug support means that the device can be added to or removed from a systemduring runtime. The Hot Plug Interface in the P-Tile Avalon-MM IP for PCIe allows anIntel FPGA with this IP to safely provide this capability.

This section describes the signals reported by the on-board hot plug components inthe Downstream Port. This interface is available only if the Slot Status Registerof the PCI Express Capability Structure is enabled.

Refer to the Slot Status Register of the PCI Express CapabilityStructure for additional information.

Table 47. Hot Plug Interface


sys_atten_button_pressed_i I Attention ButtonPressed. Indicatesthat the systemattention button waspressed, and setsthe Attention ButtonPressed bit in theSlot StatusRegister.

coreclkout_hip RP

sys_pwr_fault_det_i I Power FaultDetected. Indicatesthe power controllerdetected a powerfault at this slot.

coreclkout_hip RP

sys_mrl_sensor_chged_i I MRL SensorChanged. Indicatesthat the state of theMRL sensor haschanged.

coreclkout_hip RP

sys_pre_det_chged_i I Presence DetectChanged. Indicatesthat the state of thecard presencedetector haschanged.

coreclkout_hip RP

sys_cmd_cpled_int_i I CommandCompleted Interrupt.Indicates that theHot Plug controllercompleted acommand.

coreclkout_hip RP

sys_pre_det_state_i I Indicates whether ornot a card is presentin the slot.0 : slot is empty.1 : card is present inthe slot.

coreclkout_hip RP

continued...

4. Interfaces

UG-20237 | 2020.07.13


66



sys_mrl_sensor_state_i I MRL Sensor State.Indicates the stateof the manuallyoperated retentionlatch (MRL) sensor.0 : MRL is closed.1 : MRL is open.

coreclkout_hip RP

sys_eml_interlock_engaged_i

I Indicates whetherthe systemelectromechanicalinterlock is engaged,and controls thestate of theelectromechanicalinterlock status bit inthe Slot StatusRegister.

coreclkout_hip RP

sys_aux_pwr_det_i I Auxiliary PowerDetected. Used toreport to the hostsoftware thatauxiliary power(Vaux) is present.Refer to the DeviceStatus Registerin the PCI ExpressCapabilityStructure.

coreclkout_hip RP

4.8. Power Management Interface

Note: The Power Management interface is not available in the 20.2 release of Intel QuartusPrime. However, it may be available in a future release.

Software programs the device into a D-state by writing to the Power ManagementControl and Status register in the PCI Power Management CapabilityStructure. The power management output signals indicate the current power state.The IP core supports the two mandatory power states: D0 (full power) and D3(preparation for a loss of power). It does not support the optional D1 and D2 low-power states.

The correspondence between the device power states (D states) and link power states(L states) is as follows:

Table 48. Relationship Between Device and Link Power States

Device Power State Link Power State

D0 L0

D1 (not supported) L1

D2 (not supported) L1

D3 L1, L2/L3 Ready

P-Tile does not support ASPM.

4. Interfaces

UG-20237 | 2020.07.13


67


Table 49. Power Management Interface


pm_state_o[2:0] O Indicates the current powerstate. coreclkout_hip EP/RP

x16/x8:pm_dstate_o[31:0]

x4:pm_dstate_o[3:0]

O

Power management D-statefor each function.• 0001b : D0• 0010b : D1• 0100b : D2• 1000b : D3• 0000b : uninitialized or

invalid

Async EP/RP

x16/x8:apps_pm_xmt_pme_i[7:0]

x4: NA

I

The application logic assertsthis signal for one cycle towake up the PowerManagement Capability (PMC)state machine from a D1, D2,or D3 power state. Uponwake-up, the IP core sends aPM_PME message.

coreclkout_hip EP

x16/x8:app_ready_entr_l23_i

x4: NA

I

The application logic assertsthis signal to indicate that it isready to enter the L2/L3Ready state. Theapp_ready_entr_l23_isignal is provided forapplications that must controlthe L2/L3 Ready entry (in casecertain tasks must beperformed before going intoL2/L3 Ready). The core delayssending PM_Enter_L23 (inresponse to PM_Turn_Off) untilthis signal becomes active.This is a level-sensitive signal.

coreclkout_hip EP

x16:app_req_retry_en_i[7:0]

x8:app_req_retry_en_i

x4: NA

I

When these signals areasserted, the P-Tile Avalon-MMIP will respond toConfiguration TLPs with aConfiguration Retry Status(CRS) if it is not ready torespond with non-CRS statussince the last reset.For x4 ports, this signal is notused and needs to be driven tozero.

Async EP

4.9. Configuration Output Interface

The Transaction Layer configuration output (tl_cfg) bus provides a subset of theinformation stored in the Configuration Space. Use this information in conjunction withthe app_err* signals to understand TLP transmission problems.

4. Interfaces

UG-20237 | 2020.07.13


68


Table 50. Configuration Output Interface


tl_cfg_ctl_o[15:0]

O

Multiplexed data output from theregister specified bytl_cfg_add_o[4:0]. Thedetailed information for each fieldin this bus is defined in thefollowing table.

coreclkout_hip EP/RP

tl_cfg_add_o[4:0] O

This address bus contains theindex indicating whichConfiguration Space registerinformation is being driven ontothe tl_cfg_ctl_o[15:0] bits.


x16/x8:tl_cfg_func_o[2:0]

x4: NA

O

Specifies the function whoseConfiguration Space register valuesare being driven out ontl_cfg_ctl_o[15:0].• 3'b000: Physical Function 0

(PF0)• 3'b001: PF1and so on

Note:

In the 19.4 release of IntelQuartus Prime, the P-TileAvalon-MM IP only supportsPF0.


The table below provides the tl_cfg_add_o[4:0] to tl_cfg_ctl_o[15:0]mapping.

Table 51. Multiplexed Configuration Information Available on tl_cfg_ctl

tl_cfg_add_o[4:0] tl_cfg_ctl_o[15:8] tl_cfg_ctl_o[7:0]

5'h00

[15]: memory space enable[14]: IDO completion enable[13]: perr_en[12]: serr_en[11]: fatal_err_rpt_en[10]: nonfatal_err_rpt_en[9]: corr_err_rpt_en[8]: unsupported_req_rpt_en

Device control:[7]: bus master enable[6]: extended tag enable[5:3]: maximum read request size[2:0]: maximum payload size

5'h01

[15]: IDO request enable[14]: No Snoop enable[13]: Relaxed Ordering enable[12:8]: Device number

bus number

5'h02

[15]: pm_no_soft_rst[14]: RCB control[13]: Interrupt Request (IRQ) disable[12:8]: PCIe Capability IRQ messagenumber

[7:5]: reserved[4]: system power control[3:2]: system attention indicatorcontrol[1:0]: system power indicator control

5'h03 Number of VFs [15:0]

5'h04[15]: reserved[14]: AtomicOP Egress Block field(cfg_atomic_egress_block)

[7]: ARI forward enable[6]: Atomic request enable[5:3]: TPH ST mode[2:1]: TPH enable

continued...

4. Interfaces

UG-20237 | 2020.07.13


69



[13:9]: ATS Smallest Translation Unit(STU)[4:0][8]: ATS cache enable

[0]: VF enable

5'h05

[15:12]: auto negotiation link speed.Link speed encoding values are:• Gen1 : 0x1• Gen2 : 0x2• Gen3 : 0x4• Gen4 : 0x8[11:1]: Index of Start VF [10:0][0]: reserved

5'h06 MSI Address [15:0]




5'h0A MSI Mask [15:0]

5'h0B MSI Mask [31:16]

5'h0C

[15]: cfg_send_f_err[14]: cfg_send_nf_err[13]: cfg_send_cor_err[12:8]: AER IRQ message number

[7]: Enable extended message data forMSI (cfg_msi_ext_data_en)[6]: MSI-X func mask[5]: MSI-X enable[4:2]: Multiple MSI enable[1]: 64-bit MSI[0]: MSI enable

5'h0D MSI Data [15:0]

5'h0E AER uncorrectable error mask [15:0]

5'h0F AER uncorrectable error mask [31:16]

5'h10 AER correctable error mask [15:0]

5'h11 AER correctable error mask [31:16]

5'h12 AER uncorrectable error severity [15:0]

5'h13 AER uncorrectable error severity [31:16]

5'h14 [15:8]: ACS Egress Control Register(cfg_acs_egress_ctrl_vec)

[7]: ACS function group enable(cfg_acs_func_grp_en)[6]: ACS direct translated P2P enable(cfg_acs_p2p_direct_tranl_en)[5]: ACS P2P egress control enable(cfg_acs_egress_ctrl_en)[4]: ACS upstream forwarding enable(cfg_acs_up_forward_en)[3]: ACS P2P completion redirectenable(cfg_acs_p2p_compl_redirect_en)[2]: ACS P2P request redirect enable(cfg_acs_p2p_req_redirect_en)[1]: ACS translation blocking enable(cfg_acs_at_blocking_en)

continued...

4. Interfaces

UG-20237 | 2020.07.13


70



[0]: ACS source validation enable (RP)(cfg_acs_validation_en)

5'h15

[15]: reserved[14]: 10-bit tag requester enable(cfg_10b_tag_req_en)[13]: VF 10-bit tag requester enable(cfg_vf_10b_tag_req_en)[12]: PRS_RESP_FAILURE(cfg_prs_response_failure)[11]: PRS_UPRGI (cfg_prs_uprgi)[10]: PRS_STOPPED(cfg_prs_stopped)[9]: PRS_RESET (cfg_prs_reset)[8]: PRS_ENABLE (cfg_prs_enable)

[7:3]: reserved[2:0]: ARI function group(cfg_ari_func_grp)

5'h16PRS_OUTSTANDING_ALLOCATION(cfg_prs_outstanding_allocation) [15:0]

5'h17PRS_OUTSTANDING_ALLOCATION(cfg_prs_outstanding_allocation) [31:16]

5'h18

[15:10]: reserved[9]: Disable autonomous generation ofLTR clear message(cfg_disable_ltr_clr_msg)[8]: LTR mechanism enable(cfg_ltr_m_en)

[7]: Infinite credits for Posted header[6]: Infinite credits for Posted data[5]: Infinite credits for Completionheader[4]: Infinite credits for Completion data[3]: End-end TLP prefix blocking(cfg_end2end_tlp_pfx_blck)[2]: PASID enable(cfg_pf_pasid_en)[1]: Execute permission enable(cfg_pf_passid_execute_perm_en)[0]: Privileged mode enable(cfg_pf_passid_priv_mode_en)

5'h19

[15:9]: reserved[8]: Slot control attention buttonpressed enable(cfg_atten_button_pressed_en)

[7]: Slot control power fault detectenable (cfg_pwr_fault_det_en)[6]: Slot control MRL sensor changedenable (cfg_mrl_sensor_chged_en)[5]: Slot control presence detectchanged enable(cfg_pre_det_chged_en)[4]: Slot control hot plug interruptenable (cfg_hp_int_en)[3]: Slot control command completedinterrupt enable(cfg_cmd_cpled_int_en)[2]: Slot control DLL state changeenable (cfg_dll_state_change_en)[1]: Slot control accessed(cfg_hp_slot_ctrl_access)[0]: PF’s SERR# enable(cfg_br_ctrl_serren)

5'h1A LTR maximum snoop latency register(cfg_ltr_max_latency[15:0])

continued...

4. Interfaces

UG-20237 | 2020.07.13


71



5'h1BLTR maximum no-snoop latencyregister(cfg_ltr_max_latency[31:16])

5'h1C [15:8]: enabled Traffic Classes (TCs)(cfg_tc_enable[7:0])

[5:0]: auto negotiation link width6’h01 = x16’h02 = x26’h04 = x46’h08 = x86’h10 = x16

5'h1D MSI Data[31:16]

5'h1E N/A

5'h1F N/A

Note: The information on the Configuration Output (tl_cfg) bus is time-divisionmultiplexed (TDM).

• When tl_cfg_func[2:0] = 3'b000, tl_cfg_ctl[31:0] drive out the PF0Configuration Space register values.

• Then, tl_cfg_func[2:0] are incremented to 3'b001.

• When tl_cfg_func[2:0] = 3'b001, tl_cfg_ctl[31:0] drive out the PF1Configuration Space register values.

• This pattern repeats to cover all enabled PFs.

• The P-Tile Avalon-MM IP for PCIe only supports PF0.

Figure 25. Configuration Output Interface Timing Diagram

coreclkout_hip

tl_cfg_add_0[4:0]

tl_cfg_ctl_0[15:0]

tl_cfg_func_0[2:0]

0x00 0x01 0x02 0x00 0x01 0x020x03

PF0 DATA0 PF0 DATA1 PF0 DATA2 PF1 DATA0 PF1 DATA1 PF1 DATA2PF0 DATA3

0 1

4.10. Hard IP Reconfiguration Interface

The Hard IP reconfiguration interface is an Avalon-MM slave interface with a 21-bitaddress and an 8-bit data bus. It is also sometimes referred to as the User Avalon-MMInterface. You can use this interface to dynamically modify the value of configurationregisters. Note that after a warm reset or cold reset, changes made to theconfiguration registers of the Hard IP via the Hard IP reconfiguration interface are lostas these registers revert back to their default values.

Note: This interface can be used in Endpoint and Root Port modes. It must be enabled ifRoot Port mode is selected.

In Root Port mode, the application logic uses the Hard IP reconfiguration interface toaccess its PCIe configuration space to perform link control functions (such as HotReset, link disable, or link retrain).

4. Interfaces

UG-20237 | 2020.07.13


72


Table 52. Hard IP Reconfiguration Interface


hip_reconfig_clk I

Reconfigurationclock50 MHz - 125 MHz(Range)100 MHz(Recommended)

EP/RP

hip_reconfig_readdata_o[7:0] O Avalon-MM readdata outputs

hip_reconfig_clk

EP/RP

hip_reconfig_readdatavalid_o O

Avalon-MM readdata valid. Whenasserted, the dataonhip_reconfig_readdata_o[7:0] isvalid.

hip_reconfig_clk

EP/RP

hip_reconfig_write_i I Avalon-MM writeenable

hip_reconfig_clk

EP/RP

hip_reconfig_read_i I Avalon-MM readenable

hip_reconfig_clk

EP/RP

hip_reconfig_address_i[20:0] I Avalon-MM address hip_reconfig_clk

EP/RP

hip_reconfig_writedata_i[7:0] I Avalon-MM writedata inputs

hip_reconfig_clk

EP/RP

hip_reconfig_waitrequest_o O

When asserted, thissignal indicates thatthe IP core is notready to respond toa request.

hip_reconfig_clk

EP/RP

dummy_user_avmm_rst I

Reset signal. Youcan tie it to groundor leave it floatingwhen using the HardIP ReconfigurationInterface.

EP/RP

Reading and Writing to the Hard IP Reconfiguration Interface

Reading from the Hard IP reconfiguration interface of the P-Tile Avalon-MM IP for PCIExpress retrieves the current value at a specific address. Writing to the reconfigurationinterface changes the data value at a specific address. Intel recommends that youperform read-modify-writes when writing to a register, because two or more featuresmay share the same reconfiguration address.

Modifying the PCIe configuration registers directly affects the behavior of the PCIedevice.

4. Interfaces

UG-20237 | 2020.07.13


73


Figure 26. Timing Diagram to Perform Read and Write Operations Using the Hard IPReconfiguration Interface

4.10.1. Address Map for the User Avalon-MM Interface

The User Avalon-MM interface provides access to the configuration registers and the IPcore registers. This interface includes an 8-bit data bus and a 21-bit address bus(which contains the byte addresses).

There are two methods to access the configuration registers:

• Using direct User Avalon-MM interface (byte access)

• Using the Debug (DBI) register access (dword access). This method is useful whenyou need to read/write the entire 32 bits at one time (Counter/ Lane Margining,etc.)

The following diagram and table show the address offsets for physical function 0(PF0), User Avalon-MM Port Configuration Register and Debug (DBI) Register.

4. Interfaces

UG-20237 | 2020.07.13


74


Figure 27. Address Map for the User Avalon-MM Interface

User Avalon-MM Port Configuration Register

PF0 PCie Configuration Registers 0x000000

0x001000

0x104068

0x1fffff

Debug_DBI_Data/Debug_DBI_Addr0x104200

4. Interfaces

UG-20237 | 2020.07.13


75


Table 53. Configuration Space Offsets

Registers User Avalon-MM Offsets Comments

Physical function 0 0x0000 Refer to Appendix A for more details ofthe PF configuration space. This PF isavailable for x16, x8 and x4 cores.

User Avalon-MM Port ConfigurationRegister

0x104068 Refer to User Avalon-MM PortConfiguration Register (Offset0x104068) for more details.

Debug (DBI) Register 0x104200 to 0x104204 Refer to Using the Debug RegisterInterface Access on page 76 for moredetails.

4.10.2. Configuration Registers Access

4.10.2.1. Using Direct User Avalon-MM Interface (Byte Access)

Targeting PF Configuration Space Registers

User application needs to specify the offsets of the targeted PF registers.

For example, if the application wants to read the MSI Capability Register of PF0, it willissue a Read with address 0x0050 to target the MSI Capability Structure of PF0.

Figure 28. PF Configuration Space Registers Access Timing DiagramHIP reconfig clk

HIP reconfig addr 0x000050

HIP reconfig read

HIP reconfig readdata PF0 MSI Cap

HIP reconfig readdatavalid

Targeting VSEC Registers

User application needs to program the VSEC field (0x104068 bit[0]) first. Then allaccesses from the user Avalon-MM interface starting at offset 0xD00 will be translatedto VSEC configuration space registers.

Figure 29. VSEC Registers Access Timing Diagram

HIP reconfig clk

HIP reconfig addr 0x104068 0xD00

HIP reconfig write

HIP reconfig writedata 0x01

HIP reconfig read

HIP reconfig waitrequest

HIP reconfig readdata VSEC Cap


4.10.2.2. Using the Debug Register Interface Access

DEBUG_DBI_ADDR register is located at user Avalon-MM offsets 0x104204 to0x104207 (corresponding to byte 0 to byte 3). For example, the d_done bit is bit 7 atbyte address 0x104207.

4. Interfaces

UG-20237 | 2020.07.13


76


Table 54. DEBUG_DBI_ADDR Register

Names Bits R/W Descriptions

d_done 31 RO 1: indicates debug DBI read/write access done

d_write 30 R/W 1: write access0: read access

d_warm_reset 29 RO 1: normal operation0: warm reset is on-going

d_vf 28:18 R/W Specify the virtual functionnumber.

d_vf_select 17 R/W To access the virtual functionregisters, set this bit to one.

d_pf 16:14 R/W Specify the physical functionnumber.

reserved 13:12 R/W Reserved

d_addr 11:2 R/W Specify the DW address forthe P-Tile Avalon-MM IP DBIinterface.

d_shadow_select 1 R/W Reserved. Clear this bit foraccess to standard PCIeconfiguration registers.

d_vsec_select 0 R/W If set, this bit allows accessto Intel VSEC registers.

DEBUG_DBI_DATA register is located at user Avalon-MM offsets 0x104200 to0x104203 (corresponding to byte 0 to byte 3).

Table 55. DEBUG_DBI_DATA Register

Names Bits R/W Descriptions

d_data 31:0 R/W Read or write data for the P-Tile Avalon-MM IP registeraccess.

To write all 32 bits in a Debug register at a time:

1. Use the user_avmm interface to access 0x104200 to 0x104203 to write the datafirst.

2. Use the user_avmm interface to access 0x104204 to 0x104206 to set the addressand control bits.

3. Use the user_avmm interface to write to 0x104207 to enable the read/write bit(bit[30]).

4. Use the user_avmm interface to access 0x104207 bit[31] to poll if the write iscomplete.

4. Interfaces

UG-20237 | 2020.07.13


77


Figure 30. DBI Register Write Timing DiagramHIP reconfig clk

HIP reconfig addr 0x104200 0x104201 0x104202 0x104203 0x104204 0x104205 0x104206 0x104207

HIP reconfig write

HIP reconfig writedata 0x01 0x23 0x45 0x67 ADDR CTRL 0x4


HIP reconfig read

HIP reconfig readdata 0x4 0xC


To read all 32 bits in a Debug register at a time:

1. Use the user_avmm interface to access 0x104204 to 0x104206 to set the addressand control bits.

2. Use the user_avmm interface to write to 0x104207 to enable the read bit(bit[30]).

3. Use the user_avmm interface to access 0x104207 bit[31] to poll if the read iscomplete.

4. Use the user_avmm interface to access 0x104200 to 0x104203 to read the data

Figure 31. DBI Register Read Timing DiagramHIP reconfig clk

HIP reconfig addr 0x104204 0x104205 0x104206 0x104207 0x104207 0x104200 0x104201 0x104202 0x104203

HIP reconfig write

HIP reconfig writedata ADDR CTRL 0x0


HIP reconfig read

HIP reconfig readdata 0x0 0x8 D0 D1 D2 D3


4.11. PHY Reconfiguration Interface

The PHY reconfiguration interface is an optional Avalon-MM slave interface with a26-bit address and an 8-bit data bus. Use this bus to read the value of PHY registers.Refer to Table 60 on page 87 for details on addresses and bit mappings for the PHYregisters that you can access using this interface.

These signals are present when you turn on Enable PHY reconfiguration on theTop-Level Settings tab using the parameter editor.

Please note that the PHY reconfiguration interface is shared among all the PMA quads.

Table 56. PHY Reconfiguration Interface


xcvr_reconfig_clk IReconfiguration clock50 MHz - 125 MHz (Range)100 MHz (Recommended)

EP/RP

xcvr_reconfig_readdata[7:0]

O Avalon-MM read dataoutputs

xcvr_reconfig_clk

EP/RP

xcvr_reconfig_readdatavalid

O

Avalon-MM read data valid.When asserted, the data onxcvr_reconfig_readdata[7:0] is valid.

xcvr_reconfig_clk

EP/RP

xcvr_reconfig_write I Avalon-MM write enable xcvr_reconfig_clk

EP/RP

continued...

4. Interfaces

UG-20237 | 2020.07.13


78



xcvr_reconfig_read I

Avalon-MM read enable.This interface is notpipelined. You must wait forthe return of thexcvr_reconfig_readdata[7:0] from the currentread before starting anotherread operation.

xcvr_reconfig_clk

EP/RP

xcvr_reconfig_address[25:0]

I

Avalon-MM address[25:21] are used to indicatethe Quad.5'b00001 : Quad 05'b00010 : Quad 15'b00100 : Quad 25'b01000 : Quad 3[20:0] are used to indicatethe offset address.

xcvr_reconfig_clk

EP/RP

xcvr_reconfig_writedata[7:0]

I Avalon-MM write data inputs xcvr_reconfig_clk

EP/RP

xcvr_reconfig_waitrequest O

When asserted, this signalindicates that the PHY is notready to respond to arequest.

xcvr_reconfig_clk

EP/RP

Reading from the PHY Reconfiguration Interface

Reading from the PHY reconfiguration interface of the P-Tile Avalon-MM IP for PCIExpress retrieves the current value at a specific address.

Figure 32. Timing Diagram to Perform Read Operations Using the PHY ReconfigurationInterface

0x000006

0x01

xcvr_reconfig_clk

xcvr_reconfig_address

xcvr_reconfig_read

xcvr_reconfig_readdatavalid

xcvr_reconfig_readdata


4. Interfaces

UG-20237 | 2020.07.13


79


5. Advanced Features

5.1. PCIe Port Bifurcation and PHY Channel Mapping

Note: Port bifurcation for Gen3 x8, Gen4 x8 and Gen4 x4 will be available in a future releaseof Intel Quartus Prime.

The PCIe controller IP contains a set of port bifurcation muxes to remap the fourcontroller PIPE lane interfaces to the shared 16 PCIe PHY lanes. The table below showsthe relationship between PHY lanes and the port mapping.

Table 57. Port Bifurcation and PHY Channel Mapping

Bifurcation Mode Port 0 (x16) Port 1 (x8) Port 2 (x4) Port 3 (x4)

1 x16 0 - 15 NA NA NA

2 x8 0 - 7 8 - 15 NA NA

4 x4 4 - 7 8 - 11 0 - 3 12 - 15

Note: For more details on the bifurcation modes, refer to the Architecture section in chapter2.

UG-20237 | 2020.07.13

Send Feedback







6. Troubleshooting/DebuggingAs you bring up your PCI Express system, you may face issues related to FPGAconfiguration, link training, BIOS enumeration, data transfer, and so on. This chaptersuggests some strategies to resolve the common issues that occur during bring-up.

You can additionally use the P-Tile Debug Toolkit to identify the issues.

6.1. Hardware

Typically, PCI Express link-up involves the following steps:

1. Link training

2. BIOS enumeration and data transfer

The following sections describe the flow to debug link issues during the hardwarebring-up. Intel recommends a systematic approach to diagnosing issues as illustratedin the following figure.

Additionally, you can use the P-Tile Debug Toolkit for debugging the PCIe links whenusing the P-Tile Avalon-MM IP for PCI Express. The P-Tile Debug Toolkit includes thefollowing features:

• Protocol and link status information.

• Basic and advanced debugging capabilities including PMA register access and Eyeviewing capability.

UG-20237 | 2020.07.13

Send Feedback







Figure 33. PCI Express Debug Flow Chart

No

No

Yes

Yes

Go to “8.1.2 Debugging data transfer and

performance issues”

Go to “8.1.1 Debugging Link training issues”

System Reset

Start

End

Is the data transfer

successful?

Is the link training

successful?

6.1.1. Debugging Link Training Issues

The Physical Layer automatically performs link training and initialization withoutsoftware intervention. This is a well-defined process to configure and initialize thedevice's Physical Layer and link so that PCIe packets can be transmitted.

Some examples of link training issues include:

• Link fails to negotiate to expected link speed.

• Link fails to negotiate to the expected link width.

• LTSSM fails to reach/stay stable at L0.

Flow Chart for Debugging Link Training Issues

Use the flow chart below to identify the potential cause of the issue seen during linktraining when using the P-Tile Avalon-MM IP for PCI Express.

6. Troubleshooting/Debugging

UG-20237 | 2020.07.13


82


Figure 34. Link Training Debugging Flow

Observation: ltssm_state_o signal toggles between Detect.Quiet and Detect.Active. Check the Receiver detection status from the registers for successful receiver detection

Issue: Far end receiver not detected by the FPGA TX

Resolution: Check coupling capacitance, far end termination resistance and TX OCT values are in accordance to the spec

Observation: ltssm_state_o signal transitions from Detect.Quiet –> Detect.Active –> Polling.Active –> Polling.Compliance states.

Issue: Far end device failing receiver detection

Resolution: Check far end coupling capacitance, near end termination resistance and TX OCT valuesare in accordance to the spec

Observation: ltssm_state_o signal stuck at Detect.Quiet state

Issue: IP is in reset state

Resolution: Check if the pin_perst_n reset signal is in reset

No

NoNo

Yes

Yes

YesIs the linkout of reset?

Does the linktrain to L0 at

reduced speed?

Is therereceiver detected

at the far end?

A.Observation: Loop of Detect.Quiet –> Detect.Active –> Polling.Active –> Recovery.rcvrlock transitions observed on ltssm_state_o signal

Issue: Poor refclk quality

Resolution: Check the reference clock quality is good. (e.g. Jitter, phase noise, etc). Ensure that the clocks used are in accordance with the guidelines described in the User Guide

B.Observation: Timeout during EQ Phases on few lanes when monitoring ltssm_state_o signal

Issue: Signal Integrity issues/Sub optimal EQ settings on few lanes

Resolution: Redo the Equalization (*)

Yes Yes

No

No

No

Yes

A.Observation: Frequent transitions on ltssm_state_o signal between L0 and Recovery.rcvrlock states

Issue: Signal Integrity issues (or) sub optimal EQ settings


A.Observation: Wrong lane numbers encoded in TS1/TS2 (Observed using Protocol Analyzer)

Issue: Improper lane reversal

Resolution: Check the lane routing

B.Observation: Timeout during EQ Phases on few lanes when monitoring ltssm_state_o signal

Issue: Signal Integrity issues/Sub optimal EQ settings on few lanes


Begin

OR

OR

Does the LTSSM

enter L0?

Does link go to L0 at

advertised speed with frequent recoveries?

Does the link train to

L0 with reduced lane width?

End

Note: (*) Redo the equalization using the Link Equalization Request 8.0 GT/s bitof the Link Status 2 register for 8.0 GT/s or Link Equalization Request 16.0 GT/sbit of the 16.0 GT/s Status Register.

Use the following debug tools for debugging link training issues observed on the PCIExpress link when using the P-tile Avalon-MM IP for PCI Express.


UG-20237 | 2020.07.13


83


6.1.1.1. Generic Tools and Utilities

You can use utilities like lspci, setpci to obtain general information of the device likelink speed, link width etc.

Example: To read the negotiated link speed for the P-Tile device in a system, you canuse the following commands:

sudo lspci –s $bdf -vvv

-s refers to “slot” and is used with the bus/device/function number (bdf) information.Use this command if you know the bdf of the device in the system topology.

sudo lspci –d <1172>:$did -vvv

-d refers to device and is used with the device ID (vid:did). Use this command tosearch using the device ID.

Figure 35. lspci Output

The LnkCap under Capabilities indicates the advertised link speed and widthcapabilities of the device. The LnkSta under Capabilities indicates the negotiatedlink speed and width of the device.

6.1.1.2. SignalTapII Logic Analyzer

Using the SignalTapII Logic Analyzer, you can monitor the following top-level signalsfrom the P-Tile Avalon-MM IP for PCI Express to confirm the failure symptom for anyport type (Root port, Endpoint or TLP Bypass) and configuration (Gen4/Gen3).


UG-20237 | 2020.07.13


84


Table 58. Top-Level Signals to be Monitored for Debugging

Signals Description Expected Value for Successful Link-up

pin_perst_n Active-low asynchronous input signalto the PCIe Hard IP.Implements the PERST# functiondefined by the PCIe specification.

1'b1

p0_reset_status_n Active-low output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Held low until pin_perst_n isdeasserted and the PCIe Hard IPcomes out of reset, synchronous tocoreclkout_hip.When port bifurcation is used, there isone such signal for each Avalon-MMinterface.

1'b1

ninit_done Active-low output signal from the PCIeHard IP. High indicates that the FPGAdevice is not yet fully configured, andlow indicates the device has beenconfigured and is in normal operatingmode.

1'b0

link_up_o Active-high output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Indicates that the Physical Layer link isup.

1'b1

dl_up_o Active-high output signal from the PCIeHard IP, synchronous tocoreclkout_hip.Indicates that the Data Link Layer isactive.

1'b1

ltssm_state_o[5:0] Indicates the LTSSM state,synchronous to coreclkout_hip.

6'h11 (L0)

6.1.1.3. Additional Debug Tools

Use the Hard IP reconfiguration interface and PHY reconfiguration interface on the P-Tile Avalon-MM IP for PCI Express to access additional registers (for example, receiverdetection, lane reversal etc.).

Figure 36. Register Access for Debug

PCIe Controllers

Port N

Registers

MACPMA

Quad Nx16PCIePCS

Hard IP ReconfigInterface

PHY ReconfigInterface

PLLAPCIe

x16 Lanes

PHY

PLLBDLL TL

PHYRegisters

Using the Hard IP Reconfiguration Interface


UG-20237 | 2020.07.13


85


Refer to the section Hard IP Reconfiguration Interface for details on this interface andthe associated address map.

The following table lists the address offsets and bit settings for the PHY statusregisters. Use the Hard IP Reconfiguration Interface to access these read-onlyregisters.

Table 59. Hard IP Reconfiguration Interface Register Map for PHY Status

Offset Bit Position Register

0x0003E9 [0] RX polarity

[1] RX detection

[2] RX Valid

[3] RX Electrical Idle

[4] TX Electrical Idle

0x0003EC [7] Framing error

0x0003ED [7] Lane reversal

Follow the steps below to access registers in Table 59 on page 86 using the Hard IPreconfiguration interface:

1. Enable the Hard IP reconfiguration interface (User Avalon-MM interface) using theIP Parameter Editor.

2. Set the lane number for which you want to read the status by performing a read-modify-write to the address hip_reconfig_addr_i[20:0] with write data oflane number on hip_reconfig_writedata_i[7:0] using the Hard IPreconfiguration interface signals.

• hip_reconfig_write_i = 1’b1

• hip_reconfig_addr_i[20:0] = 0x0003E8

• hip_reconfig_writedata_i[3:0] = <Lane number>, where Lane number= 4’h0 for lane 0, 4’h1 for lane 1, 4’h2 for lane 2, …

3. Read the status of the register you want by performing a read operation from theaddress hip_reconfig_addr_i[20:0] using the Hard IP reconfigurationinterface signals.

• hip_reconfig_read_i = 1’b1

• hip_reconfig_addr_i[20:0] = <offset>

Offset = Refer to Table 59 on page 86 for the offset mapping.

• hip_reconfig_readdata_o[7:0] = Refer to Table 59 on page 86 for thebit position mapping.

Example 1: To read the RX detection status of Lane0 using the registers

1. Enable the Hard IP reconfiguration interface using the IP Parameter Editor.

2. Perform read-modify-write to address 0x0003E8 to set the lane number to 0 usingthe Hard IP reconfiguration interface signals.


UG-20237 | 2020.07.13


86


• hip_reconfig_write_i = 1’b1


• hip_reconfig_writedata_i[3:0] = 4'h0

3. Read the status of the RX detection register by performing a read operation fromthe address 0x0003E9[1] using the Hard IP reconfiguration interface signals.

• hip_reconfig_read_i = 1’b1


• hip_reconfig_readdata_o[1] = 1'b1 (Far end receiver detected)

Using the PHY Reconfiguration Interface

Refer to the section PHY Reconfiguration Interface for details on how to use thisinterface.

Follow the steps below to access registers in Table 60 on page 87 using the PHYreconfiguration interface.

1. Enable the PHY reconfiguration interface using the IP Parameter Editor.

2. Set the Quad and address offset from which you want to read the status byperforming a read operation from the address xcvr_reconfig_addr_i[25:0]using the PHY reconfiguration interface signals.

• xcvr_reconfig_read_i = 1’b1

• xcvr_reconfig_addr_i[25:0] = {5-bit Quad mapping, 21-bit addressoffset}. Refer to Table 60 on page 87 for the address offset and bit mapping.

• xcvr_reconfig_readdata_o[7:0] = Refer to Table 60 on page 87 for theaddress offset and bit mapping.

Table 60. PHY Reconfiguration Interface Register Map for PHY Status

PHY Offset Bit Position Register

0x000006 [7] PLLA state output status signal.1'b1 indicates that PLLA is locked.

0x00000a [7] PLLB state output status signal.1'b1 indicates that PLLB is locked.

Example 2: To read the PLLA status using the registers

1. Enable the PHY reconfiguration interface using the IP Parameter Editor.

2. Perform a read from address 0x000006 to read the PLLA status output of Quad0using the PHY reconfiguration interface signals.

• xcvr_reconfig_read_i = 1'b1

• xcvr_reconfig_addr_i[25:0] = 0x000006

• xcvr_reconfig_readdata_o[7:0] = 8'h80

• xcvr_reconfig_readdata_i = 1'b1 (PLLA state output high indicating PLLlock)


UG-20237 | 2020.07.13


87


6.1.2. Debugging Data Transfer and Performance Issues

There are many possible reasons causing the PCIe link to stop transmitting data. ThePCI Express base specification defines three types of errors, outlined in the tablebelow:

Table 61. Error Types Defined by the PCI Express Base Specification

Type Responsible Agent Description

Correctable Hardware While correctable errors may affectsystem performance, data integrity ismaintained.

Uncorrectable, non-fatal Device software Uncorrectable, non-fatal errors aredefined as errors in which data is lost,but system integrity is maintained. Forexample, the fabric may lose aparticular TLP, but it still works withoutproblems.

Uncorrectable, fatal System software Errors generated by a loss of data andsystem failure are considereduncorrectable and fatal. Software mustdetermine how to handle such errors:whether to reset the link or implementother means to minimize the problem.

Table 62. Correctable Error Status Register (AER)

Observation Issue Resolution

Receiver error bit set Physical layer error which may be dueto a PCS error when a lane is in L0, ora Control symbol being received in thewrong lane, or signal Integrity issueswhere the link may transition from L0to the Recovery state.

Use the Hard IP reconfigurationinterface and the flow chart in Figure34 on page 83 to obtain moreinformation about the error.

Bad DLLP bit set Data link layer error which may occurwhen a CRC verification fails.

Use the Hard IP reconfigurationinterface to obtain more informationabout the error.

Bad TLP bit set Data link layer error which may occurwhen an LCRC verification fails or whena sequence number error occurs.


Replay_num_rollover bit set Data link layer error which may be dueto TLPs sent without success (no ACK)four times in a row.


replay timer timeout status bit set Data link layer error which may occurwhen no ACK or NAK was receivedwithin the timeout period for the TLPstransmitted.


Advisory non-fatal Transaction layer error which may bedue to higher priority uncorrectableerror detected.

Corrected internal error bits set Transaction layer error which may bedue to an ECC error in the internalHard IP RAM.

Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information about the error.


UG-20237 | 2020.07.13


88


Table 63. Uncorrectable Error Status Register (AER)


Data link protocol error Data link layer error which may be dueto transmitter receiving an ACK/NAKwhose Seq ID does not correspond toan unacknowledged TLP or ACKsequence number.


Surprise down error Data link layer error which may be dueto link_up_o getting deassertedduring L0, indicating the physical layerlink is going down unexpectedly.

Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information about the error.

Flow control protocol error Transaction layer error which can bedue to the receiver reporting morethan the allowed credit limit.This error occurs when a componentdoes not receive updated flow controlcredits with the 200 μs limit.

Use the TX/RX flow control interface,Hard IP reconfiguration interface toobtain more information about theerror.

Poisoned TLP received Transaction layer error which can bedue to a received TLP with the EP bitset.

Use the Hard IP reconfigurationinterface to obtain more information onthe error and determine theappropriate action.

Completion timeout Transaction layer error which can bedue to a completion not received withinthe required amount of time after anon-posted request was sent.

Use the Hard IP reconfigurationinterface to obtain more information onthe error.

Completer abort Transaction layer error which can bedue to a completer being unable tofulfill a request due to a problem withthe requester or a failure of thecompleter.


Unexpected completion Transaction layer error which can bedue to a requester receiving acompletion that doesn’t match anyrequest awaiting a completion.The TLP is deleted by the Hard IP andnot presented to the Application Layer.


Receiver overflow Transaction layer error which can bedue to a receiver receiving more TLPsthan the available receive buffer space.The TLP is deleted by the Hard IP andnot presented to the Application Layer.

Use the TX/RX flow control interfaceand Hard IP reconfiguration interfaceto obtain more information on theerror.

Malformed TLP Transaction layer error which can bedue to errors in the received TLPheader.The TLP is deleted by the Hard IP andnot presented to the Application Layer.


ECRC error Transaction layer error which can bedue to an ECRC check failure at thereceiver despite the fact that the TLP isnot malformed and the LCRC check isvalid.The Hard IP block handles this TLPautomatically. If the TLP is a non-posted request, the Hard IP blockgenerates a completion with acompleter abort status. The TLP isdeleted by the Hard IP and notpresented to the Application Layer.


continued...


UG-20237 | 2020.07.13


89



Unsupported request Transaction layer error which can bedue to the completer being unable tofulfill the request.The TLP is deleted in the Hard IP blockand not presented to the ApplicationLayer. If the TLP is a non-postedrequest, the Hard IP block generates acompletion with Unsupported Requeststatus.


ACS violation Transaction layer error which can bedue to access control error in thereceived posted or non-posted request.


Uncorrectable internal error Transaction layer error which can bedue to an internal error that cannot becorrected by the hardware.

Use the Hard IP reconfigurationinterface and DBI registers to obtainmore information on the error.

Atomic egress blocked Use the Hard IP reconfigurationinterface to obtain more information onthe error.

TLP prefix blocked EP or RP only Use the Hard IP reconfigurationinterface to obtain more information onthe error.

Poisoned TLP egress blocked EP or RP only Use the Hard IP reconfigurationinterface to obtain more information onthe error.

Use the debug tools mentioned in the next two sections for debugging link trainingissues observed on the PCI Express link when using the P-Tile Avalon-MM IP for PCIExpress.

6.1.2.1. Advanced Error Reporting (AER)

Each PCI Express compliant device must implement a basic level of error managementand can optionally implement advanced error management. The PCI ExpressAdvanced Error Reporting Capability is an optional Extended Capability that may beimplemented by PCI Express device functions supporting advanced error control andreporting.

The P-Tile Avalon-MM IP for PCI Express implements both basic and advanced errorreporting. Error handling for a Root Port is more complex than that of an Endpoint. Inthis P-Tile Avalon-MM IP for PCI Express, the AER capability is enabled by default.

Use the AER capability of the PCIe Hard IP to identify the type of error and theprotocol stack layer in which the error may have occurred. Refer to the PCI ExpressCapability Structures section of the Configuration Space Registers appendix for theAER Extended Capability Structure and the associated registers.

6.1.2.2. Second-Level Debug Tools

Use the following debug tools for second-level debug of any issue observed on the PCIExpress link when using P-Tile:

Using the Hard IP Reconfiguration Interface

• Refer to the section Hard IP Reconfiguration Interface on page 72 for details onthis interface and the address map.


UG-20237 | 2020.07.13


90


Using the PHY Reconfiguration Interface

• Refer to the section PHY Reconfiguration Interface on page 78 for details on thisinterface and the address map.

6.2. Debug Toolkit

6.2.1. Overview

The P-Tile Debug Toolkit is a System Console-based tool for P-Tile that provides real-time control, monitoring and debugging of the PCIe links at the Physical, Data Linkand Transaction layers.

The P-Tile Debug Toolkit allows you to:

• View protocol and link status of the PCIe links per port.

• View PLL and per-channel status of the PCIe links per port.

• Control the channel analog settings.

• View the receiver eye and measure the eye height and width.

• Indicate the presence of a re-timer connected between the link partners.

The following figure provides an overview of the P-Tile Debug Toolkit in the P-TileAvalon-MM IP for PCI Express.

Figure 37. Overview of the P-Tile Debug Toolkit

P-Tile Debug Toolkit

NPDME

hip-reconfig_*

intel_pcie_ptile_avmm

AVMMPCle Config Space Registers

(Port 0)

AVMMPCle Config Space Registers

(Port 3)

AVMM PHY Registers (Quad 0)

AVMM PHY Registers (Quad 3)

...

...

System Console

GUI

xcvr_reconfig_*

When you enable the P-Tile Debug Toolkit, the intel_pcie_ptile_avmm module ofthe generated IP includes the Debug Toolkit modules and related logic as shown in thefigure above.

Drive the Debug Toolkit from a System Console. The System Console connects to theDebug Toolkit via an Native PHY Debug Master Endpoint (NPDME). Make thisconnection via an Intel FPGA Download Cable.


UG-20237 | 2020.07.13


91


This PHY reconfiguration interface clock (xcvr_reconfig_clk) is used to clock thefollowing interfaces:

• The NPDME module

• PHY reconfiguration interface (xcvr_reconfig)

• Hard IP reconfiguration interface (hip_reconfig)

Provide a clock source (50 MHz - 125 MHz, 100 MHz recommended clock frequency) todrive the xcvr_reconfig_clk clock. Use the output of the Reset Release Intel FPGAIP to drive the ninit_done, which provides the reset signal to the NPDME module.

Note: When using the port bifurcation feature, always connect the xcvr_reconfig_clk ofPort0 to a clock source. This signal is used to provide the clock to the Debug Toolkit.

Note: When you enable the P-Tile Debug Toolkit, the Hard IP reconfiguration interface isenabled by default.

When you run a dynamically-generated design example on the Intel Development Kit,make sure that clock and reset signals are connected to their respective sources andappropriate pin assignments are made. Here are some sample .qsf assignments forthe Debug Toolkit for Intel Stratix 10 DX devices:

• set_location_assignment PIN_A31 -to p0_hip_reconfig_clk_clk

• set_location_assignment PIN_C23 -to xcvr_reconfig_clk_clk

• #set_instance_assignment -name VIRTUAL_PIN ON -to *p0_hip_reconfig*

• #set_instance_assignment -name VIRTUAL_PIN ON -to *xcvr_reconfig*

6.2.2. Enabling the P-Tile Debug Toolkit

To enable the P-Tile Debug Toolkit in your design, enable the option Enable DebugToolkit in the PCIe Configuration, Debug and Extension options tab of the IntelFPGA P-Tile Avalon-MM IP for PCI Express.

When using bifurcated ports, you can enable the Debug Toolkit for each bifurcatedport by enabling the option Enable Debug Toolkit on each of the bifurcated ports.

Note: When you enable the P-Tile Debug Toolkit in the IP, the Hard IP reconfigurationinterface and the PHY reconfiguration interface will be used by the Debug Toolkit.Hence, you will not be able to drive logic on these interfaces from the FPGA fabric.

6.2.3. Launching the P-Tile Debug Toolkit

Use the design example you compiled by following the Quick Start Guide to familiarizeyourself with the P-Tile Debug Toolkit. Follow the steps in the Generating the DesignExample and Compiling the Design Example to generate the SRAM Object File, (.sof)for this design example.

To use the P-Tile Debug Toolkit, download the .sof to the Intel Development Kit. Then,open the System Console and load the design to the System Console as well. Loadingthe .sof to the System Console allows the System Console to communicate with thedesign using NPDME. NPDME is a JTAG-based Avalon-MM master. It drives Avalon-MMslave interfaces in the PCIe design. When using NPDME, the Intel Quartus Primesoftware inserts the debug interconnect fabric to connect with JTAG.


UG-20237 | 2020.07.13


92


Here are the steps to complete these tasks:

1. Use the Intel Quartus Prime Programmer to download the .sof to the Intel FPGADevelopment Kit.

Note: To ensure correct operation, use the same version of the Intel QuartusPrime Programmer and Intel Quartus Prime Pro Edition software that youused to generate the .sof.

2. To load the design into System Console:

a. Launch the Intel Quartus Prime Pro Edition software.

b. Start System Console by choosing Tools, then System Debugging Tools,then System Console.

c. On the System Console File menu, select Load design and browse to the .soffile.

d. Select the .sof and click OK. The .sof loads to the System Console.

3. The System Console Toolkit Explorer window will list all the DUTs in the designthat have the P-Tile Debug Toolkit enabled.

a. Select the DUT with the P-Tile Debug Toolkit you want to view. This will openthe Debug Toolkit instance of that DUT in the Details window.


UG-20237 | 2020.07.13


93


b. Click on the ptile_debug_toolkit_avmm to open that instance of the Toolkit.Once the Debug Toolkit is initialized and loaded, you will see the followingmessage in the Messages window: “Initializing P-Tile debug toolkit –done”.

c. A new window Main view will open with a view of all the channels in thatinstance.


UG-20237 | 2020.07.13


94


6.2.4. Using the P-Tile Debug Toolkit

The following sections describe the different tabs and features available in the DebugToolkit.

A. Main View

The main view tab lists a summary of the transmitter and receiver settings perchannel for the given instance of the PCIe IP.

The following table shows the channel mapping when using bifurcated ports.

Table 64. Channel Mapping for Bifurcated Ports

Toolkit Channel X16 Mode 2X8 Mode 4x4 Mode

Lane 0 Lane 0 Lane 0 Lane 0









continued...


UG-20237 | 2020.07.13


95


Toolkit Channel X16 Mode 2X8 Mode 4x4 Mode








B. Toolkit Parameters

The Toolkit parameters window has 2 sub-tabs.

B.1. P-Tile Information

This lists a summary of the P-Tile PCIe IP parameter settings in the PCIe IP ParameterEditor when the IP was generated, as read by the P-Tile Debug Toolkit when initialized.

When using bifurcated ports, you will see all the P-Tile information for each port forwhich the Debug Toolkit has been enabled.

All the information is read-only.

Use the Get P-tile Info button to read the settings.

Table 65. P-Tile Available Parameter Settings

Parameter Values Descriptions

Intel Vendor ID 1172 Indicates the Vendor ID as set in the IPParameter Editor.

Protocol PCIe Indicates the Protocol.

HIP Type Root Port, End Point Indicates the Hard IP Port type.

Intel IP Type intel_pcie_ptile_ast,intel_pcie_ptile_avmm Indicates the IP type used.

Advertised speed Gen3, Gen4 Indicates the advertised speed asconfigured in the IP Parameter Editor.

Advertised width x16, x8, x4 Indicates the advertised width asconfigured in the IP Parameter Editor.

Negotiated speed Gen3, Gen4 Indicates the negotiated speed duringlink training.

Negotiated width x16, x8, x4 Indicates the negotiated link widthduring link training.

Link status Link up, link down Indicates if the link (DL) is up or not.

Retimer 1 Detected, not detectedIndicates if a retimer was detected

between the Root Port and theEndpoint.

Retimer 2 Detected, not detectedIndicates if a retimer was detected

between the Root Port and theEndpoint.


UG-20237 | 2020.07.13


96


Figure 38. Example of P-Tile Parameter Settings

B.2. PCIe Configuration Space

This lists a summary of the P-Tile PCIe configuration settings of the PCIe configurationspace registers, as read by the P-Tile Debug Toolkit when initialized.

All the information is read-only.

Use the Read cfg space button to read the settings.


UG-20237 | 2020.07.13


97


Figure 39. Example of P-Tile PCIe Configuration Settings

C. Channel Parameters

The channel parameters window allows you to monitor and control the transmitter andreceiver settings for a given channel. It has the following 2 sub-windows.

C.1. TX Path

This tab allows you to monitor and control the transmitter settings for the channelselected. Use the TX Refresh button to read the settings, TX Apply Ch to apply thesettings to the selected channel, and TX apply all to apply the settings to allchannels.


UG-20237 | 2020.07.13


98


Table 66. Transmitter Settings

Parameters Values Descriptions

PHY Status

Refclk enable Enable, Disable

Indicates reference clock isenabled for the PHY.

Enable: Reference clock isenabled for the PHY.

Disable: Reference clock isdisabled for the PHY.

PHY reset Normal, Reset

Indicates the PHY is in resetmode.

Normal: PHY is out of reset.Reset: PHY is in reset.

TX Status

TX Lane enable Enable, Disable

Indicates if TX lane isenabled in the PHY.

Enable: TX lane is enabled inthe PHY.

Disable: TX lane is disabledin the PHY.

TX Data enable Enable, Disable

Indicates if TX driver isenabled and serial data is

transmitted.Enable: TX driver for the

corresponding lane isenabled.

Disable: TX driver for thecorresponding lane is

disabled.

TX Reset Normal, Reset

Indicates if TX (TX datapath,TX settings) is in reset ornormal operating mode.Normal: TX is in normal

operating mode.Reset: TX is in reset.

TX PLL

TX PLL enable Enable, Disable

Indicates if the TX PLL ispowered on or powered

down. This is dependent onthe PLL selected as indicated

by TX PLL select.There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL

status corresponding to thatQuad.

• TX path for Ch0 to 3:Status of PLLs in Quad0




Enable: TX PLL is poweredon.

Disable: TX PLL is powereddown.

TX PLL selectPLLA: Gen1/Gen2PLLB: Gen3/Gen4

Indicates which PLL isselected.

continued...


UG-20237 | 2020.07.13


99



There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL






TX PLL lock Green, Red

Indicates if TX PLL is locked.This is dependent on the PLLselected as indicated by TX

PLL select.There is one set of PLLs perQuad. The TX path of eachchannel reads out the PLL






Green: TX PLL is locked.Red: TX PLL is not locked.

TX VOD

Iboost levelGen3: 15Gen4: 15

Indicates the transmittercurrent boost level when theTX amplitude boost mode is

enabled.

Vboost enGen3 EnableGen4 Enable

Indicates if the TX swingboost level is enabled.

Enable: TX swing boost isenabled.

Disable: TX swing boost isdisabled.

Vboost levelGen3: 5Gen4: 5

Indicates the TX Vboostlevel.

TX Equalization

Pre-shoot coefficientGen3: 20 (Preset 8)Gen4: 0 (Preset 0)

Indicates transmitter driveroutput pre-emphasis (pre-

shoot coefficient).

Main coefficientGen3: 30 (Preset 8)Gen4: 30 (Preset 0)

Indicates transmitter driveroutput pre-emphasis (main

coefficient).

Post coefficientGen3: 20 (Preset 8)Gen4: 40 (Preset 0)

Indicates transmitter driveroutput pre-emphasis (post

coefficient).


UG-20237 | 2020.07.13


100


Figure 40. Example of Transmitter Settings

C.1. RX Path

This tab allows you to monitor and control the receiver settings for the channelselected. Use the RX Refresh button to read the settings, RX Apply Ch to apply thesettings to the selected channel, and RX apply all to apply the settings to allchannels.

Table 67. Receiver Settings


RX Status

RX Lane enable Enable, Disable

Indicates if RX lane isenabled in the PHY.

Enable: RX lane is enabledin the PHY.

Disable: RX lane is disabledin the PHY.

RX Data enable Enable, Disable

Indicates if RX driver isenabled and serial data is

transmitted.Enable: RX driver for the

corresponding lane isenabled.

Disable: RX driver for thecorresponding lane is

disabled.

continued...


UG-20237 | 2020.07.13


101



RX Reset Normal, Reset

Indicates if RX (RX datapath,RX settings) is in reset ornormal operating mode.Normal: RX is in normal

operating mode.Reset: RX is in reset.

RX LOS <1,0>

Indicates if the receiver haslost the signal.

1: Receiver loss of signal.0: Receiver has a data

signal.

RX CDR

CDR Lock Green, RedIndicates the CDR lock state.

Green: CDR is locked.Red: CDR is not locked.

CDR Mode Locked to Reference (LTR),Locked to Data (LTD)

Indicates the CDR lockmode.

LTR: CDR is locked toreference clock.

LTD: CDR is locked to data.

RX Equalization

Adapt Mode

Gen3: Gen3 adaptationmode.

Gen4: Gen4 adaptationmode.

Indicates the RX adaptationmode.

Adapt ContinuousGen3: 1Gen4: 1

Indicates if the receiver is incontinuous adaptation.

• 0 - continuous adaptationoff.

• 1 - continuous adaptationon.

RX ATTGen3: 0Gen4: 0

Indicates the RX equalizationattenuation level.

RX CTLE BoostGen3: 12Gen4: 16

Indicates the RX CTLE boostvalue.

RX CTLE PoleGen3: 2Gen4: 2

Indicates the RX CTLE polevalue.

RX VGA1Gen3: 5Gen4: 5

Indicates the RX AFE firststage VGA gain value.

RX VGA2Gen3: 5Gen4: 5

Indicates the RX AFE secondstage VGA gain value.

RX FOM <0-255>

Indicates the Receiver Figureof Merit (FOM) / quality ofthe received data eye. A

higher value indicates betterlink equalization, with 8'd0

indicating the worstequalization setting and

8'd255 indicating the bestequalization setting.

DFE Enable Enable, Disable

Indicates DFE adaptation isenabled for taps 1 - 5.

Enable: DFE adaptation isenabled for taps 1 - 5.

continued...


UG-20237 | 2020.07.13


102



Disable: DFE adaptation isdisabled for taps 1 - 5.

DFE Tap1 adapted value <-128 to 127>

Indicates the adapted valueof DFE tap 1. This is a

signed input (two'scomplement encoded).














UG-20237 | 2020.07.13


103


Figure 41. Example of Receiver Settings

Eye Viewer

The P-Tile Debug Toolkit supports running eye tests for Intel devices with P-Tile. TheEye Viewer tool allows you to set up and run eye tests, monitoring bit errors.

1. In the System Console Tools menu option, click on Eye View Tool.


UG-20237 | 2020.07.13


104


Figure 42. Opening the Eye Viewer

2. This will open a new tab Eye View Tool next to the Main View tab. Choose theinstance and channel for which you want to run the eye view tests.

Figure 43. Opening the Instance and Channel

3. Choose the eye vertical step setting from the drop-down menu. The eye view toolallows you to choose between vertical step sizes of 1, 2, 4, 8.

Note: The time taken for the eye view tool to draw the eye varies with differentvertical step sizes (1 results in a faster eye plot when compared to 8).

Figure 44. Choosing the Step Size

4. The messages window displays information messages to indicate the eye viewtool's progress.


UG-20237 | 2020.07.13


105


Figure 45. Eye View Tool Messages

5. Once the eye plot is complete, the eye height, eye width and eye diagram aredisplayed.

Figure 46. Sample Eye Plot

6.2.5. Enabling the P-Tile Link Inspector

To enable the Link Inspector, enable the option Enable Debug Toolkit and EnablePCIe Link Inspector in the PCIe Configuration, Debug and Extension Optionstab.

Figure 47. Enabling the Link Inspector


UG-20237 | 2020.07.13


106


6.2.6. Using the P-Tile Link Inspector

The Link Inspector is found under the PCIe Link Inspector tab after opening theDebug Toolkit:

Figure 48. View of the Link Inspector

When the Dump LTSSM Sequence to Text File button is initially clicked, a text file(ltssm_sequence_dump_p*.txt) with the LTSSM information is created in thelocation from where the System Console window is opened. Depending on the PCIetopology, there can be up to four text files. Subsequent LTSSM sequence dumps willappend to the respective files.


UG-20237 | 2020.07.13


107


Figure 49. Example LTSSM Sequence Dump (Beginning)


UG-20237 | 2020.07.13


108


Figure 50. Example LTSSM Sequence Dump (End)

Each LTSSM monitor has a FIFO storing the time values and captured LTSSM states.The FIFO is written when there is a state transition. When you want to dump theLTSSM sequence, a single read of the FIFO status of the respective core is performed.Depending on the empty status and how many entries are in the FIFO, successivereads are executed.


UG-20237 | 2020.07.13


109


A. Configuration Space Registers

A.1. Configuration Space Registers

In addition to accessing the Endpoint's configuration space registers by sendingConfiguration Read/Write TLPs via the Avalon-ST interface, the application logic canalso gain read access to these registers via the Configuration Output Interface(tl_cfg*). Furthermore, the Hard IP Reconfiguration Interface (a User Avalon-MMinterface) also provides read/write access to these registers.

For signal timings on the User Avalon-MM interface, refer to the Avalon InterfaceSpecifications document.

The table PCIe Configuration Space Registers describes the registers for each PF. Tocalculate the address for a particular register in a particular PF, add the offset for thatPF from the table Configuration Space Offsets to the byte address for that register asgiven in the table PCIe Configuration Space Registers.

Table 68. Configuration Space Offsets

Registers User Avalon-MM Offsets

Physical function 0 0x00000








Port Configuration and Status Register 0x104000

Debug (DBI) Register 0x104200, 0x104204

Table 69. PCIe Configuration Space Registers for x16/x8/x4 Controllers

Byte Address Hard IP Configuration SpaceRegister

Corresponding Section in PCIeSpecification

x16 (Port 0) = 0x000 : 0x03Cx8 (Port 1) = 0x000 : 0x03C

x4 (Ports 2,3) = 0x000 : 0x03C

PCI Header Type 0/1 ConfigurationRegisters

Type 0/1 Configuration Space Header

x16 (Port 0) = 0x040 : 0x044x8 (Port 1) = 0x040 : 0x044

x4 (Ports 2,3) = 0x040 : 0x044

Power Management PCI Power Management CapabilityStructure

continued...

UG-20237 | 2020.07.13

Send Feedback









x16 (Port 0) = 0x050 : 0x064x8 (Port 1) = 0x050 : 0x064

x4 (Ports 2,3) = 0x050 : 0x064

MSI Capability MSI Capability Structure, see also PCILocal Bus Specification

x16 (Port 0) = 0x070 : 0x0A8x8 (Port 1) = 0x070 : 0x0A8

x4 (Ports 2,3) = 0x070 : 0x0A8

PCI Express Capability PCI Express Capability Structure

x16 (Port 0) = 0x0B0 : 0x0B9x8 (Port 1) = 0x0B0 : 0x0B9

x4 (Ports 2,3) = 0x0B0 : 0x0B9

MSI-X Capability MSI-X Capability Structure, see alsoPCI Local Bus Specification

x16 (Port 0) = 0x0BC : 0x0FCx8 (Port 1) = 0x0BC : 0x0FC

x4 (Ports 2,3) = 0x0BC : 0x0FC

Reserved N/A

x16 (Port 0) = 0x100 : 0x144x8 (Port 1) = 0x100 : 0x144

x4 (Ports 2,3) = 0x100 : 0x144

Advanced Error Reporting (AER) Advanced Error Reporting CapabilityStructure

x16 (Port 0) = 0x148 : 0x164x8 (Port 1) = 0x148 : 0x164

x4 (Ports 2,3) = 0x148 : 0x164

Virtual Channel Capability Virtual Channel Capability Structure

x16 (Port 0) = 0x178 : 0x17Cx8 (Port 1) = 0x178 : 0x17C

x4 (Ports 2,3) = N/A

Alternative Routing-ID Implementation(ARI)

ARI Capability Structure

x16 (Port 0) = 0x188 : 0x1B4x8 (Port 1) = 0x188 : 0x1A4

x4 (Ports 2,3) = 0x188 : 0x1A4

Secondary PCI Express ExtendedCapability Header

PCI Express Extended Capability

x16 (Port 0) = 0x1B8 : 0x1E4x8 (Port 1) = 0x1A8 : 0x1CC

x4 (Ports 2,3) = 0x1A8 : 0x1C8

Physical Layer 16.0 GT/s ExtendedCapability

Physical Layer 16.0 GT/s ExtendedCapability Structure

x16 (Port 0) = 0x1E8 : 0x22Cx8 (Port 1) = 0x1D0 : 0x1F4

x4 (Ports 2,3) = 0x1CC : 0x1E0

Margining Extended Capability Margining Extended CapabilityStructure

x16 (Port 0) = 0x230 : 0x26Cx8 (Port 1) = 0x1F8 : 0x234


SR-IOV Capability SR-IOV Capability Structure

x16 (Port 0) = 0x270 : 0x2F8x8 (Port 1) = 0x238 : 0x2C0

x4 (Ports 2,3) = 0x1E4 : 0x26C

TLP Processing Hints (TPH) Capability TLP Processing Hints (TPH) CapabilityStructure

x16 (Port 0) = 0x2FC : 0x300x8 (Port 1) = 0x2C4 : 0x2C8


Address Translation Services (ATS)Capability

Address Translation Services ExtendedCapability (ATS) in Single Root I/O

Virtualization and Sharing Specification

x16 (Port 0) = 0x30C : 0x314x8 (Port 1) = 0x2D4 : 0x2DC

x4 (Ports 2,3) = 0x280 : 0x288

Access Control Services (ACS)Capability

Access Control Services (ACS)Capability

x16 (Port 0) = 0x318 : 0x324x8 (Port 1) = 0x2E0 : 0x2EC


Page Request Services (PRS) Capability Page Request Services (PRS) Capability

continued...


UG-20237 | 2020.07.13


111




x16 (Port 0) = 0x328 : 0x32Cx8 (Port 1) = 0x2F0 : 0x2F4


Latency Tolerance Reporting (LTR)Capability

Latency Tolerance Reporting (LTR)Capability

x16 (Port 0) = 0x330 : 0x334x8 (Port 1) = 0x2F8 : 0x2FC


Process Address Space (PASID)Capability

Process Address Space (PASID)Capability Structure

x16 (Port 0) = 0x338 : 0x434x8 (Port 1) = 0x300 : 0x3FC

x4 (Ports 2,3) = 0x2AC : 0x3A8

RAS D.E.S. Capability (VSEC)

x16 (Port 0) = 0x470 : 0x478x8 (Port 1) = 0x438 : 0x440

x4 (Ports 2,3) = 0x3E4 : 0x3EC

Data Link Feature Extended Capability

x16 (Port 0) = 0xD00 : 0xD58x8 (Port 1) = 0xD00 : 0xD58

x4 (Ports 2,3) = 0xD00 : 0xD58

Intel-defined VSEC

A.1.1. Register Access Definitions

This document uses the following abbreviations when describing register accesses.

Table 70. Register Access Abbreviations

Abbreviation Meaning

RW Read and write access

RO Read only

WO Write only

RW1C Read write 1 to clear

RW1CS Read write 1 to clear sticky

RWS Read write sticky

Note: Sticky bits are not initialized or modified by hot reset.

A.1.2. PCIe Configuration Header Registers

The Corresponding Section in PCIe Specification column in the tables in theConfiguration Space Registers section lists the appropriate sections of the PCI ExpressBase Specification that describe these registers.


UG-20237 | 2020.07.13


112


Figure 51. PCIe Type 0 Configuration Space Registers - Byte Address Offsets and Layout

0x0000x0040x0080x00C0x0100x0140x0180x01C0x0200x0240x0280x02C0x0300x0340x0380x03C

Device ID Vendor IDStatus Command

Class Code Revision ID

0x00 Header Type 0x00 Cache Line SizeBAR RegistersBAR RegistersBAR RegistersBAR RegistersBAR RegistersBAR Registers

ReservedSubsystem Device ID Subsystem Vendor ID

ReservedReserved

Reserved

Capabilities Pointer

0x00 Interrupt Pin Interrupt Line

31 24 23 16 15 8 7 0

Figure 52. PCIe Type 1 Configuration Space Registers - Byte Address Offsets and Layout

0x00000x004

Device ID31 24 23 16 15 8 7 0

0x0080x00C0x0100x0140x0180x01C0x0200x0240x0280x02C0x0300x0340x038

0x03C

Vendor ID

BIST Header Type Primary Latency Timer Cache Line Size

Status CommandClass Code Revision ID

BAR RegistersBAR Registers

Secondary Latency Timer Subordinate Bus Number Secondary Bus Number Primary Bus NumberSecondary Status I/O Limit I/O Base

Memory Limit Memory Base

Prefetchable Base Upper 32 BitsPrefetchable Limit Upper 32 Bits

I/O Limit Upper 16 Bits I/O Base Upper 16 BitsReserved Capabilities Pointer

Expansion ROM Base AddressBridge Control Interrupt Pin Interrupt Line

Prefetchable Memory Limit Prefetchable Memory Base

Related Information

PCI Express Base Specification 4.0


UG-20237 | 2020.07.13


113

http://www.pcisig.com/


A.1.3. PCI Express Capability Structures

The layouts of the most basic Capability Structures are provided below. Refer to thePCI Express Base Specification for more information about these registers.

Figure 53. Power Management Capability Structure - Byte Address Offsets and Layout

0x040

0x04C

Capabilities Register Next Cap Ptr

Data

31 24 23 16 15 8 7 0Capability ID

Power Management Status and ControlPM Control/StatusBridge Extensions

Figure 54. MSI Capability Structure

0x050

0x0540x058

Message ControlConfiguration MSI Control Status

Register Field DescriptionsNext Cap Ptr

Message AddressMessage Upper Address

Reserved Message Data

31 24 23 16 15 8 7 0

0x05C

Capability ID

Figure 55. PCI Express Capability Structure - Byte Address Offsets and LayoutIn the following table showing the PCI Express Capability Structure, registers that are not applicable to adevice are reserved.

0x070

0x0740x0780x07C0x0800x0840x0880x08C0x0900x0940x0980x09C0x0A00x0A4

0x0A8

PCI Express Capabilities Register Next Cap Pointer

Device CapabilitiesDevice Status Device Control

Slot Capabilities

Root StatusDevice Compatibilities 2

Link Capabilities 2Link Status 2 Link Control 2

Slot Capabilities 2

Slot Status 2 Slot Control 2

31 24 23 16 15 8 7 0PCI Express

Capabilities ID

Link CapabilitiesLink Status Link Control

Slot Status Slot Control

Device Status 2 Device Control 2

Root Capabilities Root Control


UG-20237 | 2020.07.13


114


Figure 56. MSI-X Capability Structure

0x0B0

0x0B4

0x0B8

Message Control Next Cap Ptr

MSI-X Table Offset

MSI-X Pending Bit Array (PBA) Offset

31 24 23 16 15 8 7 0Capability ID

3 2

MSI-XTable BAR

IndicatorMSI-X

PendingBit Array

- BARIndicator

Figure 57. PCI Express AER Extended Capability Structure

0x100

0x104 Uncorrectable Error Status RegisterPCI Express Enhanced Capability Register

Uncorrectable Error Severity Register

Uncorrectable Error Mask Register0x108

0x10C

0x110

0x114

0x118

0x11C

0x12C

0x130

0x134

Correctable Error Status Register

Correctable Error Mask Register

Advanced Error Capabilities and Control Register

Header Log Register

Root Error Command Register

Root Error Status Register

Error Source Identification Register Correctable Error Source Identification Register

31 16 15 0

Related Information

PCI Express Base Specification 4.0


UG-20237 | 2020.07.13


115

http://www.pcisig.com/


A.1.4. Physical Layer 16.0 GT/s Extended Capability Structure

Figure 58. Physical Layer 16.0 GT/s Extended Capability Structure

A.1.5. MSI-X Registers

This section describes the registers previously shown in the MSI-X capability structure.

Table 71. MSI-X Control Register

Bit Location Description Access Default Value

31 MSI-X Enable: This bit mustbe set to enable the MSI-X

interrupt generation.

RW 0

30 MSI-X Function Mask: Thisbit can be set to mask allMSI-X interrupts from this

function.

RW 0

29:27 Reserved RO 0

26:16 Size of the MSI-X table(number of MSI-X interruptvectors). The value in this

field is one less than the sizeof the table set up for thisfunction. Maximum value is

0x7FF (2048 interruptvectors).

This field is shared amongall VFs attached to one PF.

RO Programmed via theprogramming interface.

15:8 Next Capability PointerPoints to the PCI Express

Capability.


7:0 Capability ID assigned byPCI-SIG.

RO 0x11


UG-20237 | 2020.07.13


116


Table 72. MSI-X Table Offset Register


2:0 BAR Indicator Register:Specifies the BAR

corresponding to thememory address range

where the MSI-X table ofthis function is located (000= VF BAR0, 001 = VF BAR1,

…, 101 = VF BAR5).This field is shared amongall VFs attached to one PF.


31:3 Offset of the memoryaddress where the MSI-X

table is located, relative tothe specified BAR. Theaddress is extended by

appending three zeroes tomake it Qword aligned.



Table 73. MSI-X Pending Bit Array Register


2:0 BAR Indicator Register:Specifies the BAR

corresponding to thememory address range

where the Pending Bit Arrayof this function is located

(000 = VF BAR0, 001 = VFBAR1, …, 101 = VF BAR5).This field is shared amongall VFs attached to one PF.


31:3 Offset of the memoryaddress where the PendingBit Array is located, relativeto the specified BAR. Theaddress is extended by

appending three zeroes tomake it Qword aligned.



A.2. Intel-Defined VSEC Capability Registers

Table 74. Intel-Defined VSEC Capability Registers (0xD00 : 0xD58)

31 : 20 19 : 16 15 : 8 7 : 0 PCIe Byte Offset

Next Cap Offset Version PCI Express Extended Capability ID 00h

VSEC Length VSEC Rev VSEC ID 04h

Intel Marker 08h

JTAG Silicon ID DW0 0Ch

JTAG Silicon ID DW1 10h


continued...


UG-20237 | 2020.07.13


117


31 : 20 19 : 16 15 : 8 7 : 0 PCIe Byte Offset


CvP Status User Configurable Device/Board ID 1Ch

CvP Mode Control 20h

CvP Data 2 24h

CvP Data 28h

CvP Programming Control 2Ch

General Purpose Control and Status 30h

Uncorrectable Internal Error Status Register 34h

Uncorrectable Internal Error Mask Register 38h

Correctable Error Status Register 3Ch

Correctable Error Mask Register 40h

SSM IRQ Request & Status 44h

SSM IRQ Result Code 1 Shadow 48h

SSM IRQ Result Code 2 Shadow 4Ch

SSM Mailbox 50h

SSM Credit 0 Shadow 54h

SSM Credit 1 Shadow 58h

A.2.1. Intel-Defined VSEC Capability Header (Offset 00h)

Table 75. Intel-Defined VSEC Capability Header

Bits Register Description Default Value Access

[31:20] Next Capability Pointer.Value is the starting addressof the next CapabilityStructure implemented, ifany. Otherwise, NULL. Referto the Configuration AddressMap.

Variable RO

[19:16] Capability Version. PCIespecification-defined valuefor VSEC Capability Version.

0x1 RO

[15:0] Extended Capability ID. PCIespecification-defined valuefor VSEC ExtendedCapability ID.

0x000B RO


UG-20237 | 2020.07.13


118


A.2.2. Intel-Defined Vendor Specific Header (Offset 04h)

Table 76. Intel-Defined Vendor Specific Header


[31:20] VSEC Length. Total length ofthis structure in bytes.

0x5C RO

[19:16] VSEC Rev. User configurableVSEC revision.

k_vsec_rev_i RO

[15:0] VSEC ID. User configurableVSEC ID. The default valueis 0x1172 (the Intel VendorID), but you can change thisID to your own Vendor ID.

0x1172 RO

A.2.3. Intel Marker (Offset 08h)

Table 77. Intel Marker


[31:0]

Intel Marker - An additionalmarker for standard Intelprogramming software to beable to verify that this is theright structure.

0x41721172 RO

A.2.4. JTAG Silicon ID (Offset 0x0C - 0x18)

This read-only register returns the JTAG Silicon ID. Intel programming software usesthis JTAG ID to ensure that is is using the correct SRM Object File (*.sof).

These registers are only good for Port 0 (PCIe Gen4 x16). They are blocked for theother Ports.

Table 78. JTAG Silicon ID Registers

Bits Register Description Default Value(6) Access

[127:96] JTAG Silicon ID DW3 Unique ID RO




A.2.5. User Configurable Device and Board ID (Offset 0x1C - 0x1D)

This register provides a user configurable device or board ID so that the user softwarecan determine which .sof file to load into the device.

This register is only available for Port 0 (PCIe Gen4 x16). It is blocked for the otherPorts.

(6) Because the Silicon ID is a unique value, it does not have a global default value.


UG-20237 | 2020.07.13


119


Table 79. User Configurable Device and Board ID Register


[15:0] This register allows you tospecify the ID of the .soffile to be loaded.

From configuration bits RO

A.2.6. General Purpose Control and Status Register (Offset 0x30)

This register provides up to eight I/O pins each for Application Layer Control andStatus requirements. This feature supports Partial Reconfiguration of the FPGA fabric.Partial Reconfiguration only requires one input pin and one output pin. The otherseven I/Os make this interface extensible.

Table 80. General Purpose Control and Status Register


[31:16] Reserved. N/A RO

[15:8] General Purpose Status. TheApplication Layer can readthese status bits. These bitsare only available for Port 0(PCIe Gen4 x16). They areblocked for the other Ports.

0x00 RO

[7:0] General Purpose Control.The Application Layer canwrite these control bits.These bits are only availablefor Port 0 (PCIe Gen4 x16).They are blocked for theother Ports.

0x00 RW

A.2.7. Uncorrectable Internal Error Status Register (Offset 0x34)

This register reports the status of the internally checked errors that are uncorrectable.When these specific errors are enabled by the Uncorrectable Internal ErrorMask register, they are forwarded as Uncorrectable Internal Errors.

Note: This register is for debug only. Only use this register to observe behavior, not to drivelogic custom logic.

Table 81. Uncorrectable Internal Error Status Register


[31:13] Reserved 0x0 RO

[12] Debug Bus Interface (DBI)access error status fromConfig RAM block.

0x0 RW1CS

[11] Uncorrectable ECC errorfrom Config RAM block.

0x0 RW1C


[8] RX Transaction Layer parityerror reported by the IPcore.

0x0 RW1CS

continued...


UG-20237 | 2020.07.13


120



[7] TX Transaction Layer parityerror reported by the IPcore.

0x0 RW1CS

[6] Uncorrectable Internal Errorreported by the FPGA.

0x0 RW1CS

[5] cvp_config_error_latched: Configuration errordetected in CvP mode isreported as an uncorrectableerror. Set wheneverssm_cvp_config_error ofthe SSM Scratch CvP Statusregister bit[1] rises in CvPmode. This bit is onlyavailable for Port 0 (PCIeGen4 x16), but not for theother Ports.

0x0 RW1CS


Note: The access code RW1CS represents Read Write 1 to Clear Sticky.

A.2.8. Uncorrectable Internal Error Mask Register (Offset 0x38)

This register controls which errors are forwarded as internal uncorrectable errors.

Table 82. Uncorrectable Internal Error Mask Register



[12] Mask for Debug BusInterface (DBI) access error.

0x1 RWS

[11] Mask for Uncorrectable ECCerror from Config RAM block.

0x1 RWS


[8] Mask for RX TransactionLayer parity error reportedby the IP core.

0x1 RWS

[7] Mask for TX TransactionLayer parity error reportedby the IP core.

0x1 RWS

[6] Mask for UncorrectableInternal error reported bythe FPGA.

0x1 RWS

[5] Mask for Configuration Errordetected in CvP mode. Thisbit is only available for Port0 (PCIe Gen4 x16), but notfor the other Ports.

0x0 RWS


Note: The access code RWS stands for Read Write Sticky, meaning that the value is retainedafter a soft reset of the IP core.


UG-20237 | 2020.07.13


121


A.2.9. Correctable Internal Error Status Register (Offset 0x3C)

The Correctable Internal Error Status register reports the status of theinternally checked errors that are correctable. When these specific errors are enabledby the Correctable Internal Error Mask register, they are forwarded ascorrectable internal errors. This register is for debug only. Only use this register toobserve behavior, not to drive custom logic

Table 83. Correctable Internal Error Status Register



[11] Correctable ECC error statusfrom Config RAM.

0x0 RW1CS


[6] Correctable Internal Errorreported by the FPGA.

0x0 RW1CS

[5] cvp_config_error_latched: Configuration errordetected in CvP mode (to bereported as correctable) -Set whenevercvp_config_error riseswhile in CvP mode. This bitis only available for Port 0(PCIe Gen4 x16), but not forthe other Ports.

0x0 RW1CS


A.2.10. Correctable Internal Error Mask Register (Offset 0x40)

This register controls which errors are forwarded as internal correctable errors.

Table 84. Correctable Internal Error Mask Register



[11] Mask for Correctable ECCerror status for Config RAM.

0x1 RWS

[10:7] Reserved 0x0 RWS

[6] Mask for CorrectableInternal Error reported bythe FPGA.

0x1 RWS

[5] Mask for Configuration Errordetected in CvP mode. Thisbit is only available for Port0 (PCIe Gen4 x16), but notfor the other Ports.

0x1 RWS

[4] Reserved 0x1 RWS

[3:0] Reserved 0x0 RWS


UG-20237 | 2020.07.13


122


B. Document Revision History

B.1. Document Revision History for the Intel FPGA P-Tile Avalonmemory mapped IP for PCI Express User Guide

Document Version Intel QuartusPrime Version IP Version Changes

2020.07.13 20.2 3.0.0

Added support for the Gen3 x8 Endpoint andGen4 x8 Endpoint modes to the Features chapter.Updated the resource utilization numbers in the

Resource Utilization chapter.Added description for the Link Inspector in the

Debug Toolkit chapter.

2020.06.22 20.2 3.0.0

Added the lane reversal and polarity inversionsupport to the Features section.

Updated the bit ranges for the Next CapabilityOffset and Version fields in the Intel-Defined VSEC

Capability Registers section.

2020.04.29 20.1 2.0.0

Added clarification that VCS is the only simulatorsupported in the 20.1 release of Intel Quartus

Prime. Also added that PIPE mode simulations arenot supported in this release.

Changed the operation mode names from DMAMode with Data Movers to Endpoint Mode withData Movers, and from Bursting Slave Mode to

Endpoint Mode.

2020.04.28 20.1 2.0.0

Updated the document title to Intel FPGA P-TileAvalon memory mapped IP for PCI Express User

Guide to meet new legal naming guidelines.Updated the list of configurations supported in the

Features section.Replaced the Configuration Slave Interface with

the Control Register Access Interface.

2019.12.16 19.4 1.1.0Added parameters in Intel Quartus Prime to

control PASID and LTR.Added MSI extended data support.

2019.11.05 19.3 1.0.0

Added resource utilization numbers for the DMAdesign example in Intel Stratix 10 DX devices.Added the step to choose Intel Stratix 10 DX

devices to the Generating the Design Examplesection.

2019.10.28 19.3 1.0.0Removed a note containing a restriction on which

normal descriptors cannot be interrupted bypriority descriptors from the section Write Data

continued...

UG-20237 | 2020.07.13

Send Feedback







Document Version Intel QuartusPrime Version IP Version Changes

Mover Avalon-ST Descriptor Sinks, because allnormal descriptors being processed cannot be

interrupted.

2019.10.23 19.3 1.0.0 Initial release.

B. Document Revision History

UG-20237 | 2020.07.13


124


Documents

Intel® FPGA P-Tile Avalon® memory mapped IP for PCI ... › content › dam › www › programmable › ...Master, Bursting Avalon memory mapped Slave and the data movers. • Support