• Home

  • Documents

  • FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro...
13
x FPGA Acceleration and Virtualization Technology in DPDK ROSEN XU TIANFEI ZHANG

FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

  • Upload
    others

  • View
    14

  • Download
    0

Embed Size (px)

Citation preview

Page 1: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

x

FPGA Acceleration and

Virtualization Technology in

DPDKROSEN XU

TIANFEI ZHANG

Page 2: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

2

Agenda

► FPGA in Networking Acceleration

► Partial Reconfiguration (PR)

► FPGA Acceleration on DPDK► DPDK High Level Design

► Scan and Probe Work Flow

► Intel FPGA Acceleration Environment

► Intel FPGA Acceleration Stack OPAE Intro

► Intel FPGA Acceleration on DPDK

► Port Representor and Virtualization Scenario

► Status & WIP

► Acknowledgement

Page 3: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

3

FPGA in Networking Acceleration

Opportunities

– Enhancing Performance: Provide NIC ASIC liked performance

– Changing dynamically: Flexible enough for adding new feature by replace Bit Stream

Problems

– Longer design cycle than software: Compilation, Analysis & Synthesis, Fitter(Place & Router), Assembler, Timing

– Update Bit Stream affecting business: PCIe rescan and driver re-probe

– How to share One FPGA resource with many users

VM VM

vSwitchSlow Path

Acc IP 1

CPU

FPGA Networking Acceleration

Acc IP 2 Acc IP 3

Page 4: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

4

Partial Reconfiguration (PR)

With Partial Reconfigure(PR) a portion of the FPGA dynamically,

FPGA not only provides one kinds of accelerator but also

provides many types of accelerators at the same time

– All FPGA vendor support PR function

How DPDK fully support FPGA?

– Which type of DPDK Device can provide FPGA PR?

– How can we bind DPDK Driver to FPGA Partial Bit Stream?

VM VM

vSwitchSlow Path

Portion 1(Acc IP 1)

CPU

FPGA Networking Acceleration

Portion 2(Acc IP 2)

Portion 3(Acc IP 3)

Page 5: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

5

DPDK High Level Design

Crypto AFU Dev

EthAFU Dev

vdev_fpga_cfg driver

IFPGA Bus

FPGARawDev Driver

EthAFU Dev

Crypto AFU Dev

Eth PMDEth PMD

Crypto PMDCrypto PMD

PCI Bus

vdev_fpga_cfg driver vdev_fpga_cfg

driver vdev_fpga_cfg driver

DPDK Framework(APIs)

FPGA AFU

FPGA AFUFPGA AFU

FPGA AFU

PMD

AFU Dev

BUS

VdevCfg

RawDevDriver

FPGA

RawDevOPS

Application(s)

Hot-plugin

Note:• AFU(Accelerated Function Unit): Partial-Bitstream for a portion of the FPGA

FPGA is divided to many portions, each portion has its own

Partial Bit Stream

Vdev Cfg takes configuration for each AFU

IFPGA Bus is a new bus for AFU devices scan and drivers probe

New added AFU Device for each portion’s Acceleration function

All AFUs’ PMD are based on AFU Acceleration

Page 6: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

6

Scan and Probe Work Flow

afu_listrte_bus_register

constructor

rte_ifpga_driver_register

afu_driver_list

afu_eth

afu_crypto

drv_crypto

drv_eth

INSERT to rte_bus_list

INSERT to afu_driver_list

ifpga pci ixgbmcp_fpga

i40e

pci_device_list

raw_dev_arra

y dev_fpga

……rawdev.ops

ENUMERATE & PRINSERT AFU Device

rte_eal_hotplug_add

PROBE AFU Driver

Ixgb_drv

mcp_drv

I40e_drv

pci_driver_listrte_bus_list

1

2

Rawdev probed as PCI Driver takes FPGA Configuration(Download/PR)

2 scans: FPGA PCI Device Scan(1st Scan) and AFU Scan(2nd Scan)

OPAE Provides Common lib and API for low level FPGA management & accelerator access

Page 7: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

7

Intel FPGA Acceleration Environment

FPGACPU

User Application& Libraries

CPU

FPGA Interface Manager (FIM)Intel® Acceleration Engine with

OPAE1 Technology

Accelerator Function(Developer created

orprovided by Intel)

UPI2/PCIe*

HSSI3

Accelerator Function Interfaces

Hypervisor & OS

OPAE

FPGA

FPGA Management Engine(FME)

– Provides: power and thermal management, error reporting, partial reconfiguration, performance reporting, and other infrastructure functions.

– Each FPGA has one FME, accessible through the physical function.

Accelerated Function Unit(AFU)

– Implements: one Acceleration, can be partial reconfiguration.

– Each FPGA can support Multiple AFUs.

Page 8: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

8

Intel FPGA SW Stack OPAE Intro

Optimized and simplified hardware and software APIs

provided by Intel

Consistent cross-platform API

Minimal software overhead and latency

Supports virtual machines and bare metal platforms

Open source code licensing and developer

community

– Intel FPGA drivers being upstreaming to Linux kernel

– Intel FPGA user space drivers have merged into DPDK

https://01.org/sites/default/files/downloads/opae/open-programmable-acceleration-engine-paper.pdf

https://01.org/sites/default/files/downloads/opae/open-programmable-acceleration-engine-paper.pdf
Page 9: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

9

Intel FPGA Acceleration on DPDK

FPGA Mgmt

(partial reconfiguration)

Network Path

(NIC)

DPDK Device PMD

DPDK Application

librte_rawdev librte_ethdev

Bare metal Application

VFIO

OPAE Hardware Layer API

Intel fpga base code

Eth driver (AFU)fpga bus

DPDK FPGA Device DriverDPDK PCI

Bus

Custom User space driver

FPGA Accelerator

OPAE High Level APIs

Kernel Space

User Space

crypto

event

DPDK Device PMD– For defined DPDK Device(Ethdev/Cryptodev/Eventdev)

Non-DPDK User-Space Driver– For Customized Device, Transparent DPDK

Rawdev Support PR– Rawdev is submited in 18.02

Note:• DPDK and NON-DPDK mode will not run at the same time

Page 10: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

10

Port Representor and Virtualization Scenario

Port Representor– Each VF port binging to one representor port

– Control Plan Application take perception of VF port by its representor port

Virtualization– FPGA can support both DPDK APP and Socket APP in VM

– FPGA Rawdev Driver take FPGA configuration

HyperVisor

REPRESENTOR PMD

ETH DEV API

FME

FPGARawdev Driver

PF PMD

PF AFU

ETH DEV API

VF AFUVF AFU

APP Socket

App

BSD Socket Syscall

REPRESENTOR PMD

ETH DEV API

REPRESENTOR PMD

ETH DEV API

REPRESENTOR PMD

ETH DEV API

Control Plan Application

FPGA Networking Acceleration

VF AFUVF AFU

Host OS Kernel

VM VM

Page 11: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

11

Status & Working in Progress

2017’Q3 OPAE 0.9 Release [DONE]– Fully support Intel FPGA Acceleration Environment

– Support FIM 6.3.0

2018’Q1 OPAE 0.13 Release, [DONE]– Support FIM 6.4.0

2018’Q2 OPAE 1.0 Release [DONE]

2018’Q1 DPDK with OPAE User Space Driver PoC [DONE]

2018’Q1 IFPGA Bus RFC patch [DONE]

2018’Q2 IFPGA Bus patch set upstream to DPDK 18.05 [DONE]

DPDK supports FPGA Acceleration is ready.

Welcome on board!

Page 12: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

12

Acknowledgement

Song Liu

Zhang Zhang

Hao Wu

Yilun Xu

Heqing Zhu

Cunming Liang

Helin Zhang

Zhihong Wang

Yanglong Wu

Jingjing Wu

Qi Zhang

Yuwei Zhang

Declan Doherty

Bruce Richardson

Ferruh Yigit

Roy Fan Zhang

Page 13: FPGA Acceleration and Virtualization Technology in DPDK · 8 Intel FPGA SW Stack OPAE Intro Optimized and simplified hardware and software APIs provided by Intel Consistent cross-platform

Thanks !


Intel FPGA Integer Arithmetic IP Cores User Guide · 1 Intel FPGA Integer Arithmetic IP Cores You can use the Intel® FPGA integer IP cores to perform mathematical operations in your

Intel FPGA Integer Arithmetic IP Cores User Guide · 1 Intel FPGA Integer Arithmetic IP Cores You can use the Intel® FPGA integer IP cores to perform mathematical operations in your

Documents
Intel® MAX® 10 FPGA Configuration User Guide. Document Revision History for the Intel MAX 10 FPGA Configuration User Guide.....65. Contents Send Feedback Intel ® MAX ® 10 FPGA

Intel® MAX® 10 FPGA Configuration User Guide. Document Revision History for the Intel MAX 10 FPGA Configuration User Guide.....65. Contents Send Feedback Intel ® MAX ® 10 FPGA

Documents
Intel® Cyclone® 10 LP FPGA Evaluation Kit User Guide · The Intel Cyclone 10 LP FPGA Evaluation Kit includes the following components: • The Intel Cyclone 10 LP FPGA evaluation

Intel® Cyclone® 10 LP FPGA Evaluation Kit User Guide · The Intel Cyclone 10 LP FPGA Evaluation Kit includes the following components: • The Intel Cyclone 10 LP FPGA evaluation

Documents
Intel MAX 10 FPGA Device Datasheet

Intel MAX 10 FPGA Device Datasheet

Documents
JESD204C Intel® Agilex™ FPGA IP Design Example …...2. JESD204C Intel FPGA IP Design Example Quick Start Guide The JESD204C Intel FPGA IP design examples for Intel Agilex devices

JESD204C Intel® Agilex™ FPGA IP Design Example …...2. JESD204C Intel FPGA IP Design Example Quick Start Guide The JESD204C Intel FPGA IP design examples for Intel Agilex devices

Documents
Intel Card (PAC) with Intel Arria FPGA Datasheet · 2.2.6. CPLD ... Developers can use the Accelerator Functional Unit ... • OPAE Intel FPGA Linux Device Driver Architecture Guide—Describes

Intel Card (PAC) with Intel Arria FPGA Datasheet · 2.2.6. CPLD ... Developers can use the Accelerator Functional Unit ... • OPAE Intel FPGA Linux Device Driver Architecture Guide—Describes

Documents
Intel FPGA SDK for OpenCL Programming Guide

Intel FPGA SDK for OpenCL Programming Guide

Documents
AN 810: Intel FPGA JESD204B IP Core and ADI AD9208 Hardware … · 2020-01-19 · Intel FPGA JESD204B IP Core and ADI AD9208 Hardware Checkout Report The Intel FPGA JESD204B IP Core

AN 810: Intel FPGA JESD204B IP Core and ADI AD9208 Hardware … · 2020-01-19 · Intel FPGA JESD204B IP Core and ADI AD9208 Hardware Checkout Report The Intel FPGA JESD204B IP Core

Documents
Remote Update Intel FPGA IP User Guide · 1. Remote Update Intel ® FPGA IP User Guide. The Remote Update Intel ® FPGA IP core implements a device reconfiguration using dedicated

Remote Update Intel FPGA IP User Guide · 1. Remote Update Intel ® FPGA IP User Guide. The Remote Update Intel ® FPGA IP core implements a device reconfiguration using dedicated

Documents
Intel® FPGA Programmable Acceleration Card …...1. Introduction The Intel FPGA Programmable Acceleration Card D5005 (Intel ® FPGA PAC D5005) is a PCI Express* Gen 3 x16 compliant

Intel® FPGA Programmable Acceleration Card …...1. Introduction The Intel FPGA Programmable Acceleration Card D5005 (Intel ® FPGA PAC D5005) is a PCI Express* Gen 3 x16 compliant

Documents
5G LDPC Intel® FPGA IP User Guide · 2. Getting Started with the 5G LDPC Intel FPGA IP. Related Information • Introduction to Intel FPGA IP • IP Catalog and Parameter Editor

5G LDPC Intel® FPGA IP User Guide · 2. Getting Started with the 5G LDPC Intel FPGA IP. Related Information • Introduction to Intel FPGA IP • IP Catalog and Parameter Editor

Documents
Intel® FPGA Power and Thermal Calculator User Guide · Calculator. 2.1. Availability. For Intel Agilex and Intel Stratix 10 devices, the Intel FPGA Power and Thermal Calculator (PTC)

Intel® FPGA Power and Thermal Calculator User Guide · Calculator. 2.1. Availability. For Intel Agilex and Intel Stratix 10 devices, the Intel FPGA Power and Thermal Calculator (PTC)

Documents
Intel FPGA PHYLite for Parallel Interfaces IP Core User · PDF fileInput Buffer Reference Voltage ... Intel FPGA PHYLite for Parallel Interfaces IP Core User Guide ... FPGA PHYLite

Intel FPGA PHYLite for Parallel Interfaces IP Core User · PDF fileInput Buffer Reference Voltage ... Intel FPGA PHYLite for Parallel Interfaces IP Core User Guide ... FPGA PHYLite

Documents
FIFO Intel® FPGA IP User Guide · 2020. 12. 27. · FIFO Intel ® FPGA IP User Guide. Intel ® provides FIFO Intel FPGA IP core through the parameterizable single-clock FIFO (SCFIFO)

FIFO Intel® FPGA IP User Guide · 2020. 12. 27. · FIFO Intel ® FPGA IP User Guide. Intel ® provides FIFO Intel FPGA IP core through the parameterizable single-clock FIFO (SCFIFO)

Documents
SDI Audio Intel FPGA IP User Guide · 2 days ago · 2. SDI Audio Intel FPGA IP Getting Started. The SDI Audio Intel FPGA IP cores are installed as part of the Intel Quartus Prime

SDI Audio Intel FPGA IP User Guide · 2 days ago · 2. SDI Audio Intel FPGA IP Getting Started. The SDI Audio Intel FPGA IP cores are installed as part of the Intel Quartus Prime

Documents
Intel FPGA Software Installation and Licensing - Altera · 6 Intel FPGA Software Installation and Licensing ... 1.1 About Intel FPGA Software Installation and ... and installing the

Intel FPGA Software Installation and Licensing - Altera · 6 Intel FPGA Software Installation and Licensing ... 1.1 About Intel FPGA Software Installation and ... and installing the

Documents
Intel FPGA SDK for OpenCL · — Intel Arria 10 FPGA Development Kit Reference Platform Scripts and its content was merged with Contents of the Intel Stratix 10 GX FPGA Development

Intel FPGA SDK for OpenCL · — Intel Arria 10 FPGA Development Kit Reference Platform Scripts and its content was merged with Contents of the Intel Stratix 10 GX FPGA Development

Documents
Intel FPGA Data Sheet

Intel FPGA Data Sheet

Documents
Intel FPGA SDK for OpenCL · 2020-07-22 · Intel® FPGA SDK for OpenCL ™ Intel® Arria® 10 GX FPGA Development Kit Reference Platform Porting Guide Updated for Intel ® Quartus

Intel FPGA SDK for OpenCL · 2020-07-22 · Intel® FPGA SDK for OpenCL ™ Intel® Arria® 10 GX FPGA Development Kit Reference Platform Porting Guide Updated for Intel ® Quartus

Documents
AN 792: Intel FPGA JESD204B IP Core and ADI AD9371 ... · Hardware Checkout Report..... Intel FPGA JESD204B IP Core and ADI AD9371 Hardware Checkout Report. The Intel FPGA JESD204B

AN 792: Intel FPGA JESD204B IP Core and ADI AD9371 ... · Hardware Checkout Report..... Intel FPGA JESD204B IP Core and ADI AD9371 Hardware Checkout Report. The Intel FPGA JESD204B

Documents
FIFO Intel® FPGA IP User Guide · FIFO Intel® FPGA IP User Guide Intel® provides FIFO Intel FPGA IP core through the parameterizable single-clock FIFO (SCFIFO) and dual-clock FIFO

FIFO Intel® FPGA IP User Guide · FIFO Intel® FPGA IP User Guide Intel® provides FIFO Intel FPGA IP core through the parameterizable single-clock FIFO (SCFIFO) and dual-clock FIFO

Documents
AN 830: Intel FPGA Triple-Speed Ethernet and On … Intel® FPGA Triple-Speed Ethernet and On-Board PHY Chip Reference Design ... qsys_top_0 1 Intel® FPGA Triple-Speed Ethernet

AN 830: Intel FPGA Triple-Speed Ethernet and On … Intel® FPGA Triple-Speed Ethernet and On-Board PHY Chip Reference Design ... qsys_top_0 1 Intel® FPGA Triple-Speed Ethernet

Documents
Simulating Intel® FPGA Designs...FPGA IP cores in your design. • Intel FPGA provides the entry-level software, along with precompiled Intel FPGA simulation libraries, to simplify

Simulating Intel® FPGA Designs...FPGA IP cores in your design. • Intel FPGA provides the entry-level software, along with precompiled Intel FPGA simulation libraries, to simplify

Documents
Intel® FPGA Programmable Acceleration Card N3000 Data …...Figure 1. Intel FPGA Programmable Acceleration Card N3000 This data sheet describes the Intel FPGA Programmable Acceleration

Intel® FPGA Programmable Acceleration Card N3000 Data …...Figure 1. Intel FPGA Programmable Acceleration Card N3000 This data sheet describes the Intel FPGA Programmable Acceleration

Documents
From Dataflow Specifications to Customised … · OpenCL FPGA Model Intel FPGA SDK for OpenCL: ... Intel FPGA SDK for OpenCL: Programming Guide ... (provided by user)

From Dataflow Specifications to Customised … · OpenCL FPGA Model Intel FPGA SDK for OpenCL: ... Intel FPGA SDK for OpenCL: Programming Guide ... (provided by user)

Documents
Introduction to Intel FPGA IP Cores

Introduction to Intel FPGA IP Cores

Documents
Intel · 4.1.4. ROM: 1-PORT Intel FPGA IP Parameters......................................................48 4.1.5. ROM: 2-PORT Intel FPGA IP Parameters

Intel · 4.1.4. ROM: 1-PORT Intel FPGA IP Parameters......................................................48 4.1.5. ROM: 2-PORT Intel FPGA IP Parameters

Documents
JESD204B Intel FPGA IP User Guide - Altera. TX Data Link Layer ... Cyclone V, Stratix V, ... • JESD204B Intel Cyclone 10 GX FPGA IP Design Example User Guide • Intel FPGA Software

JESD204B Intel FPGA IP User Guide - Altera. TX Data Link Layer ... Cyclone V, Stratix V, ... • JESD204B Intel Cyclone 10 GX FPGA IP Design Example User Guide • Intel FPGA Software

Documents
DisplayPort Intel® Arria 10 FPGA IP Design Example User Guide · • Intel Quartus Prime (for hardware testing) • ModelSim* - Intel FPGA Edition, ModelSim - Intel FPGA Starter

DisplayPort Intel® Arria 10 FPGA IP Design Example User Guide · • Intel Quartus Prime (for hardware testing) • ModelSim* - Intel FPGA Edition, ModelSim - Intel FPGA Starter

Documents
ASMI Parallel II Intel® FPGA IP Core User Guide · 2020. 7. 10. · The ASMI Parallel II Intel FPGA IP core is available for all Intel FPGA device families including the Intel MAX®

ASMI Parallel II Intel® FPGA IP Core User Guide · 2020. 7. 10. · The ASMI Parallel II Intel FPGA IP core is available for all Intel FPGA device families including the Intel MAX®

Documents