FPGA Accelerator Virtualization in an

OpenPOWER cloudFei Chen, Yonghua Lin

IBM China Research Lab

Trend of Acceleration Technology

2

• Used FPGA to accelerate Bing search on 1632 servers• A 6*8 2D-torus design for high throughput network topology

• Storage >2000PB, processing 10~100PB/day, log 100TB~1PB/day• Using FPGA for storage controller• Used GPU for Deep Learning

Acceleration in Cloud is Taking Off

• Scalable acceleration fabric

• Open framework for accelerator integration and sharing

• Accelerator resource abstraction, re-configuration and scheduling in cloud

• Modeling & advisory tool for dynamic acceleration system composition

Innovations RequiredAcceleration programming becomes

hot topics

� TB scale problem � PB scale problem

� Acceleration architecture for singlenode

� Architecture for thousands of nodes

� Dedicate acceleration resource

� Shared acceleration resources

� Proprietary accelerator framework

� Open framework to enable accelerator sharing & integration

� Close innovation model � Open innovation model through eco-system

Appliance Acceleration in Cloud

OpenCL, Sumatra (Oracle), LiMe (IBM), …

Resources on FPGA are huge

3

• Resources on FPGA– Programmable resources•Logic cells (LCs)

•DSP slices: Fixed/floating-point

•On-chip memory blocks

•Clock resources

– Miscellaneous periphrals(Xilinx Virtex as an example)

•DDR3 controllers

•PCIe Gen3 interfaces

•10G Ethernet controllers

•...

– Hard processor core•PowerPC: Xilinx Virtex-5 FXT

•ARM: Xilinx Zynq-7000

•Atom: Intel + Altera E600C Xilinx Virtex UltraScale 440 FPGA, the largest-

scale FPGA in the world delivered in2014, consists of

more than 4 million logical cells (LCs).

Using this chip, we can build up to 250 AES

crypto accelerators, or 520 ARM7 processor cores.

FPGA Capacity Trends

FPGA on Cloud – Double Win

4

� Cloud benefits from FPGA• Performance

• Power consumption

� FPGA benefits from cloud• Lower the cost

• Tenants need not purchase and maintain FPGAs.

• Tenants pay for accelerators only when using them.

• More applications

• High FPGA utilization

• Ecosystem

• Grow with the cloud ecosystem

Motivation of Accelerator/FPGA as Service in Cloud

5

Can FPGA (pool) be managed in data

center?

ID, location, re-

configuration,

performance, etc.

App.

App.

App. VM

VM

VM Container

Container

Container

How to reduce system cost through sharing FPGA

resources among applications, VMs, containers?

Dynamic, Flexible, Priority controllable

How to orchestrate FPGA/accelerator resources

with VM, network and storage resources easily,

according to the need of application?

Enable the manageability Reduce system cost

Reduce deployment complexity

VM

Containerhost

Storage

Network

Bring high value of cloud infrastructure

Could we generate new value for IaaS ?

FPGA Ecosystem in Cloud

HEAT orchestrator

POWER8/PowerKVM

FPGA

cards

Compute Network StorageFPGA

accelerator

Accelerator Market Place

Cloud Service Provider

Cloud

Tenants•Pay for the usage of accelerator, rather than license and hardware

•Get the accelerator service in self-service way

•Use the single HEAT orchestrator to finish the workload deployment with accelerator, together with compute, network, and storage.

•Cloud service provider will buy the “Cloudified” accelerators on market place

•Create the Service Category for FPGA accelerator, and sale on the cloud as service

•Companies or

individual

developers could

upload and sale

their accelerator

through market

place (e.g. on

OpenPOWER)

• Accelerator market place will do the cloudify for accelerator, through integrating the service layer with accelerator and compilation

•All the integration, compilation, test, verification and certification will be done in automatic way.

Accelerator developers

Accelerator Cloudify Tool

(in plan)

OpenStack extenstion for accelerator service

Service logics for accelerator service in FPGA

Accelerator as Service on SuperVessel

7

• Accelerator MarketPlace for developers to

upload and compile the accelerators for

SuperVessel POWER cloud

• Allow user to request different size of cluster

Fig.1 Accelerator MarketPlace for SuperVessel Cloud

Fig.2 Cloud users could apply

accelerator when creating VM

FPGA Implementation

8

AB

C D

Service Logic

Ser

vice

Sub

laye

r

Pla

tfor

m S

ubla

yer

A

B

C

D

User Sublayer : Shared FPGA resource

Service Sublayer : Job Queue, Switch, …Platform Sublayer : DRAM, PCIe, ICAP, …

FPGA chip

DRAM

Switch

Job Scheduler

Job Queue

Security Controller

DMA Engine

Reconfig

Controller

A B C D

Eth

Registers

High Bandwidth I/O

PCIe / CAPI

……

Context Controller

Hardware

OS

Apps

Computer

The FPGA subsystem is designed as a computer system.

System Implementation

9

Control Node

Compute Node

Compiler

• Control Node: Nova, Glance, Horizon, Neutron, Swift

• Compute Node: Nova Compute

• Compiler: FPGA incremental compiler environment

1. Accelerator source

code package

2. image_file

Dashboard

CompilerGlance

Scheduler

Compute

3. VM request

(with accelerator)

4. acc_file

5. Launch VM

Evaluation

10

(1) Accelerator Sharing Evaluation

90010001100120013001 2 3 4 5 6 7 8T

ota

l Ba

nd

wid

th(M

B/s

)

Number of ProcessesHost One VM

(1)

• Host: All processes run in host environment

• One VM: All processes run in one VM

• VMs : Each process runs in one VM

• AESs: Each VM uses one independent AES accelerator

0400800120016001 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61

TimeBandwidth (BM/s)

Process 0 Process 1 Total

(2) Management – Bandwidth Control

0%

20%

40%

60%

80%

1 11 21 31 41 51 61Time (second)

CV

0

1

2

3

4

1 11 21 31 41 51 61Time (second)

Average Latency (ms)

1

10

100

1000

10000

1 11 21 31 41 51 61Time (second)

Bandwidth (MB/s)

Process 0 Process 1

Process 1 begin Priority control

1194 MB/s

25 MB/s

0.21ms 0.22ms

2.3ms

(3) Management – Priority Control

(3)

� Process 0 : 256KB payload, 100 times per second

� Process 1 : 4MB payload, best effort use.

� Same priority during second 1 ~ 38.

� Raise process 0 priority at second 38.

FPGA accelerator as Services online on SuperVessel Cloud

11

SuperVessel Cloud Infrastructure

SuperVessel

Cloud

Service

SuperVessel

Big Data and HPC

Service

Super

Class

Service

OpenPOWER

Enablement

Service

Super Project

Team

Service

Super Marketplace

1.VM and

container service

2.Storage service

3.Network service

4.Accelerator

as service5. Image service

1.Big Data:

MapReduce

(Symphony),

SPARK

2.Performance

tuning service

1.X-to-P migration

2.AutoPort Tool

3.OpenPOWER

new system test

service

1.On-line video

courses

2.Teacher course

management

3.User

contribution

management

1.Project

management

service

2.DevOps

automation

Storage IBM POWER servers OpenPOWER server FPGA/GPU

Docker

(Online)(Online) (Preparing)(Preparing)

You could try it here in two week: www.ptopenlab.com

Thanks!

12