OpenStack Acceleration As A

Service

Introducing OpenStack Cyborg

Bio

● Open Source Community Manager

○ OpenStack Cyborg PTL, Interop WG Contributor, Public Cloud WG co-chair

○ OpenSDS Core Contributor

○ OPNFV committer for several projects

○ Hyperleger Contributor

● Standard Engineer

○ Rapporteur for ETSI NFV IFA 004 & 019Wechat & Line: +8618576658966

Email: zhipengh512@gmail.com

huangzhipeng@huawei.com

Outline

● Background

● OpenStack HPC from Scientific WG’s perspective

● Introduction of Cyborg Project

● Future Works

“as more and more workloads migrate to the

cloud, the mass concentration of similar

workloads make hardware acceleration a

requirement rather than an interesting option”

——James Hamilton

http://perspectives.mvdirona.com/2017/04/ten

sor-processing-unit/

Background

Background● OpenStack Acceleration Discussion Started from Telco Requirements

○ High level requirements first drafted in the standard organization ETSI NFV ISG

○ High level requirements transformed into detailed requirements in OPNFV DPACC project.

○ New project called Nomad established to address the requirements.

○ BoF discussions back in OpenStack Austin Summit.

● Transition to Cyborg Project

○ From a long period of conversation and discussion within the OpenStack community, we found that the initial goal of Nomad project to address acceleration management in Telco is too limited. From design summit session in Barcelona Summit, we have developers from Scientific WG help us understanding the need for acceleration management in HPC cloud, and we also had a lot of discussion on the Public Cloudsupport of accelerated instances.

○ We decide to formally establish a project that will work on the management framework for dedicated devices in OpenStack, and there comes the Cyborg Project.

Background

● Cyborg Project Quick Facts

○ Started Dec 2016, not an official project yet

○ Development start from zero in OpenStack, follow “four opens” principle

○ Weekly meeting UTC 1500 on Wed at #openstack-cyborg, minutes at http://eavesdrop.openstack.org/meetings/openstack_cyborg/

○ Use openstack-dev mailing list with [acceleration]

○ Wiki at https://wiki.openstack.org/wiki/Cyborg

○ Stats : http://stackalytics.com/?project_type=openstack-others&module=cyborg&metric=person-day

○ Code: https://review.openstack.org/#/q/project:openstack/cyborg

Outline

● Background

● OpenStack HPC from Scientific WG’s perspective

● Introduction of Cyborg Project

● Future Works

Current issues using GPUs in Openstack

GPUs can be used in openstack. GPU specific flavors: pci_passthrough alias in the properties field

KVM tuning is required to achieve acceptable performance

Two options:Heterogeneous hosts: GPU and CPU-only hosts mixed

Bad - scheduler does not prioritize GPU workloads.

Good - CPU resources are available for all workloads

GPU only Host-Aggregate: GPU hosts are segregated

Good - GPU hosts are only used for GPU workloads

Bad - CPU-only workloads unable to use underutilized GPU hosts

Outline

● Background

● OpenStack HPC from Scientific WG’s perspective

● Introduction of Cyborg Project

● Future Works

What is Cyborgcyborg-api

cyborg-agent

cyborg-db

local cache

cyborg-generic-drivervendor-a-driver vendor-b-driver

vendor-a-fpga vendor-c-ipsec vendor-b-gpgpu

Control Node

Compute Node

cyborg-conductor

nova-api

What will Cyborg do with Nova?

cyborg-db

cyborg-api

cyborg-agent

placement-api

Acc driver

nova-compute

Libvirt driver

nova-conductor

nova-db

cyborg-conductor nova-scheduler

Outline

● Background

● Introduction of Cyborg Project

● OpenStack HPC from Scientific WG’s perspective

● Future Works

Future Work

Cyborg Development Goal:

• Finish the code development for the initial specs in Pike

• Have more contributors (especially from vendors to help us understand real case)

• Fulfill requirements from SWG and other related WGs

• Looking for PoC opportunities

• FPGA and GPGPU basic support in Queen

Future Work

Cross Community Efforts:

• Initial Container Device Interface (CDI) proposal in Kubernetes:

https://github.com/kubernetes/community/pull/545/

• OpenCL

• OPNFV DPACC project