Xen on ARMThe Gears of the Cloud

Stefano Stabellini - 17 September 2014

© 2013 Citrix | Confidential – Do Not Distribute

“Citrix is a cloud computing companythat enables mobile workstyles”

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Citrix and Open Source

© 2014 Citrix

Created as a Linux Foundation Collaborative Project in April 2013

ARM joined the Xen Project Advisory Board in December 2013

What is Xen?

© 2013 Citrix | Confidential – Do Not Distribute

What is Xen?

● a type-1 hypervisor

● small footprint (less than 100K LOC)

● GPLv2

● 10 million individual users

● powers the largest public clouds in production

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen Architecture

Hardware

Xen

Dom0 DomU

HW drivers

PV backends PV Frontends

DomU

PV Frontends

DomU

PV Frontends

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen Architecture

Hardware

Xen

Dom0 DomU

NetFront

Disk Driver Domain

Toolstack Disk Driver

BlockBack

Network Driver Domain

Network Driver

NetBack BlockFront

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen Architecture: driver domains

● scalability

● security

● isolation

● resilience

● disaggregation

● componentization

© 2014 Citrix

Xen on ARM

© 2013 Citrix | Confidential – Do Not Distribute

Xen on ARM: not just a port

● a lean and simple architecture● we removed all the cruft accumulated during the years● no emulation, no QEMU● one type of guest● exploit the hardware as much as possible

● a very good match for the hardware

● clean architecture = small code base

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen on ARM: architecture

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen on ARM: architecture

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Code size: smaller is better!

● X86_64 ~= 140,000LOC● ARMv7 + ARMv8 ~= 1/6 x86_64 code

Common ARMv7 ARMv8 Total

xen/arch/arm 11,767 3,503 1,812 17,082

C 11,587 954 813 13,354

ASM 180 2,549 999 3,728

xen/include/asm-arm 4,786 984 1,050 6,820

TOTAL 16,553 4,487 2,826 23,902

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Achievements

11/11 08/12 11/12 03/13 07/13

First Xen on ARM talk atXen Summit 2012

Xen support for ARM upstream in Linux 3.7

Xen running on real ARM hardware

09/12

Xen 64-bit on ARM64

01/13

Citrix announces that will be joining Linaro

Xen support for ARM64 upstream in Linux 3.11

Xen 4.3 released with ARM and ARM64 support

Part-time Xen ARM hacking starts

Xen 4.4 release

06/13 03/14

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Hypervisor: Commits and Authors

© 2014 Citrix

Xen-devel : Messages per Month Contributors / Authors per Month

Highest ever

Website Traffic and Social Media

Average monthly web visitsgrew by 30%

Social Media following grew by 47%

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Press Mentions

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

The role of Xen on ARM

25-30% of xen-devel traffic is ARM specific

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen on ARM: new horizons

● embedded● set-top boxes● smarphones● internet of things● automotive● aviation industry● high reliability

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Xen Project 4.4 release: statusFeatures:● 64-bit guest support in ARMv8● stable hypercall ABI● basic lifecycle operations● memory ballooning● scheduler configurations, vcpu pinning, cpu pools

HCL (not inclusive):● ARMv8 FVP● Arndale board● Allwinner Sunxi (Cubieboard 2)● TI OMAP5● Applied Micro X-Gene

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Linux v3.13: status

● dom0 and domU● 32-bit and 64-bit● SMP● PV drivers

○ disk○ network○ console○ framebuffer○ keyboard○ mouse

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

Coming in Xen 4.5● irq latency reduction● interrupt priorities● irq migration● SMMU support● Xen UEFI host booting● GICv3 support● support for up to 1TB of guest RAM● super pages● support for 48-bit physical addresses● QEMU PV backend on ARM: framebuffer,

mouse, keyboard, multi-console

© 2014 Citrix

Benchmarks

© 2013 Citrix | Confidential – Do Not Distribute

Execution environment

● Linux v3.15-rc8dom0, domU and native kernel

● Xen v4.5-unstable+

● QEMU-KVM v2.0.0

● OpenSUSE 13.1

© 2014 Citrix

Applied Micro X-Gene Intel Xeon CPU X5650

ARMv8 64-bit 8 cores 2.4 Ghz x86_64 2 sockets x 6 cores 2.67 Ghz

© 2013 Citrix | Confidential – Do Not Distribute

Kernbenchvirtualization overhead (percentage) - Lower is better

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

PBZIP2virtualization overhead (percentage) - Lower is better

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

FIO rand-rwvirtualization overhead (percentage) - Lower is better

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

SPECjbb2005virtualization overhead (percentage) - Lower is better

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

SpecINT 2006Xen on ARM virtualization overhead (percentage) - Lower is better

2%

© 2014 Citrix

Why Xen?

© 2013 Citrix | Confidential – Do Not Distribute

● strong in the cloud

● strong community

● successful in automotive and embedded

● strong security guarantees

● easy to deploy● from source, Linux distributions, products, etc.

● minimal overhead

Xen on ARM: the gears of the cloud

© 2014 Citrix

© 2013 Citrix | Confidential – Do Not Distribute

More information

● http://www.xenproject.org

● Xen on ARM @wiki.xenproject.org http://goo.gl/FKNXe

● Xen on ARM whitepaper http://goo.gl/vWMn7D

● http://lists.xenproject.org/mailman/listinfo/xen-devel

© 2014 Citrix

Questions?

© 2013 Citrix | Confidential – Do Not Distribute

Work better. Live Better.

Citrix

© 2014 Citrix