Sponsored by:

&

1

Xen: the Past, Present and Exciting Future

Ian Pratt Chairman of Xen.org, SVP and Founder of Bromium

Outline

• Community Update

• Xen 4 Review

• Xen and the next wave of virtualization

2

COMMUNITY UPDATE

3

2011 Highlights

• Inclusion of Xen into Linux 3 (and distros)

• New Initiatives: – Project Kronos

– Xen.org Governance

– Renewed focus on Xen for ARM

• Successful Community Initiatives – Documentation Day

– Google Summer of Code

– Hackathons: Cambridge (Citrix) and Munich (Fujitsu)

• Lars Kurth: (not so) new Community Manager

4

Contribution Statistics

5

By Change Sets Contributors to Xen.org

0.0

500.0

1000.0

1500.0

2000.0

2500.0

3000.0

3500.0

4000.0

4500.0

5000.0

XenARM**

PVOPS

XCP

Xen HV

*) End of Sept 2011 **) Activity on Development branch (not yet in xen-unstable)

0

20

40

60

80

100

120

140

160

180

200

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011*

Individuals

Orgs

2010 & 2011 Contributors (by KLOC)

6

39%

20%

8%

7%

6%

5%

5%

4% 4%

2%

Citrix HV

Citrix XCP

Oracle

Intel

Novell

Fujitsu

AMD

Individual

Misc

University

2010** 2011** ***

28%

18%

15%

13%

11%

6%

5% 3%

1%

Citrix XCP

Citrix HV

Samsung*

Novell

Oracle

AMD

Individual

Intel

Misc

*) Activity on Development branch (not yet in xen-unstable) **) Includes PVOPS ***) Until Sept 2011

Developer mailing list traffic

7

Conversations, excluding patches

0

500

1000

1500

2000

2500

Oct-

03

De

c-0

3

Feb

-04

Apr-

04

Jun-0

4

Aug-0

4

Oct-

04

De

c-0

4

Feb

-05

Apr-

05

Jun-0

5

Aug-0

5

Oct-

05

De

c-0

5

Feb

-06

Apr-

06

Jun-0

6

Aug-0

6

Oct-

06

De

c-0

6

Feb

-07

Apr-

07

Jun-0

7

Aug-0

7

Oct-

07

De

c-0

7

Feb

-08

Apr-

08

Jun-0

8

Aug-0

8

Oct-

08

De

c-0

8

Feb

-09

Apr-

09

Jun-0

9

Aug-0

9

Oct-

09

De

c-0

9

Feb

-10

Apr-

10

Jun-1

0

Aug-1

0

Oct-

10

De

c-1

0

Feb

-11

Apr-

11

Jun-1

1

xen-devel xen-api

Formalized Governance

• How to contribute (had this for a long time, but was poorly documented)

• Election of Maintainers, Committers & Project Leads – Committer Election in September

– Jan Beulich (Novell) : Committer on Xen HV project

• 2009 : 107 patches changing 11746 lines of code

• 2010 : 147 patches changing 7613 lines of code

• 2011 : 130 patches changing 27377 lines of code (as of Sept)

• Project Lifecycle – Xen HV & XCP migrated to new lifecycle

8

Xen.org Web site Activity

9

Blog Traffic/Month:

100% average increase

of traffic compared to

one year ago

Xen 4.1

Linux 3

Part of Sept

Prediction at tip of arrow

Xen 4.0

Website Traffic/Day:

Notable traffic increase since

September (almost double)

Coincides with changes to content

and new content!

Linux 3

Product and project news roundup

• Xen support in Linux 3 – More in Linux 3.1 (and subsequent releases)

• Xen support (DomU and Dom0) back in Linux distros – Debian

– Ubuntu 11.10

– Fedora 16

• Recent product releases that distribute Xen – Oracle VM 3.0

– XenServer 6.0

– XenClient 2.0

– Beta’s of QubesOS and RC’s of openSuse 12.1

10

Where do we need to improve?

11

Focus for 2012

• New website

• Better media presence

• More focus on users

(individual & commercial)

• More and better documentation

• New benchmarks, feature

comparisons, etc.

• Formalize volunteer activities such

as “documentation day”

2011 & 2012 Event Calendar

• LinuxCon Brazil, Sao Paulo, Nov 17-18

• USENIX Lisa, Boston, Dec 4-9

• Planning to co-locate XenSummit NA with LinuxCon (LinuxCon NA, San Diego, Aug 27-28)

– Not yet finalized, but should be soon

• OSCON, Portland, July 16-20

12

Community Summary • The Xen Developer community is healthy for a 10 year old project

• The inclusion of Xen support into Linux 3.x has made a big impact – Getting questions by many new users

– Building new and productive relationships with many people in the Linux and BSD communities

• Up to now Xen.org was almost exclusively looking after developers – Successful open source communities bring their users and developers together

– Xen.org needs to focus on

• Reconnecting to its users

• On making it easy to get started with Xen and engage new users

– Many opportunities in Cloud Projects

• On better communicating the Xen advantages

• Everybody can help with this!

13

XEN 4.1 REVIEW

14

Xen 4.1 Release – 21 March 2011 • Very large system support

– 4 TB; >255 CPUs

– Reliability, Availability, Scalability enhancements

• CPU Pools for system partitioning

• Page sharing enhancements

• Hypervisor emergency paging / compression

• New “xl” lightweight control stack

• Memory Introspection API

• Enhanced SR-IOV support

• Software-implemented Hardware Fault Tolerance

15

Hardware Fault Tolerance

Restart-HA monitors hosts and VMs to keep apps running

Hardware Fault Tolerance with deterministic replay or checkpointing

Xen’s Software-Implemented Hardware Fault Tolerance enables true

High Availability for unmodified applications and operating systems

Hardware Fault Tolerance

• University of British Columbia’s “Remus” project is now in Xen 4

• Smart checkpointing approach yields excellent performance

– VM executes in parallel with checkpoint transmission, with all externally visible state changes suppressed until checkpoint receipt acknowledged

– Checkpoints delta compressed

• Checkpointing possible across wide-area, even for multi-vCPU guests

17

Enhanced SR-IOV

• SR-IOV: Single Root IO Virtualization – Virtualization friendly IO devices

• High performance, high efficiency, low latency

• Enables even the most demanding applications to now be virtualized

• Compatible with live relocation via hotplug of acceleration driver “plugin” module

– Retain primary benefit of physical hardware abstraction

18

SR-IOV NIC

Dell 10G Switch

NFS Common Storage w/OpenFiler

Dell R710 Server

XenServer and Intel 10G SR-IOV NIC

Dell R710 Server

XenServer and Intel 10G SR-IOV NIC

Dell R710 Server

XenServer and Intel 10G SR-IOV NIC

• 80 Gb/s bi-directional aggregate throughput between 4 VM pairs

• Low latency, High CPU efficiency

• Live relocation between hosts - Even hosts with different NICs

Network Performance

Type-0

0

5

10

15

20

25

30

35

CP

U (

%)

usercopy

kern

xen1

grantcopy

kern0

xen0basic smart

NIC

SR-IOV

NIC

native

201%

100% 123% 103%

• New Smart NICs reduce CPU overhead substantially

• Care must be taken with SR-IOV NICs to ensure benefits

of VM portability and live relocation are not lost

• Need for an industry standard for “driver plugins”

s/w only

THE NEXT VIRTUALIZATION WAVE

21

Security will drive the Next Wave of Virtualization

• Security is key requirement for Cloud

• Security is the primary goal of virtualization on the Client – Desktop, Laptops, Smart Phones, etc

• Maintaining isolation between VMs is critical – Spatial and Temporal isolation

– Run multiple VMs with policy controlled information flow • E.g. Personal VM; Corporate VM; VM for web browsing; VM for banking

• Enables “out-of-band” management and policy enforcement – Malware detection, remote access, image update, backup, VPN, etc.

22

Xen Introspection API

• Allows a suitably privileged VM to monitor and control the execution of another VM – Interpose on disk and network IO path

– Mark VM memory as immutable, no-execute etc

– Inspect/modify CPU and memory state

• Enables robust anti-malware, anti-root kit – Cannot be disabled/bypassed by guest VM

Virtualized can be more secure than physical!

24

Secure Isolation

• Use good software engineering practice – Thin hypervisor: minimize code running with privilege

– Disaggregate and de-privilege functionality into dedicated Service VMs

– Narrow interfaces between components

– Hypervisors are simpler than OSes, simpler than OS kernels

– Use modern high-level languages where possible

• New hardware technologies help – VT-x, VT-d, EPT: reduce software complexity, enhanced protection

– TPM/TXT: Enable Dynamic Root of Trust

25

XenClient XT / Qubes OS

• First products configured to take advantage of the security benefits of Xen’s architecture

• Isolated Driver Domains

• Virtual hardware Emulation Domains

• Service VMs (global and per-guest)

• Xen Security Modules / SElinux

• Measured Launch (TXT)

26

Typical Xen Configuration

Event Channel Virtual MMU Virtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

Native

Device

Driver

GuestOS

Device

Manager &

Control s/w

VM0

GuestOS

VM1

Front-End

Device Drivers

GuestOS

Applications

VM2

Device

Emulation

GuestOS

Applications

VM3

Safe HW IF

Xen Virtual Machine Monitor

Back-End

Applications

Front-End

Device Drivers

Xen Driver Domains

Event Channel Virtual MMU Virtual CPU Control IF

Hardware (SMP, MMU, physical memory, Ethernet, SCSI/IDE)

Native

Device

Driver

GuestOS

Device

Manager &

Control s/w

VM0

Native

Device

Driver

GuestOS

VM1

Front-End

Device Drivers

GuestOS

Applications

VM2

Device

Emulation

GuestOS

Applications

VM3

IOMMU

Xen Virtual Machine Monitor

Back-End Back-End

VM3a

Advanced XenClient Architecture

29

Xen

Intel vPro Hardware

Managem

ent

Dom

ain

Netw

ork

Isola

tion User VM

Per host/device

Service VMs

Xen Security Modules

VT-d TXT

VT-x AES-NI

Policy Granularity

User VM

Policy Granularity

Devic

e

Em

ula

te

VP

N Isola

tion

Devic

e

Em

ula

tion

VP

N Isola

tion

Per guest

Service VMs C

ontr

ol

Dom

ain

Disaggregation

• Unique benefit of the Xen architecture:

• Security – Minimum privilege; Narrow interfaces

• Performance – Lightweight e.g. minios directly on hypervisor

– Exploit locality – service VMs see a subset of the machine, run close to resources with which they interact

• Reliability – Able to be safely restarted

30

Isolated Driver VMs for High Availability

• Detect failure e.g.

– Illegal access

– Timeout

• Kill domain, restart – E.g. Just 275ms outage from

failed Ethernet driver

• New work uses restarts to enhance security

0

50

100

150

200

250

300

350

0 5 10 15 20 25 30 35 40

time (s)

Proposal

• We should strive to get all Xen products and deployments to take full advantage of the Xen architecture

• We need to make this much easier!

• Proposal: define and maintain a reference architecture and implementation that embodies best practice recommendations

32

Reference Architecture

• Define using new technologies – Latest stable Xen

– Linux 3.x pvops

• Optimization effort required

– Libxl control stack

• For easy consumption by other vendor tool stacks

33

Target Features • Network restart-able driver domains

– Integrated OpenFlow vswitch

• Storage restart-able driver domains – Also allows easier deployment of new storage options e.g. vastsky, ZFS

• Qemu emulation domains

• Xen Security Modules

• Measured Launch via TXT

• Roadmap for enhanced security and performance features – E.g. the SR-IOV network plugin / vswitch architecture

34

Implementation

• Need an initial reference implementation

– Easily consumable by users

• XCP could fulfil this role

– Showcase latest Xen technologies

– Optimized for OpenStack

• Aim to be as kernel/toolstack etc agnostic to allow easy adoption by all vendors

35

Summary

• Xen project continues to thrive!

– Great success in Cloud and Client

• Key architectural security, reliability and performance benefits that are unique to Xen

– We need to do a better job of getting the message out!

– We need to do a better job of actually taking advantage of the benefits in all Xen products

36