Xen Summit at AMDApril 28-29, 2010

1

PV-ops Linux kernel update

Jeremy Fitzhardinge &Konrad Rzeszutek Wilk

Xen Summit at AMDApril 28-29, 2010

2

Agenda

Primer on para-virtualized operations Pros & cons of PV-ops vs XenLinux Upstream strategy Current status Outstanding issues Explanation of GIT branches

Xen Summit at AMDApril 28-29, 2010

3

Primer (1/2)

http://wiki.xensource.com/xenwiki/XenParavirtOps

Linux kernel can be booted on: Baremetal Hardware assisted virtualization (HVM) Para-virtualization (PV)

PV requires modifications in the Linux kernel. In the past, two variants of the kernel:

baremetal (which can also do HVM) - kernel Para-virtualized kernel (kernel-xen)

Xen Summit at AMDApril 28-29, 2010

4

Primer (2/2)

Para-Virtualization Operations (PV-ops): Detects if running in HVM, VMWare, Xen, KVM Patches all low-level privileged operations (memory

updates, spinlocks, etc) with appropiate virtualization aware calls.

End-result: One kernel to boot under different X86 platforms. Lower QA No #ifdef CONFIG_XEN (frowned on by everybody)

Xen Summit at AMDApril 28-29, 2010

5

PV-ops & XenLinux (1/2)

XenLinux (2.6.18, 2.6.27, 2.6.31, 2.6.32):

Runs only under Xen. Has much big breadth of Xen back/front drivers (PV scsi, PV

usb, Netchannel 2) and can work with proprietary drivers: nvidia, acronis snapapi, etc.

Super stable (2.6.18, 2.6.27) Not upstreamable Performance

PV-ops (2.6.31, 2.6.32):

Lower testing requirements (one kernel) Share code base with X86 common code function. New development

Xen Summit at AMDApril 28-29, 2010

6

PV-ops & XenLinux (2/2)

http://wiki.xensource.com/xenwiki/XenDom0Kernels

XenLinux distros: RHEL5, OVM OpenSUSE 11/SLES11 Debian (lenny, etch) XenSource (XCP)

PV-ops distributions: Fedora Core (unofficial)

http://myoung.fedorapeople.org/dom0 Debian squeeze

http://packages.debian.org/xen-linux-system-2.6.32

Xen Summit at AMDApril 28-29, 2010

7

Upstream strategy (1/3)

Three code ”camps”: DomU (unprivileged PV guests). Mostly frontend

drivers and Linux has many already in: Network, Disk, Framebuffer, Keyboard, and Console. Missing: PCI frontend.

Dom0 (privileged PV guest). Main guest handling PCI devices, ACPI, IOAPIC, KMS, XServer. PAT, MTRR, MSI, etc. None are upstream.

Shared between baremetal, DomU and Dom0. For example: Software IOMMU (SWIOTLB).

Xen Summit at AMDApril 28-29, 2010

8

Upstream strategy (2/3)

Jeremy git branch has DomU, Dom0 and shared code.

Konrad git has DomU and shared code. Strategy is to seperate the functionality patches

in separate branches so they can be submitted upstream as whole ”functional” pieces. For example: xen/core, xen/dom0/apic

But many are built on top of each other. For example: XenPCI front requires Xen-SWIOTLB.

Xen Summit at AMDApril 28-29, 2010

9

Upstream strategy (3/3)

Three-pronged strategy: Constantly refactor code for upstream submission. Stabilize existing Dom0 branch so that distros can

pick it up (Debian, Fedora, Qubes-OS, XenServer) Rebase existing branches against newer kernels to

keep up

Xen Summit at AMDApril 28-29, 2010

10

Status (1/3) - refactoring

SWIOTLB changes submitted upstream Xen-SWIOTLB – waiting Xen PCI and Xen PCI frontend– waiting Has been tested by community members with success pv/merge-2.6.33, pv/merge.2.6.34-rc3

PV on HVM – work in progress

PAT – being tested

IO APIC/ACPI – researching solution

MSI – researching solution

Xen Summit at AMDApril 28-29, 2010

11

Status (2/3) - stabilizing

xen/stable-2.6.[31|32|33].x: Community providing many reports And patches as well And Wiki writeups! Tracking mainline long-term stable-2.6.[31|32|33] while

incorporating Xen fixes

xen/next is the development branch: Xen bug-fixes only New ideas on solving problems (ACPI GSI IRQ 20) What is stable trickles down to xen/stable-2.6.[31|32|

33].x

Xen Summit at AMDApril 28-29, 2010

12

Status (3/3) - rebasing

Konrad's pv/pcifront-2.6.3[X] and swiotlb-0.Y rebased to keep up with upstream tree. Unprivileged PV (DomU) and X86 shared code git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git

Jeremy's xen/stable-2.6.3[X]: privileged & unprivileged PV & X86 shared code. git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git

.34 still work in progress

Xen Summit at AMDApril 28-29, 2010

13

Outstanding issues

Desktop video card (Nvidia, ATI, Intel) http://wiki.xensource.com/xenwiki/XenPVOPSDRM

DRM X Server Kernel Mode Setting Direct Rendering Manager/TTM

Network between DomU and Dom0

Xen Summit at AMDApril 28-29, 2010

14

Thanks

Thanks to Xen community who have taken the time and effort to work with and on the PV-ops kernel.

Xen Summit at AMDApril 28-29, 2010

15

Jeremy's GIT branches

git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git

xen/next – base Xen developments against 2.6.32, no kernel updates or fixes

xen/stable-2.6.[31|32|33] – tracking mainline long-term stable-2.6.[31|32|33] while incorporating Xen updates

xen/core – PV and shared code. PTE, IRQ xen/dev-evtchn – PV and shared code. Events xen/dom0/core – privileged PV (dom0) code. MTRR,

PAT, HPET, etc xen/dom0/msi – privileged PV (dom0) MSI code xen/backend/core – Xen backend base

Xen Summit at AMDApril 28-29, 2010

16

Konrad's GIT branches

git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git

pv/pcifront-2.6.[32|33|34] – Xen PCI & Xen PCI frontend driver.

xen-swiotlb-0.7 – latest Xen-SWIOTLB code pv/merge.2.6.[32|33|34] – both branches above

combined

Xen Summit at AMDApril 28-29, 2010

17

Stefano's GIT branches

git://xenbits.xen.org/people/sstabellini/linux-pvhvm

2.6.32-pvhvm – development branch