Upload
cameroon45
View
3.486
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
libvirt: A virtualization API
MARCO GUAZZONE
Distributed Computing Systems Group (DCS)Department of Computer ScienceUniversity of Piemonte Orientale
September 8, 2008
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 1 / 45
Outline
1 Hypervisor APIsHypercall APIManagement API
2 libvirtlibvirt: Overviewlibvirt: Virtualization Support
3 ExamplesQEMUXenRemote Management
4 Conclusions
5 References
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 2 / 45
Hypervisor APIs
Two types of APIs:1 Hypercall API: used by guests for para-virtualization.2 Management API: used by management tools.
I In Xen, also known as Xen API
ImportantIn the following, we will focus our attention on the Xen hypervisor.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 3 / 45
Hypervisor APIs: the Xen system
. . . From [6].
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 4 / 45
Hypervisor APIs: Hypercall API
APIs for letting a guest to perform privileged instructions
The hypervisor (not the kernel)has interrupt handlers installed.
When the application (in theguest) invokes a system call:
1 an interrupt is raised (trap)2 and is caught by the
hypervisor,3 which then passes the
control back to the guestOS, through anasynchronous eventnotification mechanism.
. . . From [6].
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 5 / 45
Hypervisor APIs: Hypercall APIExampleHow a Xen C hypercall might look like:
hypercall_ret = xen_op(operation, arg1, arg2, arg3,arg4);
ExampleResulting assembly-like routine:
_xen_op:mov eax, 4(esp)mov ebx, 8(esp)mov ecx, 12(esp)mov edx, 16(esp)mov esi, 20(esp)int 0x82ret
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 6 / 45
Hypervisor APIs: Hypercall API
Hardware Virtual Machine (HVM), also known as hardware assistedvirtualization, has recently emerged.
Intel’s VT and AMD’s AMD-V extensions are the majors hardwaresupport technologies for virtualization.With HVM the use of additional protecting rings become lesscritical.A guest in an HVM environment can use the acceleratedtransitions to ring 0 for system calls (accelerated system calls),because it has not been moved from ring 0 to ring 1 (as happensinstead with para-virtualization).
Is this the best solution?
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 7 / 45
Hypervisor APIs: Hypercall API
No! Hybrid virtualization seems to behave better.Since with HVM the guest OS is not modified for supportingpara-virtualization, it does not know that is running in a virtualenvironment and so it cannot take advantage of any of thevirtualization features.The result is that HVM might be slower than para-virtualization.Hybrid virtualization tries to take the best of each worlds:
I From HVM: makes use of accelerated system calls and exploit otherhardware assisted facilities, like nested page tables (NPTs) [3].
F With Nested Paging, a page table in the hardware takes care of thetranslation between the guest address of a virtual machine and thephysical address, reducing the overhead.
I From para-virtualization: use of light-weighted ad-hoc interfacesrather than relying on emulated hardware (e.g., for I/O).
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 8 / 45
Hypervisor APIs: (Xen) Management API
APIs used by user-space applications for management and VM’slife-cycle tasks.In Xen:
Xen API, the core API, are an XML-RPC based API.I xend listens for XML-RPC connections.I All that can be done with the xm tool is possible with Xen API.
Upon Xen API there are several bindings.I Each binding API sends an XML-RPC over the socket where an
instance of xend is listening.I xend handles the request itself or dispatch it on to the kernel’s
hypervisor interface and then on the hypervisor itself.Third party tools and libraries reside upon these bindings.
I libvirt is one of these libraries.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 9 / 45
Hypervisor APIs: (Xen) Management APICurrently supported binding languages:
libxen, C binding.pyxen, Python binding.XenSdk.net, C# binding (in Citrix XenServer 4.1).Xen-CIM, a CIM provider for the DMTF virtualization schema.
I The Common Information Model (CIM) [9] is a family of openstandards, defined and published by the Distributed ManagementTask Force (DMTF), that defines how managed elements in an ITenvironment are represented as a common set of objects andrelationships between them. This is intended to allow consistentmanagement of these managed elements, independent of theirmanufacturer or provider.
I Is a modeling language, rather than a programming language.I Provides a uniform, generic and standard interface for accessing to
management facilities.F VMware provides a CIM layer too [16].
Java binding probably no more supported.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 10 / 45
libvirt: What is it?
The libvirt toolkit provides a higher-level VM management interface fortools and applications, that is:
A set of command line utilities for interacting with the virtualizationcapabilities of the OS.A consistent set of API in C with the aim to provide support acrossdifferent virtualization tools.A CIM provider for the DMTF virtualization schema.
A project sponsored by the Red Hat’s Emerging Technology group [13].
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 11 / 45
libvirt: Goal
To provide all the operations needed to manage guests ordomains running on a single physical node.To supplies a stable interface that isolates upper-level softwarefrom changes in the underlying virtualization layer.
I Each virtualization layer would implement the libvirt interface onwhich the upper layer tools rely.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 12 / 45
libvirt: Features
Virtualization support.Management of virtual machines, virtual network and storage.Remote management.
The libvirt library does not provide high-level multi-node managementfeatures such as load balancing.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 13 / 45
libvirt: Componentslibvirt: the core C API layer.virtsh: a command line Cprogram which provides ashell environment and amanagement user interface.
I Can be used to create,pause, list, migrate andshutdown domains.
libvirtd: a C daemon formanaging guest instances andlibvirt virtual networks.
Hypervisor layer
libvirt API layer
libvirt tools layer
libvirtd virsh
libxen CIM XML
Other hypervisor drivers
API bindings
Application layer
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 14 / 45
libvirt: Virtualization support
Terminology:node: a single physical machine.hypervisor: a layer of software allowing to virtualize a node in aset of virtual machines with possibly different configurations thanthe node itself.domain: an instance of an operating system running on avirtualized machine provided by the hypervisor.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 15 / 45
libvirt: Connections
Interaction with a virtualization technology based on connection.Use of URI to specify which driver a connection refers to.driver[+transport]://[username@][hostname][:port]/[path][?extraparameters]
I driver: the virtualization technology to interact with.I transport: the transport layer to use for connecting to the driver.I username: the credentials to use for connecting to the driver.I hostname: the (possible remote) host where the virtualization
technology resides.I port: the port where the virtualization technology listens for
connections.I path: a driver dependent path (e.g. the path to a Unix domain
socket).I extraparameters: additional optional parameters.
NULL and empty string URIs means “connect to the best availablelocal hypervisor”.
I Actually tries to connect to Xen.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 16 / 45
libvirt: DriversThe virtualization technology to interact with:
ldom: the Sun LDoms virtualization technology [15].I Directly provided by Sun, starting from version 1.0.2 [14].
lxc: the LXC [1] Linux container system.openvz: the OpenVZ [2] Linux container system.qemu: the QEMU [5] emulator (also for the KVM [12] and Xenner[10] hypervisors).remote: a dummy driver for accessing to “remote” hypervisors.storage: storage on IDE/SCSI/USB disks, FibreChannel, LVM,iSCSI, NFS and filesystems.test: a dummy driver for testing purposes.xen: the Xen [7] hypervisor on Linux and Solaris hosts.Others undocumented/experimental/planned: uml (User ModeLinux), vserver (Linux V-Server, [11]), vmware (VMware), hyperv(Microsoft Hyper-V, [8]).
I Mostly found in libvirt-0.4.4/src/domain_conf.[hc].
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 17 / 45
libvirt: Transports
The protocol used for connecting to the virtualization technology:ext: use an external program which can make a connection to theremote machine by means outside the scope of libvirt.ssh: use an SSH connection (needs netcat and libvirtd on theremote machine).
I libvirt constructs an SSH command which looks like:command -p port [-l username] hostname netcat -Usocket
I port, username, hostname can be specified as part of the remoteURI.
I command, netcat and socket come from extra parameters (orsensible defaults).
tcp: use the TCP/IP transport protocol.tls: use a TLS connection (needs client and server certificates).unix: use a UNIX socket.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 18 / 45
libvirt: Extra Parameters
An optional list of parameters following the syntax of RFC-2396.
Name Transports Meaningname * Explicitly force the name of the hypervisor.
Example: ...&name=qemu:///systemcommand ssh, ext The external command.
Example: ...&command=/opt/openssh/bin/sshsocket unix, ssh The path to the Unix domain socket (overrides the default).
Example: ...&socket=/opt/libvirt/run/libvirt/libvirt-socknetcat ssh The name of the netcat command on the remote machine (default
is nc).Example: ...&netcat=/opt/netcat/bin/nc
no_verify tls If set to a non-zero value, this disables client checks of theserver’s certificate.
Example: ...&no_verify=1no_tty ssh If set to a non-zero value, this stops ssh from asking for a pass-
word if it cannot log in to the remote machine automatically (eg.using ssh-agent etc.).
Example: ...&no_tty=1
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 19 / 45
libvirt: Domain ConfigurationA virtualized element (i.e. a domain, a storage or a network) is definedthrough a configuration XML file, containing, among other things:
The type of the hypervisor used for running the domain.The symbolic name of the domain.The type of boot-loader:
I The bios boot-loader, available in full virtualization, which uses theBIOS boot order priority (e.g., floppy, hard-disk, cdrom, network) forfinding and booting the boot image.
I The host boot-loader, available in para-virtualization, where thehost is responsible for kicking off the operating system boot.
I The direct kernel boot-loader, available in full/para-virtualization,which boots directly from a kernel stored in the host OS.
The boot device.The maximum resource usage for CPUs and memory.The devices provided to the guest domain.
I Include: disks (HD, floppy, CD-ROM), USB, NIC, input devices,graphical frame-buffers (e.g. for VNC), console/serial/paralleldevices.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 20 / 45
libvirt: Local Domain Management
Hypervisor layerlibvirt API layer
xendApplication layer xenstored
Xen hypervisor
xendriver
QEMUdriver
qemu
xen:///
qemu:///
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 21 / 45
libvirt: Remote Domain Management
Hypervisor layerlibvirt API layer
xend
Application layerxenstored
Xen hypervisor
xendriver
QEMUdriver
qemu
libvirt API layer
remotedriver
xen:///
qemu:///
local host
remote host
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 22 / 45
libvirt: Bindings
Available binding languages:libvirt-python, the official Python binding.Sys::Virt: a Perl binding.ocaml-libvirt: the official OCaml binding.ruby-libvirt: the official Ruby binding.libvirt-java: the official Java binding (WIP).
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 23 / 45
Examples: QEMU
In order to use QEMU through libvirt it is necessary:1 To install QEMU and libvirt (and their dependencies)
I For QEMU is highly recommended to also install the KVM+QEMU(KQEMU) kernel module.
2 To run the proper system daemons.I On RedHat-like systems:$ /etc/init.d/kqemu start$ /etc/init.d/qemu start$ /etc/init.d/libvirtd start
3 To be able to gain root privileges.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 24 / 45
Examples: QEMU capabilities I
Going to print QEMU capabilities.1 Gain root privileges.2 With virsh command:
$ virsh -r -c "qemu:///session" capabilities
3 With the Python APIs:1 Start the Python interpreter.
$ python2 Import the libvirt module.
>>> import libvirt3 Open a (read-only) connection to the QEMU hypervisor.
>>> con = libvirt.openReadOnly("qemu:///system")4 Print capabilities to standard output.
>>> print con.getCapabilities(), "\n"
4 An XML output should appear:
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 25 / 45
Examples: QEMU capabilities II<capabilities><host><cpu><arch>x86_64</arch></cpu></host>
<guest><os_type>hvm</os_type><arch name=’i686’><wordsize>32</wordsize><emulator>/usr/bin/qemu</emulator><machine>pc</machine><machine>isapc</machine><domain type=’qemu’></domain></arch><features>
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 26 / 45
Examples: QEMU capabilities III<pae/><nonpae/><acpi default=’on’ toggle=’yes’/><apic default=’on’ toggle=’no’/></features></guest>
<guest><os_type>hvm</os_type><arch name=’x86_64’><wordsize>64</wordsize><emulator>/usr/bin/qemu-system-x86_64</emulator><machine>pc</machine><machine>isapc</machine><domain type=’qemu’></domain><domain type=’kqemu’></domain>
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 27 / 45
Examples: QEMU capabilities IV
</arch><features><acpi default=’on’ toggle=’yes’/><apic default=’on’ toggle=’no’/></features></guest>...</capabilities>
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 28 / 45
Examples: QEMU domain list
Going to print the list of the installed QEMU domains.1 Gain root privileges.2 With virsh command:
$ virsh -r -c "qemu:///session" list --allI --all includes inactive domains.
3 With the Python APIs:1 Start the Python interpreter.
$ python2 Import the libvirt module.
>>> import libvirt3 Open a (read-only) connection to the QEMU hypervisor.
>>> con = libvirt.openReadOnly("qemu:///system")4 Print capabilities to standard output.
>>> print con.listDefinedDomains(), "\n"
4 A list of 〈id, name, execution status〉 triples should appear as output.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 29 / 45
Examples: DSL on QEMU IGoing to run Damn Small Linux (DSL) [4] inside QEMU.
1 Download a DSL iso image$ GET http://.../current/current.iso > dsl.iso
2 Create a QEMU image file (100MB should suffice)$ qemu-img create -f qcow2 dsl.qcow2 100MB
3 Create the XML configuration file dsl-kqemu.xml for describing theDSL image.<?xml version="1.0"?>
<domain type=’kqemu’><name>KQEmu-DSL-i686</name><uuid>c7a5fdbd-cdaf-9455-926a-d65c16db1809</uuid><memory>65536</memory><currentMemory>32768</currentMemory><vcpu>1</vcpu><os><type arch=’i686’ machine=’pc’>hvm</type>
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 30 / 45
Examples: DSL on QEMU II
<boot dev=’cdrom’/></os><devices><emulator>/usr/bin/qemu-system-x86_64</emulator><disk type=’file’ device=’cdrom’><source file=’/path/to/vm/iso/dsl.iso’/><target dev=’hdc’/><readonly/></disk><disk type=’file’ device=’disk’><source file=’/path/to/vm/images/dsl.qcow2’/><target dev=’hda’/></disk><interface type=’network’><source network=’default’/></interface><graphics type=’vnc’ port=’-1’/>
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 31 / 45
Examples: DSL on QEMU III</devices></domain>
I For generating a UUID it is possible to use the uuidgen command.I Without libvirt, the QEMU command would be:$ qemu-system-x86_64 -M pc -m 64 -smp 1 -nameKQEmu-DSL-i686 -cdrom dsl.iso -hda dsl.qcow2-boot d
4 Run DSL on QEMUvirsh -c qemu:///system create dsl-kqemu.xml
5 Now it is possible to manage the newly created VM. For instance, open aVNC session$ virsh -c qemu:///system vncdisplay KQEmu-DSL-i686$ vncviewer 127.0.0.1:0
6 Shutdown DSL on QEMUvirsh -c qemu:///system destroy KQEmu-DSL-i686
I Note: it seems the shutdown command doesn’t work!
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 32 / 45
Examples: Xen
The syntax is very similar to the one used for QEMU:List of Xen capabilities:$ virsh -r -c "xen:///" capabilities
List of local domains:$ virsh -r -c "xen:///" list --all
. . .
What changes is the content of the XML configuration file and the URI.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 33 / 45
Example: Remote Management with SSH
1 Create your local public key pair.$ ssh-keygen -t rsa
2 Copy the public key to a remote host.$ ssh-copy-id -i ~/.ssh/id_rsa.pub root@remote-host
3 Start the libvirt daemon.$ ssh root@remote-host$ /etc/init.d/libvirtd start
4 Issue a command to the libvirt daemon.$ virtsh -r -c xen+ssh://root@remote-host/ list --all
The first three steps are to be done only once.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 34 / 45
Conclusions: Pros & Cons
Strong points:Abstraction: support different hypervisors.Isolation: isolates from hypervisor API changes.Portability: Linux, Windows and Mac OS-X clients.Security: TLS + x509, Kerberos, SSH, PolicyKit.Active community.
I Is the core of several Red-Hat virtualization softwares.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 35 / 45
Conclusions: Pros & ConsWeak points:
Abstraction: might loose some hypervisor features.Lack of documentation.
I No tutorials.I APIs poor documented.
⇒ You have to learn-by-examples (see libvirt sources andRed-Hat virt-manager)For remote management, the libvirtd daemon must be running(with root privileges) on every remote host that needs to bemanaged.Moreover, the libvirtd daemon is needed by QEMU because ithas to do lots of privileged jobs, such as starting QEMU withpermission to use /dev/kvm, accessing disks and logicalvolumes in /dev, creating TAP devices, creating bridge devices,and much more.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 36 / 45
Conclusions: libvirt Companions
The Red Hat’s Emerging Technology group includes other interestingprojects:
virt-manager: graphical desktop-based virtual machinemanagement.python-virtinst: guest installation manager.
I virt-viewer: secure guest console access.I virt-install: virtual machines provisioner.I virt-clone: virtual machine images cloner.I virt-image: virtual machine images creator (from XML files or
interactively).
oVirt: Web-based virtual machine management.All of them rely on libvirt.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 37 / 45
Conclusions
Questions?
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 38 / 45
Conclusions
Grazie!
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 39 / 45
References I
[1] The Linux Containers (LXC).http://lxc.sourceforge.net/.
[2] The OpenVZ Linux Containers.http://wiki.openvz.org/.
[3] AMD.AMD-VTM Nested Paging.Technical report, Advanced Micro Devices (AMD), Inc., July 2008.
[4] John Andrews.Damn Small Linux (DSL).http://www.damnsmalllinux.org.
[5] Fabrice Bellard.The QEMU processor emulator.http://bellard.org/qemu/.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 40 / 45
References II
[6] David Chisnall.The Definitive Guide to the Xen Hypervisor.Prentice Hall, 2007.
[7] Inc. Citrix Systems.The Xen hypervisor.http://www.xen.org/.
[8] Microsoft Corporation.Microsoft Hyper-V.http://www.microsoft.com/windowsserver2008/en/us/virtualization-consolidation.aspx?pf=true.
[9] Inc. Distributed Management Task Force (DMTF).Common Information Model (CIM).http://www.dmtf.org/standards/cim/.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 41 / 45
References III
[10] Gerd Hoffmann.Xenner: Xen emulator for KVM.http://kraxel.fedorapeople.org/xenner/.
[11] Herbert Pötzl.Linux v-server.http://linux-vserver.org/Welcome_to_Linux-VServer.org.
[12] Inc. Qumranet.The Kernel based Virtual Machine (KVM).http://kvm.qumranet.com/.
[13] Inc. Red Hat.The Red Hat’s Emerging Technology group.http://www.dmtf.org/standards/cim/.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 42 / 45
References IV
[14] Inc. Sun Microsystems.Libvirt for LDoms 1.0.1 administration guide.http://docs.sun.com/app/docs/doc/820-3838-10.
[15] Inc. Sun Microsystems.Logical domains (LDoms).http://www.sun.com/servers/coolthreads/ldoms/.
[16] Inc. VMware.VMware CIM APIs.http://www.vmware.com/support/developer/cim-sdk/.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 43 / 45
Extras: Nested Paging
Introduced by AMD [3].Virtual Machines (VMs) don’t have native direct access to the hostserver memoryAs a result, a hypervisor ends up virtualizing a “read only” layer ofmemory between physical memory and the page tables in theguest OS, which is known as shadow paging.Shadow pages requires CPU and memory, adding extraperformance overhead.With Nested Paging, a page table in the hardware takes care ofthe translation between the guest address of a VM and thephysical address, reducing the overhead.
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 44 / 45
Extras: CIM vs. libvirtCIM and libvirt both provide a hypervisor-agnostic abstractionlayer for writing tools. Nevertheless, there are a few majordifferences:
I CIM is an abstract model, with representations built on XML, andusable in a language-agnostic way.libvirt is quite closely tied to C.
I CIM is a standard defined by a working group representing multiplevendors.libvirt began life as a wrapper around xm and has grown to a moregeneral interface. It is still quite closely tied to the Xen way of doingthings, however.
I CIM is a large specification, and the virtualization parts are only asmall fraction of the whole.libvirt is designed exclusively for managing virtualization.
I CIM lets to easily and flexibly add support for managingvirtualization to a CIM-aware management tool.libvirt is extensible “only” through the bindings it provides (thoughthe set of supported languages is pretty large).
Marco Guazzone (DCS) libvirt: A virtualization API September 8, 2008 45 / 45