Upload
novell
View
167
Download
1
Tags:
Embed Size (px)
DESCRIPTION
As a technical preview, SUSE Linux Enterprise Server 11 contains KVM, which is the next-generation virtualization software delivered with the Linux kernel. In this technical session we will demonstrate how to set up SUSE Linux Enterprise Server 11 for KVM, install some virtual machines and deal with different storage and networking setups.To demonstrate live migration we will also show a distributed replicated block device (DRBD) setup and a setup based on iSCSI and OCFS2, which are included in SUSE Linux Enterprise Server 11 and SUSE Linux Enterprise 11 High Availability Extension.
Citation preview
Virtualization with (KVM)Kernel-based Virtual Machine
Thomas Korber Consultant and TrainerB1 Systems [email protected]
Bruce RogersConsulting Software EngineerNovell, [email protected]
© Novell, Inc. All rights reserved.2
KVM
First release in early 2007
Originally developed by Qumranet
Included in Linux kernel release 2.6.20
GPL v2
© Novell, Inc. All rights reserved.3
KVM – Full Virtualization
• Relies on AMD's AMD-V or Intel's VT-x virtualization technologies
• Implemented as kernel modules– kvm.ko: provides virtualization infrastructure– kvm_amd.ko and kvm_intel.ko: hardware platform specific
modules for the hardware virtualization technologies
• => Vanilla Linux kernel becomes virtual machine monitor, which can use any kernel infrastructure without modifications
• => KVM virtual machines become regular user-space processes
© Novell, Inc. All rights reserved.4
KVM ArchitectureAdds “Guest Mode” to Traditional Kernel and User Modes
UserspaceProcess
Hardware Support,vitualization technologies for x86
(AMD-V/ Intel-VT)
Linux Kernel
UserspaceProcess ...
Guest UserspaceProcesses
KVM (Module)
QEMU-KVM
Guest Kernel(e.g. Linux Kernel)
Source: “Virtualization with KVM” training, B1 Systems GmbH
© Novell, Inc. All rights reserved.5
Supported Hardware
Any i386/x86_64 CPUs that have AMD-V or VT-x:
=> Almost any server CPU sold in the last couple years
© Novell, Inc. All rights reserved.6
Supported Hardware (Continued)
Utilizes the following additional hardware virtualization features:
VPID / ASID
VT-d/IOMMU
HAP (EPT/NTP)
VMX Unrestricted Guest
SR-IOV
© Novell, Inc. All rights reserved.7
QEMU
• Community project founded in 2003
• Emulates PC hardware and CPUs
• Since v 0.10.0 support for KVM VMM
• Modified qemu-kvm is user space tool for KVM
• Communication with KVM via /dev/kvm
© Novell, Inc. All rights reserved.8
KVM Features
Supports 32 and 64 bit guests (on 64 bit hosts)
Supports hardware virtualization features
Paravirtualized drivers (virtio): blk, net, clock, balloon
Snapshots
Delta images of virtual machines
PCI passthrough
Kernel samepage merging
© Novell, Inc. All rights reserved.9
KVM Features (continued)
Sound support
CPU, memory and disk over-commit
Live migration
CPU and device hotplug
Non-kvm (emulation only) mode
PXE boot
© Novell, Inc. All rights reserved.10
KVM - Supported Guest Systems
BSD
Solaris, OpenSolaris Linux
Windows BSD Unix
© Novell, Inc. All rights reserved.11
KVM Guests Supported by Novell® (I)
Linux - both 32 and 64 bit
• SUSE Linux Enterprise Server 11 SP1 (level 3 supported)
• SUSE Linux Enterprise Server 10 SP3 (level 3 supported)
• SUSE Linux Enterprise Server 9 SP4 (level 3 supported)
• SUSE Linux Enterprise Desktop 11 SP1 (technical preview)
• Red Hat Enterprise Linux 4 (best effort)• Red Hat Enterprise Linux 5 (best effort)
© Novell, Inc. All rights reserved.12
KVM Guests Supported by Novell® (II)
Microsoft Windows – both 32 and 64 bit
(“best effort” support only)
• Microsoft Windows 2003 SP2+ plus PV drivers
• Microsoft Windows 2008+ plus PV drivers
• Microsoft Windows XP SP3+ plus PV drivers
• Microsoft Windows Vista SP1+ plus PV drivers
© Novell, Inc. All rights reserved.13
Supported Limits
Host RAM and CPU limits are the same with or without kvm modules loaded
Guest RAM size: 512 GB
Virtual CPUs per guest: 16
NICs per guest: 8
Block devices per guest: 4 emulated, 20 para-virtual (virtio-blk)
Maximum number of guests: total vCPUs <= 8 times total CPU cores in Host
© Novell, Inc. All rights reserved.14
Xen and KVM: A Comparison
KVM
• Kernel module
• Uses kernel as VMM
• In upstream kernel
• Only supports fully virtualized VMs
•
•
Xen
• VMM implementation of its own; hypervisor
• Kernel as I/O dispatcher and management domain
• Maintained and supported as a patch to mainline kernel by Novell®
• Supports fully virtualized and paravirtualized Vms
© Novell, Inc. All rights reserved.15
Virtualization in SUSE® Linux Enterprise Server 11 SP1• SUSE Linux Enterprise Server 11 SP1 will ship with
both virtualization solutions (KVM and Xen)
• Xen is the primary solution, being the proven enterprise-ready open source hypervisor
• Long term, Novell® expects KVM eventually to become equivalent to Xen
• Toolset shipped in SUSE Linux Enterprise Server 11 SP1 supports both Xen and KVM
Setting up KVM on SUSE® Linux Enterprise 11 SP1
© Novell, Inc. All rights reserved.17
Demo Setup
• Storage server and installation source:
– SUSE® Linux Enterprise Server 11 GA x86_64
– Logical volume as iSCSI target for OCFS2 file system
– Installation sources (SUSE Linux Enterprise Server 11 SP1 Beta5 and SUSE Linux Enterprise High Availability Extension Server 11 SP1 Beta5) exported via HTTP
• 2 KVM hosts– SUSE Linux Enterprise Server 11 SP1 Beta5 x86_64
– Logical volume for DRBD; DRBD primary/primary setup
© Novell, Inc. All rights reserved.18
Demo Setup – Shared Storage
OCFS2
Node 1 Node 2 Node 3
FC or iSCSI
© Novell, Inc. All rights reserved.19
Demo Setup – Replicated Storage
Node 1 Node 2
TCP/IPDRBD
Local Disk Local Disk
© Novell, Inc. All rights reserved.20
Packages
• KVM, libvirt and virt-manager as GUI
zypper in kvm virt-manager
• (optional) packages for shared storage:
– server: iscsitarget– KVM hosts: open-iscsi, ocfs2-tools, ocfs2-tools-o2cbor
– KVM hosts: drbd, drbd-kmp-default
© Novell, Inc. All rights reserved.21
iSCSI Setup
• Storage Server:storage:~ # cat /etc/ietd.conf | grep -v "#"Target iqn.2009-11.b1-systems.de:lv_shareLun 0 Path=/dev/vg_system/lv_share,Type=fileio
• KVM Hosts:node1:~ # iscsiadm -m discovery -tst -p storage192.168.2.35:3260,1 iqn.2009-11.b1-systems.de:lv_sharenode1:~ # iscsiadm -m node -T iqn.2009-11.b1-systems.de:lv_share -p 192.168.2.35 -l
© Novell, Inc. All rights reserved.22
DRBD Setup - /etc/drbd.conf (I)
Create /etc/drbd.conf and have the identical file on both nodesnode1:~ # cat /etc/drbd.confglobal { usage-count no;}resource r0 { protocol C; syncer { rate 40M; } net { allow-two-primaries; } startup { become-primary-on both; }
© Novell, Inc. All rights reserved.23
DRBD Setup - /etc/drbd.conf (II)
on node1 { device /dev/drbd0; disk /dev/mapper/storage-lv_drbd; address 192.168.2.31:7791; meta-disk internal; } on node2 { device /dev/drbd0; disk /dev/mapper/storage-lv_drbd; address 192.168.2.32:7791; meta-disk internal; }}
© Novell, Inc. All rights reserved.24
DRBD Setup
On both nodes:node1:~ # modprobe drbd && rcdrbd startnode1:~ # drbdadm create r0
On first node:node1:~ # drbdadm -- --overwrite-data-of-peer primary r0
On second node:node1:~ # drbdadm primary r0
On either node:node1:~ # cat /proc/drbd
© Novell, Inc. All rights reserved.25
OCFS2 Setup (I)
node1:~ # vi /etc/ocfs2/cluster.conf
node: name = node1
cluster = ocfs2
number = 0
ip_address = 192.168.2.31
ip_port = 7777
node: name = node2
cluster = ocfs2
number = 1
ip_address = 192.168.2.32
ip_port = 7777
cluster: name = ocfs2
node_count = 2
© Novell, Inc. All rights reserved.26
OCFS2 Setup (II)
node1:~ # rco2cb configureConfiguring the O2CB driver.
This will configure the on-boot properties of the O2CB driver.Load O2CB driver on boot (y/n) [y]: Cluster stack backing O2CB [o2cb]: Cluster to start on boot (Enter "none" to clear) [ocfs2]: Specify heartbeat dead threshold (>=7) [31]: Specify network idle timeout in ms (>=5000) [30000]: Specify network keepalive delay in ms (>=1000) [2000]: Specify network reconnect delay in ms (>=2000) [2000]: Writing O2CB configuration: OKLoading filesystem "configfs": OKMounting configfs filesystem at /sys/kernel/config: OKLoading stack plugin "o2cb": OKLoading filesystem "ocfs2_dlmfs": OKMounting ocfs2_dlmfs filesystem at /dlm: OKSetting cluster stack "o2cb": OKStarting O2CB cluster ocfs2: OK
© Novell, Inc. All rights reserved.27
OCFS2 Setup (III) - iSCSI
node1:~ # mkfs.ocfs2 /dev/disk/by-path/ip-192.168.2.35\:3260-iscsi-iqn.2009-11.b1-systems.de\:lv_share-lun-0
On both nodes:
node1:~ # mount /dev/disk/by-path/ip-192.168.2.35\:3260-iscsi-iqn.2009-11.b1-systems.de\:lv_share-lun-0 /var/lib/kvm/images/
© Novell, Inc. All rights reserved.28
OCFS2 Setup (IV) - DRBD
node1:~ # mkfs.ocfs2 /dev/drbd0
On both nodes:
node1:~ # mount /dev/drbd0 \ /var/lib/kvm/images
© Novell, Inc. All rights reserved.29
KVM VM Installation – GUI
© Novell, Inc. All rights reserved.30
KVM VM Installation: CLI
• qemu-img create \ /var/lib/kvm/images/sles11_raw_disk1.img 5G
• qemu-kvm -hda \/var/lib/kvm/images/sles11_raw_disk1.img \ -cdrom /srv/isos/SLES-11-DVD-x86_64-GM-DVD1.iso \ -boot d -m 512[installation of a “physical computer”]
• qemu-kvm -hda \/var/lib/kvm/images/sles11_raw_disk1.img -m 512
© Novell, Inc. All rights reserved.31
KVM – Installation With vm-installUnattended installation:
vm-install --background --vm-settings=/foo/bar/vm-template.xml --os-settings=/foo/bar/autoinst.xml …
© Novell, Inc. All rights reserved.32
KVM – Networking
• Usermode network stack
– Default setup
– No root permissions needed
– Integrated DHCP, DNS, SMB and DNS
• TAP device
• Bridged mode (comparable to default Xen network setup)
© Novell, Inc. All rights reserved.33
KVM – Networking (II)
Example: bridged setupnode1:~ # cat /etc/libvirt/qemu/sles11.xml cat /etc/libvirt/qemu/networks/default.xml <network> <name>default</name> [...] <bridge name="br0" /> [...]</network>
© Novell, Inc. All rights reserved.34
KVM – Selected Image Formats
Name Compression Snapshot Encryption Deltas
raw
qcow2 X X X Xvmdk X
© Novell, Inc. All rights reserved.35
KVM – Converting, Compressing and Encrypting Images• qemu-img convert -O qcow2 \ /var/lib/kvm/images/sles11_raw_disk1.img \ /var/lib/kvm/images/sles11_qcow2_disk1.img
• qemu-img convert -c -O qcow2 \ /var/lib/kvm/images/sles11_qcow2_disk1.img \ /var/lib/kvm/images/sles11_qcow2_compr_disk1.img
• qemu-img convert -e -O qcow2 \ /var/lib/kvm/images/sles11_qcow2_compr_disk1.img \ /var/lib/kvm/images/sles11_qcow2_compr_encr_disk1.img
© Novell, Inc. All rights reserved.36
Virt-Manager – Hardware Configuration
© Novell, Inc. All rights reserved.37
KVM – Snapshots
• qemu-img snapshot -l image.img
• qemu-img snapshot -a snapshot image.img
• qemu-img snapshot -c snapshot image.img
• qemu-img snapshot -d snapshot image.img
© Novell, Inc. All rights reserved.38
KVM – Live Migration
• CLI:
qemu-kvm -incoming tcp:0:4444(qemu) migrate -d tcp:192.168.3.34:4444
• Via libvirt and virt-manger instances
Demo
Unpublished Work of Novell, Inc. All Rights Reserved.This work is an unpublished work and contains confidential, proprietary, and trade secret information of Novell, Inc. Access to this work is restricted to Novell employees who have a need to know to perform tasks within the scope of their assignments. No part of this work may be practiced, performed, copied, distributed, revised, modified, translated, abridged, condensed, expanded, collected, or adapted without the prior written consent of Novell, Inc. Any use or exploitation of this work without authorization could subject the perpetrator to criminal and civil liability.
General DisclaimerThis document is not to be construed as a promise by any participating company to develop, deliver, or market a product. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Novell, Inc. makes no representations or warranties with respect to the contents of this document, and specifically disclaims any express or implied warranties of merchantability or fitness for any particular purpose. The development, release, and timing of features or functionality described for Novell products remains at the sole discretion of Novell. Further, Novell, Inc. reserves the right to revise this document and to make changes to its content, at any time, without obligation to notify any person or entity of such revisions or changes. All Novell marks referenced in this presentation are trademarks or registered trademarks of Novell, Inc. in the United States and other countries. All third-party trademarks are the property of their respective owners.