Virtualization Technology Trends

Intel Corporation21 July 2008

Agenda• Virtualization Technology evolution• VMMs

– Hybrid virtualization– Open Virtualization Format Specification– Virtual Machine Interface

• Usages evolution

Intel® Virtualization Technology Evolution

Software-only VMMs Binary translation Paravirtualization Device emulations

Simpler and more secure VMM through use of hardware VT support

Better IO/CPU perf and functionality via hardware-mediated access to memory

•Assists for IO sharing:• PCI IOV compliant devs• VMDq: Multi-context IO• End-point DMA translation caching• IO virtualization assists

Richer IO-device functionality and IO resource sharing

Core support for IO robustness & performance via DMAremapping

Richer/faster: Intel VT FlexPriority, FlexMigrationEPT, VPID, ECRR, APIC-V

Close basic processor “virtualization holes” in Intel® 64 & Itanium CPUs

Perf improvements for interrupt intensive env, faster VM boot

Interrupt filtering & remapping VT-d extensions to track PCI-SIG IOV

VT-x/i VT-x2/i2

VT-d

VT-x3/i3

VT-d2

VT-c

VMM software evolution over time with hardware supportVMM software evolution over time with hardware support

VMMSoftwareEvolution

Vector 3:IO Device Focus

Vector 1:Vector 1:Processor Focus

Vector 2:Chipset Focus

Past 2005 2010

All timeframes, dates, and products are subject to change without further notification

Intel’s Next Ecosystem of Virtualization Innovation

*Other names and brands may be claimed as the property of others.

•Designed for Multi-Core Processors

Solution

•Deliver scalable performance for Intel multi-core servers

Challenge Industry Efforts

•Flexibility and Dynamic Load Balancing for Virtualization

•Optimize I/O bottlenecks

•Unified Networking for Business Continuity and Disaster Recovery in Virtual Infrastructure

•Simplify network connectivity to the SAN

What’s Coming

Gigabit Virtualization Evolution:Next Generation Usage Models Through Virtualization Innovation

Th

rou

gh

pu

t (R

x)

0.0

2.0

4.0

6.0

10.0

w/o VMDq w/ MNDq w/ VMDq JF*

8.0

9.59.2

4.0

Source: Intel

Today’s Networking

NIC

VMMLayer 2 Software Switch

MAC/PHY

LAN

Layer 2 Sorter

…

VM1

vNIC

VM2

vNIC

VMn

vNIC…

w/ VMDq

Wire Speed Rx Side Performance With VMDq on

Intel® 82598 10 Gigabit Ethernet Controller

Virtualization PerformanceRobust Tick Tock Roadmap

55%55%

70%70%

100%100%

85%85%

45nm 45nm

Intel® Core™ Intel® Core™ uArchitectureuArchitecture

(Penryn)(Penryn)

Virtualization SW Overhead

2007 / 2008

VMDqIntel® VT-d

2008 / 2009

55%55%

70%70%

100%100%

85%85%

45nm 45nm

Next generation Next generation Intel Intel

uArchitectureuArchitecture

(Nehalem)(Nehalem)

Virtualization SW Overhead

EPT, VPID,

VMDq2

55%55%

70%70%

100%100%

85%85%

Quad-Core Intel® Quad-Core Intel® Xeon® Xeon®

ProcessorsProcessors

Intel Xeon 5100, Intel Xeon 5100, 5300, 73005300, 7300

Virtualization SW Overhead

2006 / 2007

Intel® VT-x,

FlexPriority

Roadmap will continue to deliver higher raw performance (Moore’s Law), andRoadmap will continue to deliver higher raw performance (Moore’s Law), andarchitectural enhancements to improve efficiency in virtualized environmentsarchitectural enhancements to improve efficiency in virtualized environments

All timeframes, dates, and products are subject to change without further notification

All timeframes, dates, and products are subject to change without further notification

Intel virtualization in embedded devices

All timeframes, dates, and products are subject to change without further notification

PC vs. Devices

• VM in KVM (along with Qemu) means “PC”– Legacy devices, interrupt controllers, timers, ACPI/BIOS, PCI devices,

monitor, keyboard, mouse, etc.

• There are various devices or computers that are not compatible with PC– Network routers, …, robots, …, toasters, …, PDAs/MIDs, …– Some can afford very small amount of memory (e.g. 128MB)

• And various operating systems and apps have been developed for those

• Porting such (legacy) OS, drivers, and apps to “PC” is not straightforward

All timeframes, dates, and products are subject to change without further notification

Benefits of Using Virtualization for Embedded Systems

• Portability & Maintainability– Provides simplified and uniformed VM to minimizing porting

and maintenance efforts– Once virtualized, it’s independent of H/W

• Scalability & Consolidation– Legacy operating systems often support UP only– Multiple instances of VMs

• Reliability & Protection– Tolerate and isolate fatal errors in legacy OS guest and

software to avoid system crash– Sandboxing

All timeframes, dates, and products are subject to change without further notification

What’s Mini-VM and Why?

• Bare minimum and simple VM– CPU(s), memory, abstracted (PV) devices

• Timer, front-end devices (or virtio)– Start from protected (or 64-bit) mode with paging enabled;

no real mode; No BIOS

• Protected execution environment by H/W– Run under H/W-assisted virtualization– Allow Ring-0 operations, eliminating burden of para-

virtualizingCPU

• Low virtualization overheads– Use hybrid virtualization (PV + H/W-assisted virtualization)– Real-time (e.g. direct paging mode)

Check at the Mini-VM project…

Hybrid virtualization

• Use hardware-assisted virtualization– The cost of VM exit/entry will be even lower in the future– Cost of VMCALL is lower than other VM exits

• Use para-virtualization on focused areas– Starting from hardware-assisted full-virtualization

• Easier to share the kernel binary with the native– Reduce paravirtual operations significantly

• The kernel regains the native CPU features lost in software only para-virtualization– Fast system calls– Global pages– Paging-based protection (U/S), etc.– Privileged instructions– GDT, IDT, LDT, TSS, cli/sti, etc.

• Standard exceptions/interrupts

Focus areas for hybrid virtualization

• Timer• Scheduling

– Idle handling

• Interrupt controllers• MMU

– Memory overcommit– Or hardware-assisted (i.e. EPT or NPT)

• Inter VMs communication

Usage trends• Business continuity

– High availability support through the synchronization of VMs– Reduction of unplanned downtime

• Seamless management of resources– Livemigration– Service Oriented Architectures leveraged by Virtualization

• Beginning of growth curve-expansion for desktop and application virtualization– Increased focus on security– Licensing issues/changing

• Virtual Machines mobility– Open Virtualization Format Specification– Virtual Machines Interface– Live migration

• Graphics virtualization– From a paravirtualization to a direct access approach

The future of Virtualization

Cloud computing• Cloud computing relates to the underlying architecture in

which the services are designed• Applications run somewhere on the “cloud” we don’t care

where• Big news is for application developers and IT operations.

– develop, deploy and run applications that can easily grow capacity (scalability), work fast (performance), and never — or at least rarely — fail (reliability)

• Infrastructures should have these characteristics:– Self-healing: hot backup application– SLA-driven– Multi-tenancy: built in a way that allows shared infrastructure– Service-oriented– Virtualized– Linearly Scalable: The system shall be predictable and

efficient in growing the application– Data management

Desktop/App virtualization Market expansionAccording to a recent tracker study on the Asia/Pacific excluding

Japan (APEJ) thin client market, total sales of thin clients in 1H 2007 reached 282,667 units, representing an increase of 37.3% over the previous year. Revenue likewise increased 29.2% over the same period

Across the various verticals, the predominant role of thin clients across the region has shifted away from government/education segment to financial services as the leading vertical of thin client adoption from 1H 2006 onwards

Open Virtualization Format SpecificationOVF Specification from DMTF describes an open, secure,

portable, efficient and extensible format for the packaging and distribution of software in (collections of) virtual machines

• Optimized for distribution• Optimized for a simple, automated user experience• Supports both single VM and multiple-VM configurations• Portable VM packaging• Vendor and platform independent• Extensible• Localizable• Open standard

OVF Packages

• package.ovf -> metadata descriptor (required)• package.mf -> manifest (optional) containing the SHA-1 digests

of individual files in the package• package.cert -> signature of the digest (optional) along with

the base64-encoded X.509 certificate• de-DE-resources.xml -> OVF Envelope describes VMS metadata• vmdisk1.vmdk -> no specific disk format to be used is required• vmdisk2.vmdk• resource.iso -> (optional)

• Distribution– OVF package can be made available as a set of files– OVF package can be stored as a single file using the TAR format.

The extension should be .ova (open virtual appliance or application)

Virtual Machine Interface

In 2005, VMware proposed a paravirtualization interface, the Virtual Machine Interface (VMI), as a communication mechanism between the guest operating system and the hypervisor

An implementation of this standard was merged in the main Linux kernel version 2.6.21

Motivations• Portability: it should be easy to port a guest OS to use the API• High performance: the API must not obstruct a high performance

hypervisor implementation• Maintainability: it should be easy to maintain and upgrade the guest

OS• Extensibility: it should be possible for future expansion of the API

No considerable adoption of the proposed VMI architecture, although a pending need to be satisfied (considering HVM additions as new x86 support)

What about proposing an interface from the platform?

Wrap up

• Virtualization Technology Introduction– Beginnings of Virtualization Technology in x86– Approaches to server and client virtualization

• Virtualization Usages in Servers– IT business needs behind Virtualization Technology

• VMMs / Hypervisors– Deep dive in the open source Xen hypervisor– Overview of KVM, VMware, OpenVZ

• Hardware assisted Virtualization– Software solution for x86 virtualization– Enhancements through hardware assisted virtualization– Intel VT features

• Virtualization Technology trends– VMMs standardization and VMs mobility– Virtualization in every platform

How to start…

• Do I need virtualization?– Consider the server side

• Lab in the University• Data Center consolidation• Environments for server applications development

– Consider the clients side• VMs for each student• VMs for users with low processing power needs

• Which do I choose?– Open source for teaching/learning purposes

• Choose the right one for your needs– Commercial solutions for highly dynamic data centers

• Can I do some research in the area?– Several open source projects in the software side with pending

topics– Standards to integrate virtualization through manageability

Gracias!Thank You!

Backup

Additional information sources:

• For specifications and to learn more– Intel® VT Web Site:– http://www.intel.com/technology/platform-technology/virtualization/

– Intel Virtualization Software Community:– http://www.intel.com/software/virtualization

• Online collateral on Intel® VT-x/VT-i http://www.intel.com/products/processor/manuals/index.htm?

iid=technology_virtualizationengage+body_intel64manual

References• http://www.dmtf.org/standards/published_documents/DSP0243_1.0.0

.pdf• http://www.vmware.com/pdf/vmi_specs.pdf• http://www.vmware.com/interfaces/• http://lists.xensource.com/archives/html/xen-merge/2005-08/msg000

76.html• http://download.microsoft.com/download/a/f/d/afdfd50d-6eb9-425e-8

4e1-b4085a80e34e/SYS-T312_WH07.pptx• http://wiki.xensource.com/xenwiki/Open_Topics_For_Discussion?

action=AttachFile&do=get&target=XenLoop_+A+Transparent+High+Performance+Inter-VM+pdf

• http://wiki.xensource.com/xenwiki/Open_Topics_For_Discussion?action=AttachFile&do=get&target=Memory+Overcommit.pdf

41% of new server x86 purchased in 2007 will be virtualized- IDC End User Study; Jun-06

Server Virtualization is now considered a mainstream technology among IT buyers.IT professional are bullish in future use: driving 45% server use in 12 months-IDC Directions 2007 Feb-07

>81% of business are using >81% of business are using virtualization virtualization in production environmentsin production environments- 451 Group Special Report – Dec-06- 451 Group Special Report – Dec-06

• Traditional benchmarking covers Performance, Power, ScalabilityTraditional benchmarking covers Performance, Power, Scalability– Metrics: Throughput (MB/s), Response time, #users, etcMetrics: Throughput (MB/s), Response time, #users, etc– Micro-architecture focus: cache sizing, frequency, bandwidth, etc. Micro-architecture focus: cache sizing, frequency, bandwidth, etc.

• New technology requires new areas of analysis and metricsNew technology requires new areas of analysis and metrics– Areas of focus driven by use models.Areas of focus driven by use models.

• E.g., VM migration time, VM utilizationE.g., VM migration time, VM utilization

– Need to measure how IntelNeed to measure how Intel®® Virtualization technology benefits end-users and ISVs Virtualization technology benefits end-users and ISVs

• Virtualization presents unique challengesVirtualization presents unique challenges– Which configurations to focus onWhich configurations to focus on

• Homogeneous or heterogeneous OSHomogeneous or heterogeneous OS

• Number Virtual MachinesNumber Virtual Machines

• Configuration of individual VMs (CPU, Memory, NIC, HBA, HDD)Configuration of individual VMs (CPU, Memory, NIC, HBA, HDD)

– Measuring performance Measuring performance

• Virtual clock accuracy induces platform dependent errorVirtual clock accuracy induces platform dependent error

• Availability of performance monitoring capabilitiesAvailability of performance monitoring capabilities

• Consolidation use case adds additional testing challengesConsolidation use case adds additional testing challenges– Synchronicity: Use automation scriptsSynchronicity: Use automation scripts

– Utilization: Avoid harmonic bottlenecksUtilization: Avoid harmonic bottlenecks

– Steady State: Easy, repeatable measurementsSteady State: Easy, repeatable measurements

• Only way to overcome the challenges is to develop the benchmarksOnly way to overcome the challenges is to develop the benchmarks– Tier consolidation using SAP SDTier consolidation using SAP SD

– vConsolidate: a server application consolidation benchmarkvConsolidate: a server application consolidation benchmark

• DescriptionDescription– Benchmark that represents predominant use case -> server Benchmark that represents predominant use case -> server

application consolidationapplication consolidation– Application types selected for consolidation guided by market Application types selected for consolidation guided by market

datadata

• vConsolidate providesvConsolidate provides– A methodology for measuring performance in a consolidated A methodology for measuring performance in a consolidated

environmentenvironment– A means for fellow travelers to publish virtualization performance A means for fellow travelers to publish virtualization performance

proof pointsproof points– The ability to analyze performance across VMMs and hardware The ability to analyze performance across VMMs and hardware

platformsplatforms

• Knowledge obtained Knowledge obtained SPEC virtualization workload SPEC virtualization workload

• 5 Virtual Machines5 Virtual Machines• 3 Clients: Controller, Mail, and Web3 Clients: Controller, Mail, and Web

*Other names and brands may be claimed as the property of others

• Consolidation Stack Unit – (CSU) Consolidation Stack Unit – (CSU) • Smallest granule in vConSmallest granule in vCon• Consist of 5 Virtual Machines Consist of 5 Virtual Machines

– DatabaseDatabase– Commercial MailCommercial Mail– Web ServerWeb Server– Java Application ServerJava Application Server– IdleIdle

• Each CSU represents single scoreEach CSU represents single score• Final score is aggregate of the individual CSU scoresFinal score is aggregate of the individual CSU scores

• Running vConsolidateRunning vConsolidate

– Controller applicationController application

• Starts the tests via helper scripts; Runs for 30 minutesStarts the tests via helper scripts; Runs for 30 minutes

• Stops the test and reports scoreStops the test and reports score

– Time measured in “Controller Client” Time measured in “Controller Client” external timer external timer

• ScoringScoring

– The “Controller” application The “Controller” application calculates final scorecalculates final score

– SpecJBB, Sysbench and SpecJBB, Sysbench and Loadsim - transactions/Loadsim - transactions/second second

– WebBench – throughputWebBench – throughput

• CSU Final Score = GEOMEANCSU Final Score = GEOMEAN (VM Relative Perf[i]) (VM Relative Perf[i])

• Seeding Industry with Benchmark WorkloadsSeeding Industry with Benchmark Workloads– vConsolidate– Consolidated stack of business workloads consisting of Server Side Java, Commercial vConsolidate– Consolidated stack of business workloads consisting of Server Side Java, Commercial

Database, Commercial Mail, Commercial Web Server on 4 VMsDatabase, Commercial Mail, Commercial Web Server on 4 VMs

• Collaborating with Virtualization leaders Collaborating with Virtualization leaders – Microsoft and OEMs - consolidation workloads, methodology & metricsMicrosoft and OEMs - consolidation workloads, methodology & metrics

– VMware – VMmark* consolidation stackVMware – VMmark* consolidation stack

• Establishing benchmarks with ISV/OSVs Establishing benchmarks with ISV/OSVs

• Contributing to standard benchmarks through SPEC (long term)Contributing to standard benchmarks through SPEC (long term)

*Other names and brands may be claimed as the property of others.