View
405
Download
0
Category
Tags:
Preview:
Citation preview
Virtualization: Towards More Flexible and Efficient Grids
Kate Keaheykeahey@mcs.anl.gov
Argonne National Laboratory
13/09/05 Kate Keahey, PPAM 2005
The Grid Metaphor
How do we store energy?
How do we charge for energy?
How do we reliably deliver energy?
What happens if a power station fails?
How do we ensure quality of service?
What elements make for a safe and efficient power Grid?
How do we make sure that supply meets demand?
13/09/05 Kate Keahey, PPAM 2005
Computational Grids
How do we store computing?
How do we charge for computing?
How do we reliably deliver cycles?
What happens if a power station fails?
How do we ensure quality of service?
What elements make for a safe and efficient power Grid?
How do we make sure that supply meets demand?
What is the “unit” of resource usage?
How can we manage different computing environments?
How can we ensure that disk, CPUs, network are all available?
How can we negotiate for computation?
NCSA
ANLCaltech
SDSC
Tera Grid
Grid Middleware
We need a “computon” that will combine environment and enforcement aspects
as well as a way of managing the multi-dimensional nature of the Grid
13/09/05 Kate Keahey, PPAM 2005
Grids Today Grid Middleware Tools
Security, Data Management, Resource Management & Scheduling, Monitoring
Standards: GGF, OASIS Implementations: Globus Toolkit, Condor and others Many new services are being developed
Significant deployments and use of Grid infrastructure TeraGrid, Open Science Grid (OSG), Grid 3, many
European deployments Multiple projects making production use of Grid
infrastructure. Still issues: heterogeneity, lack of satisfactory control
and accounting, no on-demand computing
13/09/05 Kate Keahey, PPAM 2005
The Virtualization Layer
Virtual Grids: virtualize computers, networks, disks, memory Overlay networks, virtual storage…
Use middleware to map the virtualized constructs onto physical hardware Trust middleware to map and remap the
virtual environment as needed Trust market forces to ensure that physical
resources are plentiful when you need them
13/09/05 Kate Keahey, PPAM 2005
Virtual Workspace
For now: focus on “virtual workspace” Unit of enforcement, a “computon” for the Grid Representation of a desired environment
Later: put all elements of the system together into a virtual Grid
We need progress in the following areas: Protocols to dynamically negotiate and describe a
workspace Ongoing work at GGF: WS-Agreement, JSDL spec
A unit of enforcement A “critical mass” implementation Recent revival in virtual machine technologies provides
potential for such an implementation
13/09/05 Kate Keahey, PPAM 2005
Virtual Machine Basics
Hardware
Virtual Machine Monitor (VMM) / Hypervisor
Guest OS(Linux)
Guest OS(NetBSD)
Guest OS(Windows)
A VM can serialize all of its state (including RAM) A VM image is simply a collection of files
Disk partitions, RAM, configuration file Such image can be easily moved (migrated) between hypervisors
of the same type Such image can also be saved and used for rollbacks
VM VM VM
AppApp AppAppApp
13/09/05 Kate Keahey, PPAM 2005
Different Hypervisor Implementations Depending on the layer you virtualize you will
end up with a different VM API: language VMs (JVM) ISA: system VMs (VMware)
Different types of system virtual machines Full virtualization (VMware)
Run multiple unmodified guest OSs Para-virtualization (Xen, UML, Denali)
Run multiple guest OSs ported to a special architecture Single OS image (Vserver)
What is the cost of using VMs? Paper: “From Sandbox to Playground: Dynamic
Virtual Environments in the Grid”, Grid 2004
13/09/05 Kate Keahey, PPAM 2005
The Need for Speed
L X V U
SPEC INT2000 (score)
L X V U
Linux build time (s)
L X V U
OSDB-OLTP (tup/s)
L X V U
SPEC WEB99 (score)
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
Benchmark suite running on Linux (L), Xen (X), VMware Workstation (V), and UML (U)
Paper: “Xen and the Art of Virtualization”, SOSP 2003
13/09/05 Kate Keahey, PPAM 2005
Licensing and Distribution
License Open source (Xen, UML)
Visible effects of open source community at work Commercial (VMware)
Also, XenSource Distribution/Installation
Para-virtualization requires kernel modifications Yes, but … everything else stays the same Xen is (or soon to be) part of multiple distributions: Fedora Core
4, Debian, inofficial: Gentoo, Mandrake and SUSE distributions Work on making Xen part of the Linux kernel
Privilege Xen (root, patch kernel, domain 0 privileges setup) VMware Workstation (root, installation only) UML: user-level
13/09/05 Kate Keahey, PPAM 2005
What Makes VMs Great Summary of VM properties:
Good isolation properties Generally enhanced security, audit forensics
Excellent enforcement potential Details depend on implementation
Customizable software configuration Library signature, OS, maybe even 64/32-bit architectures
Serialization property VM images (include RAM), can be copied
The ability to pause and resume computations Allow migration
How do we make VMs available over the network and manage them so as to leverage this potential? Challenges: security, enforcement, protocols
13/09/05 Kate Keahey, PPAM 2005
Grid Services
Web Service Resource Framework An extension of Web Services Provides standard mechanisms for
Creation Lifetime Management State management, inspection (notification)
Globus Toolkit 4 Implementation of the WSRF framework
Available since April 2005 Provides secure authentication, authorization as well as
tools for fast transfer, replica management, monitoring, and others.
13/09/05 Kate Keahey, PPAM 2005
What are Virtual Workspaces?
Virtual Workspaces: environments that can be made available dynamically the Grid well-defined properties in terms of environment definition
and resource usage enforcement Examples:
A physical cluster booted to a desired configuration (e.g. Cluster on Demand)
A Grid3 node dynamically configured using Pacman A cluster partition configured with a hypervisor A VM representing an OSG configuration enforcing memory
and CPU usage Workspaces can be implemented using a variety of
technologies VMs are the most promising
13/09/05 Kate Keahey, PPAM 2005
Virtual Workspace Environment Aspect (workspace meta-data)
Information/state that outlives its deployment Generic information (name, time to live) Attested software partition information: OS, “OSG configuration”,
“application installation”, etc. Services: ssh, GRAM, pre-configured job
Resource allocation request (deployment time) Flexibly negotiated within desired constraints
See GGF WS-Agreement standard Memory, disk, networking, etc.
See GGF JSDL standard On deployment the actual resource allocation information
becomes available for inspection Atomic workspaces and virtual clusters
Clusters are simply aggregate workspaces
13/09/05 Kate Keahey, PPAM 2005
Deploying Workspaces in the Grid
Define workspace environment
Manage workspace
Negotiate workspace deployment characteristic
WorkspaceWizard
(VW Factory)
Workspace Management
Service(VW Repository)
Workspace Service
(VW Manager)
request a workspace
workspace meta-data
manage workspace environment
workspace metadata
Workspace
terminate workspace deployment
negotiate workspace deployment
manage/monitor/renegotiate workspace deployment
manage activities within the workspace
13/09/05 Kate Keahey, PPAM 2005
Current Implementation Current prototype using Globus Toolkit 4
Leveraging standard Grid Service features Workspace Wizard
Returns workspace meta-data Very rudimentary implementation
Workspace Service Create: takes workspace meta-data and a deployment
descriptor Manage:
renegotiate resource allocation Also traditional Grid Service management: TTL, etc.
Destroy Different options: pause, shutdown or destroy
First tech preview release expected later this month
13/09/05 Kate Keahey, PPAM 2005
How dynamic is the deployment?
Automatic Protocol-based Moving towards better articulation of migration Renegotiation of resource allocation
How fast is this deployment? Deployment of workspace for EMBOSS suite:
Manual: ~45 minutes Based on pre-configured Vmware VMs: ~6 minutes Based on pre-configured Xen VM: < 1 second
How much overhead does workspace deployment add over what we have today?
13/09/05 Kate Keahey, PPAM 2005
How much deployment overhead are we adding?
Using a paused VM allows us to “save” on initiation time
8
8
8
0.7
0.7 1.7
0.8
0.8
0 2 4 6 8 10 12
a)
b)
c)
job startup scenario
time (in seconds)
VM setup
VM boot
job setup
GRAM job
a) GRAM job executionb) GRAM job execution in a paused Xen VMc) job execution in a booted Xen VM (pre-configured job)
13/09/05 Kate Keahey, PPAM 2005
Workspace Service: Virtual Clusters
13/09/05 Kate Keahey, PPAM 2005
Workspace Deployment Across Technologies
Basic node configuration (+/-boot from image) Cluster on Demand, PXE, bcfg On the order of many minutes (~30 minutes)
Refining configuration, creating access Dynamic account with workspace service: < 1s
(mostly GT4 request processing time) Refining Installation: ~2 hours to configure an ATLAS
node using Pacman Virtual machines
Deploying images Xen: ~100 ms VMware Workstation: ~ several seconds
13/09/05 Kate Keahey, PPAM 2005
Nested Workspaces
Physical machineprocure hardware
program program program
…VM
Hypervisor/OSdeploy hypervisor/OS workspace
VM VMdeploy VM workspace (with hypervisor/OS)
It is easier to maintain a few hypervisor configurations than thousands of user configurations.
Those can be deployed in virtual machines.
13/09/05 Kate Keahey, PPAM 2005
Virtual Playgrounds
Application
Virtual Grid
13/09/05 Kate Keahey, PPAM 2005
Ongoing Work on Workspaces Dynamic resource management with VMs
Virtual clusters, fine-grained resource mangement, migration, moving towards economic management
X. Zhang, T. Freeman
IP overlay network for virtual machines Management infrastructure for VM IP addresses
T. Freeman & L. Chen
Secure management of VM images Image attestation and verification Handling image distribution Managing workspace identity
W. Lu, T. Freeman, F. Siebenlist
Deployment Edge Services for OSG: with F. Wuertherwein & A. Rana
13/09/05 Kate Keahey, PPAM 2005
Related Work In-Vigo
VM-based infrastructure for the Grids VM deployment, virtual storage, virtual networks Renato Figueiredo, Jose Fortes
Virtuoso VNET: virtualizing networks Peter Dinda & lab
VIOLIN Isolated, virtual networks for VMs Dongyan Xu & lab
Cluster on Demand Clusters of VMs on demand, also networking, resource
management Jeff Chase and lab
13/09/05 Kate Keahey, PPAM 2005
The Challenges that Lie Ahead Deployment
How do I prepare a cluster for VM execution? Reserve and publish
Site-specific versus Grid-specific What security trade-offs are acceptable? How will VM usage change site configuration? And many, many others
Environment configuration management How to configure and manage a VM?
GGF CDDLM working group Packaging infrastructures
Security Huge potential: how are we going to leverage it?
Economics, Grid markets, and many others
13/09/05 Kate Keahey, PPAM 2005
Conclusions Virtual is the new real!
Virtualization is emerging as an important abstraction layer in the Grids
Virtual workspaces are cornerstone of this new abstraction layer
Rapidly developing VM technology has the potential to implement a “computon” for the Grids Fast, accessible VMs “critical mass” implementation for virtual workspaces
Two sides to providing “computation on tap” Abstractions and enforcement mechanisms Protocols
There is much ongoing work in VMs but even more challenges still like ahead
If you like a challenge, give us a call
keahey@mcs.anl.gov
Recommended