Embed Size (px)
Citation preview
The Real World of Virtual Datacenters:
The enabling technology for Cloud Computing
X. Breogan Costa
TOC
● Motivation● Introduction to virtualization and Cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options● Supporting material (after the slide 60, for free!)
3/60
Use-case I (quite trivial): old game
● You want to run an old software, let's say you absolutely love an old game made for
ZX Spectrum CPU:
Z80 8-bit
HD64180/Z180 architecture
5/60
●
But you cannot just buy a ZX Spectrum today_
Use-case I (quite trivial): old game
6/60
Use-case II: you have old servers
● 2003 Sun Fire (4800/4810)
● CPU(s): UltraSPARC III...– Architecture: SPARC V9
7/60
Use-case II: you have old servers
● (1998) Compaq ProLiant (1600r)
● CPU: Pentium II Xeon Drake (1998)– Architecture: x86
8/60
Use-case II: old software running
● And your organization depends on old software made for those architectures
● Sometimes old software not portable (proprietary or no resources to do that)
● For example...
(See Use-Case I)9/60
Problems?
● 2014 HP ProLiant (DL380 G8)
● CPU: 2013 Intel Xeon (E5-2600 v2), – Architecture: EMT64 (x86-64)
– Unsupported by old OSs
● http://www8.hp.com/us/en/products/proliant-servers/product-detail.html?oid=5177953
● http://ark.intel.com/products/series/75291/Intel-Xeon-Processor-E5-2600-v2-Product-Family#@All
Solution: a new server!
11/60
Problems?
● Installation time?
One Possible Solution:
● Fast deployment● Move (even running) VMs to new servers, no downtime● You should be able to emulate previous architectures (if they
are implemented)
13/60 Let's do it!
Intro
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options
But we need to know more
Is this new?● First implementation: 1960's, at IBM Cambridge Scientific
Center:
– Virtualization development → starts with CP-40
Is this a mature technology?
How this continued?
● IBM worked almost alone until the 1980's– VM technology in 360, 370 and 390 series
● 1980's: workstation vendors get interested in virtualization
● 1985: – V86-mode (8086)
16/60
[Wikipedia]
1998: release of the first true virtualization of the full Intel processor architecture
What we can use today?
New (big) players in the game (2000-2013)
17/60
Hypervisors tech: elements
● Hypervisor (= Virtual Machine Monitor -VMM)
● Host Machine
● Virtual Machines
What's
inside?
Hypervisors tech: elements
● Management consoleinterface
HW Emulation
– Memory address translation
– Byte ordering: little endian (Intel) vs. big endian (PowerPC, Sun, Internet)
– Totally different architecture
↓Instruction emulation
↓ Instruction set
translation
Hardware emulation
● Host-system interface– VM running in hosted mode → certain host
resources are exposed to the VM (FS's, printers, clipboard, etc)
● Virtual device subsystem– Virtual devices to real host devices mapping
21/60
Summarizing: Why Virtualization?HW independence● Generic HW architecture● + OS compatibility● Generic drivers for most OS's
Summarizing: Why Virtualization?Scalability
PerformanceImproved bymodern HW
Ecologicalbenefits
23/60
Availability
Por
tabi
lity
Server sprawl
Centralized m
anagement
Why Virtualization? Example
● The Dynamic Datacenter (according to Microsoft)
1) Physical Layer● Bare-metal HW and base SW
2) Virtual Layer● Hypervisor and VMs
3) Application Layer● Virtual servers, server consolidation
4) Model Layer● Service/application components running in more than one server● App/s requirements → App/s architecture → Deployment model
5) Management● Datacenter management, VMs management
24/60
Why Virtualization? Extra benefits
● Hardware-assisted virtualization:– CPU
● privileged instructions (generation 1 in x86): Intel VT-x, AMD-V● Memory Management Unit (generation 2 in x86): Intel EPT, AMD RVI (RVI →
+42% performance according a VMware research paper)
– Chipset: I/O (AMD-Vi and VT-d), Networking (VT-c), PCI-E (IOV), ...
● Previous States restoration– Snapshots: just for sort term: they must not be used as backups
● ...
¬¬!
25/60
Extra: Why Virtualization?
Cloud Computing!26/60
Cloud Computing Main Service Definitions
● IaaS
– Infrastructure as a Service
● PaaS
– Platform as a Service
● SaaS
– Software as a Service
● NaaS
– Network as a Service
● XaaS
– Everything as a Service
28/60
● HET (no)
Virtualization, pre-requisite?
Image by
But not all is good
● Security– Cracker gain access to:
● Management tools● Host management
– Virtual Networking
Virtualizing the
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options (Access and Safety System)
We did...
● Planification of what and how to virtualize servers in the access and safety datacenters– Nothing to do with the (great) CERN general virtual
platform
● Prototypes in testing facilities– LHC0
– PS0
● Production environments ...You can read our Paper for ICALEPCS 2013 Conference
33/60
In 2013
What our vClusters run...
● SCADA Systems– Siemens WinCC, ARC PcVue
● Access Software: Gegelec Evolynx● Video Servers● Biometric servers: LG IRIS● Distributed monitoring servers:
– Zabbix servers, Zabbix agents and Zabbix proxies
● Security auditing tools
35/60
What our vClusters run...
● Servers OS's:– SLC (Scientific CERN Linux)
● CERN + Fermilab, based on RedHat Linux.
– SuSE Linux● mainly as virtual appliances giving some service to the
virtual cluster management, as backups system
– Debian GNU/Linux: for security auditing tools
– Windows Servers (several versions)
– (sometimes) Vyatta OS (a GNU/Linux implementing a virtual router)
36/60
Requirements & classifications
37/60
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options
Requirements
● Virtual CPU architecture– At least, Intel VT-x, AMD-V
– vmx or svm in /proc/cpuinfo (egrep '(vmx|svm)' --color=always /proc/cpuinfo)
– CPU-Z in Windows
– Enabled on BIOS
● + generic/compatible hardware* (servers use to be)
38/60
Yes, you can do it at home!
(at your own ris
k ;)
Classification: Virtualization
● Partial– some but not the entire target environment is
simulated. Historical milestone● Examples: first-generation time-sharing system CTSS
(IBM M44/44X experimental paging system, 1960's)
● Full:– complete HS (HW System) emulation
● Examples: VMware ESXi/Workstation/Player, Virtualbox, Parallels Desktop
39/60
Classification: Virtualization
● Paravirtualization– Not necessarily simulate hardware,
– offers a special API that can only be used by modifying the "guest" OS.
● Examples: Win4Lin 9x, Sun's Logical Domains...
● Operating System-level virtualization– OS's Kernel allows multiple isolated user-space
instances● Examples: Parallels Virtuozzo Containers, openVZ...
40/60
Classification: Hypervisors
● Bare metal (“native” or “Type 1”)– VMware ESX/ESXi, KVM, Xen, Microsoft Hyper-V
Server (Windows Server 2012 +)
● Hosted (“Type 2”)– VMware Workstation/Player, VirtualBox, Microsoft
Windows Server Hyper-V Service (Windows Server 2008 R2 +)
41/60
What we should put in our virtual Datacenter?
42/60
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure for virtualized datacenterse● Common features, considerations● Some advanced options
Virtual Infrastructure of a virtualized datacenter
● Hosts & Hypervisors *● Storage● Virtual Network● Virtual Machines● Management platform
– Management Server
– Database
– Client platform
43/60
Important: Virtual Networking
● Defined at Datacenter level
44/60
● Defined at Datacenter level– Every VM → different virtual MAC
[Cisco Web]
Common features, considerations
45/60
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options
High Availability & redundancy
● Downtime reduction– NAS / Backups (/ Snapshots -not recommended for Backup)
– Restoration in different host
● Optional no-downtime using redundancy– Execution in parallel
● Master VM● Slave VM
46/60
Integrity
● Internal RAID disks● NAS systems
– In vSphere they must be added as datastore
● Backup complete systems● NAS servers support
– For backups
– For OS installation
47/60
Disaster recovery
● There are several backup tools to prevent this situation
● Usage of NAS servers● Programmed backups
– Commonly used snapshots as a base
● Backup keeping policy● Image sharing
48/60
Basic Security
● General risks (according Gartner researches)– Information security isn't initially involved in the
virtualization projects (40% in 2009)
– Compromise of Virtual Layer (VMM) → could compromise of all hosted workloads (VMs)...
– … adequate controls on administrative access to the Hypervisor/VMM layer and to administrative tools are lacking
49/60
Basic Security
● Recommendations:– Be careful with host system interface (shared
resources)
– VM isolation
– Don't use generic and shared administration accounts (for traceability), even delete generic admin accounts
– Restrict root access at Hypevisor level
– Use the right permissions in user roles definition
– Be careful with roles' permissions hierarchy **
50/60
Migrations & conversions
● Tools:– “P2V” tools
– “V2V” tools
● Also:– VM cloning (excepting MAC address)
– Importing:● OVF / other virtualization provider formats● Cloned images (Acronis, Norton Ghost, etc)
– Exporting:● OVF format, etc
51/60
Some advanced options
Table of Contents● Motivation● Introduction to virtualization and cloud● My experience with virtualization at CERN● Requirements & classifications● Infrastructure● Common features, considerations● Some advanced options
Advanced options
● Hardware pass-through– USB
● USB port assignation
– Real pass-through (PCI-*, etc) →● VMware VMDirectPath I/O● KVM● Xen● NOT implemented in Hyper-V
(at this moment)
53/60
If we have special
requirements...
Siemens CP1613(Industrial Ethernet)
Advanced configurations
● Embedded architectures– KVM in system-on-chip architectures:
● ARM Virtual Express (Cortex-A15 + Expansions FPGA)
● Virtualization on mobile devices– Single-core/Multi-core devices
● Cortex-A15 was the first
– Android
– Devices● Cellphones / smartphones● Tablets● Netbooks● M2M devices
54/60
Main virtualization platforms
55/60
Which virtualization
provider select?
● VMware vSphere Infrastructure– ESXi hypervisor [free*] + vCenter [proprietary + license]
● KVM hypervisor [GPL/LGPL packagesor RedHat RHEV complete suite** + license]
KVM or Xen + Management tools(RHEV and XenServer include management tools)
● Xen hypervisor [GPL packages or Citrix XenServer ** + license]
● Microsoft Hyper-V Service or Hyper-V Server [proprietary + license]
Xen and KVM are Linux kernel
customizations
Hyper-V Service runs over Windows and Hyper-V server uses a Windows based kernel
ESXi uses a VMware microkernel and depends on a Linux kernel
Takeaway
● With virtualization you can emulate different architectures
● With virtualization you can run different OSs in the same server, even made for different platforms
● Virtualization increases availability● Virtualization increases scalability
57/60
Takeaway
● Virtualization reduces power consumption: good for environment and to save many money
● Virtualization enables IaaS (Infrastructure as a Servicere), part of Cloud Computing stack
● There are several alternatives and they offer different possibilites
● NEVER, absolutely never forget about security
58/60
59/60
60/60
Questions?
The Real World of Virtual Datacenters:
The enabling technology for Cloud Computing
X. Breogán Costa
Yesss, you can do it
at home!
(at your own ris
k ;)
TOC
● An extra of Why virtualization (Microsoft things)● An extra of disaster recovery
– Just an advice: try to prevent it ;)
● An extra of basic security● An extra of virtualization platforms● An extra of... (well, we haven't spoke about this, just
introduce it) Let's speak about cloud platforms
2/28
Extra: Why Virtualization?
The Dynamic Datacenter (according to Microsoft)
Can your computer be a host machine?
● Hardware virtualization– Virtual CPU architecture
● At least, Intel VT-x, AMD-V● vmx or svm in /proc/cpuinfo (egrep '(vmx|svm)' --color=always /proc/cpuinfo)
● CPU-Z in Windows● Enabled on BIOS
– + generic/compatible hardware* (servers use to be)
4/28
Disaster recovery
● There are several backup tools to prevent this situation
● Usage of NAS servers● Programmed backups
– Commonly used snapshots as a base
● Backup keeping policy● Image sharing
5/28
Basic Security
● General risks (according Gartner researches)– Information security isn't initially involved in the
virtualization projects (40% in 2009)
– Compromise of Virtual Layer (VMM) → could compromise of all hosted workloads (VMs)...
– … adequate controls on administrative access to the Hypervisor/VMM layer and to administrative tools are lacking
6/28
Basic Security
● General risks (according Gartner researches)– Workloads of different trust levels are consolidated onto
a single physical server without sufficient separation
– vNetworks/vSwitchs: lack of visibility and controls on internal virtual networks created for VM-to-VM communications blinds existing security policy enforcement mechanisms...
– … there is a potential loss of separation of duties for network and security controls
Source article: http://bit.ly/aHzzRB
7/28
Basic Security
● Recommendations:– Be careful with host system interface (shared
resources)
– VM isolation
– Don't use generic and shared administration accounts (for traceability), even delete generic admin accounts
– Restrict root access at Hypevisor level
– Use the right permissions in user roles definition
– Be careful with roles' permissions hierarchy **
8/28
Basic Security
** About user roles– Roles → templates
– Role permissions have sense at a certain level
– An user have different views depending on his roles
– One user could have different roles at different datacenter levels
● Combine roles is normal and a good praxis● Roles combination avoid problems with permissions
hierarchy
9/28
10/28
Sec
urity
: vS
pher
e ex
ampl
e
11/28
Sec
urity
: vS
pher
e ex
ampl
e
Virtualization platforms
12/28
Datacenter Virtualization market in 2012
Note that thanks to RHEV (KVM based) expansion with Cloud
Computing platforms (i.e: OpenStack) integration and support, the market
could be different today
13/28
VMware vSphere Infrastructure
● Bare-metal hypervisor– VMware ESXi (before v. 4.0: “ESX”)
– Own microkernel: VMware vmkernel
– It uses (and depends on) a Linux kernel (service console, the 1st vm)
● Management server: – VMware vCenter Server
– Database (SQL Server / Oracle)
● Management Client– VMware vCenter Client app
● Extra Tools (HA, DRS, Operations Management, ...)– Some available in vSphere Server by default
14/28
VMware vSphere Infrastructure
● Bare-metal hypervisor– VMware ESXi (before v. 4.0: “ESX”)
– Own microkernel: VMware vmkernel,
– It uses (and depends on) a Linux kernel (service console, the 1st vm)
● Management server: – VMware vCenter Server
– Database (SQL Server / Oracle)
● Management Client– VMware vCenter Client app
● Extra Tools (HA, DRS, Operations Management, ...)– Some available in vSphere Server by default
15/28
16/28
Vmware vSphere Infrastructure
VMware ESXi hypervisor
17/28
VMware ESXi hypervisor
18/28
KVM hypervisor (GPL/LGPL)
19/28
Xen hypervisor (GPL)
20/28
Xen hypervisor (GPL)
21/28
Xen hypervisor (GPL)
● Runs in a more privileged CPU state than any other SW on the machine
● Memory management and CPU scheduling of all “domains” (VMs)
● Uses dom0 (the only VM which by default has DA to the HW.
● From Dom0 the Hypervisor can be managed and domU's could be launched.
22/28
Xen hypervisor (GPL)
● Dom0 is typically a modified version of Linux, NetBSD or Solaris
● Proprietary version of Citrix and also Citrix management tools for Citrix XenServer
23/28
KVM/Xen datacenter/virtual cluster management tools
● RHEV (Red Hat Enterprise Virtualization)● oVirt [Red Hat Inc.]
– RHEV is based in oVirt + another tools
● ConVirt [Convirture]● OpenQRM (IaaS Cloud)● ...
24/28
Microsoft Hyper-V Service & Server
● Hyper-V Windows Server Service– Released as a Windows Server 2008 R2 service
● Hyper-V Server– Released as an independent bare-metal server
based on Windows Server 2012 kernel
● Several features not supported as real pass-through
25/28
26/28
Related cloud computing platforms
Related Cloud Computing Platforms
IaaS Project started by Citrix & Cloud.comNow Apache SW Foundation
Works with KVM, Xen and vSphere
Supports AWS API
Works with KVM, Xen but also with VMware vSphere, Hyper-V
Supports AWS API
Project started by Rackspace Hosting and NASA
Works with KVM, Xen and vSphere
Open source (Eucalyptus Systems Inc) SW to build AWS
Works with vSphere
It seems vCloud Director is not as successful as vSphere
27/28
