Datacenter Virtualization: Global-scale software switches

Hakim WeatherspoonAssistant Professor, Dept of Computer Science

CS 5413: High Performance Systems and NetworkingMay 8, 2017

Slides from “Follow the Sun through the Clouds: Application Migration for Geographically Shifting Workloads “, Z. Shen, Q. Jia, E. Sela, B. Rainero, W. Song, R. van Renesse, H. Weatherspoon. In Proceedings of the ACM Symposium on Cloud Computing (SoCC), October 2016.

• Interested Topics:– SDN and programmable data planes– Disaggregated datacenters and rack-scale

computers– Alternative switch technologies– Datacenter topologies– Datacenter transports– Advanced topics

3

Where are we in the semester?

…Top of Rack Switch

Aggregate Switch

Core Switch

…

High Performance Systems and Networks

High Performance Systems and Networks

• How to optimize a global network of data centers?• And, across multiple heterogeneous clouds

Goals for Today

• Plug into the Supercloud– D. Williams, H. Jamjoom, and H. Weatherspoon. IEEE

Internet Computing, Vol. 17, No 2, March/April 2013, pages 28-34

• Follow the Sun through the Clouds: Application Migration for Geographically Shifting Workloads – Z. Shen, Q. Jia, E. Sela, B. Rainero, W. Song, R. van

Renesse, H. Weatherspoon. In the ACM Symposium on Cloud Computing (SoCC), October 2016.

Supercloud Demo

7

Nested VM

HTTP

HTTP VM

VM

Highlights• Automatic VM placement and migration• Migrated VMs are LIVE• IP addresses are not changed• TCP connections are not broken

8

Demo

9

(Full video available at http://supercloud.cs.cornell.edu)

Full Demo (http://supercloud.cs.cornell.edu)

10

Highlights• Automatic VM placement and migration• Migrated VMs are LIVE• IP addresses are not changed• TCP connections are not broken

• Appears as a unified private cloud that spans all clouds

• Controlled by the user!

11

Research Challenges• How to migrate across incompatible virtualization

platforms?

• How to keep IP addresses unchanged and TCP connections unbroken?

• How to decide when and where to migrate?

• How to make the system efficient?

12

Supercloud is the first system that supportsautomatic, efficient, and live VM migration

across heterogeneous cloud providerswithout changing IP addresses or breaking TCP connections.

13

Supercloud Architecture

14

• Computation– Nested hypervisor: Xen-Blanket– Support all major platforms

• Network– SDN overlay– Support migration with – public IP

• Storage:– Geo-replicated storage– Optimized for serving VM

images• Resource management

– OpenStack platform– Automatic scheduling framework

First Layer

KVM/virtio

Dom0

KVM/virtio

Dom0

Availability Zone 1 Availability Zone 2

Cloud Provider 1 Cloud Provider 2

Xen-Blanket Xen-Blanket Xen-Blanket

OpenStack OpenStack OpenStack

User VMsSecon

d Layer

User VMs

User VMs

User VMs

User VMs

User VMs

First-layer VM

Geo-replicated Image Storage

Second-layer VM

Second-layer hypervisor

Xen/PV-on-HVM

First-layer hypervisor

Software-Defined Network (SDN)

Supercloud Compute InstancesXen-Blanket

• Second Layer Hypervisor• Uniformity

Guest VM

Xen-Blanket

Dom0 DomU

Xen KVM Hyper-V

Second-layer provider

First-layer provider

• Move computation via Virtualization– Virtualize processor Instruction Set Architecture– Full Virtualization vs Paravirtualization (of hardware)– VMWare vs (Original) Xen

• Migrate computation among different machines

Supercloud Compute Instances

• Xen– DomU hosts guest operating system in virtual machine– Dom0 manages devices and guests– Control Transfer: Hypercalls and Events

(like syscalls and device interrupts)

How do we migrate to different virtualization system?

Dom 0 Dom U: VMBackend

FrontendPCI-Driver

Xen

Baremetal

Dom U: VM…

Supercloud Compute Instances

• Virtualization enables migration of computation across a single cloud VMM

Guest OS

App

VMM

VMM

VMM

VMM

Guest OS

App

Guest OS

App

Supercloud Compute Instances

• Need another layer of virtualization (indirection)

• Blanket drivers enable access to baremetal• Blanket drivers support memory address translation• Hypercalls provides privilege support for HVM

Xen

Dom U: VM

FrontendXen-Blanket

Dom 0 for XB

Dom 0

Backend

PCI-Driver

Baremetal

Dom U: VM

Hypercall

…DomU Dom

U…Frontend

Supercloud Compute Instances

VMM 4

VMM 3

VMM 2

VMM 1

VMM

Xen-Blanket

Guest OS

App

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

Guest OS

App

Guest OS

App

• Can create your own Cloud-within-a-Cloudaka a “Supercloud”

• Migrate computation among different cloud providers

Supercloud Compute Instances

VMM 4

VMM 3

VMM 2

VMM 1

VMM

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

Guest OS

App

• Migration of computation among different cloud providers

• Small overhead • Migration: 1 second downtime

• Is VM migration sufficient?

Supercloud Compute Instances

• VM migration is not enough for large systems– Single application includes an ecosystem– E.g. load balancer, app logic, database, and nw policies

• Network configuration is necessary– Policy based routing (e.g. firewall, load balancer, etc.)– Broadcast– Multicast

• Cloud providers’ support is limited

• We need user centric virtual wire; i.e. SDN!

Supercloud Networking

• How to build a “Supercloud” SDN?

• Blanket drivers enable access to baremetal• Blanket drivers support memory address translation• Hypercalls provides privilege support for HVM

Xen

Dom U: VM

Xen-Blanket

Dom 0 for XB

Dom 0

Backend

PCI-Driver

Baremetal

Dom U: VM

Hypercall

…DomU Dom

U…Frontend

Switch

Supercloud Networking

VMM 4

VMM 3

VMM 2

VMM 1

VMM

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

VM

VM VM

VM

Switch

Supercloud Networking

VMM 4

VMM 3

VMM 2

VMM 1

VMM

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

Xen-Blanket

VM

VM VM

VM

Switch

• Virtual Wires via SDN– Uses virtual switches– Installation of virtual connectors– Encapsulates and sends packets

• Migration of Network Configs– Migrates network topology– Obviates administrative configs

Supercloud Networking

Supercloud Networking• Challenges:

– Optimal routing without extra forwarding– Migration without changing IP addresses

• Solution:– VPN overlay with full-mesh tunnels– Frenetic SDN controller

27

vSwitch

vSwitch vSwitch

vSwitch

VM

vSwitch

VM

Cloud 1

Cloud 2

VM Migration with Public IP Address

28

VM

Pub IP front-end

54.172.26.213

54.172.26.213

Dynamic DNS?• Can be delayed due to cache• Can cause connection interrupts• Some applications might not work

VM Migration with Public IP Address

29

VM

Pub IP front-end

54.172.26.213 Pub IP front-end

52.69.94.195

Centralized VM Image Storage

30

VMImage

Long latency; Low throughput

Supercloud Geo-Replicated VM Image Storage

31

VM

Image Image

Challenges:• Strong consistency requirement• Long latency and low throughput in WAN

Supercloud VM image storage:• Decoupling consistency from data propagation.• Propagating data according to disk access patterns.

Supercloud Geo-Replicated VM Image Storage

Scheduling Framework• Application provides:

– A placement evaluation function 𝑓𝑓• 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 = 𝑓𝑓(𝑃𝑃𝑃𝑃𝑃𝑃𝑆𝑆𝑆𝑆𝑃𝑃𝑆𝑆𝑃𝑃𝑃𝑃, 𝑊𝑊𝑆𝑆𝑆𝑆𝑊𝑊𝑃𝑃𝑆𝑆𝑃𝑃𝑊𝑊, 𝐿𝐿𝑃𝑃𝑃𝑃𝑆𝑆𝑃𝑃𝑆𝑆𝐿𝐿)

– A threshold 𝑇𝑇• Minimal score change for triggering migration

• The scheduler searches for a placement that:– Maximizes the score– Outperforms the current placement by at least 𝑇𝑇

33

Benefits of the Supercloud• Case studies

– Follow the Sun– Smart Spot Market– Smart Dynamic Resource Scheduler (SDRS)

34

Benefits of the Supercloud• By leveraging cloud diversity, users can:

– Reduce cost– Reduce energy– Handle bursts– Improve availability– Reduce service response time– Improve performance– Improve security– And more…

35

Follow the Sun with Zookeeper

36

Master

Follow the Sun with Zookeeper

ZooKeeper Master in US ZooKeeper Master migrates

Benefits of Exploiting Diversity• By leveraging cloud diversity, users can:

– Reduce cost– Reduce energy– Handle bursts– Improve availability– Reduce service response time– Improve performance– Improve security– And more…

38

Spot Instance

• Amazon Spot Instance– Dramatic Price Change– Usually Cheap– Charges hourly– Ready to be terminated

Only stateless jobs!

Prices can jump up

Smart Spot Instance

• Supercloud on Spot Instances– Migrate instances before being terminated– Migrate to the cheapest location before

starting new billing hour

• Benefits– No termination– Reduced cost

Spot Instance Price History

Spot price goes very high sometimes

Regular instances have stable price

Spot price is usually low

Supercloud is migrated to the

cheapest place every hour

Migrate supercloudto regular instances

Accumulated Price

Smart spot instances save money!

Benefits of Exploiting Diversity• By leveraging cloud diversity, users can:

– Reduce cost– Reduce energy– Handle bursts– Improve availability– Reduce service response time– Improve performance– Improve security– And more…

43

• Pack VMs when loads becomes light

Resource Oversubscription

VM1

VM1VM2VM3

Migration

VM2 VM3

Cloud VM Cloud VM Cloud VM

Cloud VM

Resource Oversubscription

29% cost saving with 1.5% performance degredation.

In the Paper• Comparison with application-level migration• Placement policies for different types of

applications• Detail design of the image storage• Hierarchical network topology• Evaluations

46

Perspective

• Supercloud: application migration for geographically shifting workloads– Crossing heterogeneous cloud providers– Automatic placement and migration– Geo-replicated image storage– Wide-area SDN

• A unified private cloud that spans all clouds• Controlled by the user!

47

– Supercloud Multicloud Deployment: SoCC 2016– Supercloud Economics: CrossCloud 2016– Supercloud: Opportunities/Challenges in OSR-2015– Plug into the Supercloud in IEEE Internet Computing-2013– Supercloud/Xen-Blanket in EuroSys-2012– Supercloud in HotCloud-2011/2014– Overdriver in VEE-2011– RACS in SOCC-2010– See also, Storage: Gecko in FAST 2013 / HotStorage 2012,

Antiquity in EuroSys 2007 and SMFS in FAST 2009– Networking: SoNIC in SIGCOMM 2016, NSDI 2013/2014, Maelstrom NSDI 2008

Wireless DC in ANCS 2012, and NetSlice in ANCS 2012

• More at http://supercloud.cs.cornell.eduand also http://fireless.cs.cornell.edu

• Email: hweather@cs.cornell.edu

Paper Trail