Memory Buddies: Exploiting Page Sharing for Smart Colocation in

Virtualized Data Centers

Timothy Wood, Gabriel Tarasuk-Levin, Prashant Shenoy, Peter Desnoyers*,

Emmanuel Cecchet, and Mark D. Corner

University of Massachusetts, Amherst

*Northeastern University

Server Placement in Data Centers

• Virtualization improves resource utilization by consolidating servers

• But how to determine which servers to place together?

• Must consider many resource constraints:– CPU, Disk, Network, Memory

Why Memory?• CPU scheduling is fine grain

– Easily share among many users– Work conserving, so no waste

• Memory is much less flexible– Allocated on a large time scales– Being wrong (paging) is disastrous

• Memory is an expensive resource

• Memory capacity is increasing slower than CPU power

HypervisorPhysicalRAM

FREE

D

FREE

Content Based Page Sharing

VM 1Page Table

ABCB

C

A

A

D

B

VM 2Page Table

ADB

• If two VMs have an identical pages in memory, just keep one copy

• Supported by VMware ESX platform– Experimental tests in Xen,

further support planned

• Potential benefits– 33% in VMware ESX paper– 65% with subpage sharing

(Difference Engine) 1) Hypervisor detects identical pages

2) Copy-on-Write references created for shared pages

But what if….

• Pages change over time, breaking sharing

• If memory is being overcommitted, this can lead to hotspots

PhysicalRAM

FREE

D

FREE

B

C

A

VM 1Page Table

ABC

VM 2Page Table

ADB

A*

A*

PhysicalRAM

What’s the problem?

VM 4Page Table

ABCB

C

A

FREE

D

FREE

E

D

F

VM 3Page Table

EDF

• Only get a benefit if VMs on a machine actually have pages to share!

PhysicalRAM

VM 2Page Table

ABCB

C

A

FREE

D

FREE

E

D

F

VM 1Page Table

EDF

Host 1 Host 2

Where to place a VM?• How do you figure out which

VMs to place together?– Meet resource constraints– Maximize sharing

• Why placement is hard in large data centers?– Many applications from

different clients– Many software stacks /

platforms– Workloads change over time

Here or there?

Or there or there or there…?

?

Memory Buddies Goals

• Efficiently analyze the memory contents of multiple VMs to determine sharing potential

• Find more compact VM placement schemes

• Respond quickly to changing conditions to prevent memory hotspots

Bonus! Traces released at traces.cs.umass.eduBonus! Traces released at traces.cs.umass.edu

Outline

• Motivation

• Memory Fingerprinting & Comparison

• Sharing-aware Colocation

• Hotspot Mitigation

• Implementation & Evaluation

• Related Work & Summary

Memory Fingerprints• Hypervisor creates hash for each page

– Check hash table to see if page is sharable– Record these hashes to create fingerprint

• Hash lists are big– 32bits per 4K page = 1MB per 1GB of RAM– Need to forward fingerprint to other hosts

• Comparisons of lists is relatively slow

VM 1

AB

0x11223344

0x55667788

Bloom Filter Fingerprints• Bloom filter is a probabilistic data structure

– Stores keys by setting some bits to 1– False positive chance at lookup from hash

collisions

• Very space efficient• Tradeoff between filter size and accuracy

0 1 1 0 1… 0 0 1

M bits

Insert(key) --> set h1(key)=1 and h2(key)=1VM 1

0x11223344

0x55667788

Fingerprint Comparison• Hash list comparison

– Sort each list and then step through

• Bloom Filter– Simple method: Dot product of bit vectors

– Bloom Sharing Equation• Corrects for the expected number of false

matches in each filter• Impressively accurate!

1 0 0 0 0

0 1 0 0 11

1 1 1

1 1 11

1 01 1 1

1 1 11

1= 4

Eval: Fingerprinting

• 4GB RAM VMs– Hash: 4 sec– Sorted: 0.3 sec– Bloom: 0.02 sec

• Bloom Fingerprint 10% the size, still < 1% error

Bloom filters are smaller and 10 to 100 times fasterBloom filters are smaller and 10 to 100 times faster

Outline

• Motivation

• Memory Fingerprinting & Comparison

• Sharing-aware Colocation

• Hotspot Mitigation

• Implementation & Evaluation

• Related Work & Summary

Sharing Aware Placement• Where to place a freshly started VM?• Use staging area to find initial placement• Find feasible hosts• Estimate sharing potential• Migrate VM• Done!

Staging Host

1100011

Host 1

1010101

Host 3 Host 2

1110000

1000010

1110010+=

Compare

Consolidation & Hotspot Mitigation

• Resource usage changes over time– Sharing may not last forever

• Periodically consolidate servers– Identify candidates (least loaded hosts)– Match to destinations (hosts with best sharing)– Migrate VMs– Disable unnecessary servers

• Hotspot Mitigation– Monitor memory usage to detect hotspots– VMs may run out of memory if sharing stops– Redistribute VMs to rebalance

Offline Planning Tool

Offline Planning Tool

Host resources Resource Traces Memory Fingerprints

Dynamic programming based bin-packing tool

Finds subsets of VMs that can be placed together and

maximize sharing

Host 1

Host N

Number of hosts required = XVM to host mapping

Estimated sharing per host = Y

Outline

• Motivation

• Memory Fingerprinting & Comparison

• Sharing-aware Colocation

• Hotspot Mitigation

• Implementation & Evaluation

• Related Work & Summary

Implementation

• Memory Tracer– Tool used to gather data for trace study– Runs on Linux, OS X, and Windows– Calculates 32bit hashes for each page in

memory– Sends either a hash list or Bloom filter to

control node

• Works on physical systems or in VMs

Implementation• Nucleus

– Collects memory fingerprints for each VM

– Sends data tocontrol plane

• Control Plane– Gathers VM statistics and makes migration decisions

based on sharing– Interacts with VMware Virtual Infrastructure to

manage VMs

Eval: Trace StudySystem Mix

Total MemoryMB

% SharableUsed Mem MB (with sharing)

Linux 2.6.9

Darwin 9.0

Windows XP

4223 13.2% 3666

Darwin 9.0

Darwin 9.4

Windows XP

5248 35.3% 3397

Darwin 9.0

Darwin 9.2

Darwin 9.4

Windows XP

6272 36.8% 3966

Darwin 9.4

(3 MacBook + iMac)8192 40.0% 4917

Eval: App Placement• Try to place as many VMs onto a set of 4 hosts• Sharing Oblivious: Place on first host with sufficient

capacity• Four app types -- data contents different for each VM

instance

1 2 3 4Host

1 2 3 4Host

Sharing Oblivious

Sharing Aware

TPC-W

OFBiz

RUBiS

SpecJBB

17 VMs 20 VMs

Outline

• Motivation

• Memory Fingerprinting & Comparison

• Sharing-aware Colocation

• Hotspot Mitigation

• Implementation & Evaluation

• Related Work & Summary

Related Work

• Waldspurger, OSDI 2002– CBPS in VMware ESX Server

• Gupta, et al., OSDI 2008– Increase sharing potential by looking at

parts of pages

• VM Memory provisioning– Zhao & Wang (yesterday) has a good list!

Summary• Hypervisors already support page sharing…

• Memory Buddies makes it more useful– Identifies sharing opportunities across data center– Migrates VMs to maximize sharing– Uses efficient memory fingerprinting techniques to

scale to large data centers

• Traces will be online (soon) at:– http://traces.cs.umass.edu– Macbooks, Linux servers, and more!

• Questions? twood@cs.umass.edu