Operating Systems - Dynamic Memory Management

UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science

Emery Berger

University of Massachusetts Amherst

Operating SystemsCMPSCI 377

Dynamic Memory Management

UNIVERSITY OF MASSACHUSETTS AMHERST • Department of Computer Science 2

Dynamic Memory Management

How the heap manager is implemented

malloc, free

new, delete


Memory Management

Ideal memory manager:

Fast

Raw time, asymptotic runtime, locality

Memory efficient

Low fragmentation

With multicore & multiprocessors:

Scalable to multiple processors

New issues:

Secure from attack

Reliable in face of errors

3


Memory Manager Functions

Not just malloc/free

realloc

Change size of object, copying old contents

ptr = realloc (ptr, 10);

But: realloc(ptr, 0) = ?

How about: realloc (NULL, 16) ?

Other fun

calloc

memalign

Needs ability to locate size & object start

4


Fragmentation

Intuitively, fragmentation stems from “breaking” up heap into unusable spaces

More fragmentation = worse utilization of memory

External fragmentation

Wasted space outside allocated objects

Internal fragmentation

Wasted space inside an object

5


Classical Algorithms

First-fit

find first chunk of desired size

6



Best-fit

find chunk that fits best

Minimizes wasted space

7



Worst-fit

find chunk that fits worst

then split object

Reclaim space: coalesce free adjacent objects into one big object

8


Implementation Techniques

Freelists

Linked lists of objects in same size class

Range of object sizes

First-fit, best-fit in this context

9



Segregated size classes

Use free lists, but never coalesce or split

Choice of size classes

Exact

Powers-of-two

10



Big Bag of Pages (BiBOP)

Page or pages (multiples of 4K)

Usually segregated size classes

Header contains metadata

Locate with bitmasking

Limits external fragmentation

Can be very fast

11


Runtime Analysis

Key components

Cost of malloc (best, worst, average)

Cost of free

Cost of size lookup (for realloc & free)

Examine for first-fit, best-fit, segregated (with BiBOP)

12


Space Bounds

Fragmentation worst-case for “optimal”: O(log M/m)

M = largest object size

m = smallest object size

Best-fit = O(M * m) !

Goal: perform well for typical programs

Considerations:

Internal fragmentation

External fragmentation

Headers (metadata)

13


Performance Issues

We’ll talk about scalability later

Reliability, too

But: general-purpose allocator often seen as too slow

14


Custom Memory Allocation

Very common practice

Apache, gcc, lcc, STL, database servers…

Language-level support in C++

Widely recommended

Programmers replace new/delete, bypassingsystem allocator

Reduce runtime – often

Expand functionality –sometimes

Reduce space – rarely

“Use custom

allocators”


Drawbacks of Custom Allocators

Avoiding system allocator:

More code to maintain & debug

Can’t use memory debuggers

Not modular or robust:

Mix memory from customand general-purpose allocators → crash!

Increased burden on programmers

Are custom allocators really a win?


Class1 free list

(1) Per-Class Allocators

a

b

c

a = new Class1;

b = new Class1;

c = new Class1;

delete a;

delete b;

delete c;

a = new Class1;

b = new Class1;

c = new Class1;

Recycle freed objects from a free list

+ Fast+ Linked list operations

+ Simple+ Identical semantics

+ C++ language support

- Possibly space-inefficient


(II) Custom Patterns

Tailor-made to fit allocation patterns

Example: 197.parser (natural language parser)

char[MEMORY_LIMIT]

a = xalloc(8);b = xalloc(16);c = xalloc(8);xfree(b);xfree(c);d = xalloc(8);

a b cd

end_of_arrayend_of_arrayend_of_arrayend_of_arrayend_of_arrayend_of_array

+ Fast+ Pointer-bumping allocation

- Brittle- Fixed memory size

- Requires stack-like lifetimes


(III) Regions

+ Fast+ Pointer-bumping allocation

+ Deletion of chunks

+ Convenient+ One call frees all memory

regionmalloc(r, sz)

regiondelete(r)

Separate areas, deletion only en masse

regioncreate(r) r

- Risky- Dangling

references

- Too much space

Increasingly popular custom allocator


Custom Allocators Are Faster…

Runtime - Custom Allocator Benchmarks

0

0.25

0.5

0.75

1

1.25

1.5

1.75

197.

pars

er

boxe

d-sim

c-br

eeze

175.

vpr

176.

gcc

apac

helcc

mud

lle

No

rma

lize

d R

un

tim

e

Custom Win32

non-regions regions

As good as and sometimes much faster than Win32


Not So Fast…

Runtime - Custom Allocator Benchmarks

0

0.25

0.5

0.75

1

1.25

1.5

1.75

197.

pars

er

boxe

d-sim

c-br

eeze

175.

vpr

176.

gcc

apac

he lcc

mud

lle

No

rma

lize

d R

un

tim

e

Custom Win32 DLmalloc

non-regions regions

DLmalloc: as fast or faster for most benchmarks


The Lea Allocator (DLmalloc 2.7.0)

Mature public-domain general-purpose allocator

Optimized for common allocation patterns Per-size quicklists ≈ per-class allocation

Deferred coalescing(combining adjacent free objects) Highly-optimized fastpath

Space-efficient


Space Consumption: Mixed Results

Space - Custom Allocator Benchmarks

0

0.25

0.5

0.75

1

1.25

1.5

1.75

197.

pars

er

boxe

d-sim

c-br

eeze

175.

vpr

176.

gcc

apac

he lcc

mud

lle

No

rmalized

Sp

ace

Custom DLmalloc

regionsnon-regions


Custom Allocators?

Generally not worth the trouble:use good general-purpose allocator

Avoids risky software engineering errors

24


Problems with Unsafe Languages

C, C++: pervasive apps, but langs.memory unsafe

Numerous opportunities for security vulnerabilities, errors

Double free

Invalid free

Uninitialized reads

Dangling pointers

Buffer overflows (stack & heap)


Soundness for “Erroneous” Programs

Normally: memory errors ) ? …

Consider infinite-heap allocator:

All news fresh;ignore delete

No dangling pointers, invalid frees,double frees

Every object infinitely large

No buffer overflows, data overwrites

Transparent to correct program

“Erroneous” programs sound


Probabilistic Memory Safety

Approximate with M-heaps (e.g., M=2)

DieHard: fully-randomized M-heap

Increases odds of benign errors

Probabilistic memory safety

i.e., P(no error) n

Errors independent across heaps E(users with no error) n * |users|

? Efficient implementation…


Implementation Choices

Conventional, freelist-based heaps

Hard to randomize, protect from errors

Double frees, heap corruption

What about bitmaps? [Wilson90]

– Catastrophic fragmentation

Each small object likely to occupy one page

obj obj objobj

pages


Randomized Heap Layout

Bitmap-based, segregated size classes

Bit represents one object of given size

i.e., one bit = 2i+3 bytes, etc.

Prevents fragmentation

00000001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap


Randomized Allocation

malloc(8):

compute size class = ceil(log2 sz) – 3

randomly probe bitmap for zero-bit (free)

Fast: runtime O(1)

M=2 – E[# of probes] · 2

00000001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap


malloc(8):

compute size class = ceil(log2 sz) – 3

randomly probe bitmap for zero-bit (free)

Fast: runtime O(1)

M=2 – E[# of probes] · 2

00010001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap

Randomized Allocation


free(ptr):

Ensure object valid – aligned to right address

Ensure allocated – bit set

Resets bit

Prevents invalid frees, double frees

00010001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap

Randomized Deallocation



free(ptr):



Resets bit


00010001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap


free(ptr):



Resets bit


00000001 1010 10

size = 2i+3 2i+4 2i+5

metadata

heap



Randomized Heaps & Reliability

2 34 5 3 1 6

object size = 2i+4object size = 2i+3

…

11 6 3 2 5 4 …

My Mozilla: “malignant” overflow

Your Mozilla: “benign” overflow

Objects randomly spread across heap Different run = different heap

Errors across heaps independent


DieHard software architecture

broadcast vote

input output

execute replicas(separate processes)

replica3seed3

replica1seed1

replica2seed2

Replication-based fault-tolerance Requires randomization: errors independent


DieHard Results

Analytical results (pictures!)

Buffer overflows

Uninitialized reads

Dangling pointer errors (the best)

Empirical results

Runtime overhead

Error avoidance

Injected faults & actual applications


Analytical Results: Buffer Overflows

Model overflow as write of live data

Heap half full (max occupancy)



Model overflow as write of live data




Model overflow: random write of live data




Replicas: Increase odds of avoiding overflow in at least one replica

rep

lica

s




rep

lica

s




rep

lica

s

P(Overflow in all replicas) = (½)3 = 1/8

P(No overflow in > 1 replica) = 1-(½)3 = 7/8



F = free space H = heap size N = # objects

worth of overflow

k = replicas

Overflow one object


Empirical Results: Runtime


Empirical Results: Error Avoidance

Injected faults:

Dangling pointers (@50%, 10 allocations) glibc: crashes; DieHard: 9/10 correct

Overflows (@1%, 4 bytes over) – glibc: crashes 9/10, inf loop; DieHard: 10/10 correct

Real faults:

Avoids Squid web cache overflow Crashes BDW & glibc

Avoids dangling pointer error in Mozilla DoS in glibc & Windows


The End

47

Technology

Operating Systems - Dynamic Memory Management