Dynamic Object Sampling for Pretenuring

Dynamic Object Sampling for Pretenuring

Maria JumpDepartment of Computer SciencesThe University of Texas at Austin

mjump@cs.utexas.edu

Stephen M. BlackburnDepartment of Computer Science

Australia National UniversitySteve.Blackburn@anu.edu.au

Kathryn S. McKinleyDepartment of Computer SciencesThe University of Texas at Austin

mckinley@cs.utexas.edu

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 2

Current Trends

Object-oriented languages Lots of small objects Runtime system treats them uniformly

Objects have distinctive properties Lifetime, access frequency, locality,

calling context, thread usage, etc. PROBLEM: How can we discover and

exploit object properties?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 3

Gathering Object Statistics

Offline statistics Inconsistent with Java Obscure phase behavior

Online statistics Profiling all objects Dynamic Object Sampling

Statistics only for selected objects Use to estimate behavior of all objects

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 4

Example: Using Object Lifetimes for Pretenuring

PROBLEM: Long-lived objects are allocated into the nursery

SOLUTION: Allocate long-lived objects directly into mature space

CHALLENGE: Correlating lifetime behavior with allocation site

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 5

Outline Motivation Dynamic Object Sampling

Mechanism Dynamic Lifetime Prediction

Accuracy vs. Cost Dynamic Pretenuring

Allocation Survival Phases Coverage, Accuracy, and Performance

Conclusions

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 6

384256128

Dynamic Object Sampling Modify a bump-pointer allocator Sample one object every n bytes

Memory chunk:

Bump Pointer

Sample Tag(for n=128)

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 7

VM_Address alloc(int bytes)

throws VM_PragmaInline {

VM_Addresss oldCursor = cursor;

VM_Address newCursor = oldCursor.add(bytes);

if (newCursor.GT(sampleLimit))

return sample(bytes);

cursor = newCursor;

return oldCursor;

}

Sample Path(Intermediate) :

1) adds sample tag2) collects statistics

Slow Path

Bump-Pointer AllocatorVM_Address alloc(int bytes)

throws VM_PragmaInline {

VM_Addresss oldCursor = cursor;

VM_Address newCursor = oldCursor.add(bytes);

if (newCursor.GT(limit))

return allocSlow(bytes);

cursor = newCursor;

return oldCursor;

}

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 8

During Garbage Collection

Use GC to update statistics Trace live objects Update transient and total statistics

SAMPLE TAG FOUND!1. decode tag word2. collect statistics

survivor

What’s in the Sample Tag?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 9

Dynamic Lifetime Prediction

Encode allocation site identifier into the object sampling sample tag Allocation site is a good lifetime predictor

[Blackburn et al.]

Compiler maps identifiers to sites Accuracy vs. Cost tradeoff

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 10

Implementation and Experimental Methodology

Jikes RVM 2.3.0.1 with MMTk SPEC JVM benchmarks, SPEC JBB 2000 with

constant workload 3.2 GHz Intel Pentium 4 w/ hyperthreading,

1GB main memory, Linux 2.6.0 Generational/Mark-Sweep (GenMS)

4MB bounded nursery Pseudoadaptive compilation Report application time only [Eeckhout et al.]

2nd run methodology

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 11

Accuracy of Sampling

javac

Accuracy92.5%

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 12

Cost of Sampling

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 13

Object Sampling forDynamic Pretenuring

Accurately and efficiently estimate allocation-site lifetimes

At garbage collection time Calculate survival rate per allocation site Compute transient and total statistics

if (survival rate >= threshold) then change the allocation target

Pretenure:

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 14

if (shortLived(site))

region = nursery.alloc(…);

else

region = matureAlloc(…);

break;

Dynamic Allocation Target

case NURSERY_SPACE:region = nursery.alloc(…);

break;

table lookup

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 15

Dynamic Allocation Target Overhead

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 16

Dynamic Pretenuring Overhead

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 17

Allocation Phases

Current survival rate is not a guarantee of future survival

Sampling in older space is ineffective Collections are infrequent Small amount of allocation

Periodically allocate site in nurseryBACKSAMPLING

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 18

Backsampling

Protects the system from bad choices Reduces effectiveness of good

choicesWhen to Backsample?

BacksampleTrigger = n * f

Number of allocations for

pretenuring decision

Growth function:

ConstantLinear

Exponential

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 19

Dynamic Pretenuring Policies

Parameters

Configurations

cfg1 cfg2 cfg3sampling interval 256 256 256

minimum samples 8 4 10

pretenuring threshold 80% 80% 85%

backsampling policy linear exponential linear

backsampling shift 1 4 1

decay shift 0 0 1

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 20

Coverage: How much?

0

5

10

15

20

25

30

35

40

45

jbb javac db mtrt jack raytrace jess compress

Lon

g-L

ived O

bje

cts

(% o

f Tota

l A

llocati

on

s)

None100%95%90%85%80%75%

PretenuringThreshold

Long-LivedPretenured

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 21

0

5

10

15

20

25

30

35

40

45

jbb javac db mtrt jack raytrace jess compress

Pre

dic

ted

Lo

ng

-Liv

ed

Ob

jects

(% o

f To

tal A

llo

cati

on

s)

Accuracy: Are we right?

None100%95%90%85%80%75%

PretenuringThreshold

IncorrectCorrect

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 22

GC Time: javac

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 23

Total Time: javac

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 24

Conclusions Lifetime estimation for dynamic pretenuring

Accurately and efficiently estimates allocation-site lifetimes

Pretenuring long-lived allocation-sites can improve runtime

Few opportunities in our programs

Dynamic Object Sampling Low-overhead High accuracy Generic mechanism

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 25

Questions?

Thank You

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 26

GC Time: jess

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 27

Total Time: jess

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 28

Coverage: How much?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 29

Accuracy: Are we right?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 30

Coverage: How much?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 31

0

5

10

15

20

25

30

35

40

45

jbb javac db mtrt jack raytrace jess compress

Lon

g-L

ived O

bje

cts

(% o

f Tota

l A

llocati

on

s)Coverage: How much?

ISMM, Oct. 25, 2004 Jump, Blackburn, McKinley 32

0

5

10

15

20

25

30

35

40

45

jbb javac db mtrt jack raytrace jess compress

Lon

g-L

ived O

bje

cts

(% o

f Tota

l A

llocati

on

s)Coverage: How much?