Effective Static Race Detection for Java

Mayur, Alex, John @ CS Department Stanford University

Presented by Roy Ganor 14/2/08Point-To Analysis Seminar

Tel Aviv University

A Few Definitions (not!)

t2 = x;

t2 = t2 + 1;

x = t2;

t1 = x;

t1 = t1 + 1;

x = t1;

Motivation Concurrent = Hard

t1++ (20 total)

x==k+2

x==k+1 ××

sync (l) { sync (l) {

Definition

• Two threads access the same memory location• Without ordering constraints• At least one is a write

Non-deterministic nature of thread difficult to reproduce and fix

Dynamic vs. Static Race Detection

Dynamic (lock-set*, happens-before*)• Program is executed • Record and analyzing memory accesses and

synchronization operations • "post-mortem" - records critical events analyze

Static • Employ compile-time analysis on the program source• Reporting all potential races that could occur in any

possible program execution

Pros and ConsDynamic• Feasible execution Lower false positive rate• Not all paths are considered not sound• Cannot certify a program to be race free • Overhead on program execution

Static • False positive (reporting a potential data race when

none exists)• Scaling is also difficult • Frameworks / Open libraries• Sound?

Key Problems

• Precision• Scalability• Synchronization Idioms• Open Programs • Counterexamples

Harness Synthesis

• Problem - Detect races in open programs is important – missing callees and callers

• Solution - simulating scenarios of program’s exercise its interfaces. For each Interface:

1. declares a local variable of each type2. Assigns to each local variable of reference type T, an

object of each concrete class of type T3. Invokes each method on each combination of local

variables and assigns the return value if any to each local variable respecting the result type of the method

4. Simulates executing each pair of calls in separate threads on shared data.

Original Pairs

• F – get / set instance field x.f• G – get / set static field Class.f• A – get / set instance field x[i]

Algorithm OutlineStarting with all (possible) pairs…

JDBM - 11,189,853 pairs

Reduce pairs “Step-by-Step”JDBM – 33,443 7,511 2,756 91 pairs

Each access is reachable from a thread-spawning call site that it itself reacable from main()

Both access the same location

x.f == y.f

Both access thread-shared

Without holding a common lock

Running Example

public A{ )(f = 0} ;

public int get{ )(return rd} ;)(

public sync int inc{ )( int t = rd)( + )new A)((.wr)1(;

return wr)t(} ;

private int rd{ )(return f} ;

private int wr)int x( }f = x;return x} ;

static public void main() {

a = new A();

a.get();

a.inc(); }

All pairs of accesses such that:– Both access the same instance

field or the same static field or array elements

– At least one is a write

Computing Original Pairs

public A)( }f = 0; {

public int get)( }return rd)(; {

public sync int inc)( } int t = rd)( + )new A)((.wr)1(;

return wr)t(; {

private int rd)( }return f; {

private int wr)int x( }f = x;return x; {

Example: Original Pairs

static public void main)( }

a = new A)(;

a.get)(;

a.inc)(; {

private int rd)( } return f; {

Computing Reachable Pairs

• Step 1– Access pairs with at least one write to

same field

• Step 2– Consider access pair (e1, e2)– To have a race, e1 must be reachable

from a thread-spawning call site s1 without “switching” threads

– And s1 must be reachable from main– And similarly for e2

return wr)t(; {

a = new A)(;

a.get)(;

a.inc)(; {

Example: Reachable Pairs

return wr)t(; {

a = new A)(;

a.get)(;

a.inc)(; {

Example: Two Object-Sensitive Contexts

a = new A)(;

a.get)(;

a.inc)(; {

return wr)t(; {

Example: 1st Context

Example: 2nd Context* static public void main)( }

a = new A)(;

a.get)(;

a.inc)(; {

return wr)t(; {

a = new A)(;

a.get)(;

a.inc)(; {

Example: Reachable Pairs

Computing Aliasing Pairs

• Steps 1-2– Access pairs with at least one write to

same field– And both are reachable from some

thread

• Step 3– To have a race, both must access the

same memory location– Use alias analysis

return wr)t(; {

a = new A)(;

a.get)(;

a.inc)(; {

Example: Aliasing Pairs

Computing Escaping Pairs

• Steps 1-3– Access pairs with at least one write to same

field– And both are reachable from some thread– And both can access the same memory

location

• Step 4– To have a race, the memory location must

alsobe thread-shared

– Use thread-escape analysis

return wr)t(; {

a = new A)(;

a.get)(;

a.inc)(; {

Example: Escaping Pairs

Computing Unlocked Pairs

• Steps 1-4– Access pairs with at least one write to same field– And both are reachable from some thread– And both can access the same memory location– And the memory location is thread-shared

• Step 5– Discard pairs where the memory location is

guarded by a common lock in both accesses– Needs must-alias analysis– We use approximation of may-alias analysis,

which is unsound

a = new A)(;

a.get)(;

a.inc)(; {

return wr)t(; {

Example: Unlocked Pairs

¬ MAY-ALIAS(e1, e2)

l1 and l2 always refer to the same value

• Field f is race-free if:

Alias Analysis

// Thread 1: // Thread 2:

sync (l1) { sync (l2) {

… e1.f … … e2.f …

MUST-ALIAS(l1, l2)

e1 and e2 never refer to the same value

Must Alias Analysis

• Small body of work– Much harder problem than may alias analysis

• Impediment to many previous race detection approaches– Folk wisdom: Static race detection is intractable

Insight: Must alias analysis not necessary forrace detection!

• Field f is race-free if:

New Idea: Conditional Must Not Alias Analysis

Whenever l1 and l2 refer to different values, e1 and e2also refer to different values

MUST-NOT-ALIAS(l1, l2) => MUST-NOT-ALIAS(e1, e2)

// Thread 1: // Thread 2:

sync (l1) { sync (l2) {

… e1.f … … e2.f …

Example

a = new h0[N];

for (i = 0; i < N; i++) {

a[i] = new h1;

a[i].g = new h2;

h0a[N-1]

… h2

x2 = a[*];

sync (?) {

x2.g.f = …;

x1 = a[*];

sync (?) {

x1.g.f = …;

MUST-NOT-ALIAS(?, ?) => MUST-NOT-ALIAS(x1.g.f, x1.g.f)

static public void main() {

a = new A();

4: a.get();

5: a.inc(); }

field reference A.f (A.java:10) [Rd]A.get(A.java:4)Harness.main(Harness.java:4)

field reference A.f (A.java:12) [Wr]A.inc(A.java:7)Harness.main(Harness.java:5)

Example: Counterexample

public A() {f = 0; }

public int get() {4: return rd(); }

public sync int inc() {int t = rd() + (new A()).wr(1);

7: return wr(t); }

private int rd() {10: return f; }

private int wr(int x) {12: f = x; return x; }

Benchmarks

vect1.1htbl1.1htbl1.4vect1.4tsphedcftppooljdbmjdbfjtdsderby

classes19213663703704224933884614655531746

KLOC3375767683103124115122165646

descriptionJDK 1.1 java.util.VectorJDK 1.1 java.util.HashtableJDK 1.4 java.util.HashtableJDK 1.4 java.util.VectorTraveling Salesman ProblemWeb crawlerApache FTP serverApache object pooling libraryTransaction managerO/R mapping systemJDBC driverApache RDBMS

Pairs Retained After Each Stage

100000

1000000

10000000

100000000

original

reachable

aliasing

escaping

unlocked

Conclusions

• A scalable and precise approach to static race detection– Largest program analyzed: ~ 650 KLOC (derby)

• Handles common synchronization idioms, analyzes open programs, and generates counterexamples

• An example where precise alias analysis is key– Not just any alias analysis (k-object sensitivity)

http://www.cs.stanford.edu/~mhn/chord.html

Implementation

References• R. Agarwal, A. Sasturkar, Wang L, and S. Stoller. Optimized run-time race detection

and atomicity checking using partial discovered types. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE’05), pages 233–242, 2005

Effective Static Race Detection for Java

Documents

Java Keywords Reserved Words Part of the Java language Examples: public, static, void Pre-defined Java Identifiers Defined in Java Libraries Examples:

Static Race Detection for C using Locksmith

Static Analysis for Java Servlets and JSP

RELAY: Static Race Detection on Millions of Lines of Code

Mayur Naik Alex Aiken John Whaley Stanford University Effective Static Race Detection for Java

Locksmith: Practical Static Race Detection for C - Computer Science

Comparing Java Web Framew orks - Raible Designs :: Static Resources

RacerX: effective, static detection of race conditions and deadlocks

furqanslide.files.wordpress.com · Web viewstatic vs final java, both are known access modifiers in Java doing different functionalities (used for different jobs). static: static

A Parameterized Type System for Race-Free Java Programs

RacerX: Effective, Static Detection of Race Conditions and Deadlocks

Static Race Detection for C Jeff Foster University of Maryland

Locksmith: Practical Static Race Detection for Cjfoster/papers/toplas-locksmith.pdfLocksmith: Practical Static Race Detection for C Polyvios Pratikakis University of Maryland and Je

Object Oriented Programming using Java - Final and Static Keywords

Data races in Java and static analysis

Static and Dynamic Reverse Engineering Techniques for Java

240 core java interview questions and answers · 1 240 CORE JAVA INTERVIEW QUESTIONS AND ANSWERS Table of Contents 1) what are static blocks and static initalizers in Java ?

Finding Input Validation Errors in Java with Static Analysis

Sawja: Static Analysis Workshop for Java - IRISApeople.irisa.fr/David.Pichardie/papers/foveoos10.pdf · Sawja: Static Analysis Workshop for Java ... issue of how to program a static

Extended Static Checking for JAVA