Model checking transactional memory with Spin

Model checking transactional memory

with Spin

John O’Leary, Bratin Saha, Mark TuttleIntel Corporation

We used the Spin model checker to prove that Intel’s software transactional memory is correct.

ICDCS 2009 Slide 2 of 29

What is transactional memory?

A programming abstraction that makes it easier to write concurrent programs.


Concurrent programs are tricky

concurrent queue

enqueue(a)

enqueue(b)

enqueue(c) enqueue(v) =

if last == max return false; last := last + 1; queue[last] := v; return true;

• How do you synchronize access to tail of the queue?– What keeps two threads from writing the same queue entry?


Locks are hard

concurrent queue

enqueue(a)

enqueue(b)



acquire(lock);

release(lock);

• Locks used badly can lead to many subtle problems:– Hot-spots, blocking, dead-lock, priority inversion, preemption …


Transactional memory is easy

concurrent queue

enqueue(a)

enqueue(b)



atomic {

}

• A programming abstraction for many core– Makes it easy to implement atomic operations without locks– Makes is possible for average programmers to write correct code


Programming is easy

• Elegant code, properly synchronized, no data races

0 0 0 0head

concurrent linked list

A takes head off list

A: atomic { result := head; if head != null then head := head.next; } use result;

B increments list elements

B: atomic { node := head; while (node != null) { node.value++; node := node.head; }


A reads head0 0 01 1 101

Implementation is hard• Some implementations expose intermediate states

– A and B appear sequential, but run concurrently: data races!– A and B can exhibit the “privatization” bug:

B reads head

head headresult

result = 0 result = 1! result = 0!!

• Proving correctness is not easy– We think model checking can help


Our results

• McRT is a software transactional memory from Intel• Spin is a software model checker from AT&T + NASA

• We use Spin to prove that McRT is correct– “Every execution of every purely-transactional program with

two transactions doing three reads and writes is serializable”– We validate an implementation model of an industrial product,

not just an abstract protocol model

• We give a Spin accelerator for shared memory programs


What is McRT?



That’s it• We modeled this pseudocode exactly

– We even model pointer dereferencing with array indexing

• We do make the usual simplifying assumptions– No partial writes: modeled only whole-block loads and stores– No conflict handling: one of two conflicting transactions aborts

• Timestamps are the key to the protocol


Timestamps are everywhere• Global timestamp: global.ts

– Advances whenever a transaction tries to commit or abort– When it changes, memory may have changed, so be careful

• Transaction timestamp: txn.ts– Transaction start time (and current proposal for commit time)– Will be read by other transactions when they commit– Stored in transaction descriptor

• Along with transaction read set, write set, undo log (local data)

• Memory block timestamp: blk.ts– Commit time of last transaction writing the block– Stored in transaction record

• Along with a lock needed to write the block


Design rule 1• No transaction ever sees inconsistent data

– Not even an aborting transaction!– Requires frequent checks that the read set is still valid

• Validate() =– ts := global.ts– for each blk in my read set

• confirm blk is not locked by another transaction• confirm blk.ts my.ts• abort if either confirmation fails

– my.ts := ts

• After validation conclude– Read set has not change since transaction start


Design rule 2• No transaction commits until conflicting transactions abort

– Wait for conflicting transactions to undo changes upon abort– Avoids linked list privatization bug illustrated in introduction

• Quiesce(my.ts) =– for each active transaction txn

• block while txn.ts < my.ts and txn remains active

• After quiescence conclude– Every conflicting transaction will validate which it commits– Validation will fail, transaction will abort, and undo its changes


Protocol sketch• Commit

– Increment global ts

– Validate read set

– Set write set ts to global ts

• Abort– Increment global ts

– Undo changes to write set

– Set write set ts to global ts

• Read– Add block to read set

– … unless

• Block is locked

• blk.ts > txn.ts

• Write– Add block to write set

– Add block value to undo log

– Update block value

– … unless

• Block is locked

• blk.ts > txn.ts


What does our model look like?


An invocation/response model

pgm

mcrt

ReadI(x)

ReadR(v)AbortR

pgm

mcrt

WriteI(x,v)

WriteRAbortR

pgm

mcrt

StartICommitI StartR

CommitRAbortR

Shared memory

global timestamp

transaction timestamps

program memory block timestamps and locks


McRT environment

pgm1

mcrt1

pgm3

mcrt3

pgm2

mcrt2

Environment

send1 recv1

shared memory

send2 recv2 send3 recv3


Environment generatesprograms on the fly

pgm k

read(x,_)

----

----

pgm k

read(x,v)

read(y,_)

----

pgm k

read(x,v)

read(y,_)

----

mcrt k

ReadR(v)

mcrt k

ReadI(y)ReadI(x)


Environment simulates programsincluding aborts

pgm k

read(x,v)

read(y,w)

write(z,u)

pgm k pgm k

mcrt k

AbortR

mcrt k

StartI

read(x,_)

read(y,_)

write(z,u)

read(x,_)

read(y,_)

write(z,u)

WriteI(z,u)

pc

pc


Environment checks results

pgm 1

read(x,v)

read(y,w)

write(z,u)

pgm 2 pgm 3

mcrt 1

CommitR(ordering hint)

CommitR

read(w,a)

read(y,w)

write(w,b)

write(m,l)

write(n,p)

write(z,v)

CommitR

• CommitR carries transaction ordering hint• Environment finds a transaction ordering consistent with transaction results and program memory


We modeled pseudocode “exactly”

Let’s look at the least “exact” match: Abort


STMTxnAbort(TxnDesc* txnDesc, uint32 reason) {

for ( (addr, val, size) in txnDesc->undoLog ) { if (addr is on dead stack frames) continue; switch(size) { case 4: *(uint32*)addr = val; break; ... } }

if ((token = txnDesc->token) == 0) token = lockedIncrement(globalTimeStamp);

for ( txnRecPtr in txnDesc->writeSet ) *txnRecPtr = token;

txnDesc->localTimeStamp = 0;

backoff(); abortInternal(txnDesc); /* longjmp */ }

inline abortTransaction(txnDescPtr, ...) {

foreach adr in 0..(num_addresses)-1 { if :: txnDesc(txnDescPtr).undoLog[adr] != null_data -> memory[adr] = txnDesc(txnDescPtr).undoLog[adr]; :: else fi };

fetch_and_incr (globalTimeStamp,token,token_new);

foreach blk in 0..(num_memory_blocks)-1 { if :: txnDesc(txnDescPtr).writeSet[blk] -> txnRecHeap[blk] = token_new; :: else fi };

/* reset transaction descriptor for restart */ initTxnDesc(txnDesc(txnDescPtr),...);

txnDesc(txnDescPtr).localTimeStamp = 0;

}

Pseudocode Our model


What obstacles did we face?


Challenges• Modeling environment, abort, timestamps, …• Code-level models are hard to model check

– Too much detail, too many interleavings

• SPIN statement-merging is conservative– Intended to reduce detail by creating larger atomic blocks– Looping over data structures inhibits statement-merging

• SPIN partial-order reduction is conservative– Intended to identify and ignore “redundant” interleavings– Global variables (like shared memory) inhibit partial-order reduction


A SPIN preprocessor• Loop unrolling to help statement merging, etc.

– Loop unrolling alone gives 50% speedup

• Model rewriting to help partial order reduction (planned)– Help Spin find fewer, longer atomic blocks to reorder– Rewrite model as a set of transitions of the form

atomic{ local access; local access; … ; global access}

memory[0] := 0memory[1] := 0memory[2] := 0memory[3] := 0

adr = 0;do :: adr < num_addresses -> memory[adr] = 0 :: else -> break;od;


Related workA deep result:

Model checking TM often reduces to checking 2 threads

• Deferred update [Guerraoui, Henzinger, Jobstmann, Singh, PLDI’08]

– Applies to any TM that satisfies four structural properties– Clean, elegant result, but doesn’t apply to McRT

• Update in place [Guerraoui, Henzinger, Singh, CAV’09]

– Requires hand proof than TM satisfies four generalize properties– They prove this for an abstract model of McRT– Proof not clear for our implementation model of McRT


The abstract model

Our implementation model is 2500+ lines of Spin


Conclusion

We validated Intel’s implementation of STM.

We optimized SPIN’s performance on shared memory protocols.

Documents

Model checking transactional memory with Spin