Evaluating 3 Algorithms for Detecting Predicates Under Possibly

Modality

by

Stephan A. Lips, M.A.

Master Report

Presented to the Faculty of the Graduate School of

The University of Texas at Austin

in Partial Fulfillment

of the Requirements

for the Degree of

Master of Science in Engineering

The University of Texas at Austin

December 2003

Evaluating 3 Algorithms for Detecting Predicates Under Possibly

Modality

Approved by Supervising Committee:

Dedication

This Master Report is dedicated to my wife Stephanie and my son Kevin.

iv

Acknowledgements

I would like to thank the following individuals for their contributions and support:

• Vijay K. Garg, Ph.D., for providing the topic, and his guidance

and support during all phases of this master report.

• Kevin J. Greene, Ph.D., for being the inspiration to pursue a

graduate degree in Software Engineering, and giving this report the

final polish.

• My wife Stephanie, for her unconditional love and support.

• Troux Technologies for granting access to their test lab to run the

experiments.

December 5, 2003

v

Abstract

Evaluating 3 Algorithms for Detecting Predicates Under Possibly

Modality

Stephan A. Lips, M.S.

The University of Texas at Austin, 2003

Supervisor: Vijay K. Garg

This report provides implementations for 3 algorithms to detect global

predicates under possibly modality: breadth-first search traversal, depth-first

search traversal, and lexical-order traversal. It empirically evaluates them for

time and space requirements, using three different models of distributed

computations. Lexical-order traversal is found to be superior in both time and

space at up to two orders of magnitude in all models of computation used. The

lexical-order traversal algorithm is then evaluated for scalability by distributing it

across multiple processors.

vi

Table of Contents

INTRODUCTION 1

CONCEPTS 3

IMPLEMENTATION LANGUAGE 6

DESIGN 7

Supporting Data Structures...................................................................................... 7

OneBasedIntVector ........................................................................................ 8

TimeStamp ..................................................................................................... 8

ProcessEvent ................................................................................................... 9

ProcessEventSequence ................................................................................... 9

Algorithms for Possibly True ................................................................................ 10

Common Functionality................................................................................. 10

Template Method.......................................................................................... 11

Predicates............................................................................................................... 12

Input Format .......................................................................................................... 12

Graphical User Interface........................................................................................ 14

ALGORITHM IMPLEMENTATIONS 15

Depth-First Search Traversal................................................................................. 17

Breadth-First Search Traversal.............................................................................. 18

Lexical Order Traversal......................................................................................... 19

PROCESS SIMULATIONS AND TRACE FILE GENERATORS 20

Supporting data structures ..................................................................................... 20

Process .......................................................................................................... 20

vii

Generators.............................................................................................................. 21

Abstract Base Class ...................................................................................... 21

Implemented Generators .............................................................................. 21

EXPERIMENT SETUP 25

Hardware and Software Configurations ................................................................ 26

Measurements and Invocation............................................................................... 26

EXPERIMENT RESULTS 28

Internal Events Simulation.................................................................................... 28

Monitor Process Simulation.................................................................................. 30

Sequential Message Simulation............................................................................. 31

DISTRIBUTING THE LEXICAL ORDER TRAVERSAL ALGORITHM 32

Client Implementation........................................................................................... 32

Server Implementation .......................................................................................... 34

Client-Server Communication............................................................................... 34

DISTRIBUTED EXPERIMENT SETUP 36

Hardware and Software Configurations ................................................................ 36

Measurements and Invocation............................................................................... 37

DISTRIBUTED EXPERIMENT RESULTS 38

Internal Events Simulation.................................................................................... 38

Monitor Process Simulation.................................................................................. 39

Sequential Message Simulation............................................................................. 39

CONCLUSION 40

LIMITATIONS AND KNOWN ISSUES 42

viii

Overflow................................................................................................................ 42

Splitting the lattice of consistent global states into sub- lattices............................ 42

APPENDIX 43

Source Code: Package lips.master.alagarVenkatesan........................................... 43

PossiblyTrue ................................................................................................. 43

Source Code: Package lips.master.common.......................................................... 47

AbstractPossiblyTrue ................................................................................... 47

OneBasedIntVector ...................................................................................... 53

ProcessEvent ................................................................................................. 55

ProcessEventSequence ................................................................................. 56

TimeStamp ................................................................................................... 58

Source Code: Package lips.master.cooperMarzullo .............................................. 63

PossiblyTrue ................................................................................................. 63

Source Code: Package lips.master.garg................................................................. 68

PossiblyTrue ................................................................................................. 68

Source Code: Package lips.master.garg.distributed .............................................. 72

IPossiblyTrueServer ..................................................................................... 72

PossiblyTrueClient ....................................................................................... 73

PossiblyTrueServer....................................................................................... 80

Source Code: Package lips.master.predicates ....................................................... 84

FalsePredicate ............................................................................................... 84

GlobalStatePredicate .................................................................................... 85

IntegerAveragePredicate .............................................................................. 86

Source Code: Package lips.master.util .................................................................. 88

ProcessTraceReader ..................................................................................... 88

ix

Source Code: Package lips.master.util.generator .................................................. 90

AbstractTraceFileGenerator ......................................................................... 90

InternalEventsTraceFileGenerator ............................................................... 93

MonitorProcessTraceFileGenerator ............................................................. 94

Process .......................................................................................................... 96

Glossary................................................................................................................. 99

References ........................................................................................................... 100

Vita .................................................................................................................... 102

1

INTRODUCTION

A fundamental problem in debugging and monitoring is detecting whether

the state of a system satisfies some predicate. If the system is distributed, then the

resulting uncertainty in the state of the system makes such detection, in general,

ill-defined [2]. A technique to detect whether a predicate becomes true in a given

execution is important for testing and debugging distributed programs [1]. In a

distributed system, the problem is to detect whether in a given distributed

execution there exists a global state at which a given global predicate is true [1].

Global predicate detection is also a fundamental problem in the area of software

fault tolerance for distributed systems [3], as it allows for recording of consistent

global states (CGS), to which execution can fall back in case a fault is

encountered. An example would be a distributed application, where one of the

processes terminates unexpectedly: the execution could resume from the last

recorded consistent global state.

In their 1991 paper Consistent Detection of Global Predicates [2] Cooper

and Marzullo propose three algorithms to detect a predicate in a global state

lattice: Possibly True, Definitely True, and Currently True. Alagar/Venkatesan

[1] and Garg [3], among others, have proposed alternative algorithms for

detection of a predicate under Possibly modality, which differ both in their time

and space requirements.

As noted by Alagar/Venkatesan [1], at least some of the proposed

algorithms have not been implemented. This project provides implementations

2

for Cooper/Marzullo’s breadth-first-search traversal (BFS) algorithm,

Alagar/Venkatesan’s depth-first-search (DFS) traversal algorithm and Garg’s

lexical-order traversal algorithm; and empirically evaluates their performance in

both time and space for different models of distributed computations. It then

modifies the lexical-order traversal algorithm to run distributed, and measures

performance for a different number of participating processors.

The experiment results show the lexical-order traversal algorithm superior

in both space and time for all of the three distributed computational models used,

by up to two orders of magnitude; in a distributed environment it scales linear

with the number of processors. Performance of BFS traversal and DFS traversal

depends on the type of distributed computation that is being modeled, where for

certain models BFS can show better performance than DFS.

3

CONCEPTS

The implemented algorithms for detecting global states that satisfy a

global predicate are all based on the same data structure: the lattice of consistent

global states that correspond with an observed execution [2].

Section 2 of Alagar/Venkatesan’s paper ‘Techniques to Tackle State

Explosion in Global Predicate Detection’ [2] offers an overview of the concepts

and terminology used in the fields of distributed computing, computation of

consistent global states and and detection of global predicates. The remainder of

this section is quoted from [2], to provide an introductory overview.

A distributed system is a collection of n processes labeled P1,…,Pn. Processes are connected by point-to-point logical channels. Processes and channels are asynchronous and both are fault-free. Processes communicate by message passing only.

A process is a collection of events, which form a total order. An event in a process is an action that changes the state of the process. An event may be a send event resulting in sending of a message to other processes, a receive event resulting in receipt of a message from another process, or an internal event in which no sending or receiving of a message is involved. We use Lamport’s partial order happened before [13], denoted by →, to express the causality between two events. Let E be the set of all events in a particular execution. Then, (E, →) is a partially ordered set. An execution of a distributed program can be represented by a space-time diagram. The space-time diagram for a sample execution is shown in Fig. 1a [below].

A consistent cut C is a finite subset of E such that e ∈ C and e’ → e implies e’ ∈ C. The frontier of a cut C is <e1,…,en> such that for all i, 1) ei is in Pi, and 2) for any ei’ in Pi, if ei → e’, then ei’ ∉ C. In Fig. 1a [below], the frontier of the cut consisting of e1 in P1 and events e1 and e2 in P2 is <e1, e2>.

4

For every consistent cut there is a corresponding consistent global state (the state of the system after the execution of all the events in the cut) and vice versa. A global state is consistent if its corresponding cut is consistent. A global state S is represented as <k1,…,kn> such that, for process Pi, its state after executing ki events is included in S. We say that a global state includes an event e if e belongs to the cut corresponding to the global state.

…

The set of all global states of an execution forms a lattice [6]. Two global states of a lattice are connected by an edge, if the system can proceed from one to the other by executing one event. The initial global state of the lattice is the global state <0,…,0>. A path starting from the initial global state in the lattice is a computation of the system.

…

For a global state S, the previous and next global state can be computed using vector clocks [6], [12]. Every process maintains a vector clock consisting of n components. Let Vi be the vector clock of process Pi. Process Pi increments the ith component of its vector clock, Vi [i] whenever Pi executes an event. When a process sends a message, it timestamps the message with the current value of its vector clock. When Pi receives a message with timestamp T, Vi [j] = max(Vi [j], T [j]) for all j. The timestamp of an event e is denoted by TS(e). For an event e in Pi, TS(e) is the value of the updated vector clock Vi when e is executed.

…

The algorithm for [detection of Possibly modality] is executed by a monitor process. Each process, after executing an event, sends the timestamp of the event and the values of the local variables related to [the global predicate] to the monitor. To see if [the global predicate] holds, the monitor visits all the global states (either sequentially or in parallel), and evaluates [the global predicate] using the values of the variables stored in the visited global states.

5

from: Alagar/Venkatesan, Techniques to Tackle State Explosion in Global Predicate Detection [1]

6

IMPLEMENTATION LANGUAGE

The concepts described in section 2 can be mapped into a collection of

interacting objects with well-defined behaviors. For this reason, an object-

oriented implementation language was the natural choice.

From the available object-oriented programming languages, Sun

Microsystem’s Java Programming Language [6] was chosen for its wide-spread

acceptance, and free availability of both the language and supporting Integrated

Development Environments, such as [7].

7

DESIGN

The problem space was divided into two areas: data structures that model

the concepts described in the section Concepts to support the algorithms, and the

implementation of the algorithms themselves.

The motivation for the development of the supporting data structures was

that all three algorithms use the same conceptual entities, and an standardizing on

an implementation for those entities would simplify both development – by using

common building blocks/objects – as well as understanding of the code by future

work that expands on this project.

The implementation of the algorithms was divided into three categories:

Common Functionality, Template Methods and Predicates. Common

Functionality is all functionality that is shared by the actual implementations of

the three algorithms; note that this does not coincide with the supporting data

structures shared between the implementations. The Template Method Design

Pattern [8] is used to invoke the code for an actual algorithm in an easily

measurable unit, it also allows for extensibility. Last, Predicates are required to

implement a simple interface, through which implementors are accessed by the

system.

Supporting Data Structures

To model the concepts described in the section Concepts in an object-

oriented system, a logical choice was to develop data structures that encapsulate

the behavior and state of those concepts. This section discusses this set of data

8

structures, which is used by the implementations of all three algorithms. The

classes implementing the supporting data structures are located in the

lips.master.common package (see appendix).

ONEBASEDINTVECTOR

This class is the lowest-level data structure, and not directly referenced by

the concepts described in section 2. The motivation for this class is that entries in

a vector timestamp are integers ordered based on the process ID associated with

each element, starting at 1. Since Java vectors are zero-based and not type-safe,

OneBasedIntVector was developed to provide a type-safe implementation for

integer vectors that are one-based, to hide index calculation from clients.

TIMESTAMP

A TimeStamp contains a OneBasedIntVector. It provides methods for

Setting and getting the clock value for a given process ID.

Obtaining the sum of all components, or level, of a timestamp.

Determine if a timestamp is equal to, less than or larger than another timestamp. This

functionality is needed to identify the relationship between two timestamps: they

could be concurrent, or a happened-before relationship could exist.

All fields in a TimeStamp object have read and write access, it is the

responsibility of the client to ensure the correctness of the fields.

TimeStamp objects are used to represent vector clocks in process events,

as well as to represent (potentially) consistent global states.

9

PROCESSEVENT

A ProcessEvent contains a state variable, and a TimeStamp. The state

variable represents the state of the current process, which can be of any type. The

state variable for a process event can be null, the TimeStamp can never be null.

Both the TimeStamp and state are available read-only to clients.

PROCESSEVENTSEQUENCE

A ProcessEventSequence contains FIFO queue, in which the set of

process events for the current process are stored. It is implemented as a zero-

based vector, to which process events can be added and removed. In addition,

methods are provided to

Get the event at a given absolute and relative position. The absolute position is the

position at which the event is currently stored in the queue. The relative position

takes the number of events removed into account. For example, if 9 process events

were added and 3 were removed, the relative position for event 6 would be the

absolute position of 3, e.g. what was originally the 6th process event in the queue is

the 3rd process event after the first three events have been removed.

Get the first and last event in the sequence, respectively.

Get the first event with a specified timestamp entry for a specified process ID.

Get the maximum clock value for a specified process ID.

10

Algorithms for Possibly True

Each algorithm is implemented in its own class, which is derived from the

abstract base class lips.master.common.AbstractPossiblyTrue. The

abstract base class provides functionality used by all algorithms, as well as

defines a template method which the inheriting classes are required to implement.

This section discusses common functionality, and the mechanism for invocation

of PossiblyTrue implementations.

COMMON FUNCTIONALITY

All common functionality is implemented in the abstract base class

AbstractPossiblyTrue and the helper class ProcessTraceReader. It is

shared with all PossiblyTrue implementations through inheritance, as

implementations are derived from the AbstractPossiblyTrue base class. The

following list describes all functional features that are shared across

implementations for PossiblyTrue.

• All algorithms take as input a set of process trace files, each of which

describes the set of events for a given process. All trace files are read into

TimeStamp, ProcessEvent and ProcessEventSequence objects, which

are discussed in the section Supporting Data Structures. An instance of

ProcessTraceReader performs the transformation from flat file to an in-

memory object representation, which is stored in AbstractPossiblyTrue.

• While all algorithms construct potentially consistent global states differently,

all require the ability to check that a potential global state is consistent. Also,

all algorithms must be able to retrieve the process states for a given consistent

11

global state in order to apply a predicate. These functional features are

implemented in AbstractPossiblyTrue.

• All algorithms apply a global predicate to all consistent global states that have

been constructed. Using TimeStamp objects as common format to represent

consistent global states allowed this functional feature to be added to

AbstractPossiblyTrue.

• All algorithms report the result of a PossiblyTrue computation, which will be

the existence or non-existence of a consistent global state that satisfies the

global predicate. This reporting capability is implemented in

AbstractPossiblyTrue.

• The actual implementation of a PossiblyTrue algorithm needs to be measured

for execution length (time). The measuring mechanism is implemented in

AbstractPossiblyTrue, by measuring the execution time of the template

method.

TEMPLATE METHOD

AbstractPossiblyTrue defines a template method, which derived

classes are required to implement: compute().

An implementation for compute() is the actual algorithm, provided by a

PossiblyTrue implementation. The compute() method is invoked after all input

has been processed and the current system time has been captured. If a consistent

global state that satisfies the global predicate is found, compute() is required to

store it in a member variable of AbstractPossiblyTrue. It is then reported

by the reporting feature implemented in AbstractPossiblyTrue.

12

Predicates

Predicates used by PossiblyTrue computations are required to implement

the GlobalStatePredicate interface. This interface defines two methods,

execute() and toString(). execute() applies the predicate to a set of

process states which are passed as arguments, and returns true or false.

toString() returns a string representation of the predicate for reporting of

progress during a PossiblyTrue computation.

If a class implementing the GlobalStatePredicate interface requires

initialization parameter(s), it must provide a constructor that takes a single string

argument. This string argument constitutes the initialization parameter(s), and

will be provided at runtime.

Note that until a predicate is applied, no particular data type is required for

a process state. Although a process state is represented as string in the trace file,

it is the responsibility of a predicate to translate it into the required data type.

Input Format

The input is stored in flat files, where each file constitutes the trace from a

given process execution. Each line describes a process event: it contains the value

of a process state variable, and its associated vector clock value.

The following format is used: state=3;vectorClock=2,1,3

The entries for the process state and the vector clock value are separated

by “;”. The value of the process state variable is prefixed by “state=“. The

13

value of the vector clock value is prefixed by “vectorClock=“. The vector

clock entries are separated by “,“.

14

Graphical User Interface

A graphical user interface provides the user with the ability to

Choose between available implementations of PossiblyTrue algorithms.

Choose between available predicates, and provide initialization parameters.

Select the level of detail for reporting during computation of a PossiblyTrue

algorithm. Options are trace, which provides a trace of all detected consistent

global states, as well as the result of applying the global predicate; and debug,

which provides in-depth logging of intermediate steps during construction of

(potentially) consistent global states. By default, only the number of computed

consistent global states, result of predicate application, and time consumption will be

reported.

View and copy the results of a computation. The stdout and stderr streams are re-

directed to a text area, which is updated real-time as the computation progresses.

The graphical user interface is intended for pre-experiment trials with

different sets of data, to allow for quickly switching between algorithms and trace

files without the need to modify the invocation mechanism for all PossiblyTrue

implementations. The actual experiments were conducted bypassing the graphical

user interface, as discussed in the section Experiment Setup.

15

ALGORITHM IMPLEMENTATIONS

Each algorithm is implemented in its own class, derived from

AbstractPossiblyTrue. All implementations have three methods in

common:

A constructor that takes an argument of type GlobalStatePredicate. This

constructor is required to allow for programmatic invocation of the PossiblyTrue

implementations by the Graphical User Interface, or third-party clients. The

GlobalStatePredicate argument is the predicate used during an execution of

a PossiblyTrue algorithm.

A main() function. All classes provide a main() function to allow invocation of

the PossiblyTrue implementations from the command line, without incurring the

overhead of the Graphical User Interface.

A compute() function. All PossiblyTrue implementations are required to

implement the compute() function, which is defined as abstract by the parenting

class AbstractPossiblyTrue. The body of this function contains the

implementation of the respective algorithm.

16

The following figure from [3] provides a graphical presentation of a

distributed computation, and the order in which the consistent global states are

traversed for the three algorithms.

from: V.K. Garg, Enumerating Global States of a Distributed Computation in Lexicographic and Breadth-First Manner. [3]

17

Depth-First Search Traversal

Alagar/Venkatesan’s depth-first search traversal algorithm is presented in

the following figure; it is implemented in

lips.master.alagarVenkatesan.PossiblyTrue.

from: S. Alagar and S. Venkatesan. Techniques to Tackle State Explosion in Global Predicate Detection. [1]

18

Breadth-First Search Traversal

Cooper/Marzullo’s breadth-first search traversal algorithm is presented in the following figure; it is implemented in lips.master.cooperMarzullo.PossiblyTrue.

from: Cooper/Marzullo. Consistent Detection of Global Predicates [2].

19

Lexical Order Traversal

Garg’s lexical-order traversal algorithm is presented in the following figure; it is

implemented in lips.master.garg.PossiblyTrue.

from: V.K. Garg, Enumerating Global States of a Distributed Computation in Lexicographic and Breadth-First Manner. [3]

20

PROCESS SIMULATIONS AND TRACE FILE GENERATORS

To measure time and space complexity of the implementations of

PossiblyTrue algorithms, distributed computations with different message-passing

patterns were simulated. In each simulation, both the number of participating

processes as well as the number of events generated by each process were varied.

The simulations were implemented by configurable trace file generators, which

allowed for generation of trace files for each simulation. Configurable parameters

for each generator are the number of processes participating in the simulation, and

the number of events generated by each process.

Following the design of the PossiblyTrue implementations, a supporting

data structure Process was implemented, as well as the abstract base class

AbstractTraceFileGenerator, from which the actual trace file generators

are derived. All files used exclusively for trace files generation are located in the

lips.master.util.generator package; also utilized are the data structures

TimeStamp, ProcessEvent, and ProcessEventSequence in the

lips.master.common package.

Supporting data structures

PROCESS

The Process class simulates the behavior of a process in a distributed

system. It is capable of creating three types of events: internal events, send events

and receive events. All events increment the local vector clock entry, and store it

along with a randomly generated state in a ProcessEvent object. All process

21

events are stored in a ProcessEventSequence object, which is written to file at the

end of the simulation.

Generators

ABSTRACT BASE CLASS

Following the design of AbstractPossiblyTrue, the class

AbstractTraceFileGenerator centralizes functionality common to all

generators, and delegates the steps unique to each generator by using an

abstract/template method. The functionality common to all generators contained

in the abstract base class is

Creation of a user-defined number of Process objects.

Maintaining the sequence of process events for each Process object.

Writing the sequence of process events to file at the end of a simulation/trace file

generation run.

IMPLEMENTED GENERATORS

Generators are derived from AbstractTraceFileGenerator, which

requires them to implement a generate() method. In this method, the

Process objects communicate with each other by generating internal events and

sending inter-process messages. Generators were implemented for three different

process simulations:

22

Internal Events Simulation (InternalEventsTraceFileGenerator): every

process generates only internal events, no messages are passed between processes.

1 2 3 4 5

P1:

1 2 3 4 5

P2:

1 2 3 4 5

Pn:

23

Monitor Process Simulation (MonitorProcessTraceFileGenerator):

process 1 acts as monitoring process. Process 2 … n each sends a message to the

monitoring process, then blocks. After receiving one message from each monitored

process, the monitoring process broadcasts one message to all monitored processes.

This execution iterates until the desired number of events have been generated.

1 2 3 4 5 6

P1:

1 2 3 4

P2:

1 2 3 4

Pn:

24

Sequential Message Simulation

(SequentialMessageTraceFileGenerator): each process sends a

message to the process with the next higher process ID; the process with the highest

process ID only receives messages. This execution iterates until the desired number

of events have been generated.

1 2 3

P1:

1 2 3 4 5

P2:

1 2

Pn:

25

EXPERIMENT SETUP

Using the three trace file generators discussed in section 5.2, trace files

were generated for three experiments/simulations of distributed computations,

each with two different configurations. In one configuration, the numbers of

processes was kept constant, and the number of events generated by each process

was increased. In the other configuration, the number of processes was increased,

and the number of events generated by each process was kept constant. The only

exception were trace files for processes that generated only internal events, as the

computation time of PossiblyTrue for larger data sets was prohibitive under the

given time and hardware constraints. All resulting sets of trace files were used as

input to all three implementations of PossiblyTrue. In detail, the following sets of

process trace files were generated:

Internal Events Simulation # processes # events (total)

5 50 5 100 5 150

Monitor Process Simulation # processes # events (total)

5 250 5 500 5 750 10 500

26

Sequential Message Simulation # processes # events (total)

5 50 5 100 5 150 10 100 15 150

Hardware and Software Configurations

All experiments were conducted on Dell PowerEdge 4350 server-class

machines, configured with 1 Pentium III CPU executing at 450MHz, and 1024

megabytes of RAM. The machines were switched through a Cisco 2924 switch,

operating at 100 megabit/second; all were running Microsoft’s Windows 2000

Server operating system. Sun Microsystem’s Java Virtual Machine version 1.4.1

was used to conduct the experiments, configured to use up to 768 megabyte of

RAM (-Xmx768m).

All machines were configured to run only the minimum required set of

operating system-level services, no other users or processes were accessing the

machines while the experiments were conducted.

Measurements and Invocation

All time measurements were taken using Java’s

System.currentTimeMillis() API call, which – although not measuring the

exact operating system time due to not being a native call – provides a reasonably

approximation of the time taken for a given experiment run. Since the goal of this

27

project was to measure the performance of the algorithms relative to each other

(as opposed to the absolute performance), the use of this mechanism was

considered acceptable.

All experiment runs were initiated by calling the respective

implementation of PossiblyTrue from a .bat file, to avoid the computational

overhead of the graphical user interface. All tracing and debugging was disabled.

28

EXPERIMENT RESULTS

In all experiments, the lexical order traversal of the global state lattice

outperformed BFS and DFS traversal algorithms, at up to two orders of

magnitude. In the case of the monitor process simulation, BFS outperformed DFS

at up to one order of magnitude; in the cases of the internal message and

sequential message simulation DFS outperformed BFS at up to one order of

magnitude.

Following is a complete listing of the experiment results, which are

discussed in the section Conclusion. The legend applies to all result sets.

Legend # processes # of process tracefiles

# events (total) # of events from all processes combined time (ms) time consumed during construction of CGS

# CGS in memory max # of CGS concurrently in memory # generated CGS # of CGS constructed

Internal Events Simulation

Algorithm #

processes # events

(total) time (ms) max # CGS in

memory

# constructed

CGS Cooper/Marzullo 5 50 3254271 8801 161051 Alagar/Venkatesan 5 50 51140 51 161051 Garg 5 50 15485 2 161051 Cooper/Marzullo 5 100 86400000* N/a* n/a* Alagar/Venkatesan 5 100 1861328 101 4084101 Garg 5 100 386474 2 4084101 Cooper/Marzullo 5 150 86400000* n/a* n/a* Alagar/Venkatesan 5 150 17444131 151 28629151 Garg 5 150 2811943 2 28629151

29

*: experiment run terminated after 24 hours due to resource constraints

30

Monitor Process Simulation

algorithm #

processes # events

(total) time (ms) max # CGS in

memory

# constructed

CGS Cooper/Marzullo 5 250 687 22 2261 Alagar/Venkatesan 5 250 2047 252 2261 Garg 5 250 531 2 2261 Cooper/Marzullo 5 500 1110 22 4629 Alagar/Venkatesan 5 500 6703 503 4629 Garg 5 500 875 2 4629 Cooper/Marzullo 5 750 1547 22 6944 Alagar/Venkatesan 5 750 14016 755 6944 Garg 5 750 1328 2 6944 Cooper/Marzullo 10 500 292477 4090 503443 Alagar/Venkatesan 10 500 1795626 505 503443 Garg 10 500 132903 2 503443

31

Sequential Message Simulation

algorithm #

processes # events

(total) time (ms) max # CGS in

memory

# constructed

CGS Cooper/Marzullo 5 50 1360 122 2996 Alagar/Venkatesan 5 50 1110 53 2996 Garg 5 50 609 2 2996 Cooper/Marzullo 5 100 79873 1011 43225 Alagar/Venkatesan 5 100 18234 103 43225 Garg 5 100 4562 2 43225 Cooper/Marzullo 5 150 1778325 4016 242802 Alagar/Venkatesan 5 150 142404 153 242802 Garg 5 150 23858 2 242802 Cooper/Marzullo 10 100 433663 2124 81109 Alagar/Venkatesan 10 100 93717 103 81109 Garg 10 100 21578 2 81109 Cooper/Marzullo 15 150 55972589 16421 881115 Alagar/Venkatesan 15 150 2510077 153 881115 Garg 15 150 453963 2 881115

32

DISTRIBUTING THE LEXICAL ORDER TRAVERSAL ALGORITHM

The performance advantage of the lexical order traversal over BFS and

DFS traversal gave rise to the question to what extent the performance of the

lexical-order traversal algorithm could be improved by distributing it across

several asynchronously executing processors.

In contrast to BFS and DFS traversal, the lexical-order traversal algorithm

lends itself to distributed execution, as the set of global states can be easily split

into subsets with only negligible overhead.

Distributing the lexical-order traversal algorithm required two different

implementations: one client, and n servers. The client is responsible for

processing the process trace files, constructing the process event sequences, and

splitting the set of global states into subsets. The subsets are then processed by

the server machines. The server machines are a slightly modified version the

original lexical-order traversal algorithm, as described in the section Algorithm

Implementations.

Client Implementation

The client splits the set of global states into substates, which are assigned

to the server machines for processing. The set of global states can be split into

subsets of equal size by computing the maximum number of potentially consistent

global states, and dividing them by the number of server machines. The resulting

33

lower and upper bounds Glower and Gupper of each subset are used as input to the

nextLexCGS(G) function:

from: V.K. Garg, Enumerating Global States of a Distributed Computation in Lexicographic and Breadth-First Manner. [3]

This guarantees the lower and upper bound of each subset to be a

consistent global state.

After the bounds of the subsets had been established, each server receives

the lower and upper bound of the subset it is assigned for processing, as well as

all process event sequences. Upon completion, each server notifies the client of

the number of consistent global states computed.

The client is derived from lips.master.garg.PossiblyTrue, which

enables it to use the nextLexCGS(G) function for computation of the lower and

34

upper bounds of each subset; it is implemented in

lips.master.garg.distributed.PossiblyTrueClient.

Server Implementation

Each server is notified of the lower and upper bound of the subset of

global states that it is assigned. It also receives a copy of all process event

sequences. After the processing has completed, the controlling client is notified

of the number of consistent global states that were computed.

The server implementation is derived from

java.rmi.server.UnicastRemoteObject, and contains an instance of

lips.master.garg.PossiblyTrue, which provides access to all necessary

functionality through public member functions; it is implemented in

lips.master.garg.distributed.PossiblyTrueServer.

Client-Server Communication

Java Remote Method Invocation (RMI) [9] is used for communication

between the client and the servers. For each global state sub-lattice that is to be

processed, 5 RMI calls are being executed:

Passing of all process event sequences

Setting of the lower bound of the subset of global states

Setting of the upper bound of the subset of global states

Setting of the maximum events for each process

35

Initiate the lexical order traversal of the subset of global states, which returns the

number of consistent global states computed.

Sun’s rmic.exe RMI compiler was used to create the stub and skeleton

classes for PossiblyTrueServer, which are required in support of RMI

method calls.

36

DISTRIBUTED EXPERIMENT SETUP

Analogous to the experiment setup for the single-process computations of

PossiblyTrue (see the section Experiment Setup), trace files were generated for

three simulations of distributed computations: an internal events simulation, a

monitor process simulation, and a sequential message simulation. The given time

and hardware constraints defined the configurations that were used for each

simulation, as listed below.

Internal Events Simulation # processes # processors # events (total)

5 2 150 5 3 150 5 4 150

Monitor Process Simulation # processes # processors # events (total)

10 2 500 10 3 500 10 4 500

Sequential Message Simulation # processes # processors # events (total)

10 2 100 10 3 100 15 2 150 15 3 150

Hardware and Software Configurations

The hardware and software configuration is identical to the non-

distributed experiments, see the section Experiment Setup.

37

Measurements and Invocation

The time consumed by a distributed lexical traversal was measured on the

client side using Java’s System.currentTimeMillis() API call, see the

section Experiment Setup for limitations. It is the sum of a) the time required to

split the set of global states into subsets, b) configuring and starting the

computation on the remote PossiblyTrue servers, and c) the time consumed by the

PossiblyTrueServer instances.

All server machines were running Sun’s rmiregistry.exe naming

registry, both the client and the servers were invoked from a .bat file with all

tracing and debugging disabled.

38

DISTRIBUTED EXPERIMENT RESULTS

For the internal events and monitor process simulation, the performance

increased linearly with the number of processors added. For the sequential

message simulation, however, the computation time stayed near-constant.

Following is a complete listing of the experiment results, which are

discussed in the section Conclusion. The legend applies to all result sets.

Legend # processes # of process tracefiles

# events (total) # of events from all processes combined # processors # of processors computing CGS

time (ms) time consumed during construction of CGS memory (byte) memory consumed during construction of CGS

# generated CGS # of CGS constructed

Internal Events Simulation

Algorithm #

processes #

processors # events

(total) time (ms)

max # CGS in

memory

# constructed

CGS Garg 5 1 150 2811943 2 28629151 Garg distributed 5 2 150 1438779 4 28629151 Garg distributed 5 3 150 958144 6 28629151 Garg distributed 5 4 150 733748 8 28629151

39

Monitor Process Simulation

algorithm #

processes #

processors # events

(total) time (ms)

max # CGS in

memory

# constructed

CGS Garg 10 1 500 132903 2 503443 Garg distributed 10 2 500 72702 4 503443 Garg distributed 10 3 500 49984 6 503443 Garg distributed 10 4 500 39281 8 503443

Sequential Message Simulation

algorithm #

processes #

processors # events

(total) time (ms)

max # CGS in

memory

# constructed

CGS Garg 10 1 100 21578 2 81109 Garg distributed 10 2 100 22531 4 81109 Garg distributed 10 3 100 21594 6 81109 Garg 15 1 150 453963 2 881115 Garg distributed 15 2 150 455307 4 881115 Garg distributed 15 3 150 440166 6 881115

40

CONCLUSION

Cooper and Marzullo’s algorithm for detecting global predicates under

Possibly modality performs a breadth-first-search traversal (BFS) and requires

space proportional to the size of the biggest level of the lattice of consistent global

states [3]. The time requirement is exponential in the size of the computation:

O(mn), where m is the number of states in a process, and n is the number of

processes [5]. Alagar and Venkatesan’s algorithm performs a depth-first-search

traversal (DFS) of the lattice and requires O(nM) time and O(nE) space, where n

is the number of processes, M is the number of consistent global states, and E is

the number of intermediate global states during traversal [3]. Garg’s algorithm

performs performs a lexical traversal of the lattice, and has O(nE) space and

O(n2M) time complexity [3].

It can be seen from the results in section 8, that performance varies greatly

between the three algorithms, depending on the model of distributed computation

used. In all scenarios, however, the lexical traversal has the lowest time and

space requirements. While this contradicts the complexity analyses given above,

it can be explained from the size of the conducted experiments, where both the

number of processes and consistent global states is relatively small. Constant

computational overhead is ignored for complexity analysis; it remains to be

empirically determined at what point the computational overhead of in particular

DFS is outweighed by the higher complexity of lexical traversal, resulting in the

performance curves to cross.

41

In a distributed environment, the performance of the lexical order traversal

algorithm improves linear with the number of processors for the internal event

and the monitor process simulation, but stays constant for the sequential message

simulation. While this seems surprising, it can be explained from the comparably

low number of consistent global states computed for the sequential message

simulation. Under less stringent hardware and time constraints, it can be expected

to show improvement linear to the number of processors as well.

In conclusion, it can be said that Garg’s lexical order traversal is superior

to Cooper/Marzullo’s BFS and Alagar/Venkatesan’s DFS traversal in both time

and space for the of datasets and their size used in the experiments. Actual

performance of BFS and DFS traversal depends on the type of distributed

computation that is being modeled, where for certain models BFS can show better

performance than DFS. It remains to be empirically determined if these results

hold for larger datasets.

It must be noted, however, that the experiments were conducted with

rather small data sets, due in part to resource constraints as well as the known

issues as described in the next section. An interesting continuation of this project

would be to scale the experiment to larger-scale data, and to more models of

distributed computations.

42

LIMITATIONS AND KNOWN ISSUES

The implementations of the three algorithms as described in this report

and provided in the appendix have the following limitations.

Overflow

This implementation uses the Java primitive long data type to represent

entries in a vector clock, and to compute the levels of a consistent global state.

Using larger data sets, this would lead to overflow. Since the objective of this

project was to obtain performance measurements for all three algorithms, and the

presented results allow for unambiguous empirial classification of the algorithms’

performance, the implementations were not modified to accommodate larger-

scale data.

Splitting the lattice of consistent global states into sub-lattices

Larger data sets of certain distributed system simulations can result in

exceptions when executing the lexical-order traversal algorithm, due to the

algorithm used to split the set of global states into subsets. The algorithm splits

the set of global states into subsets of equal size, and validates the upper and

lower bounds of the subset. In most cases this results in the last/highest subset to

be smaller. Depending on the distributed computation being simulated, a subset

below the highest subset can exceed the largest consistent global state, resulting in

an exception.

43

APPENDIX

Source Code: Package lips.master.alagarVenkatesan

POSSIBLYTRUE public class PossiblyTrue extends AbstractPossiblyTrue { private int _numCGSOnStack = 0;

public PossiblyTrue( GlobalStatePredicate gsp ) throws RemoteException { super( gsp ); } public static void main ( String [] args ) throws RemoteException { PossiblyTrue pt = new PossiblyTrue( new FalsePredicate() ); pt.compute( args ); } protected void compute() { int numProcesses = _eventSequences.length; depthFirstTraversal( initialGlobalState( numProcesses ) ); } /* * Algorithm DFT(S) * begin * (1) i = 1; * (2) Let <k1,...,kn> = S * (3) while( i <= n ) do * begin * (4) S'=<k1,...,ki+1,...,kn> //S', if consistent, is a successor of S * (5) if S' is a global state then * (6) if value(S) = max( value( pred( S' ) ) ) then

44

* (7) DFT( S' ); * (8) i ++; * end; * end. */ private void depthFirstTraversal( TimeStamp state ) { // track number of CGS that are currently in memory, as well as the max # of CGS that have been/are in // memory at a given point _maxInMemoryCGSs = ++_numCGSOnStack > _maxInMemoryCGSs ? _numCGSOnStack : _maxInMemoryCGSs; if( TRACE ) { println( "--> Entering DFT for " + state.toString() ); } if ( !applyPredicate( state ) ) { TimeStamp nextState = null; int pid = 1; while( pid <= state.size() ) { nextState = (TimeStamp)state.clone(); nextState.setClockValue( nextState.getClockValue( pid ) + 1, pid); if( TRACE ) { println( "\tnext state: " + nextState.toString() ); } if( isCGS( nextState ) ) { if( TRACE ) { println( "\t\tis global state" ); } if( state.equals( getMaxPredecessor( nextState ) ) ) { depthFirstTraversal( nextState ); } }

45

else { if( TRACE ) { println( "\t\tnot a global state" ); } } pid++; } } else { _currentCGS = state; } _numCGSOnStack--; } private TimeStamp getMaxPredecessor( TimeStamp state ) { TimeStamp predecessor = (TimeStamp)state.clone(); int currentValue = -1; for( int pid = predecessor.size(); pid > 0; pid-- ) { currentValue = predecessor.getClockValue( pid ); if( currentValue > 0 ) { predecessor.setClockValue( currentValue - 1, pid ); if( isCGS( predecessor ) ) { break; } else { predecessor.setClockValue( currentValue, pid ); } } } if( DEBUG ) {

46

println( "\t\tmax predecessor: " + predecessor.toString() ); } return predecessor; } }

47

Source Code: Package lips.master.common

ABSTRACTPOSSIBLYTRUE abstract public class AbstractPossiblyTrue { public static final String TRACE_SETTING = "lips.master.TRACE"; public static final String DEBUG_SETTING = "lips.master.DEBUG"; public ProcessEventSequence [] _eventSequences = null; public TimeStamp _currentCGS = null; private GlobalStatePredicate _predicate = null; private long _time = 0L; public boolean TRACE = false; public boolean DEBUG = false; public int _numCGS = 0; public int _maxInMemoryCGSs = 0; public AbstractPossiblyTrue() { setTraceOrDebug(); } // to make global predicate configurable, use e.g. properties file as input, which specifies predicate, or require // first argument to be predicate classname, etc. For now concrete subclasses need to specify what predicate to use. public AbstractPossiblyTrue( GlobalStatePredicate p ) { _predicate = p; setTraceOrDebug(); } private void setTraceOrDebug() { TRACE = Boolean.valueOf( System.getProperty( TRACE_SETTING ) ).booleanValue(); DEBUG = Boolean.valueOf( System.getProperty( DEBUG_SETTING ) ).booleanValue();

48

if( DEBUG ) { TRACE = true; } } public void setPredicate( GlobalStatePredicate p ) { _predicate = p; } public void compute(String[] args) { if ( args.length == 0 ) { println( "\nUsage:\t <VM> [-Dlips.master.{TRACE|DEBUG}={true|false}] lips.master.{cooperMarzullo|alagarVenkatesan|garg}.PossiblyTrue processTraceFile1 ... processTraceFileN\n\n" ); return; } if ( !processTraceFiles( args ) ) { return; } Runtime rt = Runtime.getRuntime(); _time = System.currentTimeMillis(); compute(); _time = System.currentTimeMillis() - _time; reportPossiblyTrue(); } protected GlobalStatePredicate getPredicate () { return _predicate; }

49

private ArrayList getProcessStatesForGlobalState( TimeStamp globalState ) { ArrayList processStates = new ArrayList(); StringBuffer sb = new StringBuffer (); sb.append( "\tprocess states for global state " + globalState.toString() + ": " ); sb.append( "[ "); for ( int pid = 1; pid <= globalState.size(); pid ++ ) { ProcessEvent e = _eventSequences[ pid - 1 ].getEventForID( globalState.getClockValue( pid ) ); if( e == null ) { sb.append( "null, " ); processStates.add( null ); } else { sb.append( e.getState() + ", " ); processStates.add( e.getState() ); } } sb.delete( sb.length() - 2, sb.length() - 1 ); sb.append( "]" ); if( DEBUG ) { println( sb.toString() ); } return processStates; } protected TimeStamp initialGlobalState( int numProcesses ) { TimeStamp state = new TimeStamp (); for( int pid = 1; pid <= numProcesses; pid ++ ) { state.setClockValue( 0, pid ); } return state; }

50

protected void reportPossiblyTrue () { StringBuffer sb = new StringBuffer (); sb.append( "\n--------------------------------------------\n" ); sb.append( "\n\tExecution Time:\t" + _time + " ms" ); sb.append( "\n\t# of CGS computed: " + _numCGS ); sb.append( "\n\tMax # of CGSs in memory: " + _maxInMemoryCGSs ); if( _currentCGS == null ) { sb.append( "\n\t--> No global state satisfies " + getPredicate() + "\n" ); } else { sb.append( "\n\t--> Found global state that satisfies " + getPredicate() + ":\n" ); sb.append( "\t[ "); for( int pid = 1; pid <= _currentCGS.size(); pid ++ ) { sb.append( _currentCGS.getClockValue( pid ) + ", " ); } sb.delete( sb.length() - 2, sb.length() - 1 ); sb.append( "]\n" ); } println( sb.toString() ); } protected boolean isCGS( TimeStamp state ) { boolean isGlobalState = true; TimeStamp [] localStates = new TimeStamp [ state.size() ]; ProcessEventSequence pes = null; // from each process event sequence, get the event where the clock value for that process has the same // value as in the state argument for that process for( int pid = 1; pid <= state.size(); pid++ ) { pes = _eventSequences[ pid - 1 ]; if( state.getClockValue( pid ) > pes.getMaxClockValueForProcess( pid ) )

51

{ isGlobalState = false; break; } localStates[ pid - 1 ] = pes.getEventForClockValue( pid, state.getClockValue( pid ) ).getTimeStamp(); } if( isGlobalState ) { for( int pid = 1; pid <= state.size(); pid++ ) { for( int localStateNdx = 0; localStateNdx < localStates.length; localStateNdx++ ) { if( localStateNdx == pid - 1 ) { continue; } if( localStates[ localStateNdx ].getClockValue( pid ) > state.getClockValue( pid ) ) { isGlobalState = false; } } if( !isGlobalState ) { break; } } } return isGlobalState; } public boolean applyPredicate( TimeStamp cgs ) { _numCGS++; ArrayList processStates = getProcessStatesForGlobalState( cgs ); return getPredicate().execute( processStates ); } abstract protected void compute ();

52

private boolean processTraceFiles( String [] args ) { println( "" ); _eventSequences = new ProcessEventSequence [ args.length ]; ProcessTraceReader ptr = new ProcessTraceReader(); int ndx = -1; try { for( ndx = 0; ndx < args.length; ndx ++ ) { _eventSequences [ ndx ] = ptr.processFile( args [ ndx ], args.length ); } } catch ( Exception e ) { println( "Exception while processing " + args [ ndx ] + ": " + e.getMessage() ); println( "\tTrace file format: state=s;vectorClock=int1,int2,int3" ); return false; } return true; } protected void println( String s ) { System.out.println( s ); } }

53

ONEBASEDINTVECTOR public class OneBasedIntVector implements Serializable { private Vector _v = null; public OneBasedIntVector() { _v = new Vector(); } public int size() { return _v.size(); } public void addElement( Object o ) { _v.addElement( o ); } public void setElementAt( Object e, int pos ) { _v.setElementAt( e, pos - 1 ); } public Object elementAt( int ndx ) { return _v.elementAt( ndx - 1 ); } public Enumeration elements() { return _v.elements(); } public Object remove( int ndx ) { return _v.remove( ndx ); } public Object clone()

54

{ OneBasedIntVector clone = new OneBasedIntVector(); Integer clonedInt = null; int originalInt = -1; for( int ndx = 0; ndx < _v.size(); ndx++ ) { originalInt = ( (Integer)_v.elementAt( ndx ) ).intValue(); clonedInt = new Integer( originalInt ); clone._v.addElement( clonedInt ); } return clone; } public String toString() { StringBuffer sb = new StringBuffer (); sb.append( "[ "); Enumeration enum = _v.elements(); while( enum.hasMoreElements() ) { sb.append( enum.nextElement() + ", " ); } sb.delete( sb.length() - 2, sb.length() - 1 ); sb.append( "]" ); return sb.toString(); } }

55

PROCESSEVENT public class ProcessEvent implements Serializable { private String _state = null; private TimeStamp _timeStamp = null; public ProcessEvent( String state, TimeStamp timeStamp ) { _state = state; _timeStamp = timeStamp; } public String getState() { return _state; } public TimeStamp getTimeStamp() { return _timeStamp; } public int getLevel () { return _timeStamp.getComponentSum(); } public String toString () { return "state=" + _state + ";vectorClock=" + _timeStamp.toString(); } }

56

PROCESSEVENTSEQUENCE public class ProcessEventSequence implements Serializable { private Vector _events = new Vector(); private int _numPoppedEvents = 0; public void pushEvent ( ProcessEvent event ) { _events.addElement( event ); } public ProcessEvent popEvent () { if ( _events.size () == 0 ) { return null; } _numPoppedEvents ++; return (ProcessEvent)_events.remove( 0 ); } public int size () { return _events.size(); } public int getLastEventLevel () { return getLastEvent().getLevel(); } public ProcessEvent getEventAt( int pos ) { if ( _events.size() < pos ) { return null; } return (ProcessEvent)_events.elementAt( pos ); } public ProcessEvent getFirstEvent ()

57

{ return (ProcessEvent)_events.elementAt( 0 ); } public ProcessEvent getLastEvent () { return (ProcessEvent)_events.elementAt( _events.size() - 1 ); } public ProcessEvent getEventForID( int id ) { return (ProcessEvent)_events.elementAt( id - _numPoppedEvents ); } public ProcessEvent getEventForClockValue( int pid, int val ) { ProcessEvent e = null; Enumeration enum = _events.elements(); while( enum.hasMoreElements() ) { e = (ProcessEvent)enum.nextElement(); if( e.getTimeStamp().getClockValue( pid ) == val ) { break; } else { e = null; } } return e; } public int getMaxClockValueForProcess( int pid ) { return ( (ProcessEvent)_events.lastElement() ).getTimeStamp().getClockValue( pid ); } }

58

TIMESTAMP public class TimeStamp implements Serializable { private OneBasedIntVector _timeStamp = null; public TimeStamp() { _timeStamp = new OneBasedIntVector (); } public void setClockValue( int n, int pid ) { setOrAddClockValue( new Integer ( n ), pid ); } public void setClockValue( String s, int pid ) { setOrAddClockValue( new Integer( s ), pid ); } public void setClockValue( Integer n, int pid ) { setOrAddClockValue( n, pid ); } private void setOrAddClockValue( Integer n, int pid ) { if ( _timeStamp.size() == pid - 1 ) { _timeStamp.addElement( n ); } else if ( _timeStamp.size() >= pid ) { _timeStamp.setElementAt( n, pid ); } } public int getClockValue( int pid ) { int clockValue = 0; if( pid <= _timeStamp.size() )

59

{ clockValue = ( (Integer)_timeStamp.elementAt( pid ) ).intValue(); } return clockValue; } public int getComponentSum () { int sum = 0; Enumeration e = _timeStamp.elements(); while( e.hasMoreElements() ) { sum += ( (Integer)e.nextElement() ).intValue(); } return sum; } public int getComponentCount () { return _timeStamp.size(); } public boolean greaterThan( TimeStamp ts ) { boolean gt = false; if ( _timeStamp.size() == ts.getComponentCount() ) { int thisClockValue = -1; int otherClockValue = -1; for ( int pid = 1; pid <= _timeStamp.size(); pid ++ ) { thisClockValue = this.getClockValue( pid ); otherClockValue = ts.getClockValue( pid ); if( thisClockValue < otherClockValue ) { gt = false; // if one clock value from the current clock is less than the other clock, // the current clock can't be greater than the other clock break; } else if( thisClockValue > otherClockValue )

60

{ gt = true; } } } return gt; } public boolean lessThan( TimeStamp ts ) { boolean lt = false; if ( _timeStamp.size() == ts.getComponentCount() ) { int thisClockValue = -1; int otherClockValue = -1; for ( int pid = 1; pid <= _timeStamp.size(); pid ++ ) { thisClockValue = this.getClockValue( pid ); otherClockValue = ts.getClockValue( pid ); if( thisClockValue > otherClockValue ) { lt = false; // if one clock value from the current clock is greater than the other clock, // the current clock can't be less than the other clock break; } else if( thisClockValue < otherClockValue ) { lt = true; } } } return lt; } public String toString() { StringBuffer sb = new StringBuffer(); Enumeration enum = _timeStamp.elements(); while( enum.hasMoreElements() ) {

61

sb.append( ( (Integer)enum.nextElement() ).toString() + "," ); } sb.delete( sb.length() - 1, sb.length() ); return sb.toString(); } public int size() { return _timeStamp.size(); } public Object clone () { TimeStamp clone = new TimeStamp(); clone._timeStamp = (OneBasedIntVector)_timeStamp.clone(); return clone; } public boolean equals( Object o ) { boolean equals = false; if( o instanceof TimeStamp ) { TimeStamp ts = (TimeStamp)o; if( _timeStamp.size() == ts.size() ) { int thisValue = -1; int otherValue = -1; for( int pid = 1; pid <= _timeStamp.size(); pid++ ) { thisValue = ( (Integer)_timeStamp.elementAt( pid ) ).intValue(); otherValue = ts.getClockValue( pid ); if( thisValue == otherValue ) { equals = true; continue; } else { equals = false;

62

break; } } } } return equals; } }

63

Source Code: Package lips.master.cooperMarzullo

POSSIBLYTRUE public class PossiblyTrue extends AbstractPossiblyTrue { private ArrayList _currentGlobalStates = null; public PossiblyTrue( GlobalStatePredicate gsp ) throws RemoteException { super( gsp ); } public static void main ( String [] args ) throws RemoteException { PossiblyTrue pt = new PossiblyTrue( new FalsePredicate() ); pt.compute( args ); } /* * current = initial global state * level = 0 * while ( !Predicate(each current state) ) * { * last = current; * current = getAllReachableStates( last ); * } * report possiblyTrue; */ protected void compute () { int numProcesses = _eventSequences.length; _currentGlobalStates = new ArrayList(); _currentGlobalStates.add( initialGlobalState( numProcesses ) ); ArrayList last = new ArrayList(); int level = 0; while ( null == ( super._currentCGS = applyPredicate( _currentGlobalStates ) ) ) {

64

last.clear(); last.addAll( _currentGlobalStates ); getAllReachableGlobalStates( last ); if ( _currentGlobalStates.size() == 0 ) { if( TRACE ) { println( "No more global states to construct." ); } break; } } } private void getAllReachableGlobalStates( ArrayList last ) { _currentGlobalStates.clear(); Iterator stateIt = last.iterator(); TimeStamp lastState = null; TimeStamp nextState = null; while( stateIt.hasNext() ) { lastState = (TimeStamp)stateIt.next(); if( DEBUG ) { println( "\t--> Processing last global state " + lastState.toString() ); } for( int pid = 1; pid <= lastState.size(); pid++ ) { nextState = (TimeStamp)lastState.clone(); nextState.setClockValue( nextState.getClockValue( pid ) + 1, pid ); if( DEBUG ) { println( "\t\tchecking next state " + nextState.toString() ); } if( isCGS( nextState ) ) {

65

addCurrentCGS( nextState ); } else { if( DEBUG ) { println( "\t\tNot a Consistent Global State" ); } } } } // drop events where state < global state dropEventsWithClockValueLessThan( globalStatesMinClockValues () ); } private void addCurrentCGS( TimeStamp cgs ) { boolean alreadyAdded = false; Iterator it = _currentGlobalStates.iterator(); TimeStamp ts = null; while( it.hasNext() ) { ts = (TimeStamp)it.next(); if( ts.equals( cgs ) ) { alreadyAdded = true; } } if( !alreadyAdded ) { if( TRACE ) { println( "\t*** Adding global state " + cgs.toString() ); } _currentGlobalStates.add( cgs ); } }

66

// remove all events whose process clock is less than minClockValue private void dropEventsWithClockValueLessThan( TimeStamp minClockValues ) { if( minClockValues != null ) { ProcessEventSequence pes = null; TimeStamp ts = null; int minClockValue = 0; for( int pid = 1; pid <= minClockValues.size(); pid ++ ) { pes = _eventSequences[ pid - 1 ]; minClockValue = minClockValues.getClockValue( pid ); ts = pes.getFirstEvent().getTimeStamp(); int eventMinClockValue = ts.getClockValue( pid ); if( eventMinClockValue < minClockValue ) { if( DEBUG ) { println( "\t\tDropping local state " + ts.toString() + " from process " + pid ); } pes.popEvent(); } } } } // get the lowest clock value from each process in the set of all current global states private TimeStamp globalStatesMinClockValues () { TimeStamp globalState = null; TimeStamp minClockValues = null; int currentValue = -1; int minValue = -1; Iterator it = _currentGlobalStates.iterator(); if ( it.hasNext() ) {

67

minClockValues = (TimeStamp)( (TimeStamp)it.next() ).clone(); while( it.hasNext() ) { globalState = (TimeStamp)it.next(); for( int pid = 1; pid <= globalState.size(); pid ++ ) { currentValue = globalState.getClockValue( pid ); minValue = minClockValues.getClockValue( pid ); if( currentValue < minValue ) { minClockValues.setClockValue( currentValue, pid ); } } } } return minClockValues; } private TimeStamp applyPredicate( ArrayList globalStates ) { _maxInMemoryCGSs = globalStates.size() > _maxInMemoryCGSs ? globalStates.size() : _maxInMemoryCGSs; TimeStamp cgs = null; ArrayList processStates = null; Iterator it = globalStates.iterator(); while( it.hasNext() ) { cgs = (TimeStamp)it.next(); if( applyPredicate( cgs ) ) { break; } cgs = null; } return cgs; } }

68

Source Code: Package lips.master.garg

POSSIBLYTRUE public class PossiblyTrue extends AbstractPossiblyTrue { public TimeStamp _maxEvents = null; public PossiblyTrue() { super(); } public PossiblyTrue( GlobalStatePredicate gsp ) { super( gsp ); } public static void main ( String [] args ) { PossiblyTrue pt = new PossiblyTrue( new FalsePredicate() ); pt.compute( args ); } protected void compute() { // don't need to compute this, as at any given point there's never more than 2 CGS from the global // state lattice in memory _maxInMemoryCGSs = 2; _currentCGS = initialGlobalState( _eventSequences.length ); _maxEvents = getMaxEvents(); while( !_currentCGS.greaterThan( _maxEvents ) ) { if( TRACE ) { println( "--> current CGS: " + _currentCGS.toString() ); } if( applyPredicate( _currentCGS) )

69

{ break; } if( _currentCGS.equals( _maxEvents ) ) { if( TRACE ) { println( "\t\tcurrent CGS equals maxEvents" ); } _currentCGS = null; break; } _currentCGS = nextLexCGS( _currentCGS ); } } public TimeStamp nextLexCGS( TimeStamp cgs ) { TimeStamp localCGS = (TimeStamp)cgs.clone(); int numProcesses = _eventSequences.length; int k = numProcesses; TimeStamp leastCGS = null; ProcessEvent e = null; boolean found = false; while( !found ) { if( localCGS.getClockValue( k ) != _maxEvents.getClockValue( k ) ) { e = _eventSequences[ k - 1 ].getEventForClockValue( k, localCGS.getClockValue( k ) + 1 ); if( DEBUG ) { println( "\t\tevaluating event " + e.getTimeStamp().toString() + " from process " + k ); } boolean enabled = true; for( int pid = 1; pid <= numProcesses; pid++ ) { if( pid == k ) { continue;

70

} if( e.getTimeStamp().getClockValue( pid ) > localCGS.getClockValue( pid ) ) { if( DEBUG ) { println( "\t\tcurrentCGS: " + localCGS.toString() + ", pid: " + pid ); println( "\t\tevent clock for process " + pid + ": " + e.getTimeStamp().getClockValue( pid ) ); println( "\t\tcurrent CGS event clock for process " + pid + ": " + localCGS.getClockValue( pid ) ); println( "\t\t--> event not enabled" ); } enabled = false; break; } } found = enabled; if( !found ) { k--; } else { if( DEBUG ) { println( "\t\tevent is enabled" ); } } } else { k--; } } localCGS.setClockValue( localCGS.getClockValue( k ) + 1, k ); for( int pid = k + 1; pid <= numProcesses; pid++ ) { localCGS.setClockValue( 0, pid );

71

} if( DEBUG ) { println( "\t\tpossible CGS: " + localCGS.toString() ); //println( "\t\t\tvalid: " + isCGS( localCGS ) ); } leastCGS = (TimeStamp)localCGS.clone(); int leastCGSVal = -1; int currentCGSVal = -1; for( int i = 1; i <= numProcesses; i++ ) { for( int j = 1; j <= numProcesses; j++ ) { leastCGSVal = leastCGS.getClockValue( j ); currentCGSVal = _eventSequences [ i - 1 ].getEventAt( localCGS.getClockValue( i ) ).getTimeStamp().getClockValue( j ); leastCGS.setClockValue( leastCGSVal >= currentCGSVal ? leastCGSVal : currentCGSVal, j ); } } if( DEBUG ) { println( "\t\tleast CGS: " + leastCGS.toString() ); } return leastCGS; } protected TimeStamp getMaxEvents() { TimeStamp maxEvents = new TimeStamp(); TimeStamp ts = null; int pid = 0; for( int ndx = 0; ndx < _eventSequences.length; ndx ++ ) { ts = _eventSequences[ ndx ].getLastEvent().getTimeStamp(); pid = ndx + 1; maxEvents.setClockValue( ts.getClockValue( pid ), pid ); } return maxEvents; }

72

Source Code: Package lips.master.garg.distributed

IPOSSIBLYTRUESERVER public interface IPossiblyTrueServer extends Remote { public void setLowerBound( TimeStamp lower ) throws RemoteException; public void setUpperBound( TimeStamp upper ) throws RemoteException; public void setMaxEvents( TimeStamp maxEvents ) throws RemoteException; public void setEventSequences( ProcessEventSequence[] seqs ) throws RemoteException; public Integer run() throws RemoteException; }

73

POSSIBLYTRUECLIENT public class PossiblyTrueClient extends PossiblyTrue { private ArrayList _servers = null; private TimeStamp [] _lowerBoundCGSs = null; private TimeStamp [] _upperBoundCGSs = null; private long [] _processEventCounterBase = null; private long _totalEvents = 0; public PossiblyTrueClient() { super(); } public static void main( String [] args ) { PossiblyTrueClient client = new PossiblyTrueClient(); ArrayList traceFiles = new ArrayList(); client._servers = new ArrayList(); for( int ndx = 0; ndx < args.length; ndx++ ) { if( args[ ndx ].endsWith( ".trace" ) ) { traceFiles.add( args[ ndx ] ); } else { client._servers.add( args[ ndx ] ); } } client._lowerBoundCGSs = new TimeStamp[ client._servers.size() ]; client._upperBoundCGSs = new TimeStamp[ client._servers.size() ]; // any given machine computing part of the global state lattice has 2 CGSs in memory, // so total # of CGSs is 2 * # of participating machines client._maxInMemoryCGSs = 2 * client._servers.size(); client.compute( (String[])traceFiles.toArray( new String[] {} )); }

74

protected void compute() { splitGlobalStateLattice(); if (System.getSecurityManager() == null) { System.setSecurityManager(new RMISecurityManager()); } ArrayList servers = new ArrayList(); try { Iterator it = _servers.iterator(); int ndx = 0; ArrayList threads = new ArrayList(); while( it.hasNext() ) { String name = "//" + (String)it.next() + "/PossiblyTrueServer"; if( TRACE ) { println( "server name: " + name ); } IPossiblyTrueServer srv = (IPossiblyTrueServer)Naming.lookup( name ); srv.setEventSequences( _eventSequences ); srv.setLowerBound( _lowerBoundCGSs[ ndx ] ); srv.setUpperBound( _upperBoundCGSs[ ndx ] ); srv.setMaxEvents( _maxEvents ); ServerRunner srvRunner = new ServerRunner(); srvRunner.setServer( srv ); Thread t = new Thread( srvRunner ); threads.add( t ); t.start(); ndx++; } Iterator threadIt = threads.iterator(); while( threadIt.hasNext() ) { ( (Thread)threadIt.next() ).join(); }

75

} catch ( Exception e ) { System.err.println("RemotePossiblyTrue Client exception: " + e.getMessage()); e.printStackTrace(); } } synchronized void incrNumCGS( int n ) { if( TRACE ) { println( "last numCGS: " + _numCGS + ", adding " + n ); } _numCGS += n; } private void splitGlobalStateLattice() { _maxEvents = getMaxEvents(); if( DEBUG ) { println( "splitGlobalStateLattice: _maxEvents = " + _maxEvents.toString() ); } _processEventCounterBase = new long[ _maxEvents.size() ]; _processEventCounterBase[ _maxEvents.size() - 1 ] = 1; for( int ndx = _maxEvents.size() - 2; ndx >= 0; ndx -- ) { long numEvents = _maxEvents.getClockValue( ndx + 2 ) * _processEventCounterBase[ ndx + 1 ]; if( DEBUG ) { println ("splitGlobalStateLattice: ndx = " + ndx ); println( "splitGlobalStateLattice: _maxEvents.getClockValue( ndx + 2 ): " + _maxEvents.getClockValue( ndx + 2 ) ); println( "splitGlobalStateLattice: processEventCounterBase[ ndx + 1 ]: " + _processEventCounterBase[ ndx + 1 ] ); println( "numEvents: " + numEvents );

76

} _processEventCounterBase[ ndx ] = numEvents; } _totalEvents = _maxEvents.getClockValue( _maxEvents.size() ); for( int pid = _maxEvents.size() - 1; pid >= 1; pid -- ) { _totalEvents = _totalEvents + ( _processEventCounterBase [ pid - 1 ] * _maxEvents.getClockValue( pid) ); if( DEBUG ) { println( "splitGlobalStateLattice: computing _totalEvents, current value: " + _totalEvents ); } } // split into subsets long numEventsPerSubset = _totalEvents / _servers.size(); if( DEBUG ) { println( "numEvents per subset: " + numEventsPerSubset ); } TimeStamp lowerBound = initialGlobalState( _eventSequences.length ); for( int ndx = 0; ndx < _servers.size(); ndx++ ) { _lowerBoundCGSs[ ndx ] = lowerBound; _upperBoundCGSs[ ndx ] = computeNextUpperBoundCGS( lowerBound, numEventsPerSubset ); if( TRACE ) { println( "lower bound CGS for process " + ndx + ": " + _lowerBoundCGSs[ ndx ].toString() ); println( "upper bound CGS for process " + ndx + ": " + _upperBoundCGSs[ ndx ].toString() ); } if( ndx < _servers.size() - 1 ) { lowerBound = nextLexCGS( (TimeStamp)_upperBoundCGSs[ ndx ].clone() ); } } }

77

private TimeStamp computeNextUpperBoundCGS( TimeStamp lowerBound, long numEvents ) { if( DEBUG ) { println( "computeNextUpperBoundCGS: lowerBound " + lowerBound + ", numEvents: " + numEvents ); } long lowerBoundNumEvents = 0; for( int pid = 1; pid <= lowerBound.size(); pid++ ) { long lbClockValue = lowerBound.getClockValue( pid ); long counterBase = _processEventCounterBase[ pid - 1 ]; lowerBoundNumEvents += (lbClockValue * counterBase ); } long upperBoundNumEvents = lowerBoundNumEvents + numEvents; if( DEBUG ) { println( "computeNextUpperBoundCGS: lowerBoundNumEvents = " + lowerBoundNumEvents ); println( "computeNextUpperBoundCGS: upperBoundNumEvents = " + upperBoundNumEvents ); } TimeStamp nextUpperBound = new TimeStamp(); if( upperBoundNumEvents > _totalEvents ) { nextUpperBound = _maxEvents; } else { long numProcessEvents = 0; for( int pid = 1; pid < _processEventCounterBase.length; pid++ ) { numProcessEvents = upperBoundNumEvents / _processEventCounterBase[ pid - 1 ]; if( DEBUG ) { println( "computeNextUpperBoundCGS: upperBoundNumEvents: " + upperBoundNumEvents + ", processEventCounterBase: " + _processEventCounterBase[ pid - 1 ] );

78

println( "computeNextUpperBoundCGS: setting clock value for pid " + pid + " to numProcessEvents " + numProcessEvents ); } nextUpperBound.setClockValue( (int)numProcessEvents, pid ); long n = _processEventCounterBase[ pid - 1 ]; long nn = numProcessEvents * n; upperBoundNumEvents -= nn; } nextUpperBound.setClockValue( (int)upperBoundNumEvents, _processEventCounterBase.length ); if( DEBUG ) { println( "computeNextUpperBoundCGS: checking if " + nextUpperBound + " is CGS ..." ); } if( !isCGS( nextUpperBound ) ) { nextUpperBound = nextLexCGS( nextUpperBound ); } } return nextUpperBound; } public class ServerRunner implements Runnable { IPossiblyTrueServer _srv = null; public void setServer( IPossiblyTrueServer srv ) { _srv = srv; } public void run() { try { incrNumCGS( _srv.run().intValue() ); } catch ( RemoteException re ) { System.err.println( re.getMessage() ); re.printStackTrace( System.err ); } } }

79

}

80

POSSIBLYTRUESERVER public class PossiblyTrueServer extends UnicastRemoteObject implements IPossiblyTrueServer { public static final String TRACE_SETTING = "lips.master.TRACE"; public static final String DEBUG_SETTING = "lips.master.DEBUG"; public boolean TRACE = false; public boolean DEBUG = false; private TimeStamp _lowerBound = null; private TimeStamp _upperBound = null; private PossiblyTrue _possiblyTrue = null; public PossiblyTrueServer() throws RemoteException { super(); setTraceOrDebug(); _possiblyTrue = new PossiblyTrue( new FalsePredicate() ); } public Integer run() throws RemoteException { println( "\n\n------------------------\n\n" ); println( "Computing CGSs for lower bound " + _lowerBound + " and upper bound " + _upperBound ); if ( _possiblyTrue.DEBUG ) { println( "maxEvents: " + _possiblyTrue._maxEvents.toString() ); println( "lower bound: " + _lowerBound.toString() ); println( "upper bound: " + _upperBound.toString() ); } int numCGS = 1; _possiblyTrue._currentCGS = _lowerBound; while( !_possiblyTrue._currentCGS.greaterThan( _possiblyTrue._maxEvents ) ) { if( _possiblyTrue.TRACE ) { println( "--> current CGS: " + _possiblyTrue._currentCGS.toString() ); }

81

if( _possiblyTrue.applyPredicate( _possiblyTrue._currentCGS) ) { break; } if( _possiblyTrue._currentCGS.equals( _upperBound ) ) { if( _possiblyTrue.TRACE ) { println( "\t\tcurrent CGS equals upperBound" ); } _possiblyTrue._currentCGS = null; break; } _possiblyTrue._currentCGS = _possiblyTrue.nextLexCGS( _possiblyTrue._currentCGS ); numCGS++; } if( TRACE ) { println( "computed " + numCGS + " CGSs" ); } return new Integer( numCGS ); } public void setLowerBound( TimeStamp lower ) { _lowerBound = lower; } public void setUpperBound( TimeStamp upper ) { _upperBound = upper; } public void setMaxEvents( TimeStamp maxEvents ) { _possiblyTrue._maxEvents = maxEvents; } public void setEventSequences( ProcessEventSequence[] seqs ) {

82

_possiblyTrue._eventSequences = new ProcessEventSequence[ seqs.length ]; for( int ndx = 0; ndx < seqs.length; ndx++ ) { _possiblyTrue._eventSequences[ ndx ] = seqs[ ndx ]; } } public static void main ( String [] args ) { if (System.getSecurityManager() == null) { System.setSecurityManager(new RMISecurityManager()); } String name = "//" + args[ 0 ] + "/PossiblyTrueServer"; try { PossiblyTrueServer srv = new PossiblyTrueServer(); Naming.rebind(name, srv); System.out.println("PossiblyTrue Server bound"); } catch ( Exception e ) { System.err.println("PossiblyTrue Server exception: " + e.getMessage()); e.printStackTrace(); } } private void println( String s ) { System.out.println( s ); } private void setTraceOrDebug() { TRACE = Boolean.valueOf( System.getProperty( TRACE_SETTING ) ).booleanValue(); DEBUG = Boolean.valueOf( System.getProperty( DEBUG_SETTING ) ).booleanValue(); if( DEBUG ) { TRACE = true;

83

} } }

84

Source Code: Package lips.master.predicates

FALSEPREDICATE public class FalsePredicate implements GlobalStatePredicate { public boolean execute(ArrayList states) { return false; } public String toString () { return this.getClass().getName(); } }

85

GLOBALSTATEPREDICATE public interface GlobalStatePredicate extends Serializable { /* * @param states ArrayList of state values from each process for associated global state. */ public boolean execute( ArrayList states ); public String toString (); }

86

INTEGERAVERAGEPREDICATE public class IntegerAveragePredicate implements GlobalStatePredicate { private boolean TRACE = false; private boolean DEBUG = false; private int _avg = -1; public IntegerAveragePredicate( String avg ) { _avg = Integer.parseInt( avg ); TRACE = Boolean.valueOf( System.getProperty( AbstractPossiblyTrue.TRACE_SETTING ) ).booleanValue(); DEBUG = Boolean.valueOf( System.getProperty( AbstractPossiblyTrue.DEBUG_SETTING ) ).booleanValue(); if( DEBUG ) { TRACE = true; } } public boolean execute( ArrayList states ) { boolean predicateEval = false; int sum = 0; String state = null; Iterator it = states.iterator(); while( it.hasNext () ) { state = (String)it.next(); if( state != null ) { sum += Integer.parseInt( (String)state ); } } if ( sum / states.size() == _avg ) { predicateEval = true; } if( TRACE )

87

{ StringBuffer sb = new StringBuffer (); sb.append( "\tIntegerAveragePredicate: " ); if( predicateEval ) { sb.append( "TRUE" ); } else { sb.append( "FALSE" ); } sb.append ( "\t[" + sum + " / " + states.size() + " = " + sum / states.size() + "; predicate average: " + _avg + "]" ); System.out.println( sb.toString() ); } return predicateEval; } public String toString () { return this.getClass().getName() + "[" + _avg + "]"; } }

88

Source Code: Package lips.master.util

PROCESSTRACEREADER public class ProcessTraceReader { private String STATE_TIMESTAMP_DELIMITER = ";"; private String NAME_VALUE_DELIMITER = "="; private String TIMESTAMP_ENTRY_DELIMITER = ","; public ProcessEventSequence processFile(String file, int numProcesses) throws Exception { System.out.println( "input file: " + file ); ProcessEventSequence stateSequence = new ProcessEventSequence (); stateSequence.pushEvent( nullState( numProcesses ) ); LineNumberReader lnr = new LineNumberReader(new FileReader(file)); String line = null; while ((line = lnr.readLine()) != null) { stateSequence.pushEvent ( processLine(line) ); } return stateSequence; } private ProcessEvent nullState( int numProcesses ) { TimeStamp ts = new TimeStamp (); for( int pid = 1; pid <= numProcesses; pid++ ) { ts.setClockValue( 0, pid ); } return new ProcessEvent( null, ts ); } private ProcessEvent processLine( String line ) throws Exception { StringTokenizer st = new StringTokenizer( line, STATE_TIMESTAMP_DELIMITER ); if ( st.countTokens() != 2 )

89

{ throw new Exception ( "Malformed process trace: missing state or time stamp."); } String stateToken = st.nextToken(); String timeStampToken = st.nextToken (); StringTokenizer stateTokenizer = new StringTokenizer( stateToken, NAME_VALUE_DELIMITER ); if ( stateTokenizer.countTokens () != 2 ) { throw new Exception( "Malformed process trace: missing state identifier or value."); } stateTokenizer.nextToken(); //discard String state = stateTokenizer.nextToken(); StringTokenizer timeStampTokenizer = new StringTokenizer( timeStampToken, NAME_VALUE_DELIMITER ); if ( timeStampTokenizer.countTokens () != 2 ) { throw new Exception( "Malformed process trace: missing time stamp identifier or value."); } timeStampTokenizer.nextToken (); //discard StringTokenizer timeStampValueTokenizer = new StringTokenizer( timeStampTokenizer.nextToken (), TIMESTAMP_ENTRY_DELIMITER ); TimeStamp timeStamp = new TimeStamp (); int pid = 1; while ( timeStampValueTokenizer.hasMoreTokens() ) { timeStamp.setClockValue( timeStampValueTokenizer.nextToken(), pid ); pid ++; } return new ProcessEvent( state, timeStamp ); } }

90

Source Code: Package lips.master.util.generator

ABSTRACTTRACEFILEGENERATOR abstract public class AbstractTraceFileGenerator { protected PrintWriter [] _writers = null; protected Process [] _processes = null; private int _numProcesses = 0; private int _numEvents = 0; abstract protected void generate (); protected void generate( String [] args ) { if ( args.length == 0 ) { println( "\nUsage:\t <VM> lips.master.util.TraceFileGenerator numTraceFiles numEvents [fileLocation]\n\n" ); return; } _numProcesses = Integer.parseInt( args[ 0 ] ); _numEvents = Integer.parseInt( args[ 1 ] ); String location = null; String filename = null; File file = null; try { if( args.length == 3 ) { location = args[ 3 ]; } else { location = new File( "." ).getCanonicalPath(); } location = location + System.getProperty( "file.separator" );

91

_writers = new PrintWriter[ _numProcesses ]; for( int ndx = 1; ndx <= _numProcesses; ndx++ ) { filename = location + "process_" + ndx + ".trace"; file = new File( filename ); _writers[ ndx - 1 ] = new PrintWriter ( new FileWriter ( file, /*append*/false ), /*autoFlush*/ true ); } // create all processes _processes = new Process[ getNumProcesses() ]; for( int pid = 1; pid <= getNumProcesses(); pid++ ) { _processes[ pid - 1 ] = new Process( pid, getNumProcesses() ); } // generate the event sequences generate(); // write the event sequences to file for( int pid = 1; pid <= getNumProcesses(); pid++ ) { ProcessEventSequence pes = _processes[ pid - 1 ].getProcessEventSequence(); for( int ndx = 0; ndx < pes.size(); ndx++ ) { _writers[ pid - 1 ].println( pes.getEventAt( ndx ).toString() ); } _writers[ pid - 1 ].close(); } } catch( Exception e ) { println( e.getMessage() ); e.printStackTrace(); } } protected int getNumProcesses() {

92

return _numProcesses; } protected int getNumEvents() { return _numEvents; } protected static void println( String s ) { System.out.println( s ); } }

93

INTERNALEVENTSTRACEFILEGENERATOR public class InternalEventsTraceFileGenerator extends AbstractTraceFileGenerator { public static void main(String[] args) { InternalEventsTraceFileGenerator g = new InternalEventsTraceFileGenerator(); g.generate( args ); } protected void generate() { int numProcesses = getNumProcesses(); int numEvents = getNumEvents(); for( int pid = 1; pid <= numProcesses; pid++ ) { for( int evntNum = 1; evntNum <= numEvents; evntNum++ ) { _processes[ pid - 1 ].internalEvent(); } } } }

94

MONITORPROCESSTRACEFILEGENERATOR public class MonitorProcessTraceFileGenerator extends AbstractTraceFileGenerator { public static void main(String[] args) { MonitorProcessTraceFileGenerator g = new MonitorProcessTraceFileGenerator(); g.generate( args ); } protected void generate() { int numGeneratedEvents = 0; int numEvents = getNumEvents(); int numProcesses = getNumProcesses(); Process [] monitoredProcesses = new Process[ numProcesses - 1 ]; for( int pid = 1; pid < numProcesses; pid ++ ) { monitoredProcesses[ pid - 1 ] = _processes[ pid ]; } while( numGeneratedEvents <= numEvents * numProcesses ) { // every process sends an event to the monitoring process for( int pid = 1; pid < numProcesses; pid++ ) { _processes[ pid ].sendEvent( _processes[ 0 ] ); numGeneratedEvents += 2; // send and receive events are distinct if( numGeneratedEvents > numEvents * numProcesses ) { break; } } if( numGeneratedEvents > numEvents * numProcesses ) { break; }

95

// the monitoring process broadcasts an event to every monitored process _processes[ 0 ].broadcastEvent( monitoredProcesses ); numGeneratedEvents += numProcesses; // each process has one event during broadcase: 1 send, n - 1 receive events } } }

96

PROCESS public class Process { private TimeStamp _timeStamp = null; private String _state = null; private int _pid = -1; private int _numProcesses = -1; private ProcessEventSequence _pes = null; private Random _random = null; public Process( int pid, int numProcesses ) { _pid = pid; _numProcesses = numProcesses; _random = new Random( System.currentTimeMillis() ); _timeStamp = new TimeStamp(); _pes = new ProcessEventSequence(); for( int ndx = 1; ndx <= numProcesses; ndx++ ) { _timeStamp.setClockValue( 0, ndx ); } } public void internalEvent() { nextProcessEvent(); } public void sendEvent( final Process recvr ) { ProcessEvent e = nextProcessEvent(); recvr.receiveEvent( e ); } public void receiveEvent( final ProcessEvent e ) { int myVal = -1; int otherVal = -1; TimeStamp recvdTS = e.getTimeStamp(); for( int pid = 1; pid <= _numProcesses; pid++ ) {

97

if( pid == _pid ) { _timeStamp.setClockValue( _timeStamp.getClockValue( _pid ) + 1, _pid ); } else { myVal = _timeStamp.getClockValue( pid ); otherVal = recvdTS.getClockValue( pid ); _timeStamp.setClockValue( myVal >= otherVal ? myVal : otherVal, pid ); } } _state = "" + _random.nextInt( 100 ); ProcessEvent localEvent = new ProcessEvent( new String( _state ), (TimeStamp)_timeStamp.clone() ); _pes.pushEvent( localEvent ); } public void broadcastEvent( final Process[] recvrs ) { ProcessEvent e = nextProcessEvent(); for( int ndx = 0; ndx < recvrs.length; ndx++ ) { recvrs[ ndx ].receiveEvent( e ); } } public ProcessEventSequence getProcessEventSequence() { return _pes; } private ProcessEvent nextProcessEvent() { _timeStamp.setClockValue( _timeStamp.getClockValue( _pid ) + 1, _pid ); _state = "" + _random.nextInt( 100 ); ProcessEvent e = new ProcessEvent( new String( _state ), (TimeStamp)_timeStamp.clone() ); _pes.pushEvent( e ); return e; } }

98

SEQUENTIALMESSAGETRACEFILEGENERATOR public class SequentialMessageTraceFileGenerator extends AbstractTraceFileGenerator { public static void main(String[] args) { SequentialMessageTraceFileGenerator gen = new SequentialMessageTraceFileGenerator(); gen.generate( args ); } public void generate() { int numProcesses = getNumProcesses(); int numGeneratedEvents = 0; int numEvents = getNumEvents() * numProcesses; while( numGeneratedEvents <= numEvents ) { for( int pid = 1; pid < numProcesses; pid++ ) { _processes[ pid - 1 ].sendEvent( _processes[ pid ] ); numGeneratedEvents += 2; if( numGeneratedEvents > numEvents ) { break; } } } } }

99

Glossary

CGS: Consistent Global State

BFS: Breadth-First Search, in the context of this report this refers to an algorithm originally proposed by Cooper/Marzullo [2].

DFS: Depth-First Search, in the context of this report this refers to an algorithm originally proposed by Alagar/Venkatesan [1].

100

References

1. S. Alagar and S. Venkatesan. Techniques to tackle state explosion in global predicate detection. IEEE Transactions on Software Engineering, 27(8):704-714, August 2001.

2. R. Cooper and K. Marzullo. Consistent Detection of Global Predicates. In Proc. of the Workshop on Parallel and Distributed Debugging, pages 163-173, Santa Cruz, CA, May 1991.

3. V. K. Garg. Enumerating Global States of a Distributed Computation in Lexicographic and Breadth-First Manner. International Conference on Parallel and Distributed Computing and Systems (PDCS 2003)

4. V. K. Garg. Elements of Distributed Computing, Wiley and Sons, New York, 2002.

5. V. K. Garg and J.R. Mitchell. Detecting Conjunctions of Global Predicates. Information Processing Letters pages 295-302, Volume 63, Number 6, 29 September 1997.

6. F. Mattern. Virtual Time and Global States of Distributed Systems. In Parallel and Distributed Algorithms: Proceedings of the International Workshop on Parallel and Distributed Algorithms, pages 215-226. Elsevier Science Publishers B.V. (North Holland), 1989.

7. The Java Programming Language, http://java.sun.com/j2se/1.4.1

8. Eclipse Integrated Development Environment, http://www.eclipse.org/

9. S. Metsker. Design Patterns Java Workbook, Addison-Wesley, New York, 2002.

10. Java Remote Method Invocation, http://java.sun.com/docs/books/tutorial/rmi/index.html

11. K. Chandy and L. Lamport. Distributed Snapshots: Determining Global States of Distributed Systems. ACM Trans. Computer Systems, vol. 3, no. 1, pp. 63-75, 1985

101

12. J. Fidge. Timestamps in Message Passing Systems that Preserve the Partial Ordering. Proc. 11th Australian Computer Science Conf., pp. 55-66, 1988

13. L. Lamport. Time, Clocks and the Ordering of Events in a Distributed System. Comm. ACM, vol. 21, no. 7, pp558-656, 1978

102

Vita

The author was born in in Heidelberg, Germany, on June 12, 1968, to

Peter and Traudel Lips. He received a Master Degree in Music Science with

minors in business administration and law from the Friedrich Ludwigs University

in Freiburg, Germany, in December 1993. He then emigrated to the United

States, to pursue a career playing music professionally for several years before

developing an interest in software development. This newfound interest led him

to switch his career to development of commercial software. He has held project

management, quality engineering and software development position with Origin

Systems, Trilogy Software, and Vincera Software. His current position at Troux

Technologies as a Software Engineer involves management of the Quality

Engineering group, Build Management and improvement of the development

process. The author is a member of the IEEE Computer Society.

Permanent address: 7200 Holly Fern Cove, Austin, TX 78750

This report was typed by the author.