Performance of Performance of Token-based Token-based

Distributed Mutual Distributed Mutual Exclusion Exclusion

AlgorithmsAlgorithmsScott J. McCallenScott J. McCallen

Kent State UniversityKent State University

November 28 2006November 28 2006

Token-based AlgorithmsToken-based Algorithms

A single token circulates through the A single token circulates through the systemsystem

A process wants to enter CS, it A process wants to enter CS, it requests tokenrequests token

A process gets the token, it can enter A process gets the token, it can enter CS (in general)CS (in general)

Message TypesMessage Types TokenToken RequestRequest

Raymond’s Token DMXRaymond’s Token DMX

Based on a tree structureBased on a tree structure Each node holds a pointer to a node Each node holds a pointer to a node

that either holds the token or points that either holds the token or points to where the token isto where the token is

A request is held locally and is sent A request is held locally and is sent through the tree until it reaches the through the tree until it reaches the tokentoken

Token traverses the request path until Token traverses the request path until it reaches the requesterit reaches the requester

Suzuki-Kasami Token Suzuki-Kasami Token DMXDMX

Fully connected networkFully connected network Requester broadcasts requests to all Requester broadcasts requests to all

other nodes with sequence number of other nodes with sequence number of requestrequest

Before sending the token, holder Before sending the token, holder calculates which requests have not been calculates which requests have not been satisfied and enters those into the token satisfied and enters those into the token request queuerequest queue

Send token to the request at the top of Send token to the request at the top of the queue the queue

Initial Verification Initial Verification ExperimentsExperiments

SetupSetup Done to ensure correctness and begin Done to ensure correctness and begin

experimentationexperimentation AttributesAttributes

Synchronization DelaySynchronization Delay Number of messages between each CS entryNumber of messages between each CS entry

Message ComplexityMessage Complexity Total number of messages per CS entryTotal number of messages per CS entry

VariablesVariables Network sizeNetwork size

Range from 5 to 30 for Suzuki-Kasami (why?)Range from 5 to 30 for Suzuki-Kasami (why?) Range from 5 to 60 for Raymond (why?)Range from 5 to 60 for Raymond (why?)

Contention levelContention level low (sequential request/release) and high (all low (sequential request/release) and high (all

request/all release)request/all release)

Suzuki-Kasami Results Suzuki-Kasami Results (Low Load)(Low Load)

Suzuki-Kasami Results Suzuki-Kasami Results (High Load)(High Load)

Raymond Results (Low Raymond Results (Low Load)Load)

Raymond Results (High Raymond Results (High Load)Load)

Results SummaryResults Summary RaymondRaymond

Low LoadLow Load Message Complexity : log(n)/log(2) [not log(n)/2]Message Complexity : log(n)/log(2) [not log(n)/2] Synchronization Delay : 2*log(n)/log(2) [not log(n)]Synchronization Delay : 2*log(n)/log(2) [not log(n)]

High LoadHigh Load Message Complexity : 4Message Complexity : 4 Synchronization Delay : 2Synchronization Delay : 2

Suzuki-KasamiSuzuki-Kasami Low LoadLow Load

Message Complexity : NMessage Complexity : N Synchronization Delay : 1Synchronization Delay : 1

High LoadHigh Load Message Complexity : NMessage Complexity : N Synchronization Delay: 1Synchronization Delay: 1

Raymond’s ChildrenRaymond’s Children

Message Complexity in low load was Message Complexity in low load was shown to be similar to log(n)/log(2)shown to be similar to log(n)/log(2)

In initial experiments, each node had In initial experiments, each node had at most 2 childrenat most 2 children

How does the number of children How does the number of children affect the synchronization delay for affect the synchronization delay for Raymond’s DMX?Raymond’s DMX?

Raymond’s Children Raymond’s Children ExperimentExperiment

SetupSetup AttributesAttributes

Synchronization DelaySynchronization Delay VariablesVariables

Number of children per nodeNumber of children per node Range from 1 to 8 (why?)Range from 1 to 8 (why?)

ParametersParameters 50 nodes in the network (why?)50 nodes in the network (why?) 40 sequential request/releases (low 40 sequential request/releases (low

contention)contention)

ResultsResults Increasing the number of children per node Increasing the number of children per node

decreases the message complexitydecreases the message complexity

Suzuki-Kasami Average Token Suzuki-Kasami Average Token ForwardingForwarding

In high contention, nodes are put on the In high contention, nodes are put on the queue based on node identificationqueue based on node identification

A token is then released to the top A token is then released to the top request on the queuerequest on the queue

What happens when a node has the last What happens when a node has the last request and what is the average of request and what is the average of message complexity and number of token message complexity and number of token forwards before CS is allow?forwards before CS is allow?

Forwarding ExperimentsForwarding ExperimentsSetupSetup

AttributesAttributes Message Complexity + Token Forwards Message Complexity + Token Forwards

until CSuntil CS VariablesVariables

Network size Network size Range from 5 to 30 by 5Range from 5 to 30 by 5

ParametersParameters Nodes – 1 requestsNodes – 1 requests

Suzuki-Kasami ResultsSuzuki-Kasami Results Based on the number of nodes in the network, the result is Based on the number of nodes in the network, the result is

proportional to [(nodes*requests)+sum(1:requests)]/nodesproportional to [(nodes*requests)+sum(1:requests)]/nodes

ConclusionsConclusions

RaymondRaymond Better performance with many childrenBetter performance with many children Log(n)/log(c) message complexity and Log(n)/log(c) message complexity and

sync delaysync delay Suzuki KasamiSuzuki Kasami

Very low sync delayVery low sync delay Steady increase in total token forwards Steady increase in total token forwards

for high contention systemfor high contention system Neither algorithm guarantees Neither algorithm guarantees

granting CS in timestamp ordergranting CS in timestamp order

Future WorkFuture Work Test systems in continuous high-contentionTest systems in continuous high-contention

Current experiments are all-request/all-release, Current experiments are all-request/all-release, continuous high-contention may produce alternate resultscontinuous high-contention may produce alternate results

Test Raymond’s algorithm with random treeTest Raymond’s algorithm with random tree It may not be possible to create a tree with specific It may not be possible to create a tree with specific

children count, what results will be produced in the children count, what results will be produced in the random caserandom case

Test algorithms in a real (non-random) situationTest algorithms in a real (non-random) situation Perhaps there are tree algorithms which require high-Perhaps there are tree algorithms which require high-

contention critical section access within a small subgraphcontention critical section access within a small subgraph Implement message piggy-backing as suggested Implement message piggy-backing as suggested

by Raymondby Raymond Could reduce costs of duplicate messages for high-Could reduce costs of duplicate messages for high-

contention system, but should not affect low contentioncontention system, but should not affect low contention

Coding the ProjectCoding the Project ProblemsProblems

Message SizeMessage Size Limited the number of nodes for Suzuki-KasamiLimited the number of nodes for Suzuki-Kasami

Request QueueRequest Queue Incrementing lengthIncrementing length Added methods to check contents of queueAdded methods to check contents of queue copyTokenToOut didn’t copy all informationcopyTokenToOut didn’t copy all information

Knowing which node had the tokenKnowing which node had the token Required sending messages to all nodes to releaseRequired sending messages to all nodes to release

Token lossToken loss Token was lost in some tests making that test Token was lost in some tests making that test

erroneouserroneous No token loss check algorithm in place for these testsNo token loss check algorithm in place for these tests

Coding the ProjectCoding the Project

SuccessesSuccesses AlgorithmsAlgorithms

Implementation was mostly successfulImplementation was mostly successful Tython scriptsTython scripts

Learning Python rather quicklyLearning Python rather quickly Scripts can run arbitrary number of trialsScripts can run arbitrary number of trials Outputting results to file for access laterOutputting results to file for access later Scripts can be ran unattended (except changing Scripts can be ran unattended (except changing

network size)network size) MATLABMATLAB

Utilized to create graphicsUtilized to create graphics

ReferencesReferences Brewer, E., Culler, D., Gay, D., and Levis, P., Brewer, E., Culler, D., Gay, D., and Levis, P., nesC 1.1 Language nesC 1.1 Language

Reference ManualReference Manual, www.tinyos.net/tinyos-1.x/doc/nesc/ref.pdf, May , www.tinyos.net/tinyos-1.x/doc/nesc/ref.pdf, May 2003. 2003.

Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., and Culler, D. Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., and Culler, D. 2003. 2003. The The nesCnesC language: A holistic approach to networked language: A holistic approach to networked embedded systemsembedded systems. . SIGPLAN Not.SIGPLAN Not. 38, 5 (May. 2003), 1-11. 38, 5 (May. 2003), 1-11.

Hill, J., Szewczyk, R., Woo, A., Hollar, S., Culler, D., and Pister, K. 2000. Hill, J., Szewczyk, R., Woo, A., Hollar, S., Culler, D., and Pister, K. 2000. System architecture directions for networked sensorsSystem architecture directions for networked sensors. . SIGOPS Oper. SIGOPS Oper. Syst. Rev.Syst. Rev. 34, 5 (Dec. 2000), 93-104. 34, 5 (Dec. 2000), 93-104.

Levis, P., Lee, N., Welsh, M., and Culler, D. 2003. Levis, P., Lee, N., Welsh, M., and Culler, D. 2003. TOSSIM: accurate TOSSIM: accurate and scalable simulation of entire tinyOS applicationsand scalable simulation of entire tinyOS applications. In . In Proceedings of the 1st international Conference on Embedded Networked Proceedings of the 1st international Conference on Embedded Networked Sensor SystemsSensor Systems (Los Angeles, California, USA, November 05 - 07, 2003). (Los Angeles, California, USA, November 05 - 07, 2003). SenSys '03. ACM Press, New York, NY, 126-137. SenSys '03. ACM Press, New York, NY, 126-137.

Raymond, K. 1989. Raymond, K. 1989. A tree-based algorithm for distributed mutual A tree-based algorithm for distributed mutual exclusionexclusion. . ACM Trans. Comput. Syst.ACM Trans. Comput. Syst. 7, 1 (Jan. 1989), 61-77. 7, 1 (Jan. 1989), 61-77.

Suzuki, I. and Kasami, T. 1985. Suzuki, I. and Kasami, T. 1985. A distributed mutual exclusion A distributed mutual exclusion algorithmalgorithm. . ACM Trans. Comput. Syst.ACM Trans. Comput. Syst. 3, 4 (Nov. 1985), 344-349. 3, 4 (Nov. 1985), 344-349.