Performance Evaluation of a Discovery and Scheduling ...endler/paperlinks/Tadeu-Luciana-jbcs-09.pdf · Performance Evaluation of a Discovery and Scheduling Protocol for Multihop Ad

Performance Evaluation of aDiscovery and Scheduling Protocolfor Multihop Ad Hoc Mobile Grids

Antônio Tadeu A. Gomes1, Artur Ziviani 1, Luciana S. Lima2, and Markus Endler3

1National Laboratory for Scientific Computing (LNCC)Av. Getúlio Vargas 333, Petrópolis-RJ, 25651-075, Brazil

{atagomes | ziviani}@lncc.br

2Nokia Siemens Networks (NSN)Av. das Américas 3434 - Bloco 7- 3o. andar, Rio de Janeiro-RJ,22640-102, Brazil

[email protected]

3Pontifical Catholic University of Rio de Janeiro (PUC-Rio)Rua Marquês de São Vicente 225, Rio de Janeiro-RJ, 22453-900, Brazil

[email protected]

AbstractDespite the many research efforts addressing the in-

tegration of mobile nodes into grids, only a few of themhave considered the establishment of mobile grids overwireless ad hoc networks (hereafter, mobile ad hoc grids).Clearly, such grids need specialized resource discoveryand scheduling mechanisms. To the best of our knowl-edge, though, the research on these mechanisms for mo-bile ad hoc grids is still preliminary. Besides, and moreimportantly, it has approached discovery and schedulingas separate mechanisms, which, we argue, is not suit-able for mobile ad hoc grids. In this paper, we pro-pose the integration of resource discovery and schedul-ing for mobile ad hoc grids into a single protocol calledDICHOTOMY (DIscovery and sCHeduling prOTOcol forMobilitY). This protocol allows computational tasks to bedistributed appropriately in a mobile ad hoc grid, whilemitigating the overhead of discovery messages exchangedamong the nodes. Our experiments show that the proto-col: (i) does proper scheduling, allowing an efficient loadbalancing among the nodes and helping with lowering theaverage completion time of tasks; (ii) keeps the discoveryefficiency at acceptable levels in mobility scenarios; and(iii ) scales very well with respect to an increasing numberof nodes, both in the total amount of energy savings due topacket transmissions and the distribution of such savingsamong the nodes.

Keywords: Mobile Grids, Resource Management,Self-Organizing Networks

1. INTRODUCTIONThere has been an increasing amount of research over

the past few years on wireless grids [37]. These ex-tend traditional grids [2] by integrating mobile devicesin a wireless infrastructured network as either resourceconsumers or providers in the computing infrastructure.Only a few efforts [32, 46], however, have addressedthe more challenging issue of dynamically establishingspontaneous,purely mobile ad hoc grids—i.e. grids es-tablished over mobile ad hoc networks—, and even thesehave achieved only preliminary results. Further devel-opment in mobile ad hoc grids can be justified by someapplications that demand high computational power butat the same time have to be used at places or conditionswhere network infrastructure may be—or may suddenlybecome—unavailable [21]. In particular, we focus on ap-plication scenarios in which the mobility of devices isconstrained to pedestrian (walking) speeds. Examples ofsuch scenarios include:

• Emergency response systemsfor disaster handlingand crisis management situations. For instance, con-sider a large rescue and medical team working in a

Antônio Tadeu A. Gomes, Artur Ziviani, LucianaS. Lima, and Markus Endler

Perfor. Eval. of a Discovery and SchedulingProtocol for Multihop Ad Hoc Mobile Grids

natural disaster scenario, such as the one caused bythe Indian Ocean tsunami in Dec. 2004 or the hur-ricane Katrina that hit New Orleans in Aug. 2005.In such scenarios, the seamless integration of com-putational resources from on-site mobile nodes canbe crucial for rapidly achieving advanced forms ofcollaborative work, as for example to collect and au-tomatically process information about groups of in-jured people (e.g.for triage) and thus better allocaterescue teams and medical resources [13]. In this con-text, mobile ad hoc grids can be regarded as spe-cializations of the more general concept of hastilyformed networks [9].

• Field research systemsdynamically deployed on iso-lated areas. For instance, consider a team of hydro-geologists throughout a large semi-arid region, suchas the Northeastern Brazil. Such a team can ex-change data about underground water resources (e.g.hydraulic head) being collected by their mobilenodes, and may use the computational resources ofsuch nodes forin-loco, preliminary numerical analy-sis on the collected data, so as to simulate and predictsome aquifer condition of interest (e.g.the effect ofirrigation developments).

Both aforementioned application scenarios demandthe formation ofmultihopmobile ad hoc networks (here-after, MANETs). MANETs allow mobile nodes to self-organize into arbitrary and temporary topologies thatexpand the basic communication range of—and conse-quently the service offering to—these nodes. In the caseof mobile ad hoc grids, the basic service provided bythe nodes is the capability to allocate computational re-sources to execute tasks. The expanded range offered byMANETs permits that more nodes share computationalresources, but for this to be feasible specialized resourcediscovery and scheduling mechanisms are needed. In thecontext of this paper, resource discovery comprises gath-ering information about resources that mobile nodes makeavailable in the MANET, whereas resource schedulingcomprises selecting the most appropriate nodes (from thepoint of view of resource provisioning) to execute a par-ticular set of tasks. We highlight that our focus in thispaper is on the resource discovery and theinitial schedul-ing of tasks to declared available resources. Hence, we donot consider an eventual unavailability of a resourceafterit has been discovered, selected, and is already running asubmitted task. Dealing with this particular issue is actu-ally related to techniques such as task migration [24, 40],redundant submissions for increased reliability [30], andso on. Such techniques constitute another line of work ontheir own and are thus out of scope of this paper.

In this paper, we propose the integration of resourcediscovery and scheduling for mobile ad hoc grids into

a single protocol called DICHOTOMY (DIscovery andsCHeduling prOTOcol for MobilitY). This protocol al-lows the computational tasks that comprise an applica-tion to be distributed among the most resourceful nodesin a MANET. Such nodes are selected automatically bythe protocol, based on somesuitability criteria definedby the inquiring application. The suitability of a nodeis determined in terms of the resources the applicationis interested in and the relative importance among themfrom the viewpoint of the application. By always select-ing the most resourceful nodes and making advance reser-vations of the resources, our proposed protocol providesan implicit scheduling mechanism that balances the loadamong resource providers in the mobile ad hoc grid. Forcomputationally-intensive applications, such balance alsohelps with lowering the average completion time of tasksin the mobile ad hoc grid.

The DICHOTOMY protocol works in a peer-to-peerfashion, regardless of underlying routing protocols. Theprotocol can be classified as a purely query-based discov-ery protocol—requests are broadcast over the network ondemand and providers reply to these requests accordingly.Query-based protocols are known to cause waste of re-sources if consumers naively broadcast requests by flood-ing (also referred to as the broadcast storm problem [38])or if providers naively reply to such requests (a.k.a. thereply implosion problem [11, 8]). To mitigate broadcaststorms, a simple mechanism that limits the range of flood-ing is used. To avoid reply implosions, an in-network fil-tering algorithm is employed, which allows the more suit-able replies to suppress unnecessary replies from othernodes alongside the paths used for forwarding the moresuitable replies back to the inquiring node.

In our previous publications [29, 16, 15] we gave anoverview of the architecture in which the DICHOTOMYprotocol is employed and of preliminary versions of theprotocol and its mechanisms. The present paper buildsupon these, presenting an extended version of the workthat focuses specially on the details of the protocol im-plementation and its performance evaluation. In par-ticular, besides providing further details of the protocolimplementation and design choices, we also present anovel performance evaluation study that thoroughly an-alyzes the DICHOTOMY protocol covering key aspectssuch as scheduling assessment, discovery efficiency un-der mobility, and scalability analysis. Our prototypicalimplementation—deployed both in an experimental adhoc grid testbed and in the NCTUns simulation and em-ulation platform [47]—was focused on showing the fea-sibility of our approach under operating conditions. Wealso devised a simpler simulation model over the ns-2simulator [34], to evaluate the scalability of our approach.

The remainder of the paper is structured as follows.Section 2 provides some background on mobile ad hoc

2



grids. Section 3 introduces the DICHOTOMY protocolas well as its main features and mechanisms. We presentsome relevant implementation details in Section 4. Weevaluate the performance of our proposed protocol in Sec-tion 5. Section 6 discusses related work in discovery andscheduling in wireless and mobile ad hoc grids. Finally,in Section 7 we summarize the contributions of our pro-posed protocol and discuss possible future work.

2. BACKGROUND

In this section, we discuss why the discovery andscheduling mechanisms have to be different for MANETsand present the basic ideas behind our proposed DI-CHOTOMY protocol that impact discovery and schedul-ing procedures. We also introduce the MoGrid architec-ture [29], our underlying mobile ad hoc grid middleware.

2.1. RESOURCE DISCOVERY AND SCHEDULING IN

M OBILE AD HOC GRIDS

To the best of our knowledge, the research on com-bined resource discovery and scheduling for mobile adhoc grids is still rather preliminary. The mechanisms forresource discovery and scheduling in such grids must befundamentally different from those used in wired grids asthe former are much more sensitive to network behaviorthan the latter. This is due to the very dynamic natureof MANETs—for instance, nodes may move, or the QoSproperties of the wireless medium can vary over short pe-riods of time. In MANETs, there are two ways for de-vices to exchange resource information [14]:queriesandannouncements. Queries involve some form of devicessending requests throughout the network and making thedevices providing the required resource information re-ply to these requests. Announcements permit that devicesadvertise their provided resources in the network to inter-ested devices. Most MANET applications actually em-ploy a combination of the two techniques. The scope ofdiffusion of queries and announcements as well as the pe-riodicity and caching policies of announcements are im-portant parameters that determine the efficiency and ac-curacy of resource discovery in such networks.

In the particular case of mobile ad hoc grids, an-nouncements are inadequate for offering resource infor-mation. Besides the declaration of the availability ofa resource becoming time-sensitive due to node mobil-ity, the availability of highly dynamic resources, suchas CPU load and available memory, may vary consider-ably in very short periods of time [10, 3]. Therefore, weadopt a purely query-based approach in our proposed DI-CHOTOMY protocol (further details in Section 3). More-over, we argue that discovery and scheduling must be ap-proached in an integrated way in mobile ad hoc grids, in

particular for multihop scenarios. This is due to two mainreasons:

1. The discovery of highly dynamic resources shouldbe associated with some form of advance reserva-tion [43, 49], so as to ensure that these resourcesare indeed available at the resource provider whenthe resource consumer submits the intended compu-tational task.

2. The messaging overhead and the information re-quired for discovering available (and reachable) re-source providers in the mobile ad hoc grid canbe used as well for automatically selecting the setof most appropriate providers. The automatic se-lection of resource providers bears resemblance tometascheduling techniques in traditional grids [7],and is particularly important for applications tar-geted at solving computationally-intensive prob-lems.

As shown in Section 3, our proposed DICHOTOMYprotocol implements advance reservation and an in-network filtering algorithm that allows the automatic se-lection of the most suitable resource providers.

2.2. MOGRID ARCHITECTURE

To support the establishment of mobile ad hoc grids,we proposed in [29] a middleware architecture, calledMoGrid, which is depicted in Fig. 1. In our work, werefer to amobile ad hoc gridas a set of (possibly het-erogeneous) mobile nodes in a MANET that manage re-source usage and provisioning for applications in a de-centralized way. Moreover, in our architecture resource-ful nodes (e.g. laptops) may provide resources to multipleapplication tasks simultaneously, while other, resource-constrained nodes (e.g. mobile phones) may not provideresources at all—in these cases, such nodes may only askfor resources from remote nodes. We further discuss theissue of node heterogeneity in Section 7.

In the remainder of this section we provide some back-ground on this architecture, so as to identify some require-ments on the design of the DICHOTOMY protocol.

The MoGrid architecture supports applications in twophases. First, nodes acting as resource consumers dis-cover other nodes acting as resource providers by meansof services that the discovery layer offers. The DI-CHOTOMY protocol is used for implementing these ser-vices. Second, resource consumers submit applicationtasks to selected resource providers—according to the re-sources they make available—by means of services thesubmission layer offers. Protocols within this layer areout of the scope of this paper; nevertheless, we assume aresource usage model in which a resource consumer hasa groupof independent tasks (e.g.bag of tasks) which it

3



Figure 1. MoGrid architecture.

needs to submit for execution on resource providers. Theadopted resource usage model implies the use of a mech-anism during the discovery phase that selectsmultiplere-source providers simultaneously. This is a major short-coming of current discovery approaches to MANETs, asdiscussed in Section 6.

Each kind of application may have its specific re-source requirements on the mobile ad hoc grid. As anexample, for long-lived CPU-intensive applications bat-tery power is more important than highly available band-width when submitting their tasks for execution, whereasstorage capacity and connection stability have greater im-portance to data replication applications. Therefore, theselection mechanism must take into account the currentlevel of resource usage associated with a given providerand the relative importance between its resources fromthe application’s point of view.

The MoGrid architecture supports applications imple-mented directly on top of the discovery layer, or on top ofan application-specific layer (see Fig. 1). One of the pur-poses of the application-specific layers is to implementdifferent resource weightings according to different ap-plication types, as for instance computationally-intensive,communication-intensive, or data intensive applications(Section 5 provides an example of implementation of anapplication-specific layer).

The focus of the MoGrid architecture onresourcedis-covery instead ofservicediscovery also implies the useof a query-based approach to resource discovery, as op-posed to announcement-based approaches (see discussionon Section 6). This is because the highly-variable avail-ability of resources such as CPU load and available mem-ory may demand a higher announcement rate, which inturn may lead to an increased consumption of other re-sources in the MANET, such as network bandwidth andenergy.

3. THE DICHOTOMY P ROTOCOL

In this section, we present the DICHOTOMY pro-tocol for integrated discovery and scheduling in mobile

ad hoc grids. First, we discuss the basic operation ofthe proposed protocol (Section 3.1). Next (Section 3.2),we describe the operation of the adopted mechanism fordetermining the suitability of nodes (acting as resourceproviders) to answer particular requests from inquiringnodes (acting as resource consumers). Finally (Sec-tion 3.3), we describe how the protocol avoids the replyimplosion problem.

3.1. BASIC OPERATION

The DICHOTOMY protocol defines two main mes-sages: INITIATORREQUEST (IREQ) and COLLABO-RATORREPLY (CREP). An IREQ message conveys:(i) a unique request identifier used to match requests toreplies (REQID); (ii) the maximum reply delay that the in-quiring node is willing to tolerate (MAX REPLYDELAY )—c.f. Section 3.2; (iii) the number of resource providersto which the inquiring node wishes to submit tasks(NUMMAX REPLIES); (iv) information about the re-sources the application is interested in and the relativeimportance among them—we call this thecontextual in-formation(CTXTINFO) of the request; (v) the current andmaximum diameter (in number of hops) of the requestpropagation (NUMHOPS and MAX HOPS, respectively);and (vi) a unique identification of the last node (e.g. itsMAC address) sending out the message (HOPID). TheCREP message will be explained later in this section.

Fig. 2 illustrates the basic operation and involved mes-sage exchange within DICHOTOMY. An inquiring nodesends out IREQ messages to the other nodes in the mobilead hoc grid to ask them for resource provisioning. TheIREQ message is replicated at each intermediate node toform a controlled flood limited by theMAX HOPSparam-eter. This is shown in Fig. 2 as nodei, the inquiring node,sends an IREQ message to nodet, an ordinary intermedi-ate node, that replicates the IREQ message to all its neigh-bors; the process is repeated until theMAX HOPSvalue isachieved, making the IREQ message reach for instancenodesy andz.

Upon reception of an IREQ message, a noderecords it as a pending request in a local data struc-ture (PENDINGL IST). In addition to REQID, NUM-

4



Figure 2. Example of DICHOTOMY messages.

MAX REPLIES, andHOPID, which are obtained from theIREQ message, each entry ofPENDINGL IST has aNUM-REPLIESfield (initially set to 0) and two associated timers(REPLYDELAY and CLEANUP). REPLYDELAY is de-scribed in Section 3.2;NUMREPLIES andCLEANUP aredescribed in Section 3.3.

After updating PENDINGL IST, the receiver nodereplies to an IREQ message depending on its willingnessto collaborate as a resource provider, as described in Sec-tion 3.2. In addition to replying to requests, nodes mayalso forward requests to other nodes farther away from theinquiring node in the mobile ad hoc grid, ifNUMHOPS

< MAX HOPS. Such forwarding is done through sim-ple flooding to neighboring nodes—a scheme for inhibit-ing redundant rebroadcasts [38] should be employed inthis case. Before being forwarded, a request has itsNUMHOPSfield incremented and itsHOPID field updatedwith the identification of the current forwarding node.Forwarding requests with updatedHOPIDS allows nodesfarther away from the inquiring node to keep track (intheir localPENDINGL IST structures) of the path traversedby the request, which will be used for determining there-turn pathof the corresponding replies.

Nodes willing to collaborate as resource providers(hereafter,collaborating nodes) send CREP messages toinquiring nodes in response to IREQ messages. In Fig. 2,it is assumed that all nodes in the mobile ad hoc grid col-laborate by replying to the inquiring node. In particular,nodest, y, andz send CREP messages as well as possi-bly other collaborating nodes whose CREP messages arerepresented by dots in Fig. 2. A CREP message informsthe inquiring node about the collaborating node address(identified by aCOLLADDR field) as well as its resourceavailability according to the contextual information of in-terest indicated in the corresponding request (RESINFO

field). Besides theCOLLADDR and RESINFO fields, aCREPmessage also conveys: (i) theREQID matching thatof the corresponding IREQ message; and (ii) the identifi-cation of the node from which the corresponding requestwas received (RETPATH)—the replying node obtains such

an identification from theHOPID field in the correspond-ing IREQ message in thePENDINGL IST.

A collaborating node sends a CREP message towardsthe inquiring node through an application-level forward-ing mechanism that uses the already known path traversedby the request. When a node receives a CREP message,it is processed according to Algorithm 1. If the replycorresponds to a request the node has previously origi-nated, the node processes the message and prevents thismessage to be further forwarded in the network (lines 2and 3). If the reply is instead addressed to an inquir-ing node other than the receiver, the latter first checkswhether there is an entry for the corresponding request inits PENDINGL IST structure (lines 6 and 7). If not, the re-ply is silently discarded.1 Otherwise, the receiving nodedetermines whether this reply must be forwarded to theinquiring node (line 8), according to the algorithm de-scribed in Section 3.3. If this is the case, the node updatesthe reply’sRETPATH field with the value of theHOPIDfield stored in the corresponding entry ofPENDINGL IST

and forwards the reply (lines 9 and 10). This allows aneighbor node along the return path to also forward sucha reply to the inquiring node.

The application-level forwarding mechanism for replymessages avoids the use of network-level routing proto-cols that would generate additional network load on theMANET during the discovery phase. Besides, with smallmodifications in theHOPID andRETPATH fields—insteadof storing the last hop in the path, they should in this casestore the whole sequence of traversed nodes—, inquir-ing and collaborating nodes could also learn the entirepath between each other, and then use source routing—a fairly common approach used in some routing proto-cols for MANETs, such as DSR [23]—at the beginningof the task submission phase. Details on the mapping

1If MAX HOPS= ∞, this condition should not happen because everynode in the mobile ad hoc grid would in principle receive all IREQ mes-sages. Nevertheless, when a scheme for inhibiting redundant rebroad-casts is used, it is not always guaranteed that an IREQ message willreach all nodes in the network even forMAX HOPS=∞.

5



Algorithm 1 CREP message main processing.Input: msg /* Received CREP */

1: if FIRSTCOPY(msg) then2: if MY REPLY(msg) then3: PROCESS(msg)4: return5: end if6: entry ← pendingList[msg.reqID]7: if entry 6= NULL then8: if CANFORWARD(entry, msg) then9: msg.retPath← entry.hopID

10: FORWARD(entry.hopID, msg)11: end if12: return13: end if14: DISCARD(msg)15: else16: . . . /* Deal with duplicate replies */17: end if

of the application-level forwarding mechanism for replymessages onto the link level are discussed in Section 4.3.

3.2. DETERMINING SUITABILITY

In the DICHOTOMY protocol, every node willing tocollaborate with the provision of a specific resource de-lays the transmission of its CREP message according tothe REPLYDELAY timer in the corresponding entry of itsPENDINGL IST. Such a timer is set so that more resource-ful nodes reply earlier. This way, a node willing to collab-orate can detect, before sending its own CREP message,whether other, more suitable nodes have already replied tothe corresponding request. If the total number of repliesgenerated in the mobile ad hoc grid is larger than the valueNM of theNUMMAX REPLIESfield in the correspondingIREQ message, and if the inquiring node chooses theNM

first received replies, the protocol guarantees that the in-quiring node discovers theNM most suitable nodes forproviding the resource. Moreover, when used togetherwith the algorithm presented in Section 3.3, the strategyof delaying replies helps in reducing the total number ofreplies being conveyed in the mobile ad hoc grid by elimi-nating unnecessary additional replies alongside the returnpath from the replying node to the inquiring one.

It should be remarked that the determination of replydelays is flexible with regard to the resources to be takeninto account—e.g. connectivity status, CPU load, avail-able memory, remaining battery power—and the relativeimportance among them. Information about resources tobe considered when computing the suitability of a nodeis conveyed by IREQ messages in theCTXTINFO field.When a node receives a request, it gathers its current statein terms of the resources of interest to compute the re-

ply delay. For this algorithm to work correctly, all nodesin the mobile ad hoc grid must use the same criterion forthis computation. In our implementation (c.f.Section 4), acollaborating node sets theREPLYDELAY timer in the cor-responding entry of itsPENDINGL IST to τ units of time,as given by

τ =

(

1− ω

N∑

i=1

(

αiPi∑N

j=1Pj

))

(Dmax − 2HS) , (1)

where0 ≤ α ≤ 1 and 0 < ω ≤ 1. N representsthe number of different resource types the collaboratingnode should take into account.Pi is the weight that de-scribes the relative importance of each resource typei

from the viewpoint of the application on the inquiringnode,1 ≤ i ≤ N . Both N and Pi are described aspart of theCTXTINFO field in the request.Dmax is themaximum reply delay, which is also obtained from therequest (MAX REPLYDELAY field). αi is the normalizedlevel of current availability (in the interval[0, 1]) of re-source typei at the collaborating node.ω indicates thewillingness (also in the interval[0, 1]) of the collaborat-ing node to participate in the resource provisioning.τ isundefined forω = 0; such a value means that no replieswill be sent by the collaborating node—Section 4.1 pro-vides an example ofω usage in our implementation. Fi-nally,H andS are used for considering the transfer delaysthat IREQ and CREP messages may experience.H is thedistance in hops between the collaborating node and theinquiring node, andS is a tuning parameter representingthe mean transfer delay at each hop.

3.3. AVOIDING THE REPLY IMPLOSION PROBLEM

According to Algorithm 2, CREP messages are selec-tively forwarded towards the inquiring node.

Algorithm 2 FunctionCANFORWARD().Input: entry /* Entry in PENDINGL IST */ and

msg /* Received CREP */Output: boolean /* CREP is to be forwarded? */

1: entry.NR ← entry.NR + 12: if msg.retPath = localID then3: if entry.NR ≤ entry.NM then4: return true5: end if6: end if7: return false

Whenever the functionCANFORWARD() is called, itincrements the valueNR of the NUMREPLIES field inthe entry ofPENDINGL IST corresponding to the receivedCREP message (line 1),regardless of the receiving node

6



(a) Initial configuration. (b) Y receives a reply to a request.

(c) Y rebroadcasts the reply towardsW. (d) Z receives another reply to the request.

Figure 3. Scenario illustrating the suppression of CREPmessages by Algorithm 2.

being on the return path of the reply. The node’s iden-tification is then compared with the value of theRET-PATH field in the reply (line 2). If the values are equal,it means the receiver is in the return path of the reply.The receiving node, however, will only be able to for-ward the reply to the inquiring node ifNR ≤ NM ,whereNM is the value of theNUMMAX REPLIESfield inthe corresponding entry of itsPENDINGL IST (line 3). IfNR > NM , the node suppresses the reply. Note that eachintermediate node in the return path implicitly informsits own vicinity—due to the application-level forward-ing mechanism—about requests that have already beenreplied, thus allowing for further suppressions at nodesnearby the reply’s return path.

To allow further suppressions, an entry inPEND-INGL IST is kept alive until itsCLEANUP timer expires. Inour implementation, a node setsCLEANUP for entries inPENDINGL IST connected with repliesnotoriginated fromthis node toτmax = Dmax − 2HS units of time. For en-tries associated with replies originated from the node (i.e.its resource offerings),CLEANUP is set to2τmax, to takeaccount of advance reservations (c.f.Section 4.1).

Fig. 3 illustrates an example of the operation of Algo-rithm 2. In the initial configuration (Fig. 3(a)), only nodesW andZ are withinY ’s transmission range.Y receives a

reply to a request withREQID= 1000, and increments thevalueNR of theNUMREPLIESfield in the correspondingentry of itsPENDINGL IST (Fig. 3(b)). SinceY is in the re-turn path of the reply (Fig. 3(c)),Y forwards such a mes-sage towardsW, which isY ’s next hop in the return path.Z overhears this transmission due to the characteristics ofthe mapping of the application-level forwarding mecha-nism onto the link level one (discussed in Section 4.3)and then increments the valueNR of the NUMREPLIES

field in the corresponding entry of itsPENDINGL IST, butdoes not forward that reply because it is not in the reply’sreturn path.Z receives another reply to the request withREQID= 1000 (Fig. 3(d)), but although it is in this re-ply’s return path, it does not forward the message becausetheNUMMAX REPLIESfield in the corresponding entry ofits PENDINGL IST indicates that it has already either for-warded or overheardNM messages.

4. IMPLEMENTATIONWe have implemented the DICHOTOMY protocol in

Java, as part of the implementation of our MoGrid mid-dleware architecture [29].2 The implementation is done inJ2SE according to the restrictions of the CDC (Connected2The implementation is available for download at http://martin.lncc.br.

7



Device Configuration) profile of the Java ME. CDC isa standards-based framework for building and deliveringmobile applications that can be shared across a range ofnetwork-connected personal mobile devices, such as gridservices as seen in mobile ad hoc grids discussed in thispaper. Typically, these devices include a 32-bit micropro-cessor/controller and require about 2 MB of RAM and 2.5MB of ROM for the Java application environment. Ourimplementation of the protocol uses a monitoring serviceavailable as part of the MoCA architecture [42]. This ser-vice is responsible for gathering information about thecurrent state of a mobile node, including connectivity,CPU load, available energy and memory, and disk stor-age space.

4.1. SETUP OF PARAMETERS

ParametersS andω in Eq. 1 are configurable in ourimplementation. For our experiments (c.f. Section 5) wesetS = 10ms which is of the same order of magnitudeas 100 meter one-trip packet delays for 1 Mbps transmis-sion rate and 1500-byte packets (disregarding delay vari-ations due to queuing and medium access contention).ω

is computed asωusr

βL, whereωusr describes the user’s level

of interest (in the interval[0, 1]) in allowing its device tocollaborate with others on the mobile ad hoc grid,L isthe number of entries in a node’sPENDINGL IST corre-sponding to this node’s current resource offerings, andβ

is an scalar factor. The denominatorβL implements asimple mechanism for advance reservation by decreasingthe node’s willingness to collaborate when the number ofpending requests at this node increases;β in this case con-trols how fast the influence of a user’s willingness is low-ered as itsPENDINGL IST grows. We setβ = 2 arbitrarilyin our implementation. Finetuning this factor was left forfuture work.

4.2. RESOURCE DESCRIPTION AND MATCHING

In the implementation of the DICHOTOMY proto-col, the contextual information description conveyed bya request message (CTXTINFO field) and the matching ofsuch description to the resource availability on the col-laborating nodes are both based on simple attributes (key-value pairs). Semantic description languages—e.g. ontol-ogy languages [33, 35]—could be also employed. Suchlanguages support queries with more expressiveness andinference power, thereby allowing richer matching op-tions (e.g. partial matchings). Such expressiveness, how-ever, incurs some additional computational cost in termsof both processing time and memory footprint, which maybe undesirable in scenarios involving resource-poor de-vices. The trade-off analysis of attribute- and ontology-based resource description and matching for mobile adhoc grids is out of the scope of this paper.

4.3. APPLICATION -LEVEL FORWARDING OF REPLY

MESSAGES

CREP messages are sent towards the inquiringnode through an application-level forwarding mechanism.There are two alternative mappings of this scheme ontothe link level: using unicast or broadcast/multicast trans-missions.

For link-level unicast mappings, theRETPATH valueassociated with replies could be inferred from the desti-nation address field in the encapsulating packets (e.g. thedestination MAC address in IEEE 802.11 packets). Thisaddress field could convey the value of theHOPID fieldin the corresponding entry ofPENDINGL IST—which in-dicates the link-level address of the next node in the returnpath—as part of functionFORWARD() (line 1 in Algo-rithm 1). By adopting such mappings, theRETPATH fieldcarrying the link-level address of the next node in the re-turn path could be omitted from the CREP messages, andthe statementmsg.retPath ← entry.hopID (line 1) inAlgorithm 1 could be removed. For a participating nodeto overhear replies from its neighbors, however, its net-work interface would have to work in promiscuous mode.Besides the security issues involved, this alternative hasthe drawback that, in promiscuous mode, the node mustprocess the payload ofall packets (not only those pertain-ing to the DICHOTOMY protocol) at the higher levels,which results in a waste of resources (CPU, memory andenergy) that are crucial to computational tasks.

For link-level broadcast/multicast mappings, nodes donot need to set their network interfaces to work in promis-cuous mode; however, the destination link address fieldin packets encapsulating reply messages do not specify asingle recipient. Thus, theRETPATH field is necessary insuch messages and line 1 in Algorithm 1 must be kept.Even so, we argue that the additional computational over-head of promiscuous mode operation introduced by uni-cast mappings can do more harm for applications targetedat solving computationally-intensive problems than theadditional transmission overhead introduced by broadcastmappings. Therefore, we have adopted link-level broad-cast mappings for the application-level forwarding mech-anism in our implementation of the DICHOTOMY proto-col.

It is worth noting that for MANETs in which themedia access control is based on CSMA/CA (CarrierSense Multiple Acess/ Collision Avoidance), broadcasttransmissions are more unreliable and prone to collisionsin comparison with unicast transmissions [38]. This ismainly due to the lack of acknowledgments, of RTS/CTS(Request/Clear to Send) dialogues, and of a mechanismfor collision detection. The problem of collisions in link-level broadcast transmissions may be rather alleviated bythe DICHOTOMY protocol, since replies from differentresource providers are time-shifted, as discussed in Sec-

8



tion 3.2. Regarding the lack of acknowledgments, animplicit acknowledgment mechanism for broadcast trans-missions could be built upon the application-level for-warding mechanism. To understand this, consider againthe example of Fig. 3. WhenW receives the reply messagefrom Y (Fig. 3(c)), W will forward the message becauseit is in the return path. Such a transmission will be over-heard byY (since it is withinW’s range);Y could thenregard this transmission as a higher-level acknowledge-ment fromW. Nonetheless, many subtle issues arise ifa retransmission policy based on such implicit acknowl-edgments is devised to improve the reliability of the pro-tocol. We argue that such additional complexity is notworthwhile, since CREP messages are always subject tosuppression along the remaining path towards an inquir-ing node. Moreover, retransmissions add delays that mayrender the retransmitted CREP message useless at the in-quiring node if it arrives after the predefinedMAX REPLY-DELAY . In fact, the experimental results presented in Sec-tion 5.2 demonstrate that the DICHOTOMY protocol isefficient even without such a retransmission policy.

5. PERFORMANCE EVALUATION

We carried out our experiments in testbed and simula-tion environments, each of them to evaluate a different as-pect of the DICHOTOMY protocol. These are describedin the following subsections.

5.1. SCHEDULING ASSESSMENT

To assess the scheduling properties of the DI-CHOTOMY protocol under an increasing volume of re-quests, we set up an experimental ad hoc grid testbed.This testbed comprises 6 fixed, homogeneous nodes run-ning Linux. These nodes have been arranged in a vari-ety of multihop scenarios, with one node in each of suchscenarios being used as the resource consumer and the re-maining nodes as resource providers. Fig. 4 illustrates thescenario from which the results presented in this sectionhave been obtained. Other scenarios provided similar re-sults, which have been omitted for brevity.

For our experiments, we implemented a simplemaster-worker matrix-matrix multiplication applicationon top of the testbed. To easily split the applicationinto independent tasks, we employed a very simple dis-tributed multiplication algorithm: given matricesAm×n

andBn×p, a master node computesCm×p = AB byselectingp worker nodes with the DICHOTOMY proto-col and sending to each worker nodei (1 ≤ i ≤ p) acopy of matrixA along withb

in×1, i.e. the transposed

vector whose elements are those of thei-th column ofB. Each worker nodei runs a task that computes matrixc

in×1 = Ab

in×1 and returns it to the master node, which

Figure 4. Testbed scenario.

then builds thei-th column ofC fromcin×1. The selection

of the worker nodes in the experimental ad hoc grid thatwill run the tasks is made by only considering those nodeswith the most available CPU and memory resources. Weimplemented an application-specific layer for this pur-pose (c.f. Section 2.2), withN = 2, PCPU = 4, andPmem = 1 in Eq. 1.

For each testbed scenario, we ran a total of 30 experi-ments. In each experiment, the resource consumer sent atotal of 30 IREQ messages at regular intervals of 30s andset MAX REPLYDELAY = 10s. Each resource providers selected by a specific IREQ message was sent a taskto computecs

n×1. MatricesAm×n andBn×p were di-mensioned in such a way that such computation couldgenerate a cumulative load on the resource providers.Fig. 5 shows the cumulative load share for each node dur-ing one of these experiments. The other experiments onthe testbed scenario depicted in Fig. 4 provided similarresults. We observe in the figure that the slave nodeshave their loads balanced after the transient state (first10 requests). This shows that our protocol performed anefficient and dynamic load balancing between resourceproviders under an increasing volume of requests.

0

10

20

30

40

50

0 5 10 15 20 25 30

Load s

hare

(%

)

Request number

ABCDE

Figure 5. Cumulative load of slave nodes.

9



We also used the testbed experiments described aboveto analyze the scheduling quality that our protocol pro-vides to applications. We calculated the cumulative wallclock time (the time difference between the start and fin-ish of a task) of all tasks generated in the testbed applica-tion after the 30 experiments with our protocol. We thencompared it with the cumulative wall clock time achievedafter 30 experiments by a scheduler that selects workernodes randomly. As can be seen in Fig. 6, our protocolschedules tasks among worker nodes in such a way thatthe cumulative wall clock time is approximately 36.5%smaller than that achieved by a random scheduler. Fur-ther, the time overhead implied by our protocol for dis-covery and scheduling, which is upper bounded by theMAX REPLYDELAY parameter (10s in our simulation setup), is negligible with regard to the obtained cumulativewall clock time of the tasks, which is in the order of hun-dreds of thousands of seconds. These results indicate thatour protocol also has a positive impact on the applicationefficiency.

0

50000

100000

150000

200000

DICHOTOMY RANDOM

Seconds

Figure 6. Cumulative wall clock time of tasks.

5.2. DISCOVERY EFFICIENCY UNDER MOBILITY

To observe the impact of mobility on the efficiency ofthe DICHOTOMY discovery process, we employed theNCTUns simulation and emulation platform [47]. In theNCTUns platform, a single machine runs several (virtual)nodes interconnected by a simulated MANET. With thisplatform, we could use the same implementation of theprotocol and of the matrix-matrix multiplication applica-tion deployed on the testbed described in Section 5.1 forscenarios of simulated mobile ad hoc grids.

The simulation scenarios consisted of 40 nodes placedin an obstacle-free, 500m X 500m area. The initial posi-tion of each node in a specific simulation was set ran-domly, with the constraints that at the beginning of thesimulation the nodes formed a connected topology, andthe distance among the nodes was set to between 50%and 90% of the transmission range.

The first scenario consisted of a stationary topology.In the remaining scenarios, the movement of nodes fol-lowed the random walk model [4]. In such a model, eachnode moves in a random direction for some seconds—in a speed that is uniformly distributed in the range]0, Smax]—then chooses a new random direction, withno pauses in between such changes of direction. Table 1summarizes the parameters adopted in the scenarios sim-ulated with the NCTUns platform.

Table 1. Parameters for the NCTUns simulations.

Parameter Value

Number of nodes (N ) 40Number of resource providers 10

Maximum number of replies (R) 4Transmission range 100m

Maximum node speed (Smax) 0 to 5m/s

The discovery efficiency for each simulation scenariowas measured as a sample proportion calculated over 100runs. Each run consisted of a single resource consumerissuing a single IREQ message throughout the simulatedMANET. The sample proportion indicates the percentageof runs in which the protocol deliveredat leastR repliesfrom resource providers to the resource consumer, as de-termined by theNUMMAX REPLIES field in the IREQ

message. The number of collaborating nodes at each runwas fixed at 10, which corresponds to 25% of the nodesin the simulated scenarios. Such a percentage was cho-sen based on the study by Hugheset al. [19], which statesthat in Gnutella—a famous P2P, collaboration-based file-sharing system—this percentage of participants is respon-sible for 98% of all service provisions.

0

20

40

60

80

100

0 1 2 3 4 5

Dis

covery

effic

iency (

%)

Node speed (m/s)

Figure 7. Discovery efficiency in a mobile scenario.

Fig. 7 presents the discovery efficiency of the DI-CHOTOMY protocol as a function of the maximum nodespeed (Smax). The vertical error bars correspond to the95% confidence intervals for each sample proportion. The

10



results show that the protocol behaves well under sit-uations of human mobility (from 0.8 to 1.2m/s, whichcorresponds to a typical range for pedestrian walkingspeeds [41]).

As can be observed in Fig. 7, even for the stationaryscenario (Smax = 0) the protocol does not reach 100%efficiency—the sample proportion is 92%, with±4.13confidence intervals. This is due to the drawbacks statedin Section 4.3 as regards the application-level forward-ing mechanism for reply messages being mapped ontolink-level broadcast transmissions in CSMA/CA enablednodes.

5.3. SCALABILITY ANALYSIS

In principle, we could use our implementation of theDICHOTOMY protocol on the NCTUns platform to eval-uate its scalability in scenarios with an increasing num-ber of nodes. Nevertheless, any simulation of the DI-CHOTOMY protocol on the NCTUns platform was re-stricted to a maximum of 50 nodes; above these valuesthe simulations generated a load beyond the capacity ofour NCTUns simulation infrastructure. To allow a moreextensive analysis of the scalability of the DICHOTOMYprotocol, we devised a simpler simulation model over thens-2 simulator [34]. Our experiments in such simulatorconsidered fixed nodes forming topologies with a con-stant number of nodes within the same transmission range(node density), so that the impact of increasing the num-ber of nodes in the ad hoc grid could be properly eval-uated. The results presented in this section correspondto the average of 100 sample runs per simulated scenariowith a 95% confidence level. This analysis was mainly fo-cused on the evaluation of two metrics: the network loadin the ad hoc grid due to reply messages, and the suppres-sion diameter of these messages—the distance (in numberof hops) between the inquiring node and the nodes wherethese messages were suppressed. Table 2 presents the pa-rameters adopted in the scenarios simulated by ns-2.

Table 2. Parameters for ns-2 simulations.

Parameter Value

Number of nodes (N ) 10 to 240Percentage of resource providers (p) 20% to 80%

Maximum number of replies (R) 1 to 10Node density 5

Transmission range 12.5mDistance between nodes 10m

The average network load in the ad hoc grid due to re-ply messages was computed using, for each scenario, themean number of packets involving these messages. Im-portantly, this metric also allow us to imply whether thereis a significant reduction in energy consumption of de-

vices in a mobile ad hoc grid due to the suppression ofreplies, given that transmissions are known to be respon-sible for a high energy consumption. Using this metric,we compared the DICHOTOMY protocol with a hypo-thetical query-based discovery protocol in which servicereplies are sent by unicast directly to inquiring nodes (wecall it “UCast”)—in both protocols the inquiring nodesbroadcast requests by flooding, and no service announce-ments are employed. In our belief, a comparison withadvertisement- and hybrid-based approaches would notmake sense for mobile ad hoc grids because the typical re-sources involved (e.g.CPU, memory) are highly dynamic(c.f.Section 2).

Fig. 8 presents the network load due to reply messagesas a function of the number of nodes for different percent-ages of nodes willing to collaborate as resource providers.The vertical error bars indicate the confidence intervals.The results show that the adoption of the DICHOTOMYprotocol allows for an increasing reduction—with respectto the “UCast” protocol—in the total number of transmis-sions, as the number of devices in the mobile ad hoc gridincreases. We also observe an even higher level of sup-pressions when there is a larger percentage of nodes (p)in the mobile ad hoc grid with interest in collaborating onresource provisioning. These results suggest the scalabil-ity of our approach.

The suppression diameter of reply messages allow usto evaluate the degree of distribution of the mitigationof the forwarding burden provided by the DICHOTOMYprotocol among the nodes in a mobile ad hoc grid, andconsequently the distribution of energy savings amongsuch nodes due to the reduction in the amount of trans-missions. Figs. 9 and 10 present the distribution of sup-pressions as a cumulative distribution function (CDF) fordifferent numbers of nodes and percentages of replyingnodes. To better illustrate the distribution of suppressionsthrough the network, the results presented in these figuresare contrasted with a uniform CDF (represented by thestraight line in the figures).

Comparing all graphs in Figs. 9 and 10, we observe abetter distribution of suppressions as the number of nodesand the percentage of replying nodes (p) increase. Again,this suggests the scalability of our proposed approach.

Comparing the curves of each individual graph, it isalso possible to verify that the distribution of the sup-pression diameter becomes less uniform as the maximumnumber of replies (R) increases, with a tendency to sup-pressions being concentrated closer to the inquiring node.This is an expected behavior, since less suppressions oc-cur at nodes farther away from the inquiring node andreplies follow convergent paths towards such node.

11



0

50

100

150

200

250

300

350

400

50 100 150 200

num

ber

of tr

ansm

issi

ons

number of nodes

Percentage of replying nodes (p)= 20%

DICHOTOMY (R = 1)DICHOTOMY (R = 2)DICHOTOMY (R = 4)DICHOTOMY (R = 6)DICHOTOMY (R = 8)DICHOTOMY (R = 10)UCast

0

100

200

300

400

500

600

700

800

20 40 60 80 100 120 140 160 180 200 220 240

num

ber

of tr

ansm

issi

ons

number of nodes



0

150

300

450

600

750

900

1050

1200

20 40 60 80 100 120 140 160 180 200 220 240

num

ber

of tr

ansm

issi

ons

number of nodes



0

200

400

600

800

1000

1200

1400

1600

20 40 60 80 100 120 140 160 180 200 220 240

num

ber

of tr

ansm

issi

ons

number of nodes



Figure 8. Network load in the mobile ad hoc grid due to reply messages.

12



0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7

supp

ress

ion

CD

F

suppression diameter

Number of nodes = 60Percentage of replying nodes (p)= 20%

Uniform CDFDICHOTOMY (R = 1)DICHOTOMY (R = 2)DICHOTOMY (R = 4)DICHOTOMY (R = 6)DICHOTOMY (R = 8)

DICHOTOMY (R = 10)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 9 10

supp

ress

ion

CD

F




DICHOTOMY (R = 10)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

supp

ress

ion

CD

F




DICHOTOMY (R = 10)

Figure 9. Distribution of reply suppressions (20% of replying nodes inthe mobile ad hoc grid).

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7su

ppre

ssio

n C

DF




DICHOTOMY (R = 10)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 9 10

supp

ress

ion

CD

F




DICHOTOMY (R = 10)

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

supp

ress

ion

CD

F




DICHOTOMY (R = 10)

Figure 10. Distribution of reply suppressions (60% of replying nodes inthe mobile ad hoc grid).

13



6. RELATED WORKIn spite of some research efforts over the past few

years on wireless and mobile ad hoc grids, none of it—as far as we can see—has addressed the issues related tothe interplay of resource discovery and scheduling in suchgrids. In the following we survey relevant research relatedto these two areas.

6.1. RESOURCE DISCOVERY

Although some papers point out the need for resourcediscovery protocols in wireless grids [32, 36], no workhas explicitly addressed this area for multihop mobile adhoc grids. A close subject—service discovery protocols(SDPs)—has been a hot topic in the area of MANET re-search [45] and could arguably be extended or adaptedfor resource discovery purposes. Nonetheless, most ofthe SDPs for MANETs are based on announcements, al-lowing nodes to advertise services in the network; nodesinterested in such services cache the related advertise-ments. Clearly, such protocols are inadequate for offeringcomputational services that depend onhighlydynamic re-sources, such as CPU load and available memory. Theavailability of such resources may vary considerably inshort periods of time [10, 3], thus demanding frequentannouncements, which in turn may lead to an increasedconsumption of other resources in the MANET, such asnetwork bandwidth and energy. In this context, somepieces of work propose improvements to the broadcastingof service requests in multihop MANETs, so as to reducethe amount of packet transmissions related to such re-quests. Examples include Konark [27], Group-based Ser-vice Discovery (GSD) [5], and Field Theoretic Approach(FTA) [28].

Konark introduces the concept ofservice gossiping,in which a node can selectively forward both service re-quests and replies based on cached announcements fromother nodes. The efficiency of the Konark approach, how-ever, is highly dependent on caching of service informa-tion, thus being inadequate for grid-like computationalservices. The GSD architecture controls request broad-casts based on the semantic grouping of services as ontol-ogy classes, but its efficiency is also dependent on the ad-vertisement and caching of such classes. When comparedwith the two previous approaches, FTA further reducesthe amount of transmissions related to request forwardingby adopting an analogy of electrostatic fields. In the FTAapproach, an inquiring node sends out a service request (anegative test charge), which is “attracted” by the most ap-propriate service instance (the positive charge that createsthe field with highest potential). This way, FTA allows theautomatic selection of a single node as the most suitableprovider of a particular service, with smaller overhead onthe MANET as compared with Konark and GSD. Never-theless, this approach disregards the automatic selection

of multiple providers, and its efficiency is also highly de-pendent on the spread of potentials, which is based onperiodic announcements.

Other researchers have focused on extending rout-ing protocols to accomodate service discovery. Oneexample comes from Varshavskyet al. [44] and theircross-layer approach to integrating service discoveryfunctionality within the DSR [22] (query-based) andDSDV [39] (announcement-based) routing protocols.This approach—and others similar to it [26, 6]—is alsoinappropriate for mobile ad hoc grids, since either its dis-covery efficiency is again dependent on caching of an-nounced service information, or all replies from serviceproviders matching the service description of a client’squery arrive at the inquiring node for local selection bythe client, thus wasting communication and energy re-sources.

6.2. RESOURCE SCHEDULING

As far as traditional grids are concerned, many dif-ferent scheduling strategies have been developed alongthe years [1, 12, 48]. Only very recently, however, havespecific scheduling strategies for wireless and mobile adhoc grids emerged in the literature [17, 25, 31, 20, 50].Most of them employ task replication to optimize someobjective function related to the inherent limitations inprocessing, memory, battery power, and wireless commu-nications capabilities of mobile devices.

Huang et al. [17] propose a two-level schedulingmodel suitable for wireless grids. The first level oftheir model is responsible for mapping tasks to fixed gridnodes; some of these nodes act as proxies between a wire-less domain (e.g. an area covered by an access point)and the fixed grid. The second level conducts schedulingwithin each proxy-centric wireless domain. The proxyruns a revised Min-Min heuristic algorithm, which aimsat minimizing energy consumption at the mobile nodes.A similar approach based on hierarchical scheduling isproposed by Katzaros and Polyzos [25], with the differ-ence that task replication is employed to treat disconnec-tion events. Both approaches are inadequate for mobilead hoc grids due to the need for a fixed node acting as thescheduler.

Litke et al. [31] focus on shortening the overall systemresponse time with a scheme for task replication based onthe knapsack problem formulation. Their strategy aimsat maximizing the utilization of computational resourcesprovided by the mobile nodes. A probability function isused for computing the availability of each node based onits currentfailure rate (e.g.number of times it was un-reachable in the network), so that a node only receivestasks that may be completed before its mean time tofailure (MTTF). Nevertheless, Litkeet al. disregard en-ergy consumption, which—in the case of multihop mo-

14



bile ad hoc grids—may limit the operational life time ofthe whole system.

Zong et al. [50] aim at reducing schedule lengths ofprecedence-constrained parallel tasks while conservingenergy at the mobile nodes. Their strategy relies on ju-diciously replicating tasks so that energy consumption re-lated to inter-task communication is diminished; however,the process of node selection, as described in [50], impliesa centralized discovery service.

Hummel and Jelleschitz [20] propose a decentral-ized scheduler, thus bearing more resemblance to our ap-proach; each mobile node decides autonomously whetherto process a submitted task depending on its current andestimated near future capabilities. Their main focus, how-ever, is on providing fault tolerance through task replica-tion, which is coordinated by means of task queues man-aged in a distributed virtual shared memory. Crucially,their solution does not take into account the waste of com-munication and energy resources due to this coordination.

7. SUMMARY AND OUTLOOKIn this paper, we have presented the specification, im-

plementation, and performance evaluation of a novel pro-tocol (DICHOTOMY) for integrated resource discoveryand scheduling on multihop mobile ad hoc grids. Overall,our experimental results show that:

• our protocol does appropriate resource scheduling,allowing an efficient load balancing among resourceproviders and having a positive impact on the appli-cation efficiency under an increasing volume of dis-covery requests;

• the discovery efficiency—i.e. the percentage of suc-cessfully answered requests—is kept at acceptablelevels in the mobile application scenarios we are in-terested in, which involves pedestrian (walking) mo-bility;

• the proposed protocol scales very well with respectto an increasing number of nodes in comparison withthe traditional query-based solutions for service dis-covery, since it increases the total amount of energysavings due to packet transmissions;

• the suppression of replies performed by our in-network filtering algorithm is not concentrated atspecific points of a MANET—instead, the filteringis distributed among the nodes. As a consequence ofthe distributed reduction in the amount of transmis-sions, the energy savings due to nodes having fewerreplies to retransmit are also distributed throughoutthe network.

During the development of this work, some aspectshave been identified for future investigation:

• A more elaborated mechanism for advance reser-vations may be devised, including the treatment ofresource “underbookings”—a situation that mighthappen if the collaborating node reserves resourcesthat are not actually used because its reply is sup-pressed before arriving at the inquiring node. Al-though such a reservation is temporary (as definedby theCLEANUP timer), it may affect concurrent re-quests.

• TheMAX REPLYDELAY parameter has an importantimpact on the efficiency of the DICHOTOMY pro-tocol. Finetuning this parameter—e.g.as a functionof the transmission delay of messages—is essentialto increase the discovery efficiency under mobility(since messages typically take no more than a fewhundred milliseconds for delivery), and also to re-duce the discovery time without increasing the num-ber of reply collisions (which is achieved throughthe asynchronism in the transmission of these mes-sages).

• The investigation of heterogeneity-aware suitabilitycriteria is an interesting open issue. In this context,the theoretical and simulation study by Huanget al.[18] has recently brought interesting insights intohow to handle nodes with heterogeneous computa-tional power.

• Finally, in our current implementation the suitabil-ity of a resource provider is only computed by itsown context (i.e. the state of its own resources) andnot the context of intermediate nodes in its path tothe inquiring node. This did not affect our resultsin the testbed because it formed topologies with justfew hops. We are currently evaluating to what extentdisregarding the context of intermediate nodes mayaffect a mobile ad hoc grid and investigating alterna-tive solutions.

AcknowledgementsThis work has been partially funded by the Brazilian

Ministry of Science and Tecnology (MCT), CNPq, andFAPERJ.

REFERENCES[1] D. Abramson, R. Sosic, J. Giddy, and B. Hall. Nim-

rod: A tool for performing parametised simulationsusing distributed workstations. InProceedings of

15



the 4th IEEE Symposium on High Performance Dis-tributed Computing, 1995.

[2] Fran Berman, Geoffrey Fox, and Anthony J. G. Hey.Grid Computing: Making the Global Infrastructurea Reality. John Wiley & Sons, Inc., New York, NY,USA, 2003.

[3] William J. Bolosky, John R. Douceur, David Ely,and Marvin Theimer. Feasibility of a serverless dis-tributed file system deployed on an existing set ofdesktop PCs.SIGMETRICS Performance Evalua-tion Review, 28(1):34–43, 2000.

[4] T. Camp, J. Boleng, and V. Davies. A survey of mo-bility models for ad hoc network research.WirelessCommunications & Mobile Computing (WCMC):Special issue on Mobile Ad Hoc Networking: Re-search, Trends and Applications, 2(5):483–502,2002.

[5] D. Chakraborty, A. Joshi, Y. Yesha, and T. Finin. To-ward distributed service discovery in pervasive com-puting environments.IEEE Transactions on MobileComputing, 5(2):97–112, February 2006.

[6] Liang Cheng and Ivan Marsic. Service discoveryand invocation for mobile ad hoc networked ap-pliances. InProceedings of the 2nd InternationalWorkshop on Networked Appliances (IWNA), 2000.

[7] Karl Czajkowski, Ian Foster, and Carl Kesselman.Resource and service management. In Ian Fosterand Carl Kesselman, editors,The Grid 2: Blueprintfor a New Computing Infrastructure, chapter 18,pages 259–284. Morgan Kaufmann Publishers, NewYork, NY, USA, second edition, 2003.

[8] Christopher Dabrowski, Kevin L. Mills, andStephen Quirolgico. A model-based analysis offirst-generation service discovery systems. Tech-nical Report NIST Special Publication 500-260,National Institute of Standards and Technology(NIST), 2005.

[9] Peter J. Denning. Hastily formed networks.Com-munications of the ACM, 49(4):15–20, 2006.

[10] Peter A. Dinda. The statistical properties of hostload. Scientific Programming, 7(3-4):211–229,1999.

[11] A. Erramilli and R. P. Singh. A reliable and effi-cient multicast for broadband broadcast networks.ACM SIGCOMM Computer Communication Re-view, 17(5):343–352, October-November 1987.

[12] J. Frey, T. Tannenbaum, I. Foster, M. Livny, andS. Tuecke. Condor-G: A computation managementagent for multi-institutional grids.Cluster Comput-ing, 5:237–246, 2002.

[13] Mark Gaynor, Steven L. Moulton, Matt Welsh,Ed LaCombe, Austin Rowan, and John Wynne. Inte-gratng wireless sensor networks with the grid.IEEEInternet Computing, 8(4):32–39, 2004.

[14] Antônio Tadeu A. Gomes, Artur Ziviani, Lucianados S. Lima, and Markus Endler. Service dis-covery approaches to mobile peer-to-peer comput-ing. In Boon-Chong Seet, editor,Mobile Peer-to-Peer Computing for Next Generation DistributedEnvironments: Advancing Conceptual and Algorith-mic Applications, chapter 20, pages 437–459. IGIGobal, 2009.

[15] Antônio Tadeu A. Gomes, Artur Ziviani, Lucianados S. Lima, Markus Endler, and Guillaume Che-lius. Mitigating reply implosions in query-based ser-vice discovery protocols for mobile wireless ad hocnetworks. InProceedings of the 7th InternationalConference on Ad-hoc, Mobile & Wireless Networks(ADHOC-NOW), pages 29–42, 2008.

[16] Antonio Tadeu A. Gomes, Artur Ziviani, Luciana S.Lima, and Markus Endler. DICHOTOMY: A re-source discovery and scheduling protocol for mul-tihop ad hoc mobile grids. InProceedings of the 1stIEEE International Workshop on Context Awarenessand Mobility for Grid Computing (WCAMG), 2007.

[17] Chang-Qin Huang, Zhi-Ting Zhu, Yong-He Wu,and Zheng-Hong Xiao. Power-aware hierarchicalscheduling with respect to resource intermittence inwireless grids. InProceedings of the 5th IEEE Inter-national Conference on Machine Learning and Cy-bernetics (ICMLC), pages 693–698, 2006.

[18] Jun Huang and Soo-Young Lee. A heterogeneity-aware approach to load balancing of computationaltasks: a theoretical and simulation study.ClusterComputing, 11(2):133–149, 2007.

[19] Daniel Hughes, Geoff Coulson, and James Walker-dine. Free riding on gnutella revisited: the bell tolls?Distributed Systems Online, IEEE, 6(6):–, 2005.

[20] Karin Anna Hummel and Gerda Jelleschitz. A ro-bust decentralized job scheduling approach for mo-bile peers in ad-hoc grids. InProceedings of the 7thIEEE International Symposium on Cluster Comput-ing and the Grid (CCGrid), pages 461–470, 2007.

16



[21] Junseok Hwang and Praveen Aravamudham. Mid-dleware services for P2P computing in wireless gridnetworks. IEEE Internet Computing, 8(4):40–46,2004.

[22] D. Johnson, D. Maltz, and J. Broch. DSR: The dy-namic source routing protocol for multihop wirelessad hoc networks. In C.E. Perkins, editor,Ad HocNetworking, chapter 5, pages 139–172. Addison-Wesley, 2001.

[23] David Johnson, David Maltz, and Yih-Chun Hu.The dynamic source routing protocol for mobile adhoc networks for ipv4. RFC 4728 (Experimental),February 2007.

[24] Kodai Kagawa, Kazuya Yamada, Tokimasa Kamiya,and Motoyasu Nagata. Fault tolerant grid migrationusing network storage. InPDPTA, pages 241–245,2006.

[25] Konstantinos Katsaros and George C. Polyzos. Opti-mizing operation of a hierarchical campus-wide mo-bile grid for intermittent wireless connectivity. InProceedings of the 15th IEEE Workshop on Localand Metropolitan Area Networks (LANMAN), pages111–116, 2007.

[26] R. Koodli and C. Perkins. Service discovery in on-demand ad hoc networks. (Expired), July 2000.

[27] Choonhwa Lee, Abdelsalam Helal, Nitin Desai,Varun Verma, and Bekir Arslan. Konark: A sys-tem and protocols for device independent, peer-to-peer discovery and delivery of mobile services.IEEE Transactions on Systems, Man and Cybernet-ics, 33(6):682–696, November 2003.

[28] Vincent Lenders, Martin May, and Bernhard Plat-tner. Service discovery in mobile ad hoc networks:A field theoretic approach.Pervasive and MobileComputing, 1(3):343–370, 2005.

[29] Luciana S. Lima, Antonio Tadeu A. Gomes, ArturZiviani, Markus Endler, Luiz Fernando G. Soares,and Bruno R. Schulze. Peer-to-peer resource dis-covery in mobile grids. InMiddleware 2005 Com-panion: Workshop Proceedings, 2005.

[30] Antonios Litke, Dimitrios Skoutas, KonstantinosTserpes, and Theodora Varvarigou. Efficient taskreplication and management for adaptive fault tol-erance in mobile grid environments.Future Gener.Comput. Syst., 23(2):163–178, 2007.

[31] Antonios Litke, Dimitrios Skoutas, KonstantinosTserpes, and Theodora Varvarigou. Efficient task

replication and management for adaptive fault toler-ance in mobile grid environments.Future Genera-tion Computing Systems, 23(2):163–178, 2007.

[32] Antonios Litke, Dimitrios Skoutas, and TheodoraVarvarigou. Mobile grid computing: Changes andchallenges of resource management in a mobile gridenvironment. InIntl Conference on Practical As-pects of Knowledge Management (PAKM), 2004.

[33] DARPA. DAML services, 2006.

[34] Information Sciences Institute. The network simu-lator ns-2, 1995.

[35] W3C Consortium. Ontology Web Language refer-ence, 2004.

[36] L. W. McKnight and J. Howison. Towards a sharingprotocol for wireless grids. InInternational Con-ference on Computing, Communication and ControlTechnologies, 2003.

[37] Lee W. McKnight, James Howison, and Scott Brad-ner. Wireless grids: Distribute resource sharing bymobile, nomadic, and fixed devices.IEEE InternetComputing, 8(4):24–31, 2004.

[38] S. Y. Ni, Y. C. Tseng, Y. S. Chen, and J. Pl Sheu. Thebroadcast storm problem in a mobile ad-hoc net-work. In ACM/IEEE International Conference onMobile Computing and Networking (MOBICOM),pages 151–162, 1999.

[39] Charles E. Perkins and Pravin Bhagwat. Highly dy-namic destination-sequenced distance-vector rout-ing (DSDV) for mobile computers. ACMSIGCOMM Computer Communication Review,24(4):234–244, October 1994.

[40] S. Baghavathi Priya, M. Prakash, and K. K.Dhawan. Fault tolerance-genetic algorithm for gridtask scheduling using check point. InGCC ’07:Proceedings of the Sixth International Conferenceon Grid and Cooperative Computing, pages 676–680, Washington, DC, USA, 2007. IEEE ComputerSociety.

[41] Nagui Rouphail, Joseph Hummer, Joseph Milazzo,and Patrick Allen. Literature review for Chapter13, Pedestrians, of the Highway Capacity Manual.Technical report, Federal Highway Administration(FHWA), 1998.

[42] V. Sacramento, M. Endler, H. K. Rubinsztejn, L. S.Lima, K. Goncalves, F. N. Nascimento, and G. A.Bueno. MoCA: A middleware for developing col-laborative applications for mobile users.IEEE Dis-tributed Systems Online, 5(10), 2004.

17



[43] W. Smith, I. Foster, and V. Taylor. Scheduling withadvance reservations. InIEEE Int’l Parallel andDistributed Processing Symposium (IPDPS), pages127–132, 2000.

[44] A. Varshavsky, B. Reid, and E. de Lara. A cross-layer approach to service discovery and selection inMANETs. In Proceedings of IEEE InternationalConference on Mobile Adhoc and Sensor Systems(MAHSS), pages 8+, 2005.

[45] C.N. Ververidis and G.C. Polyzos. Service discov-ery for mobile ad hoc networks: A survey of issuesand techniques.Communications Surveys & Tutori-als, IEEE, 10(3):30–45, 2008.

[46] Martin Waldburger and Burkhard Stiller. Toward themobile grid: Service provisioning in a mobile dy-namic virtual organization. InIEEE InternationalConference on Computer Systems and Applications(ICCSA), pages 579–583, 2006.

[47] S.Y. Wang, C.L. Chou, C.H. Huang, C.C. Hwang,Z.M. Yang, C.C. Chiou, and C.C. Lin. The de-sign and implementation of the NCTUns 1.0 net-work simulator. Computer Networks, 42(2):175–197, 2003.

[48] C. Weng and X. Lu. Heuristic scheduling for bag-of-tasks applications in combination with QoS in thecomputational grid.Future Generation ComputingSystems, 21:271–280, 2005.

[49] L. Wolf, L. Delgrossi, R. Steinmetz, S. Schaller, andH. Wittig. Issues of reserving resources in advance.In ACM Int’l Workshop on Network and OperatingSystem Support for Digital Audio and Video (NOSS-DAV), pages 151–162, 1995.

[50] Ziliang Zong, Mais Nijim, Adam Manzanares, andXiao Qin. Energy efficient scheduling for parallelapplications on mobile clusters.Cluster Computing,11(1):91–113, 2008.

18

Documents

Performance Evaluation of a Discovery and Scheduling ...endler/paperlinks/Tadeu-Luciana-jbcs-09.pdf · Performance Evaluation of a Discovery and Scheduling Protocol for Multihop Ad