Smart Messages: A Distributed Computing Platformfor Networks of Embedded Systems ∗

Porlin Kang, Cristian Borcea, Gang Xu, Akhilesh Saxena, Ulrich Kremer, and Liviu IftodeDepartment of Computer Science, Rutgers University, Piscataway, NJ 08854

{kangp, borcea, gxu, saxena, uli, iftode}@cs.rutgers.edu

Abstract

In this paper, we present the design and imple-mentation of Smart Messages, a distributed comput-ing platform for networks of embedded systems basedon execution migration. A Smart Message (SM) isa user-defined distributed program which executes onnodes of interest, named by their properties, and usesan explicit lightweight migration to reach these nodes.During migrations, an SM carries its code and execu-tion state, and it self-routes at each intermediate nodebetween two nodes of interest. The nodes in the net-work cooperate to support the SM execution by provid-ing a virtual machine and a shared memory region ad-dressable by names (tag space). To illustrate the flexi-bility of SMs to program real world applications, wedescribe EZCab, an application for booking cabs indensely populated urban areas. We also present exper-imental results to quantify the performance achievedby the SM prototype.

1. Introduction

Following the rapid development of sensor net-work technologies [12, 26–28, 32], networks of em-bedded systems (NES) will become ubiquitous dur-ing the next decade. We are already witnessing thebeginning of this new world in which cars, cam-eras, cell phones, and even watches have wirelessnetwork interfaces and are powerful enough to runLinux [1]. NES offers the opportunity to programa large spectrum of applications, ranging from sim-ple data collection and data dissemination to com-plex distributed applications such as remote objecttracking using robots equipped with video cameras

∗ This work is supported in part by the NSF grant ANI-0121416

or inter-car collaboration to improve traffic safetyand fluidity.

To leverage the raw computing power providedby NES into new distributed applications, we haveto overcome the scale, heterogeneity, and volatil-ity that characterize these networks. Current ap-proaches to NES programming are ad hoc and pro-vide limited flexibility; they are designed for specificclasses of applications (e.g., querying the networkfor certain data) and can hardly accommodate newapplications or services after the network has beendeployed. As NES applications diversify, there willbe an increasing demand for a common distributedcomputing platform to support arbitrary applica-tions over NES.

A distributed platform for NES have to supportsimple development of new distributed applications.It also has to allow applications to cope with theuncertainty encountered in NES (e.g., the networktopology as well as the resources at nodes are un-known a priori and can vary greatly over time). Toanswer these requirements, we need new program-ming abstractions since traditional message passingdoes not work in highly dynamic network configu-rations. This model includes a number of charac-teristics that render it unusable in NES: end-to-enddata transfer between applications, fixed bindingsbetween names and node addresses, and fixed rout-ing.

A first problem with end-to-end data transfersis that they may complete very slowly, or may notcomplete at all in volatile networks [58]. Since ap-plications have no control over the network, theyare forced to wait indefinitely (or until the connec-tion times out) each time something goes wrong inthe network. To be able to adapt quickly to networkvolatility, applications would like to regain the con-trol as soon as possible. Another problem with tra-ditional end-to-end data transfer is that it does notallow in-network processing in order to reduce the

size of data transferred by applications [26]. Reduc-ing the amount of traffic in the network is impor-tant in mobile ad hoc networks, such as NES, sinceit leads to reduced bandwidth and energy consump-tion. Therefore, NES applications would also like tobe capable of performing in-network processing.

The fixed bindings between names and node ad-dresses assumed in the message passing model rep-resent also a serious obstacle for NES applications.After a fixed binding has been established duringthe name resolution phase, an application is forcedto contact the same node each time it needs to ac-cess a resource of the same type. Commonly, nameresolvers react slowly to network changes, and ap-plications would try to contact a node long time af-ter this node has become unreachable, even thoughnodes with similar resources exist in the network.To prevent such a situation, more flexible naming isneeded in NES. We believe that content-based nam-ing [9, 30] can provide a solution because it allowsapplications to contact any node has a certain re-source.

Since content-based naming makes fixed ad-dresses (e.g., IP) irrelevant, the routing and nameresolution should be integrated in NES. Addition-ally, given the diversity of applications, no sin-gle routing will provide good performance forall applications. Therefore, similar to active net-works [19, 40, 54], it would be desirable to let ap-plications use the best-suited routing for theirneeds.

In this paper, we present the design and im-plementation of Smart Messages (SMs) [15], adistributed computing platform for NES basedon execution-migration, content-based nam-ing, and self-routing [14]. Instead of passing dataend-to-end between nodes, an SM application mi-grates to nodes of interest named by contentand executes there. Each node has a virtual ma-chine for SM execution and a name-based mem-ory, called tag space. The SMs use the tag space forcontent-based naming and persistent shared mem-ory. An SM carries its own routing code and routesitself at each node in the path toward a node of in-terest. To perform routing, SMs store routinginformation in the tag space at nodes.

SMs represent an attractive alternative to tra-ditional distributed computing based on messagepassing for four reasons. First, SMs allow applica-tions to adapt to highly dynamic network condi-tions. During migrations between nodes of interest,the routing code can be instructed to return thecontrol to application as soon as a route cannot be

found or after an application-set timeout. Since ap-plication’s code as well as its execution state are al-ready at the same node, the SM can quickly adaptto changes in the network. For instance, an SM canchange dynamically its routing or its destination.Second, the content-based routing provides the flex-ibility to reach a node that offers a certain propertyin an application-controlled manner. Third, the SMprogramming model eases the deployment of newapplications in the network after the network de-ployment phase has ended. A user can inject ap-plications at any node in the network, and conse-quently the applications migrate their code at everynode where they need to execute. And fourth, SMscan significantly reduce the amount of traffic gener-ated by certain classes of applications (e.g., processthe data at the source for distributed image process-ing).

Although the SM computing platform leverageswork from various research areas, notably from mo-bile agents [22,33] and active networks [19,40,54], itsuniqueness comes from both the problem it solves,programming distributed applications in NES, andits design that allows rapid adaptation of applica-tions to volatile network configurations. A detaileddiscussion and comparison between SMs and otherwork is presented in Section 10.

To validate the proposed architecture, we havedeveloped an SM prototype in Java by modify-ing Sun’s Java KVM [2]. The prototype runs overa testbed consisting of HP iPAQs equipped with802.11 cards for wireless communication. This paperpresents the implementation and evaluation of ourSM prototype. It also describes an application de-veloped over this prototype, called EZCab, for book-ing cabs in densely populated urban areas.

The rest of this paper is organized as follows. Sec-tion 2 introduces the main SM concepts. The systemsupport provided by nodes is described in Section 3.Section 4 discusses the SM API. The self-routingmechanism is presented in Section 5. The basic se-curity architecture is explained in Section 6. Sec-tion 7 describes the implementation details of ourSM prototype, and Section 8 presents an evaluationof this prototype. Section 9 describes the EZCab ap-plication. We discuss the related work in Section 10and conclude in Section 11.

2. Smart Messages Architecture

An SM is a component of a user-defined appli-cation whose execution is distributed over a seriesof nodes using execution migration. The nodes on

Node 1(Node of interest)

Node 2(Intermediate Node)

Node 3(Node of interest)

Migration

Migration

Application

Routing

Application

CodeCode CodeCacheCache Cache

TagSpaceSpace

Machine MachineMachineVirtual Virtual Virtual

Tag TagSpace

Figure 1.DistributedComputingUsingSmartMessages

which SM applications execute, called “nodes ofinterest”, are named by content (tag names), dis-covered using application-controlled routing, andswitched when the SM application calls for exe-cution migration. The payload of an SM consistsof data “bricks”, explicitly identified in the appli-cation, and execution control state. Code “bricks”may also be transferred if the code is not cachedat destination. An SM can carry multiple data andcode bricks, and it can use them to create new SMsduring its execution. In this way, an application caneventually generate multiple SMs although it hasstarted as a single SM.

The SM computing platform assumes a decen-tralized architecture, where nodes in the networkact as peers. SMs do not make any assumptionsabout the underlying network configuration, exceptfor a minimal system support provided by nodes: avirtual machine, a name-based memory, called tagspace, and a code cache. The virtual machine of-fers a hardware abstraction layer for SM execution,which shields SMs from heterogeneous node config-urations. The tag space offers a name-based mem-ory, persistent across SM executions. It consists of(name, data) pairs, called tags, which are used fordata exchange among SMs. Special I/O tags areused as an interface to the host OS and I/O sys-tem. Tags serve also to name the destination ofSM migrations and store routing information (rout-ing tags). The code cache stores frequently accessedcode bricks in order to amortize the cost of trans-ferring code over time. Figure 1 depicts the execu-tion of an SM over three nodes. The SM applica-tion code starts on Node1 and finishes on Node3.The SM reaches Node3 by explicitly migrating fromnode to node. Node2 is used as an intermediate hop,where only the SM routing code executes.

To illustrate how NES are programmed using

// application data brickint numCabs, i;Location loc;// application code brickfor(i=0; i<numCabs; i++){

migrate("FreeCab");deleteTag("FreeCab");writeTag("Location", loc);

}

Figure 2. Smart Message Code Example

SMs, we present a very simple example consisting ofan SM that books cabs in a densely populated city.Let us consider a group of people attending a confer-ence, who wants to return to the conference venueafter an “off-site” lunch. Instead of calling a cabcompany or waiting on the street for a free cab, oneof them uses her handheld device to inject an SM inthe network to book a certain number of free cabs.Each cab provides support for SM execution and isidentified by a FreeCab tag. The code for this appli-cation is shown in Figure 2, and the SM executionpath through the network is depicted in Figure 3.The SM migrates to free cabs, changes their sta-tus from free to occupied (by removing the FreeCabtag), and instructs them to come to the client’s lo-cation (by writing to Location tag). The SM com-pletes after booking the desired number cabs.

The key operation in the SM programming modelis multi-hop, content-based migration, which imple-ments routing using tags. An SM names the nodesof interest by tags, and then calls a high-level mi-grate function to route itself to a node that has thedesired tags. In our example, migrate(“FreeCab”)routes the SM to free cabs using the occupied cabsas intermediate nodes. This high-level function usesthe low-level sys migrate primitive, provided by thesystem, for one-hop migration. After a migration,the SM resumes from the next instruction follow-ing the migrate call. It is important to notice thatmigration is explicit (i.e., the programmer calls mi-grate when needed).

Figure 3 emphasizes two major characteristics ofthe SM programming model. First, the high-level,content-based migration shields the application pro-grammer from the routing details. Although therouting code is executed at each node as the SM mi-grates hop-by-hop through the network, migrate re-turns the control to application only on nodes of in-terest (i.e., free cabs). Second, the data transferredduring a migration is specified by the programmeras data bricks; the variables numCabs, i, and loc are

Data BrickApplication CodeRouting Code

migrate("FreeCab") migrate("FreeCab")

i=1 i=2i=0i=0 i=1Mes

sage

Smar

t

sys_migratesys_migratesys_migrate sys_migrate

... ... .........

CabClient Occupied FreeCab

OccupiedCab

FreeCab

Figure 3. Smart Message Execution Path

stored in a data brick and carried from node to nodeduring migrations (the figure shows how i is modi-fied during execution).

From a user’s perspective, this model offers re-silience to dynamic network configurations and sim-ple deployment of new distributed applications inthe network. An application programmer can writesimple sequential programs that migrate to nodesnamed by content and execute there, while ignoringthe routing which is embedded in migrate functions.These are user-level functions, typically developedby system programmers. Applications can choosebetween multiple migrate functions and adapt todynamic network configurations by switching thesefunctions during execution.

To achieve good performance in networks com-posed of resource constrained nodes, we havedecided against involving the VM in determin-ing which data is needed across migrations. In ourarchitecture, the VM captures the minimal execu-tion control state required for SMs to resume at theinstruction following a migration. Although this de-cision puts clearly a burden on programmers,it avoids the overhead of having the VM col-lect the “live data” of SMs; many times thisoperation is not only time consuming, but also col-lects more data than necessary (i.e., conservativeapproach), thus increasing the amount of traf-fic in the network.

3. Cooperative Node Architecture

In order to execute SM-based applications, thenodes must cooperate to support SM execution androuting. The entire SM model is built under the as-sumption that the node architecture must be keptas simple and flexible as possible. Figure 4 showsthe system components of a cooperative node.

Virtual Machine. The virtual machine (VM)executes VM-level threads generated by incomingSMs. To migrate an SM, the VM captures the ex-

ecution state and sends it along with the code anddata bricks to the next hop. The VM at destinationresumes the SM at the instruction following the mi-grate call.

Local Injector. The local injector allows theusers to start new SMs at the local node. A VM-levelthread is generated for each new SM. This thread isstored in the SM ready queue and dispatched for ex-ecution according to the scheduling policies.

Scheduler. The SM execution is non-preemptive; other SMs can be accepted, but theyare not dispatched for execution before the cur-rent SM completes. The non-preemptive schedulingsimplifies the implementation of inter-SM syn-chronization and sharing. Additionally, we envi-sion that the overhead introduced by more complexscheduling will not be justified for NES appli-cations, which typically have short executiontime.

Admission Manager. To prevent excessive useof resources (e.g., processor cycles, tag space mem-ory, runtime memory, bandwidth), the nodes haveto perform admission control on incoming SMs. Theadmission control at nodes ensures the progress ofall SMs running in the network. It also prevents SMsfrom migrating to nodes where they cannot achieveanything due to resource constraints. SM s presenttheir resource requirements in a resource table. Theadmission manager receives the resource tables, de-cides whether to accept the SMs or not, and en-queues the accepted SMs into the SM ready queue.It also instructs an accepted SM to transfer onlythe missing code bricks (i.e., the code bricks thatare not stored locally) and stores them in the codecache upon reception.

The admission manager makes the admission de-cision based on the current state of the node and theSM’s resource requirements. This decision is basedon the admission policy in effect at that node. Anaccepted SM is guaranteed non-preemptive execu-tion as long as its resource usage does not exceed

Tag Space

Incoming SM Migrating SMNetwork Network

SM Ready Queue

Injector

Cache

Manager Scheduler Virtual

Authorization

Application I/O

Code

Admission

Tags Tags

Machine

OS & I/O

Local

SM Platform

Figure 4. Cooperative Node Architecture

certain limits defined by the admission policy. Forinstance, a node may run out of battery and decideto accept only SMs for which it is a node of inter-est, but reject all SMs that need to route throughit. If an SM is rejected, the migration call fails atthe source, and the SM regains the control.

Precise resource usage for SMs cannot be pre-dicted in advance because their computations de-pend not only on user-provided input data, but alsoon data gathered from the network during execu-tion. To be able to perform admission, the admis-sion manager needs, however, at least approximateinformation about SMs’ resource requirements. Onesolution would be to specify upper bounds for theresource requirements. We have dismissed this ideafor two reasons: computing relatively precise upperbounds is as hard as predicting the actual resourceusage (i.e., we do not have knowledge about dataacquired at runtime), and large upper bounds maylead to frequent rejections at nodes even though theSM may consume significantly less resources duringits execution.

Our solution requires each SM to specify its lowerbounds for resource requirements. The program-mers can set them before any one-hop migration,and they define the minimum amount of resourcesthat may lead to SM completion or migration. Theprogrammers may use compiler support to derivelower bounds for resource requirements. The decla-ration of these lower bounds serves two purposes:protect SMs from migrating to a node that cannotoffer enough resources for any semantically accept-able result, and protect the resources at the node

from being wasted on such SMs. Based on the ad-mission policy, the system may grant more resourcesto SMs that have exceeded their lower bounds dur-ing execution. If no more resources could be granted,the system raises an exception which, by default,terminates the SM. The SM is allowed, however,to catch this exception, to save data of interest indata bricks, and to migrate. A limited amount ofresources is reserved during admission for the ex-ception handler. To ensure a successful migrationfor this case, the SM has to declare, during admis-sion, the maximum amount of data it plans to carryto the next hop.

Tag Space. The tag space provides a name-based memory and a unique interface to the localOS and I/O system. It consists of a collection of tagsthat can be divided into two categories: (1) appli-cation tags which are created by SMs and used forinter-SM communication and synchronization, and(2) I/O tags which belong to nodes and allow SMsto access system resources. The structures of thesetags are shown in Figure 5. Each tag has a name(unique at a node, but not globally unique) whichis similar to a file name in a file system. SMs usethis name for content-based naming.

Application tags are commonly used for data ex-change among SMs because their data portion canstore application-specific data. For instance, an SMcan build a routing table in a tag, and other SMscan subsequently read the routes from this tag. Eachapplication tag has a lifetime that specifies the du-ration after which the tag expires and its memoryis reclaimed by the node.

Name Data Lifetime OwnerID Name ACL I/O Handler

I/O TagApplication Tag

SM Blocked QueueFamilyID ACL

Figure 5. Application and I/O Tag Structures

I/O tags act as a gateway between SMs and theunderlying OS and I/O system. Usually, each I/Otag is associated with an external process, whichcommunicates with the VM through a standard in-terface. Each time an I/O tag is accessed by an SM,its associated external process interacts with the lo-cal resources and returns a response to the SM.

The access to tag space is protected using an ac-cess control list (ACL). The application tags havealso ownership information (i.e., OwnerID and Fam-ilyID). We defer the description of the protectionmechanism to Section 6.

Similar to existent solutions [3,4], we use names-paces to avoid naming conflicts; a tag name is pre-ceded by a namespace (i.e., namespace:tagname).The I/O tags have a pre-defined namespace, ions,which is known by any SM. The namespaces for ap-plication tags, on the other hand, are defined by theSMs that create them. Each SM has a unique de-fault namespace which is used when a reference to atag name is not preceded by a namespace. The sys-tem where the SM is injected generates this uniquenamespace, and every SM created dynamically in-herits it from its parent SM.

An SM may use other namespaces to cooperatewith SMs that do not belong to its family. Access-ing tags in other namespaces does not create prob-lems because the access is subject to access control.Creating new tags, however, may lead to namingconflicts. For instance, two different SMs may cre-ate two tags with the same name, but with differentsemantics. A solution to this issue is to ensure thatconflicting namespaces are extremely rare in prac-tice (e.g., a namespace is a long random string ofbits). The developers that need to cooperate can ex-change these namespaces off-line.

Although simple, this solution is not bullet-proof.If an SM needs to ensure that conflicts are avoided,it has to use secure namespaces (i.e., by definition,a secure namespace is preceded by the keyword se-cure). At the compilation time, the compiler buildsthe list of secure namespaces used in tag creationinvocations throughout each code brick. The com-piler has to be able to generate the list of names-paces (i.e., the namespaces are either directly spec-ified, or the compiler is able to determine them us-ing static analysis); if the compiler is not able to find

at least one possible namespace for a tag, the com-pilation fails.

At injection time, the SM must present a capa-bility for each namespace in the compiler-generatedlist. Therefore, the developer of a code brick (orthe developer of an SM) has to acquire these ca-pabilities such that each code brick of an SM hasan associated list of capabilities. During SM injec-tion, the system verifies the capabilities and cre-ates a list of namespaces for each code brick. Thislist together with the default namespace is main-tained in the SM structure and cannot be modifiedover time. A child SM inherits the list of names-paces for the code bricks that compose it. If an SMdoes not present a capability for every namespacein the list generated by the compiler, it will be re-jected during the injection phase.

A central authority (CA) keeps track of all securenamespaces and their owners. Each time a names-pace owner decides to allow a code brick to createtags within that namespace, she associates a capa-bility, digitally signed by the CA, with this codebrick; the capability contains the hash value of thecode brick. Similar to ANTS [19], this value is ob-tained by applying a hash function on the code it-self. Each node has the public key of the CA andthe common hash function. During SM injection,the VM uses the CA’s public key and the capabil-ity to verify that the code bricks are authorized touse the secure namespaces.

Synchronization Mechanism. Given the non-preemptive SM execution, we have devised a simpleupdate-based synchronization mechanism for inter-SM communication. An SM can block on an appli-cation tag until another SM performs a write onthat tag. A blocked SM SM is appended to the SMblocked queue and yields the processor (this is theonly exception to our run-to-completion model ofexecution). After an SM SM blocks, the schedulermay dispatch other SMs for execution. When an SMwrites to an application tag with a non-empty SMblocked queue, all SMs in the queue are woken upand made ready for scheduling. To prevent infiniteblocking, if no write operation takes place within agiven timeout, SMs are unblocked and made readyfor scheduling.

Category PrimitivescreateSMFromFiles(code files);createSM(code bricks, data bricks);spawnSM();

Smart Messages sys migrate();blockSM(tag name, timeout);setResources(resources);createTag(tag name, lifetime, data);deleteTag(tag name);

Tag Space readTag(tag name);writeTag(tag name, data);

Table 1. SM API

4. Smart Messages API

The SM API is presented in Table 1. SMs are al-lowed to create new SMs dynamically, migrate one-hop to neighbor nodes, access the tag space, setlower bounds for resource requirements and syn-chronize on tags. Also, the SMs can use the uniforminterface provided by the tag space to execute sys-tem calls on the local host (i.e. through I/O tags).

SM Creation. Initially, an SM is injected at anode as a program file, and it calls createSMFrom-Files with a list of program file names to create anew SM structure. An SM may use createSM to as-semble a new, possibly smaller SM using some ofits code and data bricks. A createSM call is com-monly used to build an SM that cooperates withthe current one (e.g., a route discovery SM). An ap-plication that needs to clone itself calls spawnSM(similar to the fork system call in Unix). Typically,spawnSM is invoked when the current SM needs tomigrate a copy of itself to nodes of interest whilecontinuing the execution at the local node. A newSM generated by createSM or spawnSM is sched-uled for execution at the local node.

SM Migration. The sys migrate primitive im-plements one-hop migration. It captures the execu-tion state, sends the resource table for admission,transfers the accepted SMs, and resumes these SMsat destination. The sys migrate is used by high-levelmigrate functions to route SMs to nodes of interest.More details about the SM self-routing mechanismare presented in Section 5.

Synchronization. The blockSM primitive allowsSMs to block on a tag pending a write by anotherSM. Typically, an SM uses this primitive to waitfor a route. For instance, an SM can create a routediscovery SM and block on a routing tag until theroute discovery SM returns (i.e., the route discov-ery SM writes to the routing tag, and thus wakes

up the blocked SM).SM Resource Requirements Programmers

invoke setResources each time they need to setnew lower bounds for resource requirements. Typi-cally, this primitive is called once per high-level mi-gration invocation and specifies two categories oflower bounds: resources needed for routing, and re-sources needed for computation at the node of inter-est (i.e., the target of migration). The resources areimplementation specific, but they include at least:number of VM cycles, amount of runtime memory,amount of tag space memory and the duration forwhich this memory is needed, I/O tags to be ac-cessed, and maximum number of bytes that wouldbe generated when migrating this SM to anothernode. An SM is not required, however, to set the re-source requirements. In such a case, the admissionis based only on the size of the SM, but the nodedoes not provide any type of guarantees (i.e., theSM can be terminated or asked to migrate at anymoment). Our current prototype, described in Sec-tion 7, uses this very simple solution.

Tag Space Access. An SM can create, delete, oraccess application tags. As mentioned in Section 3,the tags are accessed subject to authorization. Thesame interface is used to access the I/O tags: SMscan issue commands to I/O devices by writing intoI/O tags, or can get I/O data by reading from I/Otags (an SM cannot create or delete I/O tags).

5. Self-Routing Mechanism

Similar to most mobile ad hoc networks, the dis-tinction between hosts and routers disappears inNES. In our approach, there is no support for rout-ing at nodes. Each application has to include at leastone routing brick among its code bricks. An applica-tion can control routing in two ways: select its rout-ing algorithms, or change the routing during exe-

1 // routing data brick2 String tag, routeToTag;3 int timeout;4 // routing code brick5 boolean migrate(String tagName, int timeOut){7 // use parameters to set tag, routeToTag, and timeout in data brick6 while(readTag(tag) == null){7 Address nextHop = readTag(routeToTag);8 if (nextHop != null)9 sys_migrate(nextHop);

10 else{11 createSM(RouteDiscoverySM, tag);12 if (blockSM(routeToTag, timeout) == TIMEOUT)13 return FALSE;14 }15 }16 return TRUE;17 }

Figure 6. Migration Example Using Simple On-demand Routing

intermediate node

SM uses routing2

TimeoutSM uses routing2

SM uses routing1 SM uses routing1

node of interest

Reached region

Change of routing due to network conditions Change of routing due to application requirements

node where routing changes

Figure 7. Dynamic Change of Routing

cution. Each routing brick defines a user-level mi-grate function, which contains the routing code andis commonly provided as a library implementation.The programmers, however, are free to implementtheir own migrate functions. Any arbitrary condi-tion on tag names and tag values can be used inmigrate to define a node of interest. For instance, asimple implementation of migrate takes a list of tagnames as parameter and migrates the SM to a nodethat contains all those tags. To cope with networkvolatility, migrate can take a timeout as parameter.If a timeout occurs (i.e., the routing algorithm hasnot been able to find a node of interest during thegiven period), the application regains the control atan arbitrary node on the path between two consec-utive nodes of interest. Consequently, it may decideto change the routing or the destination of migra-tion.

Our current prototype provides two pre-defined

routing algorithms, a content-based on-demandrouting (similar to AODV [46]) and a geograph-ical routing (similar to GPSR [35]). SMs imple-ment routing using tags and the sys migrate (4)primitive. Since tags are persistent across SM ex-ecutions, the routing information can be sharedby SMs with similar interests, thus amortiz-ing the route discovery effort. Given the volatilityof NES, most of the routing information will have ashort lifetime. Thus, the amount of tag space con-sumed for storing routing information will be lim-ited. To improve the scalability for networks withmany different tags, we have designed an al-gorithm which maintains approximate informa-tion about tags in the network using Bloom fil-ters [11,14].

Figure 6 shows a simplified version of our mi-grate function implemented using on-demand rout-ing. If a next hop toward a node of interest is avail-

able, the entire SM migrates there (lines 8-9). Oth-erwise, a route discovery SM is created, and the cur-rent SM blocks waiting for a route (lines 11-12). TheSM is woken up when the discovery SM returns witha route and writes the routing tag. If no route is ac-quired during an application-set timeout, migratereturns the control and informs the application thatit could not find a node of interest. A problem gen-erated by content-based routing is how to ensurethat an application does not end up on a node al-ready visited. We have used two solutions; either in-struct migrate to discover an un-visited node eachtime it is called, or let the application record the vis-ited nodes of interest and pass this list to migrate.

A single routing algorithm might not alwaysreach a node of interest in the presence of highly dy-namic network configurations. Therefore, the use ofmultiple routing bricks during the lifetime of an SMcan help an application complete in adverse networkconditions. The completion time of an SM can alsobe improved by using multiple routing algorithms.Two factors can determine a dynamic change ofrouting: a significant modification in the networkconfiguration, or specific application requirements.

Figure 7 illustrates both factors. In the first ex-ample, the SM needs to visit a number of nodesof interest, but it cannot complete because its cur-rent routing (suitable for dense and relatively staticnetworks) cannot find a route when the network be-comes sparse and mobile. The SM, however, has in-structed migrate to return the control after a cer-tain timeout. At this time, the SM regains the con-trol at an arbitrary node and calls another migratefunction, which implements a routing suitable forthe new network conditions (e.g., content-based on-demand routing). Using the new routing, the SM isable to make further progress.

The second example depicts a change of rout-ing determined by the application requirements. AnSM needs to visit several nodes of interest locatedin a given geographical region (the circular region).Therefore, an SM may have two routing bricks, a ge-ographical routing used to reach the region, and anon-demand content-based routing used to discoverthe nodes within that region. In this situation, theSM changes the routing when it reaches the regionof interest.

6. Security Architecture

One of the traditional pitfalls of existing systemsbased on mobile code is security. Similar to mobileagents, there are three main issues that have to be

Others

Owner

Origin

Code

Family

Figure 8. SM Protection Domains for TagSpace Access

solved: (1) protecting recipient hosts from SMs, (2)protecting SMs from each other, and (3) protect-ing SMs from malicious hosts. These problems be-come more severe for SMs due to the volatile na-ture of NES. Unlike traditional mobile agents forrelatively stable IP-based networks, the SMs haveto overcome the lack of an infrastructure or a cen-tral authority, specific to mobile ad hoc networks,which increases significantly the difficulty of key au-thentication and group management.

In this section, we present a basic security ar-chitecture for SMs, which focuses on providing pro-tected access to the tag space. This security archi-tecture offers protection against malicious SMs un-der the assumption that the SM system software atnodes is trusted (i.e., we do not protect SMs againstcompromised hosts). To protect against compro-mised systems, we plan to develop a distributedtrust mechanism [17], which helps a node assigntrust values to its one-hop neighbors; a node deemeduntrusted is simply removed from the list of neigh-bors. Optionally, an SM may ask to be migrated inan encrypted form between neighbor nodes. To sup-port this, each node carries a pair of public/privatekeys.

6.1. Access Control

A unique characteristic of SMs is that no di-rect access is allowed to system resources (i.e., theSMs access both their data and system resourcesthrough the tag space). The advantage of this designis that the tag space is a single point of access con-trol, which can be implemented and enforced uni-formly. Compared to mobile agent systems [23], thetag space simplifies greatly the control mechanisms.The SM creating a tag, called tag’s owner, deter-mines the access control policy and delegates thehost to enforce this policy on its behalf. Protect-

r=

r=

r

3N

(c) Code-based Cooperation

1(C ,C )1

SM

SM

{Code=(C ), rw}{Origin, rw}{Family, rw}

(b) Single Originator Cooperation(a) Family Cooperation

2N

SM

2SM1N5N2SM

1SM1SM

2SM2N

3N

4N

1N

1SM1

2(C ,C )2N

1N1SM

2SM

4

5N

N3

N2

N

SM2

4

N5

SM1

SM2

T T T

Figure 9. Three Access Control Scenarios for SM Group Cooperation (Ni are Nodes, SMi are SmartMessages, and T is a Tag)

ing the application tags ensures that SM executionsdo not interfere with each other, and therefore, pro-vides a secure channel for SM cooperation.

A tag incorporates the ID of its owner, the ID ofits owner’s family, the address of the node where itsowner’s family originated, and its ACL (access con-trol list). SMs are uniquely identified by the nodeaddress where they originated and the time of theircreation. We define a family of SMs as all SMs origi-nated from an SM injected in the network by a user.The family ID is the ID of the original SM. Sincean SM can migrate or spawn new SMs at intermedi-ate nodes, its family information can be used to en-force access control for an entire family of SMs. TheACL is a matrix of subjects and their access permis-sions to tags, read(r) or write(w). The ACL con-tains five protection domains: Owner, Family, Ori-gin, Code, and Others.

Each time an SM tries to execute an operationon a tag, the VM performs the authorization pro-cess. Based on the credentials presented during ad-mission and the currently executing code brick, theSM is associated with at least one protection do-main. A user or the SM itself cannot forge an SM’sidentification information because this informationis set automatically by the system. The request isgranted if the SM has the necessary permissions toaccess the tag in any of the protection domains ithas been associated with.

6.2. Protection Domains

The Owner and Others protection domains de-fine the access permissions for the SM that ownsthe tag and for any SM, respectively. The groupconcept, defined as an arbitrary relation over SMs,supports more flexible cooperation, but also requireshigh overhead of managing the group membershipon-the-fly. Currently, our architecture does not sup-port dynamic cooperation among totally indepen-dent SMs. Instead, we define three protections do-

mains that allow cooperation among well-definedgroups of SMs (i.e., Family, Origin, Code). Figure 8shows that an SM can be associated with multi-ple protection domains for a tag. In the following,we present three scenarios that illustrate the pro-tection domains for group cooperation.

Family cooperation. In Figure 9(a), all coop-erative SMs originate from a common SM ancestor.For instance, SM1 is created on N1 and migratesto N2. At this node it creates a child, SM2, whichmigrates and creates a tag T on node N5. To al-low SM1 to access this tag, SM2 sets the ACL to{Family, rw} (i.e., the familyID of T is the same asthe family ID of SM1).

Single originator cooperation. Figure 9(b)shows the scenario when the group of cooperativeSMs originate from a common node. SM1 and SM2

are created on node N1 and migrate to a target nodeN5 via different paths. SM1 arrives at N5 before SM2

and creates a tag T. It also sets the ACL as {Origin,rw} such that SM2 will be able to access T (i.e., theunique IDs of SM1 and SM2 contain the same ori-gin ID). This scenario is very likely to be encoun-tered since many nodes are small devices, such asPDAs or cell phones, owned by a single user.

Code-based cooperation. In addition to thesimple groups described before, the SM group co-operation can be coordinated more flexibly basedon code bricks. To ensure cooperation among SMsthat are aware of the code used for data sharing ordata exchange, each tag has a list of associated hashvalues for certain code bricks. These hash values de-fine the members of the Code group (they may ormay not belong to the owner of the tag). By defini-tion, an SM is a member of the Code group if thehash value of its currently executing code brick be-longs to this list. For instance, SMs using the samerouting brick can add the hash value correspond-ing to this brick to the tag’s list of hash values inorder to facilitate route sharing among them. Fig-ure 9(c) presents such an example. SM1 creates a tag

T and sets the ACL to {Code=(Cr), rw} to grant ac-cess to all the other SMs using the Cr routing brick.Hence, SM2 has the permissions to use T.

7. Implementation

To leverage on the existing user base, we have im-plemented our SM prototype in the Java program-ming environment over Linux. Specifically, we havemodified Sun Microsystem’s KVM (Kilobyte Vir-tual Machine) [2] because its source code is availableand has a small memory footprint (i.e., it is suitablefor resource constrained devices such as those en-countered in NES). The SM API is encapsulated intwo Java classes: SmartMessage and TagSpace. Forefficiency, we have implemented the API as Java na-tive methods. Besides the KVM interpreter thread,we have introduced two additional threads for ad-mission control and local code injection. The designof the SM computing platform is not specific to anyhardware or software environment. It can be im-plemented on any virtual machine (e.g., Mate [37],Scylla [56]), programming language, or underlyingoperating system.

In the rest of this section, we describe the mostimportant components of our prototype implemen-tation: the primitives for SM creation, the mem-ory management mechanism which ensures thread-safety in KVM, the lightweight migration mecha-nism, the code caching, and the I/O tags. Currently,the admission manager is very simple; it acceptsany SM as long as the destination node has enoughmemory to accommodate this SM.

7.1. Creating New Smart Messages

New SMs can be created at a node by the localinjector or the VM interpreter. Each SM in the sys-tem is associated with a VM-level thread. The ad-mission manager can also create VM-level threadsfor SMs arriving from the network.

A user can inject a new SM by passing a Javaclass name and a list of arguments to the local in-jector. The injector attempts to load, link, verify,and initialize the class file. Upon successful initial-ization, the injector creates a new VM-level threadwith an initial stack frame for the main methodof the class and inserts the thread into the readyqueue. The arguments passed by the user are pushedonto the stack as arguments of the main method. Atthis point, the VM-level thread has no associatedSM structure. When the VM-level thread starts itsexecution, it has to call createSMFromFiles to asso-ciate itself with a new SM structure.

The interpreter thread also creates new VM-levelthreads in response to createSM and spawnSM invo-cations. When an SM calls createSM, the data bricksof the new SM are cloned from the current SM, andthe code bricks of the new SM refer to the verifiedcode bricks in the code cache. The spawnSM call issimilar to createSM, except that the new SM startsits execution from the next bytecode after spawnSM.To implement this primitive, the execution stackframe associated with the VM-level thread of theoriginal SM is duplicated onto the VM-level threadof the new SM.

7.2. Memory Management

The garbage collector in KVM is designed for asingle-threaded environment. Since any of the threethreads in SM prototype (i.e., interpreter, local in-jector, admission manager) could allocate memoryfrom the dynamic heap, we protect the garbage col-lector data structures using a heap lock and re-strict the garbage collection to a limited numberof locations (i.e. GC Points [10]). We have modi-fied the mark-sweep garbage collector in KVM suchthat garbage collection is performed by the inter-preter only during context switches (i.e., the inter-preter has a single GC Point). The interpreter trig-gers a garbage collection during a context switch ifthe available memory falls below a threshold. Be-fore performing garbage collection, the VM ensuresthat the admission manager and the local injectorthreads have reached their GCPoints (defined as theregions where all valid memory references are reach-able from the garbage collector’s root set). The GCPoints of the three VM threads are demarcated us-ing a single read-write lock. During garbage collec-tion, the interpreter thread holds the write lock. Theadmission manager and the injector hold the readlock to protect the critical regions from garbage col-lection.

7.3. Lightweight Migration

One of the main obstacles in implementing an ef-ficient execution migration arises from the strongcoupling between the execution entity and the host.For example, traditional process migration needs todeal with sockets and file descriptors during migra-tion. Two key features in the design of our systemhelped us circumvent the problem of strong cou-pling.

First, the tag space shields the SMs from di-rect coupling with the underlying OS. The read andwrite operations on tags are complete and atomic

AdmissionManager

StackControlCB2 DB2DB1

CB1CB2...

...

Code Cache

Tag

Spa

ce

CB1 CB2

Running SM

DB1 DB2QueueSM Ready

AdmissionManager

CB1

...

...

Code Cache

Tag

Spa

ce

CB3

QueueSM Ready

Running SM

DB3 DB4CB1 CB3

Node1

Stack

VM

Send SM (4)

Send ResourceTable (1)

Missing = CB2 (3)Ack

VM

Stack

CheckCache (2)

Add CB2 (5)

Enqueue SM (6)

Node2

InterpreterInterpretersys_migrate

Figure 10. Smart Message Transfer (Main Operations)

transactions; no state of the underlying OS re-sources is kept in the SM structure. Hence, an SMcan be completely extracted from its execution en-vironment, migrated, and resumed at destination.

Second, an SM program never creates a commu-nication endpoint directly since it is based on exe-cution migration, not message passing. Communica-tion channels are managed implicit by the underly-ing system. In contrast, traditional message passingprograms create communication channels explicitlyto transfer data. Hence, SM programs do not haveany reference to OS network descriptors.

Our migration is lightweight in the sense that wedo not migrate the complete memory referred to bySMs. Instead, we migrate data bricks which are ex-plicitly identified in the SM. To simplify the taskof programmers, we migrate, however, the this self-reference for non-static methods. Therefore, thesemethods can use object member variables safely af-ter migration.

For clarity of exposition, we will describe the SMmigration mechanism as three logical phases: SMcapture, SM transfer and SM resumption.

7.3.1. SM Capture Phase An SM enters intothis phase when it invokes sys migrate directly oras part of a routing library. In this phase, we con-vert the SM into a machine-independent represen-tation. The code bricks are already in the machine-independent Java class format, and therefore, onlythe data bricks and execution stack frames need tobe converted.

Data Brick Capture. To implement this con-version, we have developed a simple object seri-alization mechanism (i.e., KVM does not provideone). Each data brick is serialized into values and

types representing its internal structure recursively.During serialization, we also generate a temporarystructure which provides a unique identifier for eachdata brick reference. The unique identifiers of adata brick object and its sub-objects are determinedsolely by the structure of the data bricks.

Execution Control State Capture. The exe-cution control state of an SM is represented by theexecution stack frames of its associated VM-levelthread. Each stack frame is serialized into a tuple ofsix values: current offset of instruction and operandstack pointers, method name, signature name, classname, and a flag indicating whether the method isnon-static. For non-static methods, we also encodethe machine-independent identifier for the this self-reference.

7.3.2. SM Transfer Phase Using the data brickand stack information sizes obtained during thecapture phase, the interpreter initiates a three-way handshake protocol with the destination node.The operation of this protocol is shown in Fig-ure 10. If the SM is accepted, the admission man-ager sends back a list of missing code bricks as partof the acknowledgment. Otherwise, the admissionjust drops the request. Upon the receipt of the ac-knowledgment, the source node sends the completeSM, which consists of missing code bricks, serializeddata bricks, and execution control state. To simplifythe implementation, we have used TCP for reliablesingle-hop communication between neighbors. Forbetter performance, we plan to change it into a re-liable single-hop protocol over 802.11.

7.3.3. SM Resumption Phase After the admis-sion manager successfully received the code bricks,data bricks, and execution control information from

Unix PipesGPS GPS

Device

Read Command

Handler

Interpreter

Tag Space

Serial (/dev/ttyS)

I/O Handler

Location ACL

Location l = readTag("Location");

Location Object

Figure 11. I/O Tag Example (Using GPS to Get the Current Location)

a source node, a new VM-level thread and its as-sociated SM structure are constructed. The missingcode bricks sent from the source node are verifiedby the KVM verifier and stored in the code cache bythe admission manager. We have modified the ex-isting KVM class loader to search the code cacheeach time the VM needs a class. During data brickde-serialization, the admission manager constructsa temporary structure (similar to the structure con-structed during the data brick capture at the sourcenode) which maps a unique identifier to each databrick reference. The execution stack frames are re-constructed using the tuples sent from the source.Finally, the interpreter thread is notified if it is cur-rently idle.

7.4. Code Caching

Each code cache entry consists of the Java classfile of a code brick, a reference count, and a refer-ence to the internal VM class representation. Theoriginal class format is stored for future migrationsto nodes that do not have it cached. The referencecount keeps track of the number of SMs currentlyreferring to this code brick. Each time an SM refer-ring to this code brick migrates or terminates, thereference count is decremented. When the referencecount becomes zero, the code cache entry is movedto a free list. Should the same code brick be refer-enced by a new SM, the cache entry is resurrectedfrom the free list. The memory associated with freelist entries is reclaimed according to an LRU pol-icy. When a cache entry is evicted, the code brickmemory is freed, and the corresponding internal VMclass representation is unloaded (since KVM doesnot have a class unloading capability, we have im-plemented our own class unloading mechanism).

7.5. I/O Tags for Interaction with theOS and I/O System

An application uses the readTag and writeTagprimitives to access an I/O tag. It is up to the sys-tem to define the source of the data, but readTagtypically translates to an OS call. A writeTag trans-lates to an OS call which sets certain parameters foran I/O device. Example of I/O tags currently avail-able in our prototype can be found in Table 5.

Since each I/O tag requires specific native code,adding new I/O tags involves adding new nativecode to the node. We have identified three possi-ble solutions for this issue. The first option is tostatically link the native code into the VM. This isnot viable because adding new I/O tags would in-volve shutting down the VM. The second option isto implement new I/O tags as dynamic shared li-braries. This is not viable because we cannot as-sume that every node supports dynamic linking.The third option is to implement new I/O tags asexternal processes which communicate with the VMusing a standard interface. We have chosen the thirdalternative since it enables users to dynamically ex-tend the I/O tags without requiring the VM to beshut down or the host to support dynamic sharedlibraries. For efficiency, a few basic I/O tags (e.g.,free memory and system time) are implemented andlinked permanently into the VM executable.

Commonly, an I/O tag is associated with an ex-ternal program, termed handler, which incorporatesthe code for reading and writing this I/O tag. Whenthe VM receives a request to add a new I/O tag, itcreates a new Unix process for this handler. We useUnix pipes for communication between VM and thehandler process. Figure 11 shows the interaction be-tween an SM and a handler process for a GPS de-vice. When the SM issues a read request for the Lo-cation tag, the interpreter sends a read command tothe handler and blocks waiting for an answer. Once

the handler has obtained the data from the GPSdevice (connected on the serial port in our exam-ple), the handler encodes the data and sends it backto the VM. The VM de-serializes the results into aJava Object and returns it to the SM. A write op-eration is performed similarly.

Certain SMs may have a user interface (in theform of an external process) which allow users tointeract with SMs via special I/O tags, termed UItags. Unlike regular I/O tags, a UI tag behaves sim-ilar to a producer-consumer circular buffer. Each UIprocess can communicate with multiple SMs. Thiscommunication is done through a pair of UI tags: awrite tag for passing data to SMs, and a read tagfor receiving data from SMs. These tags persist forthe entire duration of the UI process.

8. SM Prototype Evaluation

To evaluate the performance of our prototype, wehave measured the cost of the SM primitives: cre-ating new SMs, migrating SMs between two nodes,and accessing the tag space. Our testbed consists ofHP iPAQ 3870 running Linux 2.4.18. Each iPAQcontains an Intel StrongARM 206Mhz processor,32MB flash memory, and 64MB RAM memory. Forcommunication, we use Orinoco 802.11b Silver PCCards.

8.1. Cost of SM Creation

createSMFromFiles. This primitive allows auser to inject a new SM at a node. After an in-vocation, the VM loads the class files from the localfile system, unless the classes are already in the VMcode cache, and creates a new SM structure. To eval-uate its cost, we have performed two series of experi-ments. In the first, we invoke createSMFromFiles foran un-cached class of different sizes while keepingthe data brick size constant (53 bytes). Then, we re-peat the same experiment with the class cached. Inboth experiments, we have used 1KB class files andwe varied the number of class files used to create anSM. Table 2 shows that the cost of createSMFrom-Files almost doubles (when the code is not cached)as we double the size of the code brick. These re-sults show that the cost of class loading dominatesthe cost of creating a new SM structure. The cost ofcreating a new SM structure is essentially the costmeasured when the code is cached.

createSM and spawnSM. Table 3 shows thecosts of spawnSM and createSM for different databrick sizes. The code brick and stack size are fixed at

Time(ms)Size(KB)Uncached Cached

1 2.622 0.0322 5.112 0.0344 9.953 0.0428 20.151 0.063

Table 2. Effect of Code Brick Size on cre-ateSMFromFiles

Time(ms)Size(KB)spawnSM createSM

2 0.270 0.2434 0.367 0.3268 0.508 0.46916 0.913 0.822

Table 3. Effect ofDataBrick Size on spawnSMand createSM

1527 and 131 bytes, respectively. Typically, an SMhas a mixture of static and non-static call frames.Therefore, we consider a stack consisting of twostack frames, one for a static method and the otherfor a virtual method call. Although these two prim-itives are similar, the results show that the cost ofspawnSM is slightly higher than the cost of cre-ateSM. The difference is the time spent to dupli-cate the execution stack frames for spawnSM.

8.2. Cost of SM Migration

The most significant factors that determine thecost of our migration are the data brick serialization,the SM transfer, and data brick de-serialization.

Data Brick Serialization and De-Serialization. Since the code bricks need notbe serialized, we perform this operation only ondata bricks and execution stack frames. Our mea-surements indicate that the serialization costfor the execution stack frames is small com-pared to the cost of data brick serialization; itvaries from 0.204ms to 0.567ms as we vary the ex-ecution stack from 2 to 15 frames. To study theeffect of data brick serialization, we vary thedata brick size from 2KB to 16KB, while us-ing a fixed size code brick (1197 bytes) and twofixed size stack frames (131 bytes).

Figure 12. Cost of Data Brick Serialization

Figure13.CostofDataBrickDe-Serialization

Commonly, the data bricks in an SM consist ofa mixture of objects and primitive types. We usetwo types of data bricks in this evaluation: an arrayof integers, and an array of objects. The serializa-tion costs for these two data bricks provide practicallower and upper bounds for the cost of data brickserialization. The object array represents an upperbound since each of its elements causes a call to thetop level VM serialization method. The integer ar-ray represents a lower bound since it involves onlyone call to the top level VM serialization method.

Figure 12 shows that the serialization cost is be-low 6ms for data bricks as large as 16KB. Com-monly, the SMs process data at its source, andtherefore, they carry small size data. The applica-tions that we have developed carry less than 2KB,which costs less than 1ms to serialize. Figure 13presents the de-serialization cost for the same data

Figure 14. Effect of Code Brick Size on SingleHop Migration

Figure 15. Effect of Data Brick Size on SingleHop Migration

bricks. We observe that de-serialization cost is asmuch as 30% higher than the cost of serializa-tion due to memory allocation during object de-serialization.

SM Transfer. The variation of execution con-trol state size is small compared to that of codebricks and data bricks. Thus, we only consider theeffect of code bricks and data bricks in the sub-sequent experiments. We have performed two setsof experiments to evaluate the cost of migration(serialization, transfer, de-serialization) for differ-ent code brick and data brick sizes. In the first set,we vary the code brick size while keeping the databrick size and stack frame size fixed at 53 bytesand 131 bytes, respectively. In the second experi-ment, we vary the data brick size while keeping thecode brick size and stack frame size fixed at 1197

Operation Time(µs)createTag 101.781deleteTag 75.071readTag 34.548writeTag 50.289blockSM 59.844

Table 4. Cost of Tag Space Primitives for Ap-plication Tags

bytes and 131 bytes. Figures 14 and 15 show the re-sults of these two experiments.

The values in Figure 14 represent the total timefor single hop migration in two situations: the codeis not cached, and the code cached. The time totransfer the SM when the code is cached is constantand represents the overhead of the three-way hand-shake protocol. Figure 15 shows that the data bricksize contributes significantly to the total cost of mi-gration. Thus, it is important to have a serializationscheme with minimal space overhead.

8.3. Tag Space Operations

Table 4 shows the cost of the tag space operationsfor application tags. The readTag primitive has thelowest cost since it performs the least number of op-erations; when an SM reads a tag, the interpreteracquires a lock, performs a lookup in the tag space,verifies the access rights, and returns the data tothe SM. The writeTag operation costs slightly moresince the interpreter has to check for and unblockany SMs blocked on the tag. The blockSM operationcosts more than both readTag and writeTag sinceit also needs to append the SM to the SM blockedqueue and suspend the VM-level thread. The delete-Tag primitive has the second highest cost since theinterpreter needs to wake up all SMs blocked on thetag, remove the timer for the tag lifetime, and re-move the tag structure from the tag space, while thecreateTag primitive has the highest cost since it in-volves additional steps to register a timer for the taglifetime and create access control data structures.

Table 5 presents the access time to several I/Otags that are currently implemented in our proto-type: GPS location query, neighbor discovery, cam-era image capture, light sensor, and system statusinquiry (battery lifetime, system time, and amountof free memory). The gps location is updated by auser-level process which reads from the GPS se-rial interface. The location of the neighbors along

Tag Name Time(ms)gps location 0.20neighbor list 0.34image capture (32 Kb) 341.23light sensor 0.11battery lifetime 25.63system time 0.09free memory 0.12

Table 5. Cost of Reading I/O Tags

with their identifiers are returned by reading theneighbor list tag. This tag is typically used by geo-graphical routing algorithms carried and executedby SMs. To get the information about neighbornodes, we have implemented a neighbor discoveryprotocol which maintains a cache of known neigh-bors. For the image capture tag, the I/O handlerconverts the image received from camera in YUYVformat to RGB format before returning it to theSM. All the other tag values are obtained directlyfrom Linux using system calls.

9. Case Study: EZCab

We envision that the use of embedded devices incars will soon become a reality [41, 44]. To demon-strate the feasibility of the SM computing platformfor real-world applications, we have developed EZ-Cab, an application for locating and booking freecabs in densely crowded traffic environments (likeManhattan, where looking for a free cab can bean annoying experience). Instead of calling a cabcompany or merely “gesturing” to negotiate a cabfor her destination, a client can simply inject anSM through her handheld device to perform seam-lessly the same action. Unlike the existing solutionsfor inter-car communication that are based on cer-tain infrastructures (which are expensive, cannot bedeployed on every road, and provide only limitedinformation), EZCab uses a peer-to-peer approachwhose key benefits are scalability and practicality.The minimal infrastructure needed by EZCab is theavailability of the SM support in the cabs, a loca-tion service (e.g., GPS), and wireless connectivity.

The main component of EZCab is an SM that mi-grates to a cab identified by a FreeCab tag, negoti-ates the price according to a client-established limit,let the cab know the identity of the client, and in-structs the cab to go to the client’s location. Thebooking is complete after the cab sends a messagewith its identification to the client, and the client ac-

Figure 16. Route Discovery

knowledges this message. When the cab arrives atthe client’s location, a validation process takes placeto ensure that the client gets her booked cab (andthe cab takes the client that booked it). In the fol-lowing, we present a brief description of the basicoperations in EZCab: (1) discovering the routes tofree cabs, (2) booking a free cab, and (3) perform-ing the validation between the cab and the client.We conclude the section with an analysis of our ap-plication based on experimental results.

9.1. Route Discovery

The EZCab application starts at the client nodeand takes as parameter the radius of the circular ge-ographical region to be covered (the maximum num-ber of hops, maxHops, for which any EZCab SM isallowed to migrate is computed based on this ra-dius and the transmission range of nodes). To reacha free cab, the SM uses routing tables that spec-ify the next hop as the probability to reach a freecab from the current node (similar to probabilis-tic routing [50]). If the probability to find a freecab using the existing routes is too low, or thereare no routes at all, the SM creates a route discov-ery SM and blocks waiting for routes (Figure 16 il-lustrates this process). The code for route discoveryis presented in Figure 18. At each node, the discov-ery SM spawns and blocks (lines 9-11), while thechild SM migrates to all the neighbor nodes (line15). Each SM migrates through the network until itarrives at a node already visited by another discov-ery SM (i.e., it ends its execution) or it reaches themaximum number of hops that it is allowed to mi-grate. Once this threshold is reached, the SM mi-grates back one hop and reports its current infor-mation (lines 19-28). This is a recursive process that

Figure 17. Cab Booking

builds the routing tables at nodes. We have chosento wait for replies for a given period of time becauseit is difficult to wait for a fixed number of replies ina volatile network (i.e., those replies may never ar-rive).

9.2. Cab Booking

Booking a cab is a three-way handshake proto-col. If a node has routes to free cabs, the applicationcreates a booking SM to find a free cab and blocksfor a certain amount of time. If the cab is not free,the booking SM chooses the next neighbor greed-ily (i.e., using the greatest probability in the rout-ing table), as shown in Figure 17. Once a free cabis found, the SM removes the FreeCab tag, writesthe client’s location in the Location tag, and cre-ates a reporting SM to confirm the booking with theclient. Then, it blocks at the cab waiting for an ac-knowledgment from the client.

The reporting SM migrates to the client’s loca-tion using geographical routing to improve the effi-ciency. Once it has informed the client that a cab ison its way, it returns to the cab with an acknowl-edgment to let the cab know that the handshakehas succeeded. If no reply is received from a cab af-ter a timeout, EZCab will re-start with a new bestroute. Consequently, the booking SM waiting at thecab times out and re-creates a FreeCab tag to reflectthe change in the cab’s status.

9.3. Validation

Upon reaching the client’s location, the valida-tion mechanism is initiated. To make the validationpossible, the booking SM carries the public key of

1 int probability, status = 0, nHops = 0; // stored in data bricks2 Address parent, nextHop, source; // stored in data bricks3 source = SmartMessage.getLocalAddress();4 while (nHops < maxHops){5 if (tagExists("Visited"))6 System.exit();7 nHops++;8 parent = SmartMessage.getLocalAddress();9 if (SmartMessage.spawnSM()){

10 TagSpace.createTag("Visited", Lifetime, null)11 TagSpace.blockSM("Discovery", DiscoveryTimeout);12 break;13 }14 else15 SmartMessage.sys_migrate(AllNeighbors);16 }17 if (tagExists("FreeCab"))18 status = 1;19 if (nHops == maxHops)20 probability = status/2;21 else{22 if (parent.equals(source))23 return;24 probability = (status + maxProbabilityReported("routingTable"))/2;25 }26 nextHop = SmartMessage.getLocalAddress();27 SmartMessage.sys_migrate(parent);28 addEntry("routingTable", nextHop, probability);

Figure 18. Route Discovery Code

the client to the cab, and the reporting SM car-ries the public key of the cab to the client. To vali-date the client, the cab broadcasts a challenge in thezone by encrypting a text using the client’s publickey. The client, upon receiving the encrypted text,decrypts it using its private key. In turn, it uses thecab’s public key to encrypt the text again and sendit to the cab. If the reply text is identical, the clientis validated.

9.4. Analysis

For EZCab, it is of particular importance to eval-uate its completion time given realistic configura-tions. In the following, we present an analysis whichdemonstrates that EZCab can cover a circular areaup to 1km radius around the client’s location, andthe user-perceived response time is less than 2 sec-onds. Figure 19 shows our EZCab prototype.

The first part of our evaluation tries to deter-mine the maximum distance at which two movingcars can communicate and the time for which thetopology is relatively stable. Using two HP iPAQswith 802.11 cards for communication, and variousmobility scenarios (as much as 170km/h relativespeed between two cars moving in opposite direc-tions), we have experienced a substantial increase

in the packet loss rate for distances bigger than60m. We consider this distance feasible for our tar-get networks (we have also experimented with exter-nal antennas and amplifiers to increase to range asmuch as 400m). Given this distance, two cars are inthe communication range of each other for approx-imately 2 seconds at a relative speed of 120km/h(i.e., typical speed for two cars moving in oppo-site directions in a crowded city). Therefore, our ap-plication should complete faster than that in orderto reduce the effects of mobility on the establishedroutes.

The second part of our evaluation tries to see ifEZCab can finish using this time bound, and howbig the geographical region covered by our applica-tion is (i.e., a bigger region increases the probabil-ity to locate a free cab). The response time for EZ-Cab is defined as the time spent until the client re-ceives a confirmation from the cab. The design ofEZCab makes it easy to bound this response time.All the main operations (route discovery, booking acab, and reporting a booked cab) are bounded bya timeout. Therefore, the maximum response timefor a successful booking is the sum of the timeoutsfor route discovery and booking a cab.

We compute the timeouts for each SM gener-ated by EZCab as the products of the round trip

Figure 19. EZCab Prototype

Figure 20. Estimated Completion Time forEZCab

time of each SM between two nodes (RTT) andthe maximum number of hops traveled by an SM(maxHops). We further assume that all cabs havethe code cached. SMs transfer only small size databricks and execution control state when the code iscached. Hence, the measured values of the RTTsfor the three SMs are almost identical (24.3ms,25.4ms, 25.1ms). To include the costs of SM exe-cution and wireless contention, we consider, conser-vatively, a value three times greater. Since bookinga cab and reporting back to the client do not in-volve any broadcast, we just double the minimumtimeout value for booking a cab and reporting backto the client.

Figure 20 shows the response time as a functionof the size of the covered region. The results indi-

cate that EZCab can finish in less than 2 secondsfor a region radius of approximately 1km even if itneeds to perform a route discovery. As determinedby the first part of our evaluation, we expect thenetwork topology to be relatively stable during thistime period. Therefore, we conclude that SMs of-fer the flexibility to program the EZCab applicationwithout any infrastructure, and the analysis of theactual SM implementation demonstrates the feasi-bility of EZCab for densely populated cities.

10. Related Work

The SM platform shares the idea of executionmigration with process migration [7, 39, 43], mobileagents [22,33], and active networks [19,40,54].

Unlike process migration which has been used toincrease performance or availability in stable net-works, the main goal of the SM platform is to pro-vide flexible support for programming distributedapplications over highly dynamic NES. Addition-ally, process migration and SM migration differ intwo aspects. First, the SM migration is explicit (i.e.,the programmer decides when and where to mi-grate), while process migration is implicit (i.e., thesystem decides when and where to migrate a pro-cess). Second, the SM architecture avoids one of themost difficult problems in process migration: trans-ferring the kernel state (e.g., sockets, file descrip-tors). The SM platform does not transfer any kernelstate because SMs interact with local hosts throughatomic operations performed on the tag space, andthey do not open explicitly communication chan-nels.

SMs are influenced by the design of mo-bile agents. Similar to a mobile agent, an SM maybe viewed as an application that explicitly mi-grates between nodes of interest. Mobile agents,however, name nodes by fixed addresses and com-monly know the network configuration a priori,while SMs name nodes by content and discov-ers the network configuration dynamically. Incontrast to mobile agents, SMs are responsi-ble for their own routing at each node in the pathbetween two nodes of interest. This feature al-lows SMs to adapt quickly to changes that mayoccur both in the network topology and the avail-ability of resources at nodes. Furthermore, the SMsystem architecture is suitable for resource con-strained devices since it defines a lightweightsystem support at nodes, with most of the “intelli-gence” incorporated into SMs.

Although the SM computing platform (especially

the self-routing mechanism), shares some of the de-sign goals and leverages work done in active net-works (AN), it differs from AN in several key fea-tures. A first difference comes from the problemsthey try to solve: AN target improved performancefor end-to-end data transfer in relatively stable net-works, while the SM platform helps the developmentof distributed applications on top of a new comput-ing infrastructure which is significantly under-useddue to the lack of programmability support. Un-like AN, we define a computing model whereby sev-eral SMs can cooperate, exchange data, and syn-chronize with each other through the tag space. Interms of migration, AN do not transfer the execu-tion state from node to node whereas the SM modeldoes. The migration of the execution state for SMstrades off overhead for flexibility to react “on-the-spot” to adverse network conditions.

Sensor networks represent the first attempt to-ward deploying large scale NES. Most of theresearch in this area has focused on hard-ware [32, 49], operating systems [28], or net-work protocols [12, 27, 30]. Even though sen-sor networks act primarily as huge distributeddatabases [13, 38], more sophisticated applica-tions might be needed in the future. Toward thisend, SensorWare [16] and Mate [37] have pro-posed solutions for network re-programmability.The SM architecture takes one step further and pro-poses a distributed computing model that is flexibleenough to be implemented for nodes with very lim-ited resources such as those encountered in sensornetworks.

Among many projects that target the pro-grammability of ubiquitous computing environ-ments [8, 24, 48, 51], one.world [24] is similar toour work in the sense that both consider migra-tion as an essential mechanism to adapt to highlydynamic computing environments. Each applica-tion in one.world has at least one environmentthat contains tuples (similar to application tagsin SM platform), application’s components, andother nested environments. When needed, a migra-tion moves a checkpointed copy of an environmentto another node. A significant difference be-tween SMs and one.world is that our work proposesa computing model based on execution migra-tion, while one.world uses migration just as amechanism to adapt to changes (i.e., in their pro-gramming model, the applications reside on nodesand communicate through remote event pass-ing). Another difference is that the SM architectureis more suitable for resource constrained de-

vices whereas one.world is designed for more pow-erful nodes.

Smart Message is the underlying platform forSpatial Views, a high-level programming model fornetworks of embedded systems, targeting its dy-namic, space-sensitive and resource-restrained char-acteristics. The core of the model is iterative pro-gramming over a dynamic collection of nodes iden-tified by the physical spaces they are in and the ser-vices they provide. Hidden in the iteration is execu-tion migration, as the main collaboration paradigm,constrained by user specified limits on resource us-age such as response time and energy consumption.A Spatial Views prototype has been implementedand first results are reported in [42]. A Spatial Viewscompiler with Smart Messages as its target is cur-rently being implemented.

The tag space bears some similarity with tuplespaces [18, 36]. While both offer persistent sharedmemory for applications, the essential difference isthat the tag space is local to each node. Also, un-like tuple spaces, the tag space provides SMs withI/O tags for interaction with the local OS and I/Osubsystem. The concept of I/O tags share the samegoal with Linux Procfs [5] which allows user-levelprograms to access certain kernel information.

Content-based naming has been recently pre-sented for both the Internet [9, 25, 57] and sen-sor networks [26]. SMs use content-based migra-tion to reach the nodes of interest. This high-levelmigration function implements routing algorithmswhich leverage work done for mobile ad hoc net-works [31,35,46].

Although the security for both mobile agents [23,34] and ad hoc networks [29, 47] have been exten-sively studied, we have faced a new and more dif-ficult problem: how to define a security architec-ture for a system based on execution migration overmobile ad hoc networks? Given the complexity ofthis problem, our current architecture provides solu-tions for protecting the hosts against SMs and SMsagainst each other. It is much harder, however, toprevent an SM from being tampered by a malicioushost. Since SMs have to execute at any host, end-to-end authentication based on digital signatures or en-crypting the entire message are not possible. Hard-ware solutions [6, 45] represent an option, but theyinvolve extra-costs. Complete software solutions arenot known yet, but code confusion and encryptiontechniques have been investigated [20,53] in the con-text of mobile agents.

Coupled with security comes the issue of ad-mission control at nodes. A significant amount

of research has been done to solve this prob-lem for real time systems [52, 55] and active net-works [21, 40]. Given that we did not want tolimit the expressibility of the programming lan-guage (e.g., SNAP [40]), our solution is basedon user-provided lower bounds for resources andnon-preemptive execution. Each node has the flex-ibility to implement its own scheduling and re-source allocation policies which are typically inte-grated. These policies guarantee enough resourcesto satisfy the lower bounds and let the SM mi-grate in case no more resources are allocated. Aproblem that remains to be solved is how to pro-tect the network, as a whole, against malicious SMsthat waste network resources, but respect the ad-mission contract at each node. TTL-based [19]or market-based [23] schemes offer possible solu-tions.

11. Conclusions

In this paper, we have presented the Smart Mes-sages (SMs) platform for distributed computing innetworks of embedded systems (NES). SMs are dis-tributed applications which overcome the scale, het-erogeneity, and volatility encountered in NES bymigrating the execution to nodes of interest, usingapplication-controlled routing, instead of using end-to-end communication among nodes. The main fea-ture of the SM programming model is its high flexi-bility in the presence of dynamic network configura-tions. The experimental results as well as the anal-ysis of our real world application (EZCab) indicatethat the SM computing platform can be a feasiblesolution for programming NES.

Acknowledgments

The authors would like to thank Deepa Iyer forher contribution in developing the SM prototype,and Chalermek Intanagonwiwat for our useful dis-cussions regarding the SM design. We would alsolike to thank the anonymous reviewers which helpedus improve this paper.

References

[1] Linux Devices. http://www.linuxdevices.com.

[2] K Virtual Machine.http://java.sun.com/products/cldc/.

[3] XML. http://www.w3.org/XML/.

[4] JavaSpaces.http://wwws.sun.com/software/jini/specs/jini1.1html/js-title.html.

[5] Linux Kernel Procfs.http://www.kernelnewbies.org/documents/kdoc/procfs-guide/intro.html.

[6] Trusted Computing.http://www.cl.cam.ac.uk/ rja14/tcpa-faq.html.

[7] Accetta, M., Baron, R., Bolosky, W., Golub,D., Rashid, R., Tevanian, A., and Young, M.Mach: A new kernel foundation for unix develop-ment. In Proceedings of the USENIX 1986 SummerConference (Atlanta, GA, July 1986), pp. 93–113.

[8] Adhikari, S., Paul, A., and Ramachandran, U.D-Stampede: Distributed Programming System forUbiquitous Computing. In Proceedings of the 22ndInternational Conference on Distributed Comput-ing Systems (ICDCS 2002) (Vienna, Austria, July2002), pp. 209–216.

[9] Adjie-Winoto, W., Schwartz, E., Balakrish-nan, H., and Lilley, J. The Design and Imple-mentation of an Intentional Naming System. In Pro-ceedings of the 17th ACM Symposium on OperatingSystems Principles (SOSP 1999) (Charleston, SC,1999), ACM Press, New York, NY, pp. 186–201.

[10] Agesen, O. GC Points in a Threaded Environment.Tech. Rep. SMLI TR-98-70, Sun Microsystems Lab-oratories, Palo Alto, CA, December 1998.

[11] Bloom, B. Space/time trade-offs in hash codingwith allowable errors. Communication of the ACM13, 7 (July 1970), 422–426.

[12] Blum, B., Nagaraddi, P., Wood, A., Abdelza-her, T., Son, S., and Stankovic, J. An En-tity Maintenance and Connection Service for Sen-sor Networks. In Proceedings of the First Interna-tional Conference on Mobile Systems, Applications,and Services (MobiSys 2003) (San Francisco, CA,May 2003), pp. 201–214.

[13] Bonnet, P., Gehrke, J. E., and Seshadri, P.Querying the Physical World. IEEE Personal Com-munications 7, 5 (October 2000), 10–15.

[14] Borcea, C., Intanagonwiwat, C., Saxena, A.,and Iftode, L. Self-Routing in Pervasive Comput-ing Environments using Smart Messages. In Pro-ceedings of the 1st IEEE International ConferenceonPervasive Computing andCommunications (Per-Com 2003) (Dallas-Fort Worth, TX, March 2003),pp. 87–96.

[15] Borcea, C., Iyer, D., Kang, P., Saxena, A., andIftode, L. Cooperative Computing for DistributedEmbedded Systems. In Proceedings of the 22ndInternational Conference on Distributed Comput-ing Systems (ICDCS 2002) (Vienna, Austria, July2002), pp. 227–236.

[16] Boulis, A., Han, C., and Srivastava, M. De-signandImplementationofaFramework forEfficientand Programmable Sensor Networks. In Proceed-ings of the First International Conference on Mobile

Systems, Applications, and Services (MobiSys 2003)(San Francisco, CA, May 2003), pp. 187–200.

[17] Cahill, V., and et al. Using trust for secure col-laboration in uncertain environments. In PervasiveComputing, IEEE (2003), vol. 2(3), pp. 52–61.

[18] Carriero, N., and Gelernter, D. Linda in con-text. Communications of theACM32, 4 (April 1989),444–458.

[19] D. Wetherall. Active Network Vision Reality:Lessons from a Capsule-based System. In Proceed-ings of the 17th ACM Symposium on Operating Sys-tems Principles (SOSP 1999) (Charleston, SC, De-cember1999),ACMPress,NewYork,NY,pp. 64–79.

[20] F.Hohl. Time Limited Blackbox Security: Protect-ing Mobile Agents from Malicious Hosts. In Mo-bile Agents and Security, G. Vigna, Ed., vol. 1419 ofLectureNotes inComputer Science. Springer-Verlag,London, UK, 1998, pp. 92–113.

[21] Galtier, V., Mills, K., Carlinet, Y., Bush, S.,and Kulkarni, A. Predicting resource demand inheterogeneous active networks. In Military Commu-nications Conference, 2001 (MILCOM 2001). Com-munications forNetwork-CentricOperations: Creat-ing the Information Force (Washington, D.C., Octo-ber 2001), pp. 905–909.

[22] Gray, R., Cybenko, G., Kotz, D., and Rus, D.Mobile Agents: Motivations and State of the Art.In Handbook of Agent Technology, J. Bradshaw, Ed.AAAI/MIT Press, 2002.

[23] Gray, R., Kotz, D., Cybenko, G., and Rus, D.D’Agents: Security in a multiple-language, mobile-agent system. In Mobile Agents and Security, G. Vi-gna, Ed., vol. 1419 of Lecture Notes in Computer Sci-ence. Springer-Verlag, London, UK, 1998, pp. 154–187.

[24] Grimm, R., and et al. Systems Directions for Per-vasive Computing. In Proceedings of the 8th Work-shop on Hot Topics in Operating Systems (HotOS-VIII) (Elmau/Oberbayern, Germany, May 2001),IEEE Computer Society, Washington, DC, pp. 147–151.

[25] Gritter, M., and Cheriton, D. An Architecturefor Content Routing Support in the Internet. In Pro-ceedings of the 3rd USENIX Symposium on InternetTechnologies and Systems (USITS 2001) (San Fran-cisco, CA, March 2001), pp. 37–48.

[26] Heideman, J., Silva, F., Intanagonwiwat, C.,Govindan, R., Estrin, D., and Ganesan, D.Building Efficient Wireless Sensor Networks withLow-Level Naming. In Proceedings of the 18th ACMSymposium on Operating Systems Principles (SOSP2001) (Banff, Canada, October 2001), ACM Press,New York, NY, pp. 146–159.

[27] Heinzelman, W. R., Kulik, J., and Balakrish-nan,H. AdaptiveProtocols for InformationDissem-ination in Wireless Sensor Networks. In Proceedings

of the Fifth annual ACM/IEEE International Con-ference onMobileComputing andNetworking (Mobi-Com 1999) (Seattle,WA,August 1999),ACMPress,New York, NY, pp. 174–185.

[28] Hill, J., Szewczyk, R., Woo, A., Hollar, S.,Culler, D., and Pister, K. System ArchitectureDirections for Networked Sensors. In Proceedings ofthe Ninth International Conference on ArchitecturalSupport for Programming Languages and OperatingSystems (ASPLOS-IX) (Cambridge,MA,November2000), ACM Press, New York, NY, pp. 93–104.

[29] Hu, Y., Perrig, A., and Johnson, D. Ariadne: asecure on-demand routing protocol for ad hoc net-works. In Proceedings of the 8th annual ACM/IEEEInternational Conference on Mobile Computing andNetworking (MobiCom2002) (Atlanta,GA,Septem-ber 2002), ACM Press, New York, NY, pp. 12–23.

[30] Intanagonwiwat, C., Govindan, R., and Es-trin, D. Directed Diffusion: A Scalable and Ro-bustCommunicationParadigmforSensorNetworks.In Proceedings of the Sixth annual ACM/IEEE In-ternational Conference on Mobile Computing andNetworking (MobiCom 2000) (Boston, MA, August2000), ACM Press, New York, NY, pp. 56–67.

[31] Johnson, D., and Maltz, D. Dynamic SourceRouting in Ad Hoc Wireless Networks. T. Imielin-ski and H. Korth, (Eds.). Kluwer Academic Publish-ers, 1996.

[32] Juang, P., Oki, H., Wang, Y., Martonosi, M.,Peh, L., and Rubenstein, D. Energy-EfficientComputing for Wildlife Tracking: Design Tradeoffsand Early Experiences with ZebraNet. In Proceed-ings of the Tenth International Conference on Ar-chitectural Support for Programming Languages andOperating Systems (ASPLOS-X) (San Jose, CA, Oc-tober 2002), ACM Press, New York, NY, pp. 96–107.

[33] Karnik, N., and Tripathi, A. Agent Server Ar-chitecture for the Ajanta Mobile-Agent System. InProceedings of the 1998 International Conferenceon Parallel and Distributed Processing Techniquesand Applications (PDPTA’98) (Las Vegas, NV, July1998), pp. 66–73.

[34] Karnik, N., and Tripathi, A. Security in theAjanta Mobile Agent System. Software Practice andExperience 31, 4 (January 2001), 301–329.

[35] Karp, B., and Kung, H. Greedy Perimeter State-less Routing for Wireless Networks. In Proceedingsof the Sixth annual ACM/IEEE International Con-ference onMobileComputing andNetworking (Mobi-Com 2000) (Boston,MA,August 2000),ACMPress,New York, NY, pp. 243–254.

[36] Lehman, T., Cozzi, A., Xiong, Y., Gottschalk,J.,Vasudevan,V.,Landis, S.,Davis,P.,Khavar,B., and Bowman, P. Hitting the distributed com-puting sweet spot with tspaces. Computer Net-works: The International Journal of Computer andTelecommunicationsNetworking 35, 4 (March2001),457–472.

[37] Levis, P., and Culler, D. Mate: A Virtual Ma-chine for Tiny Networked Sensors. In Proceedingsof the Tenth International Conference on Architec-tural Support for Programming Languages and Oper-ating Systems (ASPLOS-X) (San Jose, CA, October2002), ACM Press, New York, NY, pp. 85–95.

[38] Madden, S., Franklin,M.,Hellerstein, J., andHong, W. The Design of an Acquisitional QueryProcessor for Sensor Networks. In Proceedings ofthe 2003ACMSIGMOD international conference onManagement of data (San Diego, CA, June 2003),ACM Press, New York, NY, pp. 491–502.

[39] Milojicic, D., Douglis, F., Paindaveine, Y.,Wheeler, R., and Zhou, S. Process migration.ACM Computing Surveys 32, 3 (September 2000),241–299.

[40] Moore, J., Hicks, M., and Nettles, S. PracticalProgrammable Packets. In Proceedings of the 20thAnnual Joint Conference of the IEEE Computer andCommunications Societies (INFOCOM 2001) (An-chorage, AK, April 2001), pp. 41–50.

[41] Morris, R., Jannotti, J., Kaashoek, F., Li, J.,and Decouto, D. CarNet: A Scalable Ad HocWireless Network System. In Proceedings of the 9thACM SIGOPS European Workshop (Kolding, Den-mark, September 2000), ACM Press, New York, NY,pp. 61–65.

[42] Ni, Y., Kremer, U., and Iftode, L. SpatialViews:space-awareprogramming fornetworks of em-bedded systems. In Proceedings of the 16th Inter-national Workshop on Languages and Compilers forParallel Computing (LCPC 2003) (College Station,TX, October 2003).

[43] Ousterhout, J., Cherenson, A., Douglis, F.,Nelson, M., and Welch, B. The sprite networkoperating system. IEEE Computer 21, 2 (February1988), 23–36.

[44] P. Koopman. Critical Embedded Automotive Net-works. IEEE Micro 22, 4 (July-August 2002), 14–18.

[45] Palmer, E. An Introduction to Citadel - A SecureCypto Coprocessor for Workstations. In Proceed-ings of IFIP SEC’94 Conference (Curacao, DutchAntilles, May 1994).

[46] Perkins, C., and Royer, E. Ad Hoc On DemandDistance Vector Routing. In Proceedings of the 2ndIEEE Workshop on Mobile Computing Systems andApplications (WMCSA 1999) (New Orleans, LA,February 1999), pp. 90–100.

[47] Perrig, A., Szewczyk, R., Wen, V., Culler,D., and Tygar, J. SPINS: Security Protocolsfor Sensor Netowrks. In Proceedings of the 7th an-nual ACM/IEEE International Conference on Mo-bile Computing and Networking (MobiCom 2001)(Rome, Italy, July2001),ACMPress,NewYork,NY,pp. 189–199.

[48] Ponnekanti, S., Lee, B., Fox, A., Hanrahan,P., and Winograd, T. ICrafter: A Service Frame-work for Ubiquitous Computing Environments. In

Proceedings of the Third International Conferenceon Ubiquitous Computing (Ubicomp) (Atlanta, GA,September 2001), Springer-Verlag, London, UK,pp. 56–75.

[49] Priyantha, N., Miu, A., Balakrishnan, H., andTeller,S.TheCricketCompass forContext-AwareMobile Applications. In Proceedings of the 7th an-nual ACM/IEEE International Conference on Mo-bile Computing and Networking (MobiCom 2001)(July 2001), ACM Press, New York, NY, pp. 1–14.

[50] Rhea, S., andKubiatowicz, J. ProbabilisticLoca-tion and Routing. In Proceedings of the 21th AnnualJoint Conference of the IEEE Computer and Com-munications Societies (INFOCOM’02) (New York,NY, June 2002), pp. 1248–1257.

[51] Roman, M., and Campbell, R. GAIA: En-abling Active Spaces. In Proceedings of the 9thACM SIGOPS European Workshop (Kolding, Den-mark, September 2000), ACM Press, New York, NY,pp. 229–234.

[52] Rosu, D., Schwan, K., and Yalamanchili, S.Fara - a framework for adaptive resource allocationin complex real-time systems. In Proceedings ofthe Fourth IEEEReal-TimeTechnology andApplica-tions Symposium (Denver, CO, May 1998), pp. 79–84.

[53] Sander, T., and Tschudin, C. Protecting MobileAgents against Malicious Hosts. In Mobile Agentsand Security, G. Vigna, Ed., vol. 1419 of LectureNotes in Computer Science. Springer-Verlag, 1998,pp. 44–60.

[54] Schwartz, B., Jackson, A., Strayer, W., Zhou,W., Rockwell, R., and Partridge, C. Smartpackets: Applying active networks to network man-agement. ACM Transactions on Computer Systems18, 1 (2000), 67–88.

[55] Stankovic, J., and Ramamritham, K. The springkernel: A new paradigm for real-time systems. IEEESoftware 8 (May 1991), 62–72.

[56] Stanley-Marbell, P., and Iftode, L. Scylla:A smart virtual machine for mobile embedded sys-tems. In 3rd IEEE Workshop on Mobile Comput-ing Systems and Applications, WMCSA2000 (Mon-terey, CA, December 2000), pp. 41–50.

[57] Vahdat, A., Dahlin, M., Anderson, T., and Ag-garwal, A. Active Names: Flexible Location andTransport of Wide-Area Resources. In Proceedingsof the Second USENIX Symposium on Internet Tech-nologies and Systems (USITS 1999) (Boulder, CO,October 1999), pp. 151–164.

[58] Wan, C., Campbell, A., and Krishnamurthy,L. PSFQ: A Reliable Transport Protocol For Wire-less Sensor Networks. In Proceedings of the 1stACMinternationalworkshoponWireless sensornet-works and applications (WSNA 2002) (Atlanta, GA,September 2002), ACM Press, New York, NY, pp. 1–11.