MaNEA A distributed architecture for enforcing norms in ... · MaNEA: A distributed architecture for enforcing norms in open MAS N. Criadoa,n, E. Argentea, P. Noriegab, V. Bottia

Engineering Applications of Artificial Intelligence 26 (2013) 76–95

Contents lists available at SciVerse ScienceDirect

Engineering Applications of Artificial Intelligence

0952-19

http://d

n Corr

E-m

eargent

vbotti@

journal homepage: www.elsevier.com/locate/engappai

MaNEA: A distributed architecture for enforcing norms in open MAS

N. Criado a,n, E. Argente a, P. Noriega b, V. Botti a

a Departamento de Sistemas Informaticos y Computacion, Universitat Polit�ecnica de Val�encia, Camino de Vera s/n. 46022, Valencia, Spainb Institut d’Investigacio en Intel.lig�encia Artificial, Consejo Superior de Investigaciones Cientıficas, Campus de la UAB, Bellaterra, Catalonia, Spain

a r t i c l e i n f o

Article history:

Received 2 December 2011

Received in revised form

17 July 2012

Accepted 21 August 2012Available online 30 September 2012

Keywords:

Norm enforcement

Agent platforms

Open MAS

76/$ - see front matter & 2012 Elsevier Ltd. A

x.doi.org/10.1016/j.engappai.2012.08.007

esponding author.

ail addresses: [email protected] (N. Criado)

[email protected] (E. Argente), [email protected]

dsic.upv.es (V. Botti).

a b s t r a c t

Norms have been promoted as a coordination mechanism for controlling agent behaviours in open

MAS. Thus, agent platforms must provide normative support, allowing both norm-aware and non-

norm-aware agents to take part in MAS that are controlled by norms. In this paper, the most relevant

proposals on the definition of norm enforcement mechanisms are analyzed. These proposals present

several drawbacks that make them unsuitable for open MAS. In response to these problems, this paper

describes a new Norm-Enforcing Architecture aimed at controlling norms in open MAS. Specifically, this

architecture supports the creation and deletion of norms on-line as well as the dynamic activation and

expiration of instances. Finally, it can dynamically adapt to different scale MAS. The efficiency of this

architecture has been experimentally evaluated and the results are shown in this paper.

& 2012 Elsevier Ltd. All rights reserved.

1. Introduction

The main applications of Multi-Agent Systems (MAS) supportlarge scale open distributed systems. These systems are characterizedby the heterogeneity of their participants; their limited trust; a highuncertainty; and the existence of individual goals that might be inconflict (Artikis and Pitt, 2001). In these scenarios, norms areconceived as an effective mechanism for achieving coordination andensuring social order; i.e., norms represent an effective tool forregulating the actions of software agents and the interactions amongthem (Lopez et al., 2006). Most of the proposals on methodologiesand guidelines aimed at developing open MAS (Argente et al., 2011b;Dignum et al., 2005) are based on organizational concepts, such asnorms. These concepts facilitate the analysis and design of coordina-tion and collaboration mechanisms for MAS. Therefore, norms shouldbe considered in the design and specification of the MAS (Criado et al.,2011b). As pointed out in Castelfranchi (2003), the use of norms inMAS allows better results to be achieved in dynamic and complexenvironments. Specifically, the fact that agents can violate normsautonomously allows a better adaptation to the environmentalchanges. Finally, the occurrence of norm violations can evidence theneed to adapt the MAS (Bernon et al., 2003). Agent platforms are thesoftware that supports the development and execution of MAS. Thus,norms must be also considered in the design and implementation ofagent platforms (Criado et al., 2011b). As a consequence, agentplatforms must implement norms in an optimized way, given thatin open MAS the internal states of agents are not accessible (Criado

ll rights reserved.

,

(P. Noriega),

et al., 2011a). Therefore, norms cannot be imposed as agent’s beliefsor goals, but they must be implemented in the platforms by means ofcontrol mechanisms (Grossi et al., 2007).

This paper shows an overview of the most relevant works onnorm implementation. This paper considers the main challenges ofopen MAS and points out the main deficiencies and drawbacks ofagent platforms and infrastructures when supporting norms. Withthe aim of overcoming some of these problems, in this paper a Norm-Enforcing Architecture, known as MaNEA, is proposed. Specifically,MaNEA has been integrated into the Magentix2 platform.1 TheMagentix2 platform allows the management of open MAS in a secureand optimized way. Its main objective is to bring agent technology toreal domains: business, industry, e-commerce, among others. Thisgoal entails the development of more robust and efficient mechan-isms for enforcing norms that control these complex applications.

This paper is organized as follows: Section 2 contains theanalysis of the main proposals on norm enforcement; Section 3describes briefly the Magentix2 platform; Section 4 describes themain components of MaNEA; Section 5 illustrates the perfor-mance of MaNEA through a case study; Section 6 contains anevaluation of this architecture; and, finally, Section 7 containsconclusions and future works.

2. Related work

Most of the proposals on norms for controlling MAS tackle thisissue from a theoretical perspective (Boella, 2004; Sergot, 1998).However, there are also works on norms from a computational pointof view. These works propose control mechanisms for norms to have

1 http://magentix2.gti-ia.upv.es/.

www.elsevier.com/locate/engappai

www.elsevier.com/locate/engappai

dx.doi.org/10.1016/j.engappai.2012.08.007



mailto:[email protected]




http://magentix2.gti-ia.upv.es/


N. Criado et al. / Engineering Applications of Artificial Intelligence 26 (2013) 76–95 77

an effective influence on agent behaviours (Grossi et al., 2007). Thesecontrol mechanisms are classified into two categories (Grossi et al.,2007): regimentation mechanisms, which consist of making theviolation of norms impossible; and enforcement mechanisms, whichare applied after the detection of norm violations, reacting upon them.

2.1. Norm regimentation

Regimentation forces ideality (expressed as norms) and reality(defined by agents’ behaviour) to coincide (Jones and Sergot,1993). Thus, regimentation mechanisms prevent agents fromperforming forbidden actions by mediating the resources andthe communication channel, such as Electronic Institutions (EIs)(Esteva et al., 2004). However, the regimentation of all actions canbe not only difficult or impossible, but also it is sometimespreferable to allow agents to violate norms (Castelfranchi,2003). In fact, Open MAS that operate in complex and changingenvironments may benefit from the occurrence and detection ofnorm violations. The reasons behind desirability of norm viola-tions are that it is impossible to take a thorough control of allagents’ actions; or agents could obtain higher personal benefitswhen norms are violated; or norms may be violated by functionalor cooperative motivations. For example, changes in the environ-ment may cause norms to lose their validity. In this situation,autonomous agents may deviate from norms in order to achievetheir own goal, which may imply a better performance of thewhole MAS. All these situations require norms to be controlledbut allowing norm violations to occur. In response to this need,the enforcement mechanisms have been developed, which areexplained in the following subsection.

2.2. Norm enforcement

Proposals on the enforcement of norms can be classifiedaccording to the entity that observes whether norms are fulfilledor not. Specifically, norm compliance must be observed by agentsor the infrastructure may provide mechanisms for monitoringagent activities according to norms. Each one of these approachesto norm enforcement is explained below.

2.2.1. Agent observability

This approach is characterized by the fact that norm violationsmay be observed by agents that are involved in an interaction inwhich the norm has been violated (second-party observability), orother agents that observe an interaction in which they are notdirectly involved (third-party observability).

Second-party observability

There are proposals (such as Venkatraman and Singh, 1999;Daskalopulu et al., 2002) in which the agents involved in aninteraction are responsible for monitoring norms. In theseapproaches, agents evaluate their interaction partners subjec-tively. In accordance with this evaluation, agents may punish orreward their partners (Boella, 2003) or they may start a grievanceprocedure (Criado et al., 2010b). For example, Venkatraman andSingh (1999) propose an approach for testing the compliance ofagents with respect to a commitment. Commitments are specifiedin temporal logic and their compliance is evaluated with respectto locally constructed models for the given observer. The workcontained in Daskalopulu et al. (2002) proposes a framework forcontract performance arbitrating. In particular, it uses subjectivelogic (Jøsang, 2001) as the formal basis for evidence-basedreasoning. Subjective logic addresses the problem of forming ameasurable belief about a proposition on the basis of insufficientevidence, or in the presence of uncertainty and ignorance.

Third-party observability

If there are agents that are not directly involved in an interactionbut that are capable of observing it, they would be also capable offorming an own image about the interacting participants. Moreover,these evaluations or reputations may be exchanged. Thus, agents arepersuaded to obey norms because their non-normative behaviour canbe observed by others. In this case, the society as a whole acts as normenforcer (Sen and Airiau, 2007). These non-compliant agents mighteven be excluded from the society (de Pinninck et al., 2007). The roleof emotions in social enforcement (Elster, 1996) is also interesting.For example, the work described in Fix (2006) models the emotion-based enforcement of norms in agent societies. In this approach, thewhole society observes compliance of norms and generates socialemotions such as contempt or disgust, in case of norm violation; andadmiration or gratefulness, in case of norm fulfilment. In the sameway, agents observe the expression of these emotions and are alsoable to generate emotions such as shame or satisfaction in response.

The main drawback of proposals on second-party and third-party observability is the fact that the underlying infrastructuredoes not offer support for enforcing norms. Thus, the normmonitoring and the reaction to violations must be implementedby agent programmers. In this sense, agent programmers areresponsible for watching over norm compliance. Even if theinfrastructure provides authority entities that act as arbiters orjudges in grievance processes, agents must be endowed withcapabilities for both detecting norm violations and participatingin these dispute resolution processes.

2.2.2. Infrastructural observability

Normative agent platforms provide entities that are in chargeof both observing and enforcing norms. Cardoso and Oliveira(2007) propose a norm-enforcing architecture in which themonitoring and enforcement of norms is made by a singleinstitutional entity, named as normative environment. This entityreceives all messages that have been exchanged among agentsand determines if an agent has violated (vs. fulfilled) a norm.In this case, the normative environment sends a sanctioning(vs. rewarding) notification to this agent. As argued by Cardosoand Oliveira the implementation of the normative environment asa centralized component represents a performance limitationwhen dealing with a considerable number of agents.

To address the performance limitation of centralizedapproaches, distributed mechanisms for an institutional enforce-ment of norms are proposed in Minsky and Ungureanu (2000) andGaertner et al. (2007). These works propose languages for expres-sing norms and software architectures for the distributed enfor-cement of these norms. Minsky and Ungureanu (2000) present anenforcement mechanism that is implemented by the Mosestoolkit (Minsky and Ungureanu, 1998). Its performance is asgeneral (i.e., it can implement all norms that are controllable bya centralized enforcement) and more scalable and efficient thancentralized approaches. However, one of the main drawbacks ofthis proposal is the fact that norms can only be expressed in termsof the messages sent or received by an agent; i.e., this frameworkdoes not support the definition of norms that affect an agent as aconsequence of an action carried out independently by anotheragent. This problem is overcome by Gaertner et al. (2007). In theirapproach, Gaertner et al. propose a distributed architecture forenforcing norms in EI. Specifically, this architecture only controlsdialogical actions. Thus, the dialogical actions performed byagents cause the propagation of normative propositions(i.e., obligations, permissions, and prohibitions). These normativepropositions are taken into account by the normative level; i.e., ahigher level in which norm reasoning and management processesare performed in a distributed manner.

N. Criado et al. / Engineering Applications of Artificial Intelligence 26 (2013) 76–9578

In a more recent work, Modgil et al. (2009) propose a generalarchitecture for monitoring norm-governed systems. Specifically,it is a two layer architecture in which observers (i.e., the lowestlayer) are capable of reporting to monitors (i.e., the highest layer)on states of interest relevant to the activation, fulfilment, viola-tion and expiration of norms. Monitors determine if a violation orfulfilment has occurred and they take remedial actions accord-ingly. The proposal of Modgil et al. does not give any detail of howthe monitoring and observation levels can be dynamically dis-tributed into a set of coordinated entities in response to achanging environment. Thus, this architecture is not capable ofdynamically adapting to situations in which the number of normsand agents to be controlled may change drastically.

2.3. Mixed approaches

Finally, there are works that use a mixed approach for controllingnorms. They propose the usage of regimentation mechanisms toensure compliance with norms that preserve the integrity of theapplication; However, infrastructural enforcement is also used tocontrol norms that cannot be regimented due to the fact that theyare not verifiable or their violation may be desirable. An example ofthis mixed approach is proposed by Criado et al. (2011b). This workshows how those norms that define permissions and prohibitionsrelated to the management of the organizational infrastructureare regimented through mediation, whereas obligation norms areenforced by the infrastructure that perform sanctions and rewards.However, these remedial actions can only be expressed in terms ofactions that are under the control of the organizational infrastruc-ture. Therefore, Criado et al. proposal performs a partial control ofnorms since only those norms that regulate the access to theorganizational infrastructure are controlled. The ORA4MAS(Hubner et al., 2010) is another well-known proposal that makesuse of a mixed approach for implementing norms.

The ORA4MAS proposal defines artifacts as first-class entities toinstrument the organization to support agent activities within it.Artifacts are resources and tools that agents can create and use toperform their individual and social activities (Omicini et al., 2008).Regarding the implementation of norms in the ORA4MAS framework,regimentation mechanisms are implemented in artifacts that agentsuse to access the organizational infrastructure (i.e., mediation). Theenforcement of norms is implemented by artifacts, which detectnorm violations, and agents, which are informed about norm viola-tions and carry out the evaluation and judgement of these situations.In these two proposals only the norms that regulate the access to theplatform are controlled whereas other domain dependent norms arenot automatically controlled (i.e., evaluated and judged).

2.4. Requirements for norm enforcing architectures

As being illustrated by the previous section, proposals that providesupport to norm-enforcing present some drawbacks that make them

Table 1Summary of distributed proposals on infrastructural enforcement.

Violationdetection

Remedialapplication

Normativeinformation

Messageexchange

Actioperfo

Cardoso and

Oliveira (2007)O O – O O

Minsky and

Ungureanu (2000)O O – O –

Gaertner et al.

(2007)O O – O O

Modgil et al.

(2009)O O – O O

unsuitable for controlling norms in open MAS. In summary, the mostimportant requirements for norm-enforcing architectures are:

�

nrma

Automatic enforcement. It must provide support for the detection ofnorm violations and the application of remedial mechanisms. Itimplies that agents can trust the enforcement system that willsanction their partners if they behave dishonestly. Moreover, theenforcement architecture must provide normative information inorder to allow agents to realize that they or other agents haveviolated a norm. Thus, agents are persuaded to obey norms notonly by a material system of sanctions but also since their non-normative behaviour can be observed by others, who may rejectto interact with them in the future.
� Control of general norms. It must control complex and general
norms. Thus, it must allow the definition and management ofnorms that control not only the messages exchanged amongagents but also other actions carried out by agents. In addition,it must support the enforcement of norms that control statesof affairs. Finally, it must control norms that are defined interms of actions and states of affairs that occur independently(e.g., actions that are performed by different agents).
� Dynamic enforceable content. Dynamic situations may cause norms
to lose their validity or to need to be adapted. Thus, norm-enforcing mechanisms should provide solutions to open MAS inwhich the set of norms evolves along time. Moreover, it mustprovide support for the enforcement of unforeseen norms thatcontrol activities and actions that are defined on-line.
� Efficient, distributed and robust. Finally, enforcement mechanisms
must provide the possibility of performing this task in a distributedway. This distributed architecture must be capable of operatingquickly, effectively and orderly in changing environments in whichthe number of agent, norms, etc. may change drastically.

Table 1 summarizes the performance of the proposals oninfrastructural enforcement with respect to these requirements.In particular, the automatic enforcement feature consists of threedifferent activities: (i) the detection of norm violations (Violation

Detection column of Table 1), the application of sanctions andrewards (Remedial Application column) and the provision ofnormative information (Normative Information column). Withregard to the type of norms that these proposals control, theyhave been evaluated according to four criteria: the possibility ofcontrolling the messages exchanged by agents (Message Exchange

column), the possibility of controlling actions performed byagents (Action Performance column), the possibility of controllingstates of affairs (States of Affairs column) and the possibility ofcontrolling norms that affect an agent due to a certain actionor message sent by other agent (Independent Situations column).The suitability of these infrastructures for controlling dynamicenvironments has been evaluated according to two criteria: theconsideration of norms that are only active under specific circum-stances (Norm Evolution column) and the explicit consideration of

nceStates ofaffairs

Independentsituations

Normevolution

Normmodification

Distributedarchitecture

O O – – –

– – – – O

– O O – O

O O O – O


norm change (i.e., creating and deleting norms on-line) (Norm

Modification column). Finally these proposals have been evaluatedaccording to the possibility of distributing the norm enforcingarchitecture (Distributed Architecture column). As illustrated inthis table, issues such as the provision of normative information

and the explicit consideration of the norm modification problemhave not been properly addressed by the existing proposals. Withthe aim of meeting these pending requirements and improvingthe efficiency of existing approaches in terms of the messagesthat are required to control norms, we propose in Section 4 aNorm-Enforcing Architecture for controlling norms in the Magen-tix2 platform. Specifically, the Norm-Enforcing Architecture baseson the organization and interaction support offered by Magen-tix2. Next, the Magentix2 platform is briefly described.

3. The Magentix2 platform

Magentix2 is an agent platform for open MAS in which hetero-geneous agents interact and organize themselves into Virtual Organi-zations (VOs) (Foster et al., 2001). VOs are open systems formed by thegrouping and collaboration among heterogeneous entities. In VOsthere is a separation between form and function that requires defininghow behaviour will take place (Ferber et al., 2005). VOs are socialentities formed by agents that try to achieve the organizational goals.These agents are organized in groups that are controlled by norms.

Magentix2 is formed by different building blocks that providesupport for VOs at two levels:

�
Organization level. Magentix2 incorporates the THOMAS frame-work (Argente et al., 2011a), allowing users to manage organiza-tional and service aspects easily. The THOMAS frameworkprovides access to the organizational infrastructure through aset of services included on two main components: the Service
Facilitator (Val et al., 2009), which registers the services providedby entities and facilitates service discovering for potential clients;

Fig. 1. Overview of the Tracing Service su

and the Organization Management System (OMS) (Criado et al.,2010c), which is in charge of the management of VOs, takingcontrol of their underlying structure, the roles played by agents,and the register of the norms that govern the VO.
� Interaction level. Magentix2 provides support to: agent com-
munication, supporting asynchronous reliable messageexchanges and facilitating the interoperability betweenheterogeneous entities; agent conversations (Fogues et al.,2010), which are automated Interaction Protocols; tracing

service support (Burdalo et al., 2011), which allows agents ina MAS to share information in an indirect way by means oftrace events; and, finally, Magentix2 incorporates a security

module (Such et al., 2011) that provides features regardingsecurity, privacy, openness and interoperability.

Norms define what is considered as permitted, forbidden orobligatory in an abstract way. However, norm compliance mustbe controlled considering the actions and messages exchangedamong agents at the interaction level. The Norm-EnforcingArchitecture proposed in this paper tries to fill the gap betweenthe organizational level, at which norms are registered by theOMS; and the interaction level, at which actions and communica-tions between agents can be traced. Next, the Tracing ServiceSupport and the storage of norms, provided by the OMS, aredescribed.

3.1. Tracing service support

In order to facilitate indirect communication (i.e., indirectways of interaction and coordination), Magentix2 providesTracing Service Support (Burdalo et al., 2011) (see Fig. 1). Thisservice is based on the publish/subscribe software pattern,which allows subscribers to filter events satisfying the valuesof some attributes (content-based filtering), so that agents onlyreceive the information in which they are interested and onlyrequested information is transmitted. In addition, security

pport (Burdalo et al., 2011).


policies define which entities are authorized to receive whichspecific events. These tracing facilities are provided by a set ofcomponents named Trace Manager (TM). There can be threetypes of tracing entities (i.e., those elements of the systemcapable of generating and/or receiving events): agents, artifactsor groups of agents.

A trace event or event is a piece of data representing an action,message exchange or situation that has taken place during theexecution of an agent or any other component of the MAS. Generic

events, which represent application independent information, areinstrumented within the code of the platform. Application eventsare domain dependent information.

Definition 1 (Event). An event e is defined as a tuple e¼/Type,Time,Origin,DataS, where:

�
Type is a constant that represents the nature of the informationrepresented by the event; � Time is a numeric value that indicates the global time at which
the event is generated;
� Origin is a constant that identifies the tracing entity that
generates the event;
� Data¼c14 � � �4cn is a conjunction of possibly negated first-
order grounded atomic formulae that contains extra attacheddata required for interpreting the event.

Trace events can be processed or even combined to generatecompound trace events, which can be used to represent morecomplex information.

Any tracing entity is provided with mail boxes for receiving ordelivering events (EIn and EOut). Entities that want to receivecertain trace events request the subscription to these events bysending to the TM a subscription event that contains the templateof those events they are interested in.

Definition 2 (Template). A template t is a tuple t¼/Type,Origin,DataS that contains the filtering specified criteria forevents, where:

�
Type is a constant that represents the nature of the informationrepresented by the event; � Origin is a constant that identifies the entity that generates
the event;
� Data¼c14 � � �4cn is a conjunction of possibly negated
first-order atomic formulae that may contain free variables.

Let us consider the standard notion of substitution as afinite and possibly empty set of pairs X/y where X is a variableand y is a term. Let us also define the application of a substitutions as:

1.
sðcÞ ¼ c if c is a constant. 2. sðXÞ ¼ y if X=yAs; otherwise sðXÞ ¼ X. 3. sðc14 � � �4cnÞ ¼ sðc1Þ4 � � �4sðcnÞ. 4. sð/r0, . . . ,rnSÞ ¼/sðr0Þ, . . . ,sðrnÞS.
Therefore, the application of a substitution on a template isdefined as follows:

sð/Type,Origin,DataSÞ ¼/Type,Origin,sðDataÞS

since Type and Origin take constant values.According to the definitions of events and templates the

matching relationship between events and templates is definedas follows:

Definition 3 (Matching Function). Given an event e¼/Type,Time,Origin,DataS and a template t¼/Type0,Origin0,Data0S, their

matching is a boolean function defined as follows:

matchingðe,tÞ ¼

true ifðType¼ Type0Þ4ððOrigin¼ Origin0Þ

3ðOrigin0 is undefinedÞÞ

4ð8ciAData0 : ciADataÞ

false otherwise

8>>>>>><>>>>>>:

Definition 4 (Unification Function). Given an event e and atemplate t, their unification is a boolean function defined asfollows:

unificationðe,tÞ ¼

true if exists a substitution

of variables s such that

matchingðe,sðtÞÞis true

false otherwise

8>>>>>><>>>>>>:

3.2. Organization Management System (OMS)

The Organization Management System (OMS) (Criado et al.,2010c) is responsible for the management of VOs and theirconstituent entities. The OMS provides a set of services: structural

services, which comprise services for adding/deleting norms(registerNorm and deregisterNorm services allow entities to modifythe norms that are in force or applicable within a VO), and foradding/deleting roles and groups; informative services, whichprovide information of the current state of the organization;and dynamic services, which allow agents to enact/leave rolesinside VOs (acquireRole and leaveRole services). Moreover, agentscan be forced to leave a specific role (expulse service). When theOMS provides any of these services successfully, then it generatesan event for informing about the changes produced in the VO.

3.2.1. Virtual Organization model

VOs have been employed as an abstraction for modelling openMAS. VOs include the integration of organizational and individualperspectives and also the dynamic adaptation of models to organi-zational and environmental changes (Dignum and Dignum, 2002). InCriado et al. (2011b) the model of VO considered by the OMS isdescribed in detail. It is based on the Human Organization Theory(Daft, 2003) and classifies the main aspects of a VO into fourdimensions as follows:

�
The Structural dimension, which describes components of thesystem and their relationships. Thus, it describes the roles andgroups that form a VO. Roles allow dividing the VO function-alities in an abstract way. Groups represent the context inwhich these activities take place. � The Functional dimension, which describes the system function-
ality in terms of the services provided by agents and groups.
� The Normative dimension, which describes the norms defined to
control the society members. Norms are a coordination mechan-ism that attempt to: (i) promote behaviours that are satisfactoryto the organization (i.e., actions that contribute to achievement ofglobal goals); and (ii) avoid harmful actions (i.e., actions thatprompt the system to be unsatisfactory or unstable).
� The Environmental dimension, which describes the environ-
ment in terms of its resources and how agents can perceiveand act on them.

3.2.2. Norm definition

According to the normative definitions provided in Oren et al.(2009), in Magentix2 a distinction among norms and instances is

In Force DeletedregisterNorm deregisterNorm

Fig. 2. Norm life-cycle.


made. Norms define patterns of behaviours by means of deontic

modalities: obligations, which define which actions or states of affairsshould be performed or satisfied by agents; and prohibitions, whichdefine which actions or states of affairs should not be performed orachieved. Magentix2 takes a closed world assumption where every-thing is considered as permitted by default. Therefore, permissionsare not considered in this paper, since they can be defined asnormative operators that invalidate the activation of an obligationor prohibition. Therefore, norms define a pattern of behaviour (ornorm condition in our terminology) as obligatory or prohibited. Thisnorm condition can be represented as actions to be performed orstates of affairs to be achieved. In fact, we make no sharp distinctionbetween actions and states of affairs, since what in one situation isbest described as an action may be best described in another situationas a state of affairs (Lopez et al., 2006). Also inspired by therepresentation of Oren et al. (2009), we define norms as conditionalrules that are relevant to a set of agents under specific circumstances.Thus, the set of agents that is affected by a specific norm are the onesthat are playing the target role of this norm. Thus, norms representthe responsibilities, rights and duties of roles. In general, norms arenot applied at all times, but include the notions of activation andexpiration conditions. Specifically, the activation condition defineswhen obligations and prohibitions must be instantiated and mustbe fulfilled by all agents playing the target role. These instancesremain active, even if the activation condition ceases to hold.Specifically, the expiration condition defines the validity period ordeadline of an instance. Finally, inspired by Lopez et al. (2006), normsalso include information about the enforcement mechanisms:sanctions, to punish agents that do not obey the norm and rewards,for rewarding norm fulfilment.

Definition 5 (Norm). A norm n is defined as a tuple n¼/id,D,T ,A,E,C,S,RS, where:

�
id is the norm identifier; � DAfF ,Og is the deontic modality of the norm, F represents
prohibition and O represents obligation;
� T is the target of the norm, the role to which the norm is
addressed;
� A is the norm activation condition, it defines under which
circumstances the norm is active and must be instantiated;
� E is the norm expiration condition that determines when the
norm expires and no longer affects agents;
� C is the norm condition that represents the action or state of
affairs that is forbidden or obliged;
� S and R describe the sanctioning and rewarding actions that
will be carried out in case of norm violation or fulfilment,respectively.

Since MaNEA builds on the event tracing approach to monitoring,the conditions A, E and C are expressed in terms of eventtemplates.

As previously mentioned, the OMS offers provides agents withservices for creating and deleting norms on-line. Once norms havebeen registered they are in force or applicable. Similarly, normsbecome deleted when they are unregistered. Fig. 2 shows anoverview of the norm life-cycle.

In Magentix2 norms can be classified into two main cate-gories: organizational and functional norms. Examples of normsbelonging to each category are provided below.

Organizational norms

Organizational norms (Criado et al., 2010c) are related toservices offered by the OMS to members of the organization.They establish organizational dynamics, e.g. role management

(role cardinalities, incompatibility between roles) and the proto-col by which agents are enabled to acquire roles. For example, anorganizational norm that forbids any agent to register new normswhen there is a critical situation is defined as follows:

/n1,F ,member,

/restrictedNormativeChange,�,�S,

/freeNormativeChange,�,�S,

/registerNorm,�,normðNÞS,�,�S

According to norm n1 once the restrictedNormativeChange

event is sent, any agent that enacts the member role (it is aspecial role that is implicitly played by all agents in Magentix2) isforbidden to request the register of any norm (i.e., any event thematches the template /registerNorm,�,normðNÞS will be consid-ered as forbidden). This norm will remain active until thefreeNormativeChange event is received.

Similarly, an incompatibility constraint between two roles(r1 and r2), which define that agents cannot play simultaneouslyroles r1 and r2, is modelled by the two following norms:

/n2,F ,r1,

/incompatibilityActivation,�,�S,

/incompatibilityExpiration,�,�S,

/aquireRole,�,roleðr2ÞS,�,�S

/n3,F ,r2,

/incompatibilityActivation,�,�S,

/incompatibilityExpiration,�,�S,

/aquireRole,�,roleðr1ÞS,�,�S

For example, norm n3 defines that once the incompatibility

Activation event has been sent, then any agent that enacts role r1

is forbidden to request the acquisition of role r2. This norm willremain active since the incompatibilityExpiration event is sent.

Functional norms

Functional norms (Criado et al., 2010c) are domain dependentnorms that define the functionality of roles. For example, let ussuppose the case of an assembly line that has been implementedas a hierarchy of agents. Thus, there is a set of robots that performthe different assembly tasks; i.e., the subordinated. These robotsare controlled by a set of agents that monitor and evaluate theirperformance; i.e., the supervisors. Supervisors are responsible fordynamically reorganizing robots in the assembly line to improvethe productivity. To this aim, robot agents are asked for reportinginformation about their performance to auditor agents that willanalyze the performance of the assembly line. This situation canbe modelled as a functional norm defined as follows:

/n4,O,subordinated,

/auditStart,�,taskðTÞS,/auditEnd,�,taskðTÞS,

/taskPerformance,�,performanceðPÞ4taskðTÞS,

�,�S

When an audit stage of a given task (T) starts (i.e., theauditStart event is sent), subordinated agents are obliged to informabout their performance on this task before the audit stage ends(i.e., the auditEnd event is sent).


3.2.3. Instance definition

When the activation condition of a norm holds; i.e., theactivation event is detected, then it becomes active and severalnorm instances (or instances for short) are created, according tothe possible groundings of the activation condition.

Definition 6 (Instance). Given a norm n¼/id,D,T ,A,E,C,S,RS anda perceived event e, an instance i of n is the tuplei¼/id0,D0,T 0,E0,C0,S0,R0S, where:

�

a

unification(e, A) is true, i.e., there is a substitution s such thatmatchingðe,sðAÞÞ is true (the norm is active);
� C0 ¼ sðCÞ, E0 ¼ sðEÞ, S0 ¼ sðSÞ, and R0 ¼ sðRÞ; � id0 ¼ id,D0 ¼D and T 0 ¼ T .
Fig. 3 shows an overview of the instance life-cycle. Instancesare active when their activation conditions hold. When theexpiration condition of an instance is true or the norm that givesrise to the instance is deleted, then the instance expires.

For example, let us suppose that the event

/auditStart,t,s1,taskðassemblingÞS

is sent by an agent (s1). Thus, norm n4 will be instantiated asfollows:

iassembling ¼/n4,O,subordinated,

/auditEnd,�,taskðassemblingÞS,

/taskPerformance,�,

performanceðPÞ4taskðassemblingÞS,�,�S

Definition 7 (Instantiation Function). Given an event e¼/Type,Time,Origin,DataS and a norm n¼/id,D,T ,A,E,C,S,RS, instantiation

is a function that instantiates norm n as follows:

instantiationðe,nÞ ¼/id0,D0,T 0,E0,C0,S0,R0S

where

�
there is a substitution s such that matchingðe,sðAÞÞ is true; Violated
iration
� xp C0 ¼ sðCÞ, E0 ¼ sðEÞ, S0 ¼ sðSÞ, and R0 ¼ sðRÞ; instanceE
le
�
Fulfilled

Expired

expulse

PendingacquireRoleleaveRo

norm condition

Fig. 4. Obligation life-cycle.

Fulfilled

InstanceExpiration

id0 ¼ id,D0 ¼D and T 0 ¼ T .

3.2.4. Power definition

Once the norm activation event has been detected and a newinstance is created, all agents playing the target role are under theinfluence of the new instance. Thus, a normative power (or powerfor short) represents the control over a concrete agent that isplaying the target role.

Definition 8 (Power). Given an instance i¼/id0,D0,T 0,E0, C 0,S0,R0S,a power p is a tuple p¼/id00,D00,T 00,C00,S00,R00,W 00S where:

PendingacquireRole
� leaveRole | expn
id00 ¼ id0,D00 ¼D0,T 00 ¼ T 0,S00 ¼ S0,R00 ¼ R0 are defined as in theinstance;
ulseorm �
Expired condi ation

C00 ¼/C0Type,AgentID,C0DataS such that C0 ¼/C0Type,�,C 0DataS andAgentID is a constant that identifies the agent affected bythe power;

tio Expirulse
� n
Instance

leaveRole | exp
W 00 is a boolean constant that expresses if the event C 00 hasbeen received.
Active Expiredctivation condition expiration conditionnormDeletion

Fig. 3. Instance life-cycle.

The dynamics of powers depends on their deontic modality. Fig. 4shows an overview of the obligation power life-cycle. Specifically, anobligation power is created when an agent enacts the target role ofthis power. From that moment on, the obligation is pending. Whenthe agent affected by the power achieves the norm condition, thenthe obligation is fulfilled, the agent is rewarded, and the power isdeleted. On the contrary, if this agent leaves the role without fulfillingits pending obligations or the instance that gives rise to the powerexpires, then the obligation is violated, the agent is sanctioned, andthe power is deleted. If the agent is forced to leave the role (i.e., it isexpulsed), then the obligation is expired, the power is deleted, andthe agent is neither rewarded nor sanctioned. The dynamics ofprohibition powers is depicted by Fig. 5. Prohibition powers are alsocreated when any agent enacts the target role. From that moment on,every time this agent achieves the norm condition, the prohibition isviolated, and the agent is sanctioned. If the instance that gives rise tothe power expires and the prohibition has not been violated, then theprohibition is fulfilled, the power is deleted, and the agent isrewarded. On the contrary, if the instance that gives rise to thepower expires and the prohibition has been violated, then the poweris deleted, and the agent is not rewarded. Finally, if the agent leavesthe target role, then the power is deleted, and the agent is notrewarded.

For example, let us suppose that there is a robot agent r1 that isplaying the subordinated role. Thus, a new power for controllingthe behaviour of r1 according to n4 will be created as follows:

pr1 ,assembling ¼/n4,O,subordinated,

/taskPerformance,r1,

performanceðPÞ4taskðassemblingÞS,

�,�,falseS

The next section describes the Norm-Enforcing Architectureproposed in this paper. It is a two layer architecture formed by:a higher level in charge of detecting the instantiation of norms; anda lower level in charge of enforcing powers on agents. The operationalsemantics of norms, instances and powers (i.e., how they are created,deleted, fulfilled and violated) is explained below.

Violated

norm condition

Fig. 5. Prohibition life-cycle.


4. Norm-enforcing architecture: MaNEA

The main purpose of MaNEA (Magentix2 Norm-Enforcing

Architecture) is to endow the Magentix2 platform with an infra-structure capable of controlling norms in open MAS whereunforeseen scenarios may occur. Therefore, the number of agentsand the situations that must be controlled through norms maychange at runtime. For this reason, MaNEA has been designed as adistributed architecture. Specifically, MaNEA has been distributedover two layers (see Fig. 6):

�

Figstor

info

The highest layer of MaNEA is formed by Norm Manager (NM)entities. Entities belonging to this level are informed by theOMS about the norms that have been registered and dereg-istered. With this information the NMs observe the behaviourof agents and detect the activation and expiration of instances.Thus, NMs perform two different processes related to thecreation and elimination of norms (named Norm Management

process) and instances (named Instance Management process).
� The lowest layer of MaNEA is formed by Norm Enforcer (NE)
entities that are responsible for controlling the agents’ behaviours.The NEs are informed by the NMs about the instances that havebeen created at a given moment. Moreover, the OMS informs theNEs about the roles that are enacted by each agent. The NEsrequire this information to create powers for controlling all agentsthat are affected by instances. NEs observe the behaviour of these

Norm Manager

Norm Enforcer

PowerInstance

instanceActivation

instanceExpiration| normDeletion

NormInstance

OMS

Role

Norm

Group

RoleEnactment

registerNorm

deregisterNorm

acqu

ireR

ole

| lea

veR

ole

expu

lse

sanc

tion

rew

ard

normconditio

n

RoleEnactment

expi

ratio

nco

nditi

onac

tivat

ion

cond

ition

. 6. MaNEA architecture (ovals denote agents, cylinders denote lists or data

es, boxes denote the NM and the NE and links represent the exchanges of

rmation).

agents and detect the fulfilment and violation of powers. Thus,NEs perform three different processes related to: the maintenanceof the instance list (named Instance Management process), themaintenance of the role enactment list (named Role Enactment

Management process), and the observation of agent behaviours todetect the fulfilment and violation of powers (named Observation

of Behaviours process).

The communication among the NMS, the NEs, the OMS and theagents is carried out in an indirect way by means of trace servicesupport provided by Magentix2 (see Section 3.1).2 Next, both theNM and the NE entities are described in detail.

4.1. Norm Manager

The Norm Manager (NM) is responsible for determining whichnorms are active (i.e., have to be instantiated) at a given moment.Algorithm 1 is the pseudocode of the control loop performed bythe NM. When the NM receives an event (e), then it handles theevent according to the event type. Mainly, the NM carries out aprocess that can be divided into two differentiated tasks: normmanagement and instance management. Thus, the NM maintainsa list (N) that contains all norms that have been registered inMagentix2 and a list (I) that contains all instances that remainactive at a given moment.

4.1.1. Norm Management

In order to maintain the norm list, the NM subscribes to thoseevents sent by the OMS related to the creation and deletion ofnorms (i.e., registerNorm and deregisterNorm events). Thus, whenthe NM receives an event informing about the creation of a newnorm, then it adds this norm into its norm list and subscribes tothe event that activates the norm (i.e., it sends the subscription

event to the TM with the event template A).3

When a norm is deregistered, then the NM removes it fromits norm list. Moreover, it removes all instances that havebeen created out of this norm. For each deleted instance, theNM unsubscribes from its expiration event (i.e., it sends theunsubscription event to the TM with the event template E0)and generates an event for informing about the deletion of thisinstance (i.e., a normDeletion event is sent through the eventsending box).

4.1.2. Instance Management

Once the activation event of a norm is received (i.e., unification

(e, A) is true), then the NM instantiates the norm (i.e., instantiation

(e, n)) and adds it to the instance list. At this moment, the NMsubscribes to the expiration event and informs about the activationof the norm (i.e., the instanceActivation event is sent by the NM).

Algorithm 1. Norm Manager Control Loop.

2 M

the VO

has be

service

(Miche3 F

Require: Event reception box EIn

Require: Event sending box EOut

Require: Norm list N

Require: Instance list I

1:
Add /subscription,NM,/registerNorm,OMS,�SS to EOut
//where NM stands for Norm Manager

2:
Add /subscription,NM,/deregisterNorm,OMS,�SS to EOut
aNEA needs to be informed by means of events about the current state of

and about the actions and messages exchanged by agents. Since MaNEA

en integrated in Magentix2, it has been designed according to the trace

support provided by Magentix2. However, other event driven architectures

lson, 2006) could also be used.

or simplicity we omit the time at which events are generated.


3:
while EIn is not empty do 4: Retrieve e from
EIn // e¼/Type,Time,Origin,DataS
//Norm Management
5:
if Type¼registerNorm then // Data¼/id,D,T ,A,E,C,S,RS
6:
Add Data to N
7:
Add /subscription,NM,AS to EOut
8:
end if 9: if Type¼deregisterNorm and Data in N then
//Data¼/id,D,T,A,E,C,S,RS
10: Remove Data from N
11:
Add /unsubscription,NM,AS to EOut
12:
for all i in I do
//i¼/id0,D0,T 0,E0,C0,S0,R0S
13: if id0 ¼ id then 14: Remove i from I
15:
Add /unsubscription,NM,E0S to EOut
16:
Add /normDeletion,NM,iS to EOut
17:
end if 18: end for 19: end if
//Instance Management

20:
for all n in N do // n¼/id,D,T,A,E,C,S,RS
21:
if unification(e, A)¼true then //the norm is active
22:
i¼ instantiation(e, n)
//i¼/id0,D0,T 0,E0,C 0,S0,R0S is an instance

23:
if i not in I then 24: Add i to I
25:
Add /instanceActivation,NM,iS to EOut
26:
Add /subscription,NM,E0S to EOut
27:
end if 28: end if 29: end for 30: for all i in I do
//i¼/id0,D0,T 0,E0,C0,S0,R0S
31: if unificationðe,E0Þ ¼ true then 32: Remove i from I
33:
Add /unsubscription,NM,E0S to EOut
34:
Add /instanceExpiration,NM,iS to EOut
35:
end if 36: end for 37: end while
Similarly, when the NM receives the expiration event of anyinstance (i.e., unificationðe,E0Þ is true), then it removes the instancefrom the instance list, unsubscribes from the expiration event andinforms about the expiration of this instance (i.e., the instance

Expiration is sent by the NM).Initially, there is a single NM registered in the Magentix2

platform. However, the NM is capable of simple adaptationbehaviours (i.e., replication and death) in response to changingsituations. For example, before the NM collapses (i.e., its eventreception box is full), it might replicate itself and unsubscribefrom the registerNorm event. Thus, the new NM is responsible forcontrolling the activation of the new norms. Similarly, if the NMreaches a state in which it has no norm to control and it is not thelast NM subscribed to the registerNorm event, then it removesitself. These replication and death mechanisms are a simpleexample that illustrates how the highest layer of MaNEA can bedynamically distributed into several NMs. However, the definitionof more elaborated procedures for adapting dynamically to

changing environments (Nakano and Suda, 2005) is a complexissue that is out the scope of this paper.

4.2. Norm Enforcer

The Norm Enforcer (NE) is responsible for controlling agentbehaviour. Thus, it detects violations and fulfilments of norms,and reacts upon it by sanctioning or rewarding agents. Algorithm2 is the control loop executed by the NE. As illustrated by thisalgorithm, the NE maintains a list (I) with the instances that holdat a given moment. Thus, it subscribes to the events sent by theNM that inform about the activation and expiration of instances,and the deletion of norms. Besides that, the NE is also in charge ofcontrolling agents affected by the instances. Thus, it maintains alist P that contains all powers that have been created out ofinstances. To determine which agents are controlled by theseinstances, it also maintains a list (RE) containing informationabout role enactment (i.e., the set of roles that each agent isplaying at a given moment). Thus, the NE subscribes to the eventssent by the OMS that inform about the fact that an agent hasacquired or left a role (acquireRole and leaveRole events). Inaddition, the NE also subscribes to the expel event, which informsabout the fact that a particular agent has been forced to leave arole as a disciplinary measure.

Algorithm 2. Norm Enforcer Control Loop.

Require: Event reception box EIn

Require: Event sending box EOut

Require: Instance list I

Require: Power list P

Require: Role enactment list RE

1:
Add /subscription,NE,/instanceActivation,NM,�SS to EOut
//where NE stands for Norm Enforcer

2:
Add /subscription,NE,/instanceExpiration,NM,�SS to EOut
3:
Add /subscription,NE,/normDeletion,NM,�SS to EOut
4:
Add /subscription,NE,/acquireRole,OMS,�SS to EOut
5:
Add /subscription,NE,/leaveRole,OMS,�SS to EOut
6:
Add /subscription,NE,/expel,OMS,�SS to EOut
7:
while EIn is not empty do 8: Retrieve e from EIn //e¼/Type,Time,Origin,DataS
//y
//Role enactment management
//y
//Instance management
//y
//Observation of Behaviour
75:
end while
As in case of the NM, the NE starts retrieving an event from itsevent reception box. Then, different operations are performedaccording to the type of the event received. Specifically, the NEcarries out a process that can be divided into three differentactivities: role enactment management, instance managementand observation of behaviours.

4.2.1. Role Enactment Management

Algorithm 3 is the pseudocode corresponding to the roleenactment management process. Specifically, when the OMSinforms that an agent (identified by AgentID) has acquired anew role (identified by RoleID), then the NE updates the roleenactment list. Moreover, the list of instances is also checked fordetermining which instances affect the role RoleID. For each oneof these instances, the NE creates a new power addressed to theagent identified by AgentID. In addition, the NE subscribes to the


event expressed in the norm condition in order to be aware ofthe fulfilment or violation of this norm; i.e., it requests itssubscription to the events that match the template C00 ¼/C0Type,AgentID,C0DataS.

When the NE is informed by the OMS about the fact that anagent (identified by AgentID) is not longer playing a role (identi-fied by RoleID) (i.e., leaveRole or expel events are received by theNE), then the role enactment list is updated. Similarly, all powersthat affect the agent AgentID as a consequence of being playingthe role RoleID are removed. Therefore, the NE does not have toobserve the norm condition anymore and unsubscribes from thisevent. Finally, if any agent leaves a role voluntarily (i.e., theleaveRole event is received) before fulfilling its pending obliga-tions, then it is sanctioned (i.e., the NE performs the sanctioningaction S00). The definition of actions that are applied as sanctionsand rewards are domain dependent. For example, these sanctionsmight consist on a degradation of the public evaluation of a seller(as occurs in eBay),4 or malicious agents may be expelled from theorganization by the infrastructure itself, or there may be otherdomain agents in charge of performing sanctions. Besides that,the NE informs about the fact that an agent has been sanctionedfor violating an obligation (i.e., the sanction event is sent throughthe EOut box).

Algorithm 3. Role Enactment Management.

4 h

9:
if Type¼acquireRole then // Data is a pair (AgentID,RoleID)
10:
Add Data to RE
11:
for all i in I do
//i¼/id0,D0,T 0,E0,/C0Type,�,C 0DataS,S0,R0S

12:
if T 0 ¼ RoleID then 13: C 00 ¼/C0Type,AgentID,C0DataS
14:
Add /id0,D0,T 0,C00,S0,R0,falseS to P
15:
Add /subscription,NE,C00S to EOut
16:
end if 17: end for 18: end if 19: if Type¼ leaveRole or Type¼expel then
//Data is a pair (AgentID,RoleID)
20: Remove Data from RE
21:
for all p in P do
//p¼/id00,D00,T 00,C00,S00,R00,W 00S and

C 00 ¼/C00Type,AgentID00,C00DataS

22:
if T 00 ¼ RoleID and AgentID00 ¼ AgentID then 23: Remove p from P
24:
Add /unsubscription,NE,C00S to EOut
25:
if D00 ¼O and W 00¼ False and Type¼ leaveRole
then
//O stands for obligation
26:
Perform S00//against AgentID
27:
Add /sanction,NE,violatedðid00,AgentIDÞS to EOut
28:
end if 29: end if 30: end for 31: end if
4.2.2. Instance Management

This process is depicted in Algorithm 4. When the NE isinformed by the NM about the creation of a new instance (i.e.,the NE receives the instanceActivation event), then the NE updatesits instance list and creates new powers for controlling all the

ttp://www.ebay.com/.

agents that are playing the target role at that moment. The watchcondition (W 00) of powers is initially set to false. Thus, for each oneof the new powers the NE starts to observe indirectly normcompliance by subscribing to the event C 00.

Algorithm 4. Instance Management.

32:
if Type¼ instanceActivation then
// Data¼/id0,D0,T 0,E0,/C0Type,�,C0DataS,S0,R0S

33:
Add Data to I
34:
for all (AgentID,RoleID) in RE do 35: if RoleID¼ T 0 then 36: C00 ¼/C0Type,AgentID,C0DataS
37:
Add /id,D0,T 0,C00,S0,R0,falseS to P
38:
Add /subscription,NE,C 00S to EOut
39:
end if 40: end for 41: end if 42: if (Type¼ instanceExpiration or Type¼normDeletion) and
Data in I then
//Data¼/id,D,T ,E,C,S,RS
43:
Delete Data from I
44:
for all p in P do
//p¼/id00,D00,T 00,C00,S00,R00,W 00Sand

C00 ¼/C 00Type,AgentID,C00DataS

45:
if id0 ¼ id00 then 46: Remove p from P
47:
48:
if Type¼ instanceExpiration then //The agent is responsible for norm fulfilment
49:
if W 00¼ false and D00 ¼O then
//The obligation has not been fulfilled before it has

expired

50:
Perform S00 // against AgentID
51:
52:
end if 53: if W 00
¼ false and D00 ¼F then
//The prohibition has been observed
54:
Perform R00 // in favour of AgentID
55:
Add /reward,NE,fulfilledðid00,AgentIDÞS to EOut
56:
end if 57: end if 58: end if 59: end for 60: end if
When an instance has no longer effect (i.e., the NE receives theinstanceExpiration or normDeletion event), then the NE updates theinstance list and removes all powers created out of this instance.An instance becomes ineffective whenever its expiration condi-tion holds or the norm that has given rise to it is abolished. In thefirst case (i.e., the NM receives the instanceExpiration event),the agents controlled by this instance are responsible for fulfillingthe norm. Thus, if the instance obliges agents to reach some stateof affairs (e.g., agents are obliged to perform an action) and thisstate has not been observed yet (i.e., the watch condition W 00 ofpowers is false), then the offender agents are sanctioned. On thecontrary, if agents are prohibited to reach some situation and theforbidden state has not been observed (i.e., W 00 of powers is false),then agents are rewarded. Finally, if an instance becomes ineffec-tive due to the deletion of a norm, then agents are not responsiblefor the fulfilment of the norm and enforcement actions are notperformed.

http://www.ebay.com/.


Algorithm 5. Observation of Behaviours.

61:
for all p in P do
// p¼/id00,D00,T 00,C00,S00,R00,W 00S and

C00 ¼/C00Type,AgentID,C00DataS

62:
if unificationðe,C 00Þ ¼ true then 63: Remove p from P
64:
if D00 ¼F then //The prohibition has been violated
65:
Add /id00,D00,T 00,C00,S00,R00,trueS to P
66:
Perform S00 // against AgentID
67:
68:
else // The obligation has been fulfilled and it expires
69:
Perform R00 // in favour of AgentID
70:
Add /reward,NE,fulfilledðid00,AgentIDÞS to EOut
71:
72:
end if 73: end if 74: end for
4.2.3. Observation of Behaviours

This functionality is implemented by Algorithm 5. The NE checksfor each one of the powers whether the C 00 event has been detected(i.e., unificationðe,C00Þ is true). If it is the case, then the power isupdated. The watch condition is registered as true indicating that thenorm condition has been perceived. Then, enforcement actions areperformed according to the deontic modality of the power. Forexample, if the power is an obligation, then the obligation isconsidered as fulfilled (i.e., the power is deleted from P) and theagent is rewarded. Similarly, if it is a prohibition, then the agent issanctioned. However, in case of a prohibition the power is notremoved until the norm expires, since sanctions must be applied ifthe forbidden event is detected again.

As in case of the NM, the lowest level of MaNEA has beendescribed assuming that there is a single NE. However, this layermay be formed by a set of specialized NEs. For example, the set ofinstances can be distributed among NEs according to the target role.Thus, each NE is responsible for controlling actions in which a specificset of roles is involved. It is also possible to specialize NEs forcontrolling a specific group of agents independently of the roles thatthey play. Finally, it is also possible to dynamically adapt the amountof NEs by performing cloning and self-deletion operations.

5 EMFGormas developers are currently working on extending it with a new

module to generate code automatically from the model to Magentix2 (Garcia et al.,

2010). In the near future, engineers will be able to directly execute the specifica-

tion of any VO in Magentix2.

5. Case study

The first step that any engineer must perform to implement aMAS in Magentix2 is to model the MAS based on the VO approach.To facilitate this task, engineers are encouraged to use theGORMAS guidelines (Argente et al., 2011b). Specifically, GORMAS(Guidelines for ORganizational Multi-Agent Systems) proposes aset of activities for the analysis and the design of VOs. Theseguidelines cover the requirement analysis, the structure designand the design of the organization-dynamics. Software designersspecify the services must be provided by the system, the internalstructure of this system and the norms that control its behaviour,taking into account the specific features of open MAS. For acomplete description of GORMAS see Argente (2008). Forexample, in the assembly line VO, which has been introduced inSection 3.2.2, the engineer starts with the analysis of the systemrequirements. Specifically, the engineer must identify the usecases (e.g., the assembly line VO may be used by an expert thatwants to reconfigure the assembly line) and the global goals ofthe assembly line (e.g., improve the productivity). Then GORMAS

helps the engineer to select the most suitable topology accordingto the system requirements. In the assembly line, agents can begrouped on the basis of the processes that they carry out. More-over, most of agents perform specialized tasks that are controlledand evaluated by a subset of agents. Thus, the engineer issuggested to model the assembly line as a hierarchy in whichagents enact three main roles (i.e., supervisor, auditor and sub-ordinated). The next step consists of defining the VO function-ality; i.e., the services provided and offered by the different roles.Finally, GORMAS provides the engineer with recommendations todefine the most suitable norms that control the VO according tothe organizational goals, topology, etc. In this phase the engineerdefines the norms that control the assembly line. For example,GORMAS suggests the engineer to create norms to define theprecedence relationships between tasks, to control the access toresources, and so on. Thus, the norm n4 is identified and definedin this step. Engineers are encouraged to use EMFGormas (Garciaet al., 2009) as a tool for modelling MAS. EMFGormas is anengineering tool for open MAS based on GORMAS. It is a CASEtool that simplifies the modelling task and checks the interde-pendencies between the GORMAS models and their coherence.5

The norms must be registered in the OMS and then they will beautomatically monitored and enforced by Magentix2 (i.e., byMaNEA’s entities). Thus, the engineer will only need to pro-gramme the concrete behaviour of the domain agents. Thesedomain agents must take into account norms if they want toavoid sanctions. Thus, the engineer can follow either the approachdescribed in Kollingbaum and Norman (2003) so that he hard-wires norms on agents, or the approach of endowing these agentswith capabilities for accepting norms while maintaining theirautonomy as described in Criado et al. (2010a).

To illustrate the performance of MaNEA at execution time, aconcrete execution of the assembly line is explained below. Let usassume the existence of four domain agents: the supervisor (s1),the auditor (a1) and two robot agents (r1 and r2) that play thesubordinated role. Moreover, there are infrastructural agents: thenorm manager (NM), the norm enforcer (NE), the trace manager(TM) and the organization management system (OMS).

Fig. 7 shows the exchange of events among the agents TM,OMS, NM, NE and s1 corresponding to the activation of norm n4,which has been defined in Section 3.2.2 as follows:

/n4,O,subordinated,

/auditStart,�,taskðTÞS,/auditEnd,�,taskðTÞS,

/taskPerformance,�,performanceðPÞ4taskðTÞS,

�,�S

Specifically, agent OMS sends an event for informing the NMabout the register of norm n4. According to Algorithm 1(described in Section 4.1), the NM sends an event to the TM forsubscribing to the norm activation event (/auditStart,�,taskðTÞS).The supervisor agent (s1) is responsible for initiating an auditstage. Specifically, agent s1 sends an event for starting the audit ofthe assembling task. Then the NM receives this event and informsthe NE about the creation of an instance named iassembling:

iassembling ¼/n4,O,subordinated,

/auditEnd,�,taskðassemblingÞS,

/taskPerformance,�,

performanceðPÞ4taskðassemblingÞS,�,�S

∧∧

Fig. 7. Norm activation.

∧∧

∧

∧

Fig. 8. Observation of behaviours.

Fig. 9. Norm expiration.


Moreover, the NM requests to the TM its subscription to theexpiration event (/auditEnd,�,taskðassemblingÞS). Since there aretwo agents (r1 and r2) that are playing the subordinated role, twopowers are created. Thus, the NE executes Algorithm 4 (describedin Section 4.2.2) and sends two subscription requests to the TMfor controlling these two powers.

Fig. 8 corresponds to the exchange of events that occur whenthe norm n4 is active. When the audit stage of the assembling taskstarts, agents r1 and r2 send events for informing about this task.Since the auditor agent (a1) is subscribed to this information, itreceives these events. On the one hand, agent r1 informs about thecost of this task. This event does not match the norm conditiontemplate and, as a consequence, agent NE does not receive thisevent. On the other hand, agent r2 informs about the performanceof this task. In this case, agent NE receives this event and itconsiders that agent r2 has fulfilled the norm n4. Agent NEexecutes Algorithm 5 (described in Section 4.2.3), rewards agentr2 and unsubscribes from the event that controls the powercorresponding to r2.

Finally, Fig. 9 illustrates the events that are sent when theinstance iassembling expires. This process starts when the agent s1

sends the expiration event (/auditEnd,�,taskðassemblingÞS). Thenthe NM unsubscribes from this event, deletes the instance andinforms the about the expiration of the instance iassembling. Sincethe NE is subscribed to the instanceExpiration event, then theevent sent by the NM is directly delivered to the NE. The NEexecutes Algorithm 4 (described in Section 4.2.2) and checksall powers related to the instance that remain unfulfilled.Specifically, the NE removes the power that controls agentr1 and unsubscribes itself from the norm condition event

(/taskPerformance,r1,performanceðPÞ4taskðassemblingÞS) and sendsa sanctioning event to agent r1, since this agent has not fulfilled theobligation.

As illustrated by this example, the supervisor agent is respon-sible for starting and ending the audit stages. Similarly, theauditor agent gathers the information provided by subordinatedagents to analyze the assembly line. Thus, neither the supervisornor the auditor must control norms. The engineer of this casestudy does not have to program any agent that is responsible forcontrolling norms since the infrastructure itself provides thisfunctionality. This is simple case study in which a small numberof agents participate. However, in a real example of an assemblyline there may be a large and changeable number of agentsdepending on the number of products, sub-assemblies, compo-nents and raw materials (Caridi and Sianesi, 2000). This type ofdynamic and complex applications requires efficient platformscapable of adapting dynamically to different scale MAS. As aresponse to this need, MaNEA has been proposed for controllingnorms in an effective way allowing open MAS to be executed inMagentix2.

6. Evaluation

Bringing agent technology to real domains such as industryor business entails the development of efficient platforms andinfrastructures capable of engineering open distributed systems.Efficiency is one of the main requirements that have motivatedthe design of MaNEA. To this goal, MaNEA is based on a tracingservice, which has been implemented following a publish/subscribe


metaphor. Traditionally, Norm-Enforcing Architectures have beenbuilt using overhearing approaches. Overhearing is defined as anindirect interaction whereby an agent receives information forwhich it is not addressee (Kaminka et al., 2002; Legras andTessier, 2003). In this section the MaNEA proposal is evaluatedtheoretically and experimentally in order to illustrate its perfor-mance6 with respect to these overhearing approaches. Specifically,MaNEA is compared with two other proposals on norm enforcementthat provide similar functionalities: Cardoso and Oliveira’ approach,and Modgil et al. framework (Modgil et al., 2009). Other proposals,such as ORA4MAS,7 have been not selected since they do notprovide many of the functionalities provided by MaNEA.

Cardoso and Oliveira’ approach is a centralized overhearingapproach (or centralized approach for short) to control norms inVOs. In Cardoso and Oliveira’ approach there is a centralized entity(known as norm environment) that receives information about all themessages exchanged by agents. This entity considers these messagestogether with the norms to determine if any agent has violated orfulfilled a norm. The norm environment has been implemented as arule system in Jess.8 Thus, the enforcement of norms is carried out bya single rule engine, which implies a limitation on the scalability ofthis approach (i.e., if the number of agents or norms to be controlledincreases drastically, the enforcement of norms may become a bottleneck). The normative language used by Cardoso and Oliveira’approach allows the definition of obligation norms (prohibitionscan be defined as an obligation to achieve the negation of theforbidden condition) that are addressed to a specific role (i.e., thesenorms also include a target). These obligations become active undercertain circumstances (i.e., they allow the definition of an activationcondition that can be expressed in terms of actions, messages andstates of affairs). Moreover, obligation norms have a deadline thatdefines the validity period of the norm (i.e., the expiration conditioncan only be expressed in terms of time constraints). The maindifference between the normative languages used Cardoso andOliveira’ approach and MaNEA is that Cardoso and Oliveira’ languagedoes not allow the explicit definition of sanctions and rewards in thenorms. In Cardoso and Oliveira’ approach the sanctions (vs. rewards)are expressed as rules that are applied after the violation(vs. fulfilment) of norms by the norm environment. This proposalhas been selected to evaluate the performance of MaNEA for severalreasons: the normative languages used by Cardoso and Oliveira’approach and MaNEA have a similar expressiveness; Cardoso andOliveira’ approach also performs the automatic enforcement ofnorms; and it is a centralized approach, centralized approaches aresupposed to require the optimum number of messages.

Modgil et al. proposal is a mixed approach that takes over-hearing and subscription approaches. In particular, the monitor

subscribes to observers that report states that are relevant to normreasoning (i.e. states that are included in the norms). All messagesand actions performed by agents are reported to a set of trustedobservers. A prototype of Modgil et al. proposal has beenimplemented in AgentSpeak (Bordini et al., 2008) using Jason.9

The normative language used by Modgil et al. proposal allows thedefinition of obligation and prohibition norms that are addressedto a specific role (i.e., these norms also include a target). These

6 Notice that we evaluate the performance considering the number of

messages that are exchanged due to norm enforcement.7 Specifically, ORA4MAS takes a mixed approach to control norms. The norms

that define the access to the organizational infrastructure are regimented, whereas

work domain norms are enforced. Moreover, the enforcement functionalities

provided by ORA4MAS are a small subset of the functionalities provided by

MaNEA. ORA4MAS just detects norm violations but the evaluation, judgement and

repair of these situations must be performed by agents. All of these functionalities

are provided by MaNEA’s entities.8 http://herzberg.ca.sandia.gov/.9 http://jason.sourceforge.net/Jason/Jason.html.

norms become active under certain circumstances (i.e., they allowthe definition of an activation condition that can be expressed interms of actions, messages and states of affairs). Similarly, normshave an expiration condition. The main difference between thenormative languages used Modgil et al. proposal and MaNEA isthat the former does not allow the explicit definition of sanctionsand rewards in the norms. Sanctions and rewards are specified asobligations that become active when norm are violated orfulfilled. As Table 1 (in Section 2.4) shows, the functionalityprovided by Modgil et al. framework is the most similar toMaNEA; i.e., it also enforces general norms that control actions,messages and states of affairs that may occur independently.Moreover, it also proposes a two-layer architecture for enforcingnorms. Therefore, this proposal has been selected among thedistributed norm-enforcing architectures to evaluate the perfor-mance of MaNEA with respect to the number of messagesexchanged due to norm enforcement.

The norm languages used by Cardoso and Oliveira’ approachand Modgil et al. proposal do not include sanctions and rewardsin the norm definition. Specifically, sanctions and rewards aredefined in terms of enforcement rules or norms that are appliedwhen norms are violated and fulfilled, respectively. To avoid thatdifferences on the expressiveness of the norm languages influencethe comparison between MaNEA and these two proposals, weassume that in Cardoso and Oliveira’ approach and Modgil et al.proposal any norm has enforcement rules or norms for definingboth sanctions and rewards. We also assume that these enforce-ment rules or norms are automatically applied by the normenvironment, in case of Cardoso and Oliveira’ approach; or thenorm monitors, in case of Modgil et al. proposal. Finally, weassume that Cardoso and Oliveira’ language allows the definitionof the expiration condition not only in terms of time constraintsbut also in terms of states of affairs.

6.1. Theoretical results

Neither Cardoso and Oliveira’ approach nor Modgil et al.proposal provide detailed information about the interaction pro-tocols followed by agents and the enforcing entities (i.e., normenvironment, monitors and observers). However, the two propo-sals provide information about the algorithms executed by eachone of the enforcing entities. Our comparison is based on thenumber of messages that are exchanged in each of the proposal,and this number is obtained from an analysis of these algorithms.

Let us consider a general scenario in which we have a set ofagents (A) that interact over the course of I iterations. In eachiteration, each agent performs one action. Each agent plays one ormore roles. For simplicity we make the following assumptions:(i) we do not take into account the fact that agents may playdifferent roles during their execution; (ii) each agent is randomlyassigned to a set of roles in each execution; (iii) this scenariois controlled by a set of norms; and (iv) norms are not changedat runtime; i.e., we assume that there is a set of norms(N ¼ fn1, . . . ,njg) that remains static.

6.1.1. Cardoso and Oliveira

In Cardoso and Oliveira’ approach (Cardoso and Oliveira, 2007)agents report all messages that they sent to a single entity, whichwill be named agent C, which is responsible for norm enforce-ment. For simplicity, we assume that agents also report all actionsthat they perform to C. This entity processes the reportingmessages together with the norms to infer norm violations andfulfilments.

To determine the sequence and number of messages that areexchanged in this approach, let us start with a simple example

http://herzberg.ca.sandia.gov/.

http://jason.sourceforge.net/Jason/Jason.html


depicted in Fig. 10. This figure illustrates the message exchangeamong a set of agents ðfC,A1,A2,A3,A4gÞ along the different stages ofnorm monitoring (Initialization, Activation, Fulfilment and Expiration).Agents A1, A2, A3 and A4 are domain agents. We consider the case inwhich C is only responsible for monitoring an obligation norm(/id,O,T ,A,E,C,�,�S). When agent A2 performs the action thatactivates the norm (Fig. 10 message action(A)) then the norm isinstantiated (/id0,O,T 0,E0,C0,�,�S). From that moment on, agent C

controls all of the agents that are playing role T 0. Let us assume thatagents A3 and A4 are playing that role so that they are under theinfluence of this norm and two powers are created for controllingthese agents (/id00,O,T 00,E00,C00,�,�,W 00S). When agent A3 performsthe obliged action, it sends a reposting message to agent C (Fig. 10message actionðC00Þ). Then the norm has been fulfilled and C rewardsagent A3. Finally, when agent A1 performs the expiration action(Fig. 10 messages actionðE0Þ), then C sanctions A4 (Fig. 10 messagesanction), since this agent has not complied with the norm. At anystep agent C receives messages that are not related to norms. Theseother messages report all the actions that have been performed byagents (e.g., the message reporting action X1 in Fig. 10).

As illustrated by this example, the number of messages thatare exchanged for controlling norms in Cardoso and Oliveira’proposal is:

�

I

Fignor

Initialization. Agent C receives messages informing about allthe actions that have been performed. Thus, c does not have tosubscribe to the activation event of norms. Therefore, nomessage is sent for this purpose in the initialization step.
� Activation. When norms are activated it is not necessary to
send any extra message.
� Expiration. Similarly to the activation stage, no extra message
is sent when norms expire.
� Fulfilment. If an agent fulfils a norm, then agent C sends
1 rewarding message.
� Violation. When an agent violates a norm, then C sends
1 message for sanctioning the offender agent.
� Reporting. All actions that are performed by agents are
reported to agent C. Therefore, agent C receives one reportingmessage for each action that each agent performed.

Thus, the number of messages required for controlling a singlenorm n when a set of agents (A) interact along I iterations is:

0|{z}Initialization

þ 0|{z}Activation

þ 0|{z}Expiration

þ ynF|{z}

Fulfilment

þ ynV|{z}

Violation

þ 9A9 � I|fflfflffl{zfflfflffl}Reporting

C A1 A2 A4A3

action(A)

action(C’’)reward

sanctionaction(E’)

nitialization

Activation

Fulfilment

Expiration

action(X )

action(X )action(X )

action(X )action(X )

action(X )

action(X )

. 10. Messages exchanged in Cardoso and Oliveira’ approach when a single

m is controlled.

where ynV and yn

F are the number of times that a norm has beenviolated or fulfilled, respectively. The total number of messages thatare required for controlling a set of norms N ðN ¼ fn1, . . . ,njgÞ is:

ð9A9 � IÞþXj

i ¼ 1

ðyni

F þyni

V Þ

6.1.2. Modgil et al.

In the case of the proposal of Modgil et al., agents report toobservers all the actions that they perform, as in case of thecentralized approach. However, observers only report informationto monitors when the information is relevant to the activation,expiration, violation or fulfilment of norms. Therefore, the num-ber of messages that is required for controlling norms dependson the number of times that observers report information tomonitors. To illustrate the messages that are necessary forcontrolling norms, we start with a simple example that is shownin Fig. 11. This figure illustrates the message exchange among aset of agents ðfM,O,A1,A2,A3,A4gÞ along the different stages ofnorm monitoring. M and O are the monitor and observer agents,respectively. Agents A1, A2, A3 and A4 are domain agents. Again,we will consider the case in which M is responsible for monitor-ing the same obligation norm (/id,O,T ,A,E,C,�,�S). M subscribesto the observer entrusted with reporting on the states of interestidentified by the norm (A, E, C). When agent A2 reports to theobserver that it has performed the action that activates the norm,it sends the event that indicates that the norm is active (Fig. 11message action(A)). Then O sends this information to M and thenorm is instantiated (/id0,O,T 0,E0,C0,�,�S). From that moment on,M controls all agents that are under the influence of the norm.Again, agents A3 and A4 are affected by the norm; i.e., two powersare created (/id00,O,T 00,E00,C00,�,�,W 00S). Agent A4 performs actionE000. E000 is an instantiation of E (i.e., exits a substitution s such assðEÞ ¼ E000) and O informs agent M about it. However, E000 does notmatch the expiration condition of the instance (i.e., E000aE0) andthe instance does not expire. When agent A3 performs the obligedaction, then it sends a reporting message to O(Fig. 11 messageactionðC 00Þ). The observer sends this information to the monitor M.As previously mentioned, in case of norm violations and fulfil-ment the monitor takes remedial actions accordingly. In thisexperiment we have assumed that these remedial actions consistin sending sanctioning and rewarding actions. Thus, M rewardsagent A3. Finally, when agent A1 performs the expiration action(Fig. 11 message actionðE0Þ), then the observer sends this informa-tion to M. M is aware of the expiration of the obligation instanceand sanctions A4 (Fig. 11 message sanction).

As illustrated by this example, the number of messages thatare exchanged in Modgil et al. framework is:

�
Initialization. Agent M must subscribe to the observers thatinform about the states of interest specified by the norm.Therefore, 3 messages are sent in the initialization step. � Activation. When a norm is activated 1 message is sent by an
observer to the monitor.
� Expiration. Every time that an observer detects that the
expiration condition (E) of a norm has been instantiated itsends 1 message to the monitor. If the instantiation is equal tothe expiration condition of an instance (E0) then this instanceexpires. If the instantiation is not equal to the expirationcondition of any instance then all instances remain active.
� Fulfilment. Every time that an observer detects that an agent
has performed an action that is an instantiation, or the normcondition (C), then it sends one message to the monitor. If theinstantiation is equal to the norm condition of an instance (C0)and the agent that has performed the action is under the

M O A1 A2 A4A3

subscription(A)

action(A)action(A)

subscription(E)

action(C’’)

reward

sanction

action(E’)

Initialization

Activation

Fulfilment

Expiration

subscription(C)

action(C’’)

action(E’)

action(X1)

action(X2)action(E’’’)

action(E’’’)

action(X3)

action(C’’’)action(C’’’)

Fig. 11. Messages exchanged in Modgil et al. approach when a single norm is controlled.

NM NE A1TM A2 A4A3

subscription(A)

event(A)

instanceActivationsubscription(E’)

subscription(C’’ )subscription(C’’ )

event(C’’)

reward

sanction

event(E’)unsubscription(E’)

instanceExpirationunsubscription(C’’ )

Initialization

Activation

Fulfilment

Expiration

unsubscription(C’’ )

Fig. 12. Messages exchanged in MaNEA when a single norm is controlled.


influence of the instance (i.e., it plays the target role) then theinstance has been fulfilled. In this case, agent M sends1 rewarding message. On the contrary, if the instantiation isdifferent from C00 or the agent is not under the influence of theinstance then the instance remains unfulfilled.
� Violation. When an agent violates a norm, then 1 message is
sent for sanctioning the offender agent.
� Reporting. All actions that are performed by agents are
reported to observers. Therefore, the observer agent receives1 reporting message for each action that has been performed.

Thus, the number of messages required for controlling a singlenorm n when a set of agents (A) interact along I iterations is:

3|{z}Initialization

þ ynA|{z}

Activation

þ ynE|{z}

Expiration

þynCþy

nF|fflfflfflffl{zfflfflfflffl}

Fulfilment

þ ynV|{z}

Violation

þ 9A9 � I|fflfflffl{zfflfflffl}Reporting

where ynA, yn

E and ynC are the number of times that an observer

detects that an instantiation of the activation, the expiration ofthe norm condition of a given norm holds, respectively. yn

V and ynF

are the number of times that a norm has been violated or fulfilled,respectively. The total number of messages that are required for

controlling a set of norms ðN ¼ fn1, . . . ,njgÞ is:

ð9A9 � IÞþXj

i ¼ 1

ð3þyni

A þyni

E þyni

C þyni

F þyni

V Þ

6.1.3. MaNEA

In MaNEA, the number of messages exchanged depends on thenumber of actions that are relevant to the activation, expirationand fulfilment of norms. To illustrate the exchange of messagesthat occurs in MaNEA we also use a simple example that is shownin Fig. 12. This figure illustrates the message exchange among aset of agents {NM, NE, TM, A1, A2, A3, A4} along the differentstages of norm monitoring. NM, NE and TM are the NormManager, Norm Enforcer and Trace Manager. Agents A1, A2, A3and A4 are domain agents. Again, we consider the case in whichMaNEA is only responsible for monitoring the same obligationnorm (/id,O,T ,A,E,C,�,�S). Thus, the NM subscribes to theactivation event of this norm (Fig. 12 message subscription(A)).When agent A2 sends the event that indicates that the norm isactive (Fig. 12 message event(A)) then the norm is instantiated


(/id0,O,T 0,E0,C0,�,�S) and the NM subscribes to the expirationevent (Fig. 12 message subscriptionðE0Þ). Moreover, the NM sends amessage to the NE for informing about the creation of a newinstance (Fig. 12 message instanceActivation). Again, agents A3and A4 are under the influence of the norm and two powers arecreated (/id00,O,T 00,E00,C00,�,�,W 00S). Thus, the NE sends two mes-sages to the TM for subscribing to the events sent by A3 and A4that inform about the fulfilment of the norm (Fig. 12 messagessubscriptionðCA3

00Þ and subscriptionðCA4

00Þ). When agent A3 per-

forms the obliged action, then it sends a message to the NE(Fig. 12 message eventðC00Þ). Then the norm has been fulfilled andthe NE rewards agent A3 and unsubscribes from event C00A3.Aspreviously mentioned, the NE sends an event for informing allsubscribed agents about the fulfilment of a norm. Here, we willassume that only one event is sent for rewarding the agent thathas complied with the norm, as occurs in the other two proposals.Finally, when agent A2 sends the expiration event (Fig. 12messages eventðE0Þ), then the NM unsubscribes from this eventand sends an instanceExpiration event to the NE. The NE unsub-scribes from those events that are pending (Fig. 12 messageunsubscriptionðC00A4Þ) and sanctions A4 (Fig. 12 message sanction),since this agent has not complied with the norm.

According to Fig. 12 the number of messages that are requiredfor controlling norms in MaNEA is:

�
Initialization.1 message for subscribing to the activation condition; � Activation.1 message for sending the activation event, 1 mes-
sage for subscribing to the expiration condition and 1 messagefor informing the NE about the new instance. Moreover, foreach agent that is playing the target role 1 message is sent forsubscribing to the norm condition. Thus, the total number ofmessages that are sent per activation is 3þyn

T , where ynT is the

number of agents that are under the influence of norm n (i.e.,the number of agents that enact role T).
� Expiration. When the norm expires 3 messages are sent: 1 that
corresponds to the expiration event; 1 sent by the NM forunsubscribing from this event and 1 for informing the NEabout the expiration of the instance.
� Fulfilment. When the obliged action is performed by an agent that
is affected by the norm 3 messages are sent: 1 for sending theevent informing about performance of the norm condition, 1 forunsubscribing from this event and 1 for rewarding the agent.
� Violation. When an agent violates a norm, then the NE sends
2 messages: 1 for unsubscribing from the norm condition and1 for sanctioning the offender agent.
�
Table 2Parameters used in the experiments.

Parameter Fixed value Experimentation interval

# of iterations 100 [100, 1000]

# of actions 100 [10, 200]

# of norms 20 [1, 100]

# of instantiations 10 [1, 20]

# of agents 100 [0, 500]

# of roles 10 [1, 100]

Reporting. No message is sent for this purpose in MaNEA.

The total number of messages required in MaNEA for controlling asingle norm n:

1|{z}Initialization

þð3þynT Þny

nA|fflfflfflfflfflfflfflffl{zfflfflfflfflfflfflfflffl}

Activation

þ 3nynE0|fflffl{zfflffl}

Expiration

þ 3nynF|ffl{zffl}

Fulfilment

þ 2nynV|fflffl{zfflffl}

Violation

þ 0|{z}Reporting

where ynE0 is the number of times that instances of a given norm n

have expired. ynA, yn

F and ynV are the number of times that an

observer detects that an instantiation of the activation holds, thenumber of times that a norm has been fulfilled, and the number oftimes that a norm has been violated, respectively.

Finally if we consider an application scenario that is controlledby a set of norms ðN ¼ fn1, . . . ,njgÞ the number of messagesexchanged is:

Xj

i ¼ 1

ð1þð3þyni

T Þnyni

A þ3nyni

E0þ3nyni

F þ2nyni

V Þ

In the three proposals the number of messages depends onseveral factors such as: the number of times that norms are

activated, expired violated and fulfilled. As far as we know, thereis not any work that analyses the occurrence of norm activations,expirations, fulfilments, violations and the number of agents thatare affected by norms in average. In order to compare empiricallythe three proposals, we have developed a set of experiments thatare described below.

6.2. Experimental results

As explained before, neither Cardoso and Oliveira’ approachnor Modgil et al. proposal provide detailed information about theinteraction protocols followed by agents and the enforcing enti-ties. Moreover, implementations of these two proposals are notavailable and we cannot evaluate the performance of the imple-mentations of these other proposals. However, to compare thenumber of messages exchanged by each proposal we haveimplemented a simulator that computes the number of messagesthat are required to control norms. This simulator implements theenforcing algorithms specified by Cardoso and Oliveira’ approachand Modgil et al. proposal. Thus, the results presented in thissection are not affected by the interaction protocols used in eachproposal (regardless of the interaction protocol being used thiswill finally imply a specific number of messages exchanged).

In this section we describe the set of simulations that wecarried out to experimentally evaluate the performance of MaNEAwith respect to Cardoso and Oliveira’ approach and Modgil et al.framework. We compute the number of messages that is requiredfor controlling norms in each approach. Therefore, we comparethe number of messages that are sent on average in each one ofthe three proposals.

6.2.1. Simulation description

We considered a scenario with the parameters that we sum up inTable 2. Each simulation is populated by 100 agents. Agents enactone or more roles randomly in each simulation. Specifically, 10different roles have been considered. In order to specify the desiredbehaviour of these roles, 20 norms are created in each simulation.Each norm is also randomly assigned to a single role, which is thetarget of the norm. As previously mentioned, norms are defined interms of three conditions, which correspond to the activation,expiration and normative condition (i.e., A, E and C). We assumethat these conditions are expressed in terms of actions that agentsperform or events that inform about the performance of actions (incase of MaNEA). Specifically, we consider that agents are able tocarry out 100 different actions. Thus, in each simulation theactivation, expiration and normative condition of each norm aredefined as actions that are randomly selected from this set of 100actions. Therefore, there are 60 (i.e., 20n3) actions (or normative

actions) that are selected randomly from a set of 100 actions (60% ofthe actions have normative consequences).

Each simulation is executed during 100 iterations. Each agentperforms one action in each iteration. All agents have the sameacting capabilities. Thus, they are able to perform any of the 100actions. Each action can be performed (i.e., instantiated) in 10different ways. Since MaNEA is aimed to control open MAS, which

0

20000

40000

60000

80000

100000

120000

140000

0 50 100 150 200 250 300 350 400 450 500Iterations

Messages

Cardoso & OliveiraModgil et al.MaNEA

Fig. 13. Performance of MaNEA, Cardoso and Oliveira’ approach and Modgil et al.

frameworks with respect to the number of iterations.

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

0 20 40 60 80 100 120 140 160 180 200Actions

Messages



frameworks with respect to the number of actions.


are populated by heterogeneous agents; we do not want to makeany assumption about the agents’ capabilities to reason aboutnorms and goals. Thus, in each iteration each agent executes anaction that is randomly selected from the 1000 concrete actions(these concrete actions correspond to the 10 ways in which eachone of the 100 actions can be instantiated).10 These actions maybe or not relevant to the activation, expiration, violation orfulfilment of norms. Thus, the performance of actions may bereported or not, which may entail the exchange of more mes-sages, depending on the specific features of each norm-enforcingapproach. In each iteration, we simulate the exchange of mes-sages that occurs in the three proposals (i.e., MaNEA, Cardoso andOliveira’ approach and Modgil et al. framework) due to normenforcement. With this information, we compute the number ofmessages that is required for controlling norms by each approachin the whole simulation.

We have performed six different experiments to illustrate thenumber of messages with respect to: the number of iterations;the number of actions; the number of norms; the number ofinstantiations; the number of agents; and the number of roles. Inthe experiments the values of the parameters range as indicatedby the Experimentation Interval column in Table 2. The results ofthese experiments are described below.

6.2.2. Number of iterations

Fig. 13 illustrates the number of messages that are sent tocontrol norms with respect to the number of iterations that thescenario is executed. As the results show, in the three proposalsthe number of messages increases linearly with the number ofiterations. When the number of iterations increases, more actionsare executed and more actions must be reported. As one couldexpect, in the proposals in which all actions are reported (i.e.,Cardoso and Oliveira’ and Modgil et al. frameworks) the line has ahigher slope. The number of messages in Modgil et al. frameworkis slightly higher than Cardoso and Oliveira’ approach.11 More-over, as the number of iterations increase more actions areperformed and there are more probabilities of agents performingany of the 60 normative actions. As mentioned before, these

10 For simplicity we assume that there is a perfect execution of actions: i.e.,

agents never fail when they perform actions.11 As one could expect from the theoretical results:

ð9A9nIÞþXj

i ¼ 1

ðyni

F þyni

V Þoð9A9nIÞþXj

i ¼ 1

ð3þyni

A þyni

E þyni

C þyni

F þyni

V Þ:

normative actions may cause the activation, expiration, fulfilmentand violation of norms.12 As the theoretical results illustrate, thesesituations determine the number of messages that are sent inthe three proposals. As a consequence, in MaNEA the number ofmessages also increases as the interactions increase. We can concludethat MaNEA performs better than the other two proposals in theconditions of this experiment. In the rest of the experiments we onlyshow the results that are obtained with 100 iterations. However,there are not significant differences among the results obtained whenthe number of iterations changes.

6.2.3. Number of actions

Fig. 14 illustrates the performance of the three proposals withrespect to the number of actions that can be executed by agents.As the number of actions increases the performance of MaNEAgets better than the other two proposals. This is explained by thefact that if there are more actions that can be performed byagents, the probability of agents performing normative actions islower.13 For a number of actions higher than 60, the performanceof MaNEA is better than Cardoso and Oliveira’ approach andModgil et al. framework. This can be considered as a good resultsince there are 20 norms and, as a consequence, there are 60normative actions. When the number of actions is less than 60 allof them are normative. Thus, all of the actions that can beperformed by agents are controlled by norms and it is better touse an overhearing approach or, even better, a regimentationframework such as an EI. This also means that MaNEA performsworse when MAS are over-regulated.

6.2.4. Number of norms

Fig. 15 illustrates the performance of the three proposals withrespect to the number of norms. In this experiment, the numberof actions increases linearly with the number of norms tomaintain the ratio between the number of norms and actions.14

In the three proposals, the number of messages that are sentremains quite stable regardless of the number of norms. In light ofthese results, we can conclude that MaNEA performs better than

12 This increases the values ofPj

i ¼ 1 yni

A ,Pj

i ¼ 1 yni

E ,Pj

i ¼ 1 yni

E0,Pj

i ¼ 1 yni

C ,Pji ¼ 1 y

ni

F andPj

i ¼ 1 yni

V .13 This decreases the values of

Pji ¼ 1 y

ni

A ,Pj

i ¼ 1 yni

E ,Pj

i ¼ 1 yni

E0,Pj

i ¼ 1 yni

C ,Pji ¼ 1 y

ni

F andPj

i ¼ 1 yni

V .14 If there are more norms than actions, then all actions are controlled by

norms. As previously mentioned, in this case it is better to use a regimentation

system.

0

5000

10000

15000

20000

25000

0 10 20 30 40 50 60 70 80 90 100Norms

Messages



frameworks with respect to the number of norms.

0

10000

20000

30000

40000

50000

60000

70000

80000

90000

100000

0 2 4 6 8 10 12 14 16 18 20Instantiations

Messages



frameworks with respect to the number of instantiations.

0

20000

40000

60000

80000

100000

120000

0 50 100 150 200 250 300 350 400 450 500Agents

Messages



frameworks with respect to the number of agents.

0

10000

20000

30000

40000

50000

60000

70000

0 10 20 30 40 50 60 70 80 90 100Roles

Messages



frameworks with respect to the number of roles.


Modgil et al. framework and Cardoso and Oliveira’ approachregardless of the number of norms that are controlled.

6.2.5. Number of instantiations

Fig. 16 illustrates the performance of the three proposals withrespect to the number of ways in which each action can beinstantiated. As the number of instantiations increases the per-formance of MaNEA gets better than the other two proposals.Thus it is more scalable for an increasing number of instantia-tions. If actions can be instantiated in more ways, the probabilityof an agent performing the concrete instance that causes thefulfilment, violation or expiration of an instance is lower.15 For anumber of instantiations higher than 7 the performance ofMaNEA is better than Cardoso and Oliveira’ approach and Modgilet al. frameworks. This can be considered as a good result sincethere are 100 agents and it seems reasonable that these 100agents are able to execute each action in seven different ways.

6.2.6. Number of agents

Fig. 17 illustrates the performance of the three proposals withrespect to the number of agents. In this experiment, the numberof instantiations also increases linearly with the number of agentsto maintain the ratio between the number of agents and

15 This decreases the values ofPj

i ¼ 1 yni

F ,Pj

i ¼ 1 yni

V andPj

i ¼ 1 yni

E0.

instantiations. It makes sense to assume that if there are moreagents, there will be more different kinds of agents that will beable to execute actions in more different ways. In the threeproposals the number of messages increases linearly with thenumber of agents; which is consistent with the theoreticalresults. If there are more agents, there are more actions to bereported.16 Moreover, as the number of agents increases moreactions are performed and there are more possibilities of agentsperforming any of the normative actions.17 In light of theseresults we can conclude that the MaNEA performs better thanCardoso and Oliveira’ approach and Modgil et al. frameworksregardless of the number of agents.

6.2.7. Number of roles

Fig. 18 illustrates the performance of the three proposals withrespect to the number of roles. In the three proposals, as thenumber of roles increases, fewer messages are sent. Reasonsbehind this decrement are related to the fact that if there is ahigher number of roles, each role will be affected by fewer norms.The target of any norm is defined as a role that is randomlyselected from the set of roles. Thus, the more roles, the less

16 This increases the value of ð9A9nIÞ.17 This increases the values of

Pji ¼ 1 y

ni

A ,Pj

i ¼ 1 yni

E ,Pj

i ¼ 1 yni

E0,Pj

i ¼ 1 yni

C ,Pji ¼ 1 y

ni

F andPj

i ¼ 1 yni

V .


probability of roles being selected as targets. Similarly, the moreroles, the less probability of agents enacting a specific role.18 As aconsequence, fewer agents will enact the target roles and feweragents will be under the influence of norms. Therefore, there is alower probability of norm fulfilment and violation.19 For anumber of roles higher than 3, which seems reasonable consider-ing the number of agents and norms, MaNEA performs better thanModgil et al. framework.

In general, the performance of MaNEA, which has beenmeasured in terms of the number of messages, is better thancentralized and distributed proposals with the same capabilities.MaNEA uses trace event system based on a publish/subscriptionprocedure. In light of the results described above, we havedemonstrated that under certain circumstances the use of atracing service implies an outstanding reduction of the numberof messages exchanged for controlling norms.

7. Conclusions and future works

In this paper, we have described a Norm-Enforcing Architec-ture (MaNEA) that has been developed considering the facilitiesprovided by the Magentix2 platform. The main aim of MaNEA isto overcome problems of existing proposals on norm enforce-ment. MaNEA enforces norms providing support to those agentsthat are not endowed with normative reasoning capabilities. Inaddition, the generation of events for informing about sanctionsand rewards allows norm-aware agents to use this informationfor selecting the most suitable interaction partners. MaNEAprovides support to norm evaluation according to dynamic situa-tions; i.e., the norms and instances can be created or deleted on-line. Moreover, MaNEA is endowed with mechanisms to controlthe dynamic enactment of roles. Finally, MaNEA is built upon atrace event system, which provides support for indirect commu-nication in a more efficient way than overhearing approaches. Ingeneral situations, the use of a tracing service reduces the numberof messages required to control norms. Besides that, MaNEA candynamically adapt to changes in the scale MAS (i.e., situations inwhich the number of agents or norms to be controlled changesdramatically) by performing cloning and self-deletion operations.

As future work, we plan to deal with complex scenarios inwhich there are norms whose violation cannot be directlyobserved, since they regulate situations that take place out ofthe institution boundaries. Or even more, norms can be inter-preted ambiguously. This entails the development of intelligentand proactive norm-enforcer entities (i.e., agents) (Criado et al.,2011c) endowed with capabilities for applying techniques such asnegotiation or conflict resolution procedures.

Acknowledgement

We acknowledge ITMAS 2011 as the forum in which the mainideas behind this paper were preliminary discussed. This paperwas partially funded by the Spanish government under projectsCONSOLIDER-INGENIO 2010 CSD2007-00022, TIN2009-13839-C03-01, TIN2012-36586-C03-01 and by the FPU grant AP-2007-01256 awarded to N.Criado. This research has also been partiallyfunded by the Generalitat de Catalunya under the grant 2009-SGR-1434.

18 This decreases the value ofPj

i ¼ 1 yni

T .19 This decreases the values of

Pji ¼ 1 y

ni

F andPj

i ¼ 1 yni

V .

References

Argente, E., 2008. Gormas: Guidelines for Organization-Based Multiagent Systems.Ph.D. Thesis, Universidad Politecnica de Valencia.

Argente, E., Botti, V., Carrascosa, C., Giret, A., Julian, V., Rebollo, M., 2011a. Anabstract architecture for virtual organizations: the THOMAS approach. Knowl.Inf. Syst., 1–35.

Argente, E., Botti, V., Julian, V., 2011b. Gormas: an organizational-orientedmethodological guideline for open MAS. In: Agent-Oriented SoftwareEngineering X, pp. 32–47.

Artikis, A., Pitt, J., 2001. A formal model of open agent societies. In: Proceedings ofthe International Conference on Autonomous Agents and Multiagent Systems(AAMAS), pp. 192–193.

Bernon, C., Gleizes, M., Peyruqueou, S., Picard, G., 2003. ADELFE: a methodology foradaptive multi-agent systems engineering. In: Engineering Societies in theAgents World III, pp. 70–81.

Boella, G., van der Torre, L., 2003. Norm governed multiagent systems: thedelegation of control to autonomous agents. In: Proceedings of the IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT).IEEE, pp. 329–335.

Boella, G., Van der Torre, L., 2004. Regulative and constitutive norms in normativemultiagent systems. In: Proceedings of the International Conference on thePrinciples of Knowledge Representation and Reasoning (KR), pp. 255–265.

Bordini, R., Hubner, J., Wooldridge, M., 2008. Programming Multi-Agent Systems inAgentSpeak using Jason, vol. 8. Wiley-Interscience.

Burdalo, L., Garcia-Fornes, A., Julian, V., Terrasa, A., 2011. TRAMMAS: a tracingmodel for multiagent systems. Eng. Appl. Artif. Intell. 24 (7), 1110–1119.

Cardoso, H., Oliveira, E., 2007. Institutional reality and norms: specifying andmonitoring agent organizations. Int. J. Coop. Inf. Syst. 16 (1), 67–95.

Caridi, M., Sianesi, A., 2000. Multi-agent systems in production planning andcontrol: an application to the scheduling of mixed-model assembly lines. Int. J.Prod. Econ. 68 (1), 29–42.

Castelfranchi, C., 2003. Formalising the informal? Dynamic social order, bottom-upsocial control, and spontaneous normative relations. J. Appl. Logic 1 (1–2),47–92.

Criado, N., Argente, E., Botti, V., 2010a. A BDI architecture for normative decisionmaking (extended abstract). In: Proceedings of the International Conferenceon Autonomous Agents and Multiagent Systems (AAMAS), pp. 1383–1384.

Criado, N., Argente, E., Botti, V., 2011a. Open issues for normative multi-agentsystems. AI Commun. 24 (3), 233–264.

Criado, N., Argente, E., Botti, V., 2011b. Thomas: an agent platform for supportingnormative multi-agent systems. J. Logic Comput. http://dx.doi.org/10.1093/logcom/exr025.

Criado, N., Argente, E., Botti, V., 2011c. Towards norm enforcer agents. In:Proceedings of the International Conference on Practical Applications ofAgents and Multi-Agent Systems (PAAMS). Advances in Intelligent and SoftComputing, vol. 89. Springer, pp. 135–142.

Criado, N., Argente, E., Garrido, A., Gimeno, J.A., Igual, F., Botti, V.J., Noriega, P., Giret, A.,2010b. Norm enforceability in electronic institutions? In: Coordination, Organi-zations, Institutions and Norms in Agent Systems VI, pp. 250–267.

Criado, N., Julian, V., Botti, V., Argente, E., 2010c. A norm-based organizationmanagement system. In: Coordination, Organizations, Institutions and Normsin Agent Systems V, pp. 19–35.

Daft, R., 2003. Organization Theory and Design. South-Western College.Daskalopulu, A., Dimitrakos, T., Maibaum, T., 2002. Evidence-based electronic

contract performance monitoring. In: Group Decision and Negotiation, vol. 11,no. 6, pp. 469–485.

de Pinninck, A., Sierra, C., Schorlemmer, M., 2007. Friends no more: normenforcement in multiagent systems. In: Proceedings of the InternationalConference on Autonomous Agents and Multiagent Systems (AAMAS), p. 92.

Dignum, V., Dignum, F., 2002. Towards an agent-based infrastructure to supportvirtual organisations. In: Proceedings of the Working Conference on Infra-structures for Virtual Enterprises: Collaborative Business Ecosystems andVirtual Enterprises, pp. 363–370.

Dignum, V., Vazquez-Salceda, J., Dignum, F., 2005. Omni: introducing socialstructure, norms and ontologies into agent organizations. In: ProgrammingMulti-Agent Systems, pp. 181–198.

Elster, J., 1996. Rationality and the emotions. Econ. J. 106 (438), 1386–1397.Esteva, M., Rosell, B., Rodrıguez-Aguilar, J.A., Arcos, J.L., 2004. Ameli: an agent-

based middleware for electronic institutions. In: Proceedings of the Interna-tional Conference on Autonomous Agents and Multiagent Systems (AAMAS),pp. 236–243.

Ferber, J., Michel, F., Baez, J., 2005. AGRE: integrating environments with organiza-tions. In: Environments for Multi-Agent Systems, pp. 48–56.

Fix, J., von Scheve, C., Moldt, D., 2006. Emotion-based norm enforcement andmaintenance in multi-agent systems: foundations and petri net modeling. In:Proceedings of the International Conference on Autonomous Agents andMultiagent Systems (AAMAS), pp. 105–107.

Fogues, R.L., Alberola, J.M., Such, J.M., Espinosa, A., Garcıa-Fornes, A., 2010.Towards dynamic agent interaction support in open multiagent systems. In:Proceedings of the International Conference of the Catalan Association forArtificial Intelligence (CCIA), vol. 220. IOS Press, pp. 89–98.

Foster, I., Kesselman, C., Tuecke, S., 2001. The anatomy of the grid: enablingscalable virtual organizations. Int. J. High Perform. Comput. Appl. 15 (3), 200.

dx.doi.org/10.1093/logcom/exr025

dx.doi.org/10.1093/logcom/exr025


Gaertner, D., Garcia-Camino, A., Noriega, P., Rodriguez-Aguilar, J., Vasconcelos, W.,2007. Distributed norm management in regulated multiagent systems. In:Proceedings of the International Conference on Autonomous Agents andMultiagent Systems (AAMAS), pp. 624–631.

Garcia, E., Argente, E., Giret, A., 2009. A modeling tool for service-oriented openmultiagent systems. In: Proceedings of the International Conference onPrinciples of Practice in Multi-Agent Systems (PRIMA), pp. 345–360.

Garcia, E., Argente, E., Giret, A., 2010. EMFGormas: a CASE tool for developingService-oriented Open MAS. In: Proceedings of the International Conferenceon Autonomous Agents and Multiagent Systems (AAMAS), pp. 1623–1624.

Grossi, D., Aldewereld, H., Dignum, F., 2007. Ubi lex, ibi poena: designing normenforcement in e-institutions. In: Coordination, Organizations, Institutions,and Norms in Agent Systems II. Springer, pp. 101–114.

Hubner, J., Boissier, O., Kitio, R., Ricci, A., 2010. Instrumenting multi-agentorganisations with organisational artifacts and agents. J. Autonomous AgentsMulti-Agent Syst. 20 (3), 369–400.

Jones, A.J.I., Sergot, M., 1993. On the Characterization of Law and ComputerSystems: The Normative Systems Perspective. John Wiley and Sons Ltd.,Chichester, UK, pp. 275–307.

Jøsang, A., 2001. A logic for uncertain probabilities. Int. J. Uncertainty FuzzinessKnowl. Based Syst. 9 (3), 279–311.

Kaminka, G., Pynadath, D., Tambe, M., 2002. Monitoring teams by overhearing:a multi-agent plan-recognition approach. J. Artif. Intell. Res. 17 (1), 83–135.

Kollingbaum, M., Norman, T., 2003. Noa—a normative agent architecture. In:Proceedings of the International Joint Conferences on Artificial Intelligence(IJCAI), vol. 18, pp. 1465–1466.

Legras, F., Tessier, C., 2003. Lotto: group formation by overhearing in large teams.In: Proceedings of the International Conference on Autonomous Agents andMultiagent Systems (AAMAS), pp. 425–432.

Lopez, F., Luck, M., d’Inverno, M., 2006. A normative framework for agent-basedsystems. Comput. Math. O. Theory 12 (2), 227–250.

Michelson, B., 2006. Event-driven architecture overview. Patricia Seybold Group.Minsky, N., Ungureanu, V., 1998. A mechanism for establishing policies for

electronic commerce. In: Proceedings of the International Conference on

Distributed Computing Systems (ICDCS), pp. 322–331.Minsky, N., Ungureanu, V., 2000. Law-governed interaction: a coordination

and control mechanism for heterogeneous distributed systems. ACM Trans.Software Eng. Methodol. 9 (3), 273–305.

Modgil, S., Faci, N., Meneguzzi, F., Oren, N., Miles, S., Luck, M., 2009. A frameworkfor monitoring agent-based normative systems. In: Proceedings of the Inter-national Conference on Autonomous Agents and Multiagent Systems

(AAMAS), pp. 153–160.Nakano, T., Suda, T., 2005. Self-organizing network services with evolutionary

adaptation. IEEE Trans. Neural Networks 16 (5), 1269–1278.Omicini, A., Ricci, A., Viroli, M., 2008. Artifacts in the A& A meta-model for multi-

agent systems. J. Autonomous Agents Multi-Agent Syst. 17 (3), 432–456.Oren, N., Panagiotidi, S., Vazquez-Salceda, J., Modgil, S., Luck, M., Miles, S., 2009.

Towards a formalisation of electronic contracting environments. In: Coordination,Organizations, Institutions and Norms in Agent Systems IV, pp. 156–171.

Sen, S., Airiau, S., 2007. Emergence of norms through social learning. In: Proceedings ofthe International Joint Conferences on Artificial Intelligence (IJCAI), pp. 1507–1512.

Sergot, M., 1998. Normative positions. In: Norms, Logics and Information Systems,pp. 289–308.

Such, J.M., Espinosa, A., Garcıa-Fornes, A., Botti, V., 2011. Partial identities as afoundation for trust and reputation. Eng. Appl. Artif. Intell. 24 (7), 1128–1136.

Val, E.D., Criado, N., Rebollo, M., Argente, E., Julian, V., 2009. Service-orientedframework for virtual organizations. In: Proceedings of the InternationalConference on Artificial Intelligence (ICAI), vol. 1, pp. 108–114.

Venkatraman, M., Singh, M., 1999. Verifying compliance with commitmentprotocols. J. Autonomous Agents Multi-Agent Syst. 2 (3), 217–236.

Documents

MaNEA A distributed architecture for enforcing norms in ... · MaNEA: A distributed architecture for enforcing norms in open MAS N. Criadoa,n, E. Argentea, P. Noriegab, V. Bottia