LECTURE 7: Cooperation in MAS (I)

Artificial Intelligence II – Multi-Agent Systems

Introduction to Multi-Agent Systems

URV, Winter–Spring 2010

Outline of the lecture

CoordinationKinds of cooperation in MAS

Emergent cooperationCooperation with explicit communication

DeliberativePartial Global Planning

NegotiationContract NetAuctions [specific lecture]

What is Coordination?““Coordination is the process of managing

interdependencies between activities ””

-- Malone and Malone and CrowstonCrowston, 1991, 1991

Coordination problems occur when:

• An agent has a choice in its actions within some task, and the choice affects its and other agents’ performance

• The order in which actions are carried out affects performance

• The time at which actions are carried out affects performance

Subproblem Interdependencies (I)

Subproblems are the same/overlapping, but different agents have either alternative methods or data that can be used to generate a solutionSubproblems are part of a larger problem in which a solution to the larger problem requires that certain constraints exist among the solutions to its subproblems

Subproblem Interdependencies (II)

It is not possible to decompose the problem into a set of subproblems such that there is a perfect fit between the location of information, expertise, processing, and communication capabilities in the agent network and the computational needs for effectively solving each subproblemContention for resources among the subproblems

Implications of subproblem

interdependencies

It may be impossible to completely solve one subproblem without first partially solving another subproblem

Solving or partially solving one subproblem may simply make it easier to solve another subproblem

Knowing the solution to one subproblem may obviate the need to solve another

How an agent orders which subgoals to do and when to communicate the (partial) results of solving a

subgoal can significantly affect global performance

Another vision of coordinationDeciding for each agent in the context of other agent activities

What activities it should do, when it should do them and how it should do them (planning, scheduling)What it should communicate, when it should communicate and to whom (cooperation)

Domain information and Control information

Communication and scheduling of activities are intimately connectedThis is a highly complex computational problem, especially if optimal solutions are required

Many approaches to coordinationType of information available about static and dynamic behaviour of agents

Cost of acquiring current state of other agentsCost of finding out the abilities of other agents

Importance of optimal solutionCost of computing coordination decisionsImplications of generating non-optimal coordination

Real-time requirementsHow long you have to make a decision

There is no one best approach to CoordinationComplex Multi-attributed Optimization Problem

MAS

Cooperative Benevolent

Independent Self-interested

EmergentDiscrete Without communication -

Implicit

With communication -

Explicit

NegotiatorsDeliberative

Cooperation hierarchy (Franklin)

Benevolent AgentsIf we “own” the whole system, we can design agents to help each other whenever askedIn this case, we can assume agents are benevolent: our best interest is their best interestProblem-solving in benevolent systems is cooperative distributed problem solvingBenevolence simplifies the system design task enormously!

[ Example: practical exercise]

Self-Interested AgentsIf agents represent individuals or organizations, then we cannot make the benevolence assumption

E.g. buyers/sellers in e-commerceAgents will be assumed to act to further their own interests, possibly at the expense of othersPotential for conflictIt may complicate the design task enormously!

MAS

Cooperative Benevolent

Independent Self-interested

EmergentDiscrete Without communication -

Implicit

With communication -

Explicit

NegotiatorsDeliberative

Cooperation hierarchy (Franklin)

Discrete MAS

Independent agentsEach agent pursues its own agendaThe agendas of the agents bear no relation to one another

For example, one agent can filter e-mail while another one gathers information from the Web

No cooperation

MAS with emergent behaviourAgents can cooperate with no intention of doing soThe system can exhibit high-level, complex, intelligent, coordinated behaviour without any designed coordination mechanisms, just as a side effect of the interactions among agentsExample [recall reactive architectures]

Puck gathering robots – Beckers

Puck gathering robots

World with a set of pucksSet of autonomous, independent robots, that can pick up pucks and drop themA simple rule-based individual behaviour of each robot, without any communication or coordination with the other robots, can lead to a complex global system behaviour (putting all the pucks together in a pile)

Behavioural rules (I)

Rule 1 – Pick up pucksIf (there is not a puck in the gripper) & (there is a puck ahead) then take the puck in the gripperRule 2 – Drop pucks togetherIf (there is one puck in the gripper) & (there is a puck ahead) then drop the puck, go backward for one second and turn at a random angle

Behavioural rules (II)

Rule 3 – ExplorationIf there are no pucks ahead then go forward

Rule 4 – Avoid obstaclesIf there is an obstacle (wall or other robot) ahead then avoid the obstacle (turn at a random angle and go forward)

Important FactorsAgents

Don't need to know about each other, don’t communicate with each otherDon't have special rolesLoosing a few doesn't matter

EnvironmentActs as a communication mechanism Is affected by the actions of all individuals

Result: surprisingly coherent group (coordinated) action

Subsumption as Coordination Robotics as a Multi-Agent System

Brooks Subsumption Architecture:Layers of controllersEach layer creates a competenceHigher layers subsume lower layers

Can be seen, at a high level of abstraction,as coordination of autonomous entities

Each layer takes its own decisions

Intelligence as emergent behaviour

Basic element of human intelligence: neural activityNeurons: very small element with very little computational powerThe interaction between an enormous amount of neurons leads to human-level complex thought patterns and intelligence !

MAS

Cooperative Benevolent

Independent Self-interested

EmergentDiscrete Without communication -

Implicit

With communication -

Explicit

NegotiatorsDeliberative

Cooperation hierarchy (Franklin)

Cooperative agents

The agendas of the agents include cooperating with other agents in the system in some way

Explicitly: intentional sending and receiving of communicative signals (e.g. via a common blackboard or via messages)Implicitly: without explicit messages (e.g. observing and reacting to the behaviour of the other agents of the system)

MAS

Cooperative Benevolent

Independent Self-interested

EmergentDiscrete Without communication -

Implicit

With communication -

Explicit

NegotiatorsDeliberative

Cooperation hierarchy (Franklin)

Deliberative agents

Agents with inference and planningcapabilitiesSome kind of explicit distributed planning mechanism, based on information exchange, addressed to solve collectively a given problem

Partial

Global Planning (PGP) – Durfee, Lesser

Distributed planning techniqueIntegrates planning and execution

Not the usual Planning-Scheduling-Action cycle

Dynamic domains with unpredictable, unreliableinformationThe tasks are inherently distributed; each agent performs its own taskInitially applied in the Distributed Vehicle Monitoring (DVM) problem, then extended to be domain independent

Distributed

Vehicle Monitoring

Goal in the DVM problemThe agents are not aware of the global state of the system; however, there is a common goal: converge on a consistent map of vehicle movements by integrating the partial tracks formed by different agents into a single complete map or into a consistent set of local maps distributed among agentsCoordination by means of partial plans exchange

Difficulties in DVM (I)

The data sensed in an area cannot be exhaustively processed in a timely manner

Huge volume of incoming dataMany noisy data generated by sensors/environment, which should not be processedCorrelations between data sensed in nearby locations provide constraints on whether/how that data should be processed

Difficulties in DVM (II)Sensor overlap implies possible processing duplication

The same data should not be processed by different agents

Sensing demands in an area vary heavilydynamically

The allocation of work should be done in a very dynamic way, depending on the workload of each agent at each moment

Partial Global Planning phases

1. Create local plans2. Exchange local plans3. Generate Partial Global Plans4. Optimize Partial Global Plans

PGP steps (I)

1- Create the local plan of each agentEach agent represents its own expected activity (to solve its assigned tasks) with a local (tentative) plan, at two levels:

higher level – most important steps (actions) to be followed to solve the problem, abstract planlower level –it specifies primitive operations to achieve the next step in the abstract plan; as the plan is executed, new details are added incrementally

Schematic view of a local plan

Next operations(short term details)

A1 A2 A3

A4Local Plan(Actions)

A5

O1 O2 O3

Length1 Length2

Local plan characteristics

Local plans may involve alternative actions depending on the results of previous actions and changes in the environment [conditional plans]

See actions A4 and A5 in the previous slideThey have to be dynamically modifiable in an easy way

Plan componentsNameCreation timeSet of objectives to achieveList of planned actions for data processing

Major steps in the planSet of primitive problem-solving operations (short-term details)Predictions of how long each action will take and the expected outcome of each actionRating (plan importance)

Basic components of an action

Preconditions for the actionResults of the actionSet of data to be processed by the actionSet of procedures to be applied to the dataEstimated start time of the actionEstimated end time of the actionEstimated degree of confidence in the result

Ordering actions within a plan

Prefer actions that concurrently achieve multiple goalsPrefer actions expected to require less resources (especially time)Prefer actions that will strongly verify or refute that some goals are worth pursuing

E.g. analysis that confirms that a signal follows a vehicle trace detected by a neighbour agent

PGP steps (II)

2- Exchange plansThe agents exchange

information about their local plans with other agents (usually high-level information)

GoalsLong-term strategyPlan rating

Distribution of local plansEach agent must have knowledge about the MAS organisational structure, so that it can infer the role of each agent in the problem solving process and decide which informationto send to which agentsIt would be very expensive and inefficient to send all the local plans to all the agents in the system !!!

E.g. a police car probably doesn’t need to exchange its plans with all the fire trucks and ambulances

Nodes meta-level organisation

Information that each node must know to infer the organisational structure

Nodes it has authority overNodes that have authority over itNodes with equal authority

Coordination possibilities

Centralised coordinationA node has authority over all other nodes

Hierarchical coordinationEach node has 1 “boss” and some “subordinate”nodes

Lateral coordinationAll nodes have the same authority

PGP steps (III)3- Generate Partial Global Plans (PGPs)

Each agent models the collective activity of the system, by combining the received local partial plans into a Partial Global Plan

Check dependencies between the received information and its own local plan

Identify when the goals of one or more agents can be considered subgoals of a single global goal: partial global goal

Identify opportunities to improve coordination

E.g. two agents having to solve the same subproblem

Components of a PGP (I)Partial global goal: final aim of the global planPlan activity map: plan actions to be executed concurrently by itself and the other agents, including costs and expected results of actions

Initially, it will contain the union of all the actions of all local plans

Criteria for rating the actions in the Plan Activity Map

The action extends a partial resultE.g. vehicle tracking hypothesis

The action produces a partial result that might help some other agents in forming partial resultsHow long the action is expected to take

Components of a PGP (II)

From the modified Plan Activity Map, the agent builds a Solution Construction Graph: how the agents should interact, including specifications about

what partial results to exchangewhen to exchange themwho to exchange them with

It must take into account the estimated time of the actions, the results they will provide, etc.

PGP steps (IV)

4- Optimize Partial Global PlansEach agent has a Planner Module, especialised in analyzing the received information to detect if there are several agents working on the same goal. This information is put in the Plan Activity Map, along with the expected future behaviour and expected results of the other agents

Optimizing Plans

Local plan + Plan Activity Map =>New modified Local Plan

Optimized using the updated knowledge of the system

Solution Construction GraphConcrete details about when to send particular results to specific agents in the future

Possible optimizations of Local Plan

Task reorderingChange the order of the actions in the plan

Task reallocationMove some actions to nodes without assigned work

Weight of authorityChange local plan according to the decisions of the nodes that have more authority

This information is obtained by analyzing the local plans of nodes with higher authority

Benefits of PGP

Systems with highly dynamic behaviourAll plans can be adapted to dynamic changes in the environment => flexibilityHowever, if an agent changes its local plan, it has to inform other agents (e.g. those that were waiting for a partial result)

EfficiencyIf different agents work on the same/similar subproblems, they will notice that fact in their local plans and reassign their tasks appropriately

Negotiation techniquesCooperation mechanisms with explicit information exchange in which there is some sort of competition between the agents

Auctions [specific lecture]English / Dutch / Vickrey / FPSBMulti-attribute auctionsCombinatorial auctions

Contract NetOthers …

Task Sharing and Result Sharing

Two main modes of cooperative problem solving:

task sharing:activities are distributed among the agents of the systemresult sharing:information (partial or final results) is distributed

The Contract Net

A well known task-sharing protocol for task allocation is the Contract Net:

1. Recognition2. Announcement3. Bidding4. Awarding5. Expediting

[More general and detailed presentation than in the lecture about communication protocols]

1-Recognition

In this stage, an agent recognizesit has a problem it wants help withAn agent has a goal, and either…

realizes it cannot achieve the goal in isolation —does not have capabilityrealizes it would prefer not to achieve the goal in isolation (typically because of solution quality, deadline, use of resources, etc.)

2-AnnouncementIn this stage, the agent with the task sends out an announcement of the task which includes a specification of the task to be achievedSpecification must encode:

Description of the task itself Any constraints (e.g. deadlines, quality constraints)Meta-task information (e.g. preference on attributes)

The announcement is then broadcast

3-BiddingAgents that receive the announcement decide for themselves whether they wish to bid for the taskFactors:

agent must decide whether it is capable of expediting taskagent must evaluate the cost of making the task and the benefits it can get from making it

If an agent chooses to bid, then it submits a tender, detailing the conditions on which it can execute the task

4-AwardingThe agent that sent the taskannouncement must choose between bids & decide who to “award the contract” toThe result of this process is communicated to the agents that submitted a bid

5-Expediting

The successful contractor then expedites the task

That may involve generating further manager-contractor relationships: sub-contracting

Contract NetThe collection of nodes is the “contract net”Each node on the network can, at different times or for different tasks, be a manager or a contractorWhen a node gets a composite task (or for any reason can’t solve its present task), it breaks it into subtasks (if possible) and announces them (acting as a manager), receives bids from potential contractors, and then awards the job

Issues for Implementing Contract Net

How to…… specify tasks?… specify quality of service?… select between competing offers?… differentiate between offers based on multiple criteria?

Manager

Task Announcement

Node Issues Task Announcement

Manager

Manager

Manager

PotentialContractor

Idle Node Listening to Task Announcements

Manager

PotentialContractor

Bid

Node Submitting a Bid

Manager

PotentialContractor

PotentialContractor

Bids

Manager listening to bids

Manager

Contractor

Award

Manager Making an Award

Manager

Contractor

Contract

Contract Established

FIPA-Contract Net protocol

Contract Net node modulesLocal database

Knowledge base, info. on the state of negotiations and the state of the solution of tasks

Interface moduleSends/receives messages, deals with the communication with the other nodes

Task processorExecutes the tasks assigned to the node

Contract processorStudies new offered tasks, submits bids, formalizes contracts

Domain-Specific Evaluation

Task announcement message prompts potential contractors to use domain specifictask evaluation procedures

E.g. identifying important attributesThere is deliberation going on, not just selection —perhaps no tasks are suitable at present

Manager considers the submitted bids using a domain specific bid evaluation procedure

Efficiency ModificationsFocused addressing — when general broadcast isn’t required

Agents could automatically learn which are the most appropriate nodes for common tasks

Directed contracts — when manager already knows which node is appropriate

For instance when a very similar task has already been done in the past

The nodes can make proactive offers to potential managers of the kind of tasks they are able to execute

FIPA- Iterated Contract Net protocol

Features of Contract Net Protocol

Two-way dynamic transfer of informationMutual selection

Bidders select from among task announcementsManagers select from among bids

Local evaluationPreserving autonomony and private information of agents

Suitable Applications

Hierarchy of TasksSubtasks are large enough (and it’s worthwhile to spend effort to distribute them wisely)Primary concerns are distributed control, achieving reliability, avoiding bottlenecks

Limitations

Other stages of distributed problem solvingare non-trivial:

Problem DecompositionSolution Synthesis

Computational overheadMessagesTime – deliberation, analyze offer/bid, wait for decisions

Ideas for the practical exercise (I)

Emergent cooperationDifferent firemen could be made to fight together a certain fire, without explicit coordination mechanisms

E.g. each fire truck goes to the nearest fireReactive agents, rule-based behaviour of each firemanDifferent police cars could be working on the same road blocking

Ideas for the practical exercise (II)

Explicit cooperationExchange of messages between firemen/policemen/ambulances (or higher-level coordinators) to distribute fires, form teams, coordinate joint movements, etc.Some kind of global planning

Ideas for the practical exercise (III)

Negotiation Different firemen could “negotiate” who fights a certain fire

Negotiation via auctions or via Contract NetIt could even be a multi-attribute auction, depending on their distance to the fire, whether they are already fighting other fires or on their way to other fires, ...

Readings for this week

Chapter 8 of the book An introduction to MultiAgent Systems (M. Wooldridge), 2nd ed.Chapter 4 of the book Agentes Software y Sistemas Multi-Agente (A. Mas)Paper on cooperation hierarchy (Doran et al.)Paper on Partial Global Planning (Durfee, Lesser)Paper on Contract Net (Smith)