LECTURE 7: Cooperation in MAS (I)
Artificial Intelligence II – Multi-Agent Systems
Introduction to Multi-Agent Systems
URV, Winter–Spring 2010
Outline of the lecture
CoordinationKinds of cooperation in MAS
Emergent cooperationCooperation with explicit communication
DeliberativePartial Global Planning
NegotiationContract NetAuctions [specific lecture]
What is Coordination?““Coordination is the process of managing
interdependencies between activities ””
-- Malone and Malone and CrowstonCrowston, 1991, 1991
Coordination problems occur when:
• An agent has a choice in its actions within some task, and the choice affects its and other agents’ performance
• The order in which actions are carried out affects performance
• The time at which actions are carried out affects performance
Subproblem Interdependencies (I)
Subproblems are the same/overlapping, but different agents have either alternative methods or data that can be used to generate a solutionSubproblems are part of a larger problem in which a solution to the larger problem requires that certain constraints exist among the solutions to its subproblems
Subproblem Interdependencies (II)
It is not possible to decompose the problem into a set of subproblems such that there is a perfect fit between the location of information, expertise, processing, and communication capabilities in the agent network and the computational needs for effectively solving each subproblemContention for resources among the subproblems
Implications of subproblem
interdependencies
It may be impossible to completely solve one subproblem without first partially solving another subproblem
Solving or partially solving one subproblem may simply make it easier to solve another subproblem
Knowing the solution to one subproblem may obviate the need to solve another
How an agent orders which subgoals to do and when to communicate the (partial) results of solving a
subgoal can significantly affect global performance
Another vision of coordinationDeciding for each agent in the context of other agent activities
What activities it should do, when it should do them and how it should do them (planning, scheduling)What it should communicate, when it should communicate and to whom (cooperation)
Domain information and Control information
Communication and scheduling of activities are intimately connectedThis is a highly complex computational problem, especially if optimal solutions are required
Many approaches to coordinationType of information available about static and dynamic behaviour of agents
Cost of acquiring current state of other agentsCost of finding out the abilities of other agents
Importance of optimal solutionCost of computing coordination decisionsImplications of generating non-optimal coordination
Real-time requirementsHow long you have to make a decision
There is no one best approach to CoordinationComplex Multi-attributed Optimization Problem
MAS
Cooperative Benevolent
Independent Self-interested
EmergentDiscrete Without communication -
Implicit
With communication -
Explicit
NegotiatorsDeliberative
Cooperation hierarchy (Franklin)
Benevolent AgentsIf we “own” the whole system, we can design agents to help each other whenever askedIn this case, we can assume agents are benevolent: our best interest is their best interestProblem-solving in benevolent systems is cooperative distributed problem solvingBenevolence simplifies the system design task enormously!
[ Example: practical exercise]
Self-Interested AgentsIf agents represent individuals or organizations, then we cannot make the benevolence assumption
E.g. buyers/sellers in e-commerceAgents will be assumed to act to further their own interests, possibly at the expense of othersPotential for conflictIt may complicate the design task enormously!
MAS
Cooperative Benevolent
Independent Self-interested
EmergentDiscrete Without communication -
Implicit
With communication -
Explicit
NegotiatorsDeliberative
Cooperation hierarchy (Franklin)
Discrete MAS
Independent agentsEach agent pursues its own agendaThe agendas of the agents bear no relation to one another
For example, one agent can filter e-mail while another one gathers information from the Web
No cooperation
MAS with emergent behaviourAgents can cooperate with no intention of doing soThe system can exhibit high-level, complex, intelligent, coordinated behaviour without any designed coordination mechanisms, just as a side effect of the interactions among agentsExample [recall reactive architectures]
Puck gathering robots – Beckers
Puck gathering robots
World with a set of pucksSet of autonomous, independent robots, that can pick up pucks and drop themA simple rule-based individual behaviour of each robot, without any communication or coordination with the other robots, can lead to a complex global system behaviour (putting all the pucks together in a pile)
Behavioural rules (I)
Rule 1 – Pick up pucksIf (there is not a puck in the gripper) & (there is a puck ahead) then take the puck in the gripperRule 2 – Drop pucks togetherIf (there is one puck in the gripper) & (there is a puck ahead) then drop the puck, go backward for one second and turn at a random angle
Behavioural rules (II)
Rule 3 – ExplorationIf there are no pucks ahead then go forward
Rule 4 – Avoid obstaclesIf there is an obstacle (wall or other robot) ahead then avoid the obstacle (turn at a random angle and go forward)
Important FactorsAgents
Don't need to know about each other, don’t communicate with each otherDon't have special rolesLoosing a few doesn't matter
EnvironmentActs as a communication mechanism Is affected by the actions of all individuals
Result: surprisingly coherent group (coordinated) action
Subsumption as Coordination Robotics as a Multi-Agent System
Brooks Subsumption Architecture:Layers of controllersEach layer creates a competenceHigher layers subsume lower layers
Can be seen, at a high level of abstraction,as coordination of autonomous entities
Each layer takes its own decisions
Intelligence as emergent behaviour
Basic element of human intelligence: neural activityNeurons: very small element with very little computational powerThe interaction between an enormous amount of neurons leads to human-level complex thought patterns and intelligence !
MAS
Cooperative Benevolent
Independent Self-interested
EmergentDiscrete Without communication -
Implicit
With communication -
Explicit
NegotiatorsDeliberative
Cooperation hierarchy (Franklin)
Cooperative agents
The agendas of the agents include cooperating with other agents in the system in some way
Explicitly: intentional sending and receiving of communicative signals (e.g. via a common blackboard or via messages)Implicitly: without explicit messages (e.g. observing and reacting to the behaviour of the other agents of the system)
MAS
Cooperative Benevolent
Independent Self-interested
EmergentDiscrete Without communication -
Implicit
With communication -
Explicit
NegotiatorsDeliberative
Cooperation hierarchy (Franklin)
Deliberative agents
Agents with inference and planningcapabilitiesSome kind of explicit distributed planning mechanism, based on information exchange, addressed to solve collectively a given problem
Partial
Global Planning (PGP) – Durfee, Lesser
Distributed planning techniqueIntegrates planning and execution
Not the usual Planning-Scheduling-Action cycle
Dynamic domains with unpredictable, unreliableinformationThe tasks are inherently distributed; each agent performs its own taskInitially applied in the Distributed Vehicle Monitoring (DVM) problem, then extended to be domain independent
Distributed
Vehicle Monitoring
Goal in the DVM problemThe agents are not aware of the global state of the system; however, there is a common goal: converge on a consistent map of vehicle movements by integrating the partial tracks formed by different agents into a single complete map or into a consistent set of local maps distributed among agentsCoordination by means of partial plans exchange
Difficulties in DVM (I)
The data sensed in an area cannot be exhaustively processed in a timely manner
Huge volume of incoming dataMany noisy data generated by sensors/environment, which should not be processedCorrelations between data sensed in nearby locations provide constraints on whether/how that data should be processed
Difficulties in DVM (II)Sensor overlap implies possible processing duplication
The same data should not be processed by different agents
Sensing demands in an area vary heavilydynamically
The allocation of work should be done in a very dynamic way, depending on the workload of each agent at each moment
Partial Global Planning phases
1. Create local plans2. Exchange local plans3. Generate Partial Global Plans4. Optimize Partial Global Plans
PGP steps (I)
1- Create the local plan of each agentEach agent represents its own expected activity (to solve its assigned tasks) with a local (tentative) plan, at two levels:
higher level – most important steps (actions) to be followed to solve the problem, abstract planlower level –it specifies primitive operations to achieve the next step in the abstract plan; as the plan is executed, new details are added incrementally
Schematic view of a local plan
Next operations(short term details)
A1 A2 A3
A4Local Plan(Actions)
A5
O1 O2 O3
Length1 Length2
Local plan characteristics
Local plans may involve alternative actions depending on the results of previous actions and changes in the environment [conditional plans]
See actions A4 and A5 in the previous slideThey have to be dynamically modifiable in an easy way
Plan componentsNameCreation timeSet of objectives to achieveList of planned actions for data processing
Major steps in the planSet of primitive problem-solving operations (short-term details)Predictions of how long each action will take and the expected outcome of each actionRating (plan importance)
Basic components of an action
Preconditions for the actionResults of the actionSet of data to be processed by the actionSet of procedures to be applied to the dataEstimated start time of the actionEstimated end time of the actionEstimated degree of confidence in the result
Ordering actions within a plan
Prefer actions that concurrently achieve multiple goalsPrefer actions expected to require less resources (especially time)Prefer actions that will strongly verify or refute that some goals are worth pursuing
E.g. analysis that confirms that a signal follows a vehicle trace detected by a neighbour agent
PGP steps (II)
2- Exchange plansThe agents exchange
information about their local plans with other agents (usually high-level information)
GoalsLong-term strategyPlan rating
Distribution of local plansEach agent must have knowledge about the MAS organisational structure, so that it can infer the role of each agent in the problem solving process and decide which informationto send to which agentsIt would be very expensive and inefficient to send all the local plans to all the agents in the system !!!
E.g. a police car probably doesn’t need to exchange its plans with all the fire trucks and ambulances
Nodes meta-level organisation
Information that each node must know to infer the organisational structure
Nodes it has authority overNodes that have authority over itNodes with equal authority
Coordination possibilities
Centralised coordinationA node has authority over all other nodes
Hierarchical coordinationEach node has 1 “boss” and some “subordinate”nodes
Lateral coordinationAll nodes have the same authority
PGP steps (III)3- Generate Partial Global Plans (PGPs)
Each agent models the collective activity of the system, by combining the received local partial plans into a Partial Global Plan
Check dependencies between the received information and its own local plan
Identify when the goals of one or more agents can be considered subgoals of a single global goal: partial global goal
Identify opportunities to improve coordination
E.g. two agents having to solve the same subproblem
Components of a PGP (I)Partial global goal: final aim of the global planPlan activity map: plan actions to be executed concurrently by itself and the other agents, including costs and expected results of actions
Initially, it will contain the union of all the actions of all local plans
Criteria for rating the actions in the Plan Activity Map
The action extends a partial resultE.g. vehicle tracking hypothesis
The action produces a partial result that might help some other agents in forming partial resultsHow long the action is expected to take
Components of a PGP (II)
From the modified Plan Activity Map, the agent builds a Solution Construction Graph: how the agents should interact, including specifications about
what partial results to exchangewhen to exchange themwho to exchange them with
It must take into account the estimated time of the actions, the results they will provide, etc.
PGP steps (IV)
4- Optimize Partial Global PlansEach agent has a Planner Module, especialised in analyzing the received information to detect if there are several agents working on the same goal. This information is put in the Plan Activity Map, along with the expected future behaviour and expected results of the other agents
Optimizing Plans
Local plan + Plan Activity Map =>New modified Local Plan
Optimized using the updated knowledge of the system
Solution Construction GraphConcrete details about when to send particular results to specific agents in the future
Possible optimizations of Local Plan
Task reorderingChange the order of the actions in the plan
Task reallocationMove some actions to nodes without assigned work
Weight of authorityChange local plan according to the decisions of the nodes that have more authority
This information is obtained by analyzing the local plans of nodes with higher authority
Benefits of PGP
Systems with highly dynamic behaviourAll plans can be adapted to dynamic changes in the environment => flexibilityHowever, if an agent changes its local plan, it has to inform other agents (e.g. those that were waiting for a partial result)
EfficiencyIf different agents work on the same/similar subproblems, they will notice that fact in their local plans and reassign their tasks appropriately
Negotiation techniquesCooperation mechanisms with explicit information exchange in which there is some sort of competition between the agents
Auctions [specific lecture]English / Dutch / Vickrey / FPSBMulti-attribute auctionsCombinatorial auctions
Contract NetOthers …
Task Sharing and Result Sharing
Two main modes of cooperative problem solving:
task sharing:activities are distributed among the agents of the systemresult sharing:information (partial or final results) is distributed
The Contract Net
A well known task-sharing protocol for task allocation is the Contract Net:
1. Recognition2. Announcement3. Bidding4. Awarding5. Expediting
[More general and detailed presentation than in the lecture about communication protocols]
1-Recognition
In this stage, an agent recognizesit has a problem it wants help withAn agent has a goal, and either…
realizes it cannot achieve the goal in isolation —does not have capabilityrealizes it would prefer not to achieve the goal in isolation (typically because of solution quality, deadline, use of resources, etc.)
2-AnnouncementIn this stage, the agent with the task sends out an announcement of the task which includes a specification of the task to be achievedSpecification must encode:
Description of the task itself Any constraints (e.g. deadlines, quality constraints)Meta-task information (e.g. preference on attributes)
The announcement is then broadcast
3-BiddingAgents that receive the announcement decide for themselves whether they wish to bid for the taskFactors:
agent must decide whether it is capable of expediting taskagent must evaluate the cost of making the task and the benefits it can get from making it
If an agent chooses to bid, then it submits a tender, detailing the conditions on which it can execute the task
4-AwardingThe agent that sent the taskannouncement must choose between bids & decide who to “award the contract” toThe result of this process is communicated to the agents that submitted a bid
5-Expediting
The successful contractor then expedites the task
That may involve generating further manager-contractor relationships: sub-contracting
Contract NetThe collection of nodes is the “contract net”Each node on the network can, at different times or for different tasks, be a manager or a contractorWhen a node gets a composite task (or for any reason can’t solve its present task), it breaks it into subtasks (if possible) and announces them (acting as a manager), receives bids from potential contractors, and then awards the job
Issues for Implementing Contract Net
How to…… specify tasks?… specify quality of service?… select between competing offers?… differentiate between offers based on multiple criteria?
Manager
Task Announcement
Node Issues Task Announcement
Manager
Manager
Manager
PotentialContractor
Idle Node Listening to Task Announcements
Manager
PotentialContractor
Bid
Node Submitting a Bid
Manager
PotentialContractor
PotentialContractor
Bids
Manager listening to bids
Manager
Contractor
Award
Manager Making an Award
Manager
Contractor
Contract
Contract Established
FIPA-Contract Net protocol
Contract Net node modulesLocal database
Knowledge base, info. on the state of negotiations and the state of the solution of tasks
Interface moduleSends/receives messages, deals with the communication with the other nodes
Task processorExecutes the tasks assigned to the node
Contract processorStudies new offered tasks, submits bids, formalizes contracts
Domain-Specific Evaluation
Task announcement message prompts potential contractors to use domain specifictask evaluation procedures
E.g. identifying important attributesThere is deliberation going on, not just selection —perhaps no tasks are suitable at present
Manager considers the submitted bids using a domain specific bid evaluation procedure
Efficiency ModificationsFocused addressing — when general broadcast isn’t required
Agents could automatically learn which are the most appropriate nodes for common tasks
Directed contracts — when manager already knows which node is appropriate
For instance when a very similar task has already been done in the past
The nodes can make proactive offers to potential managers of the kind of tasks they are able to execute
FIPA- Iterated Contract Net protocol
Features of Contract Net Protocol
Two-way dynamic transfer of informationMutual selection
Bidders select from among task announcementsManagers select from among bids
Local evaluationPreserving autonomony and private information of agents
Suitable Applications
Hierarchy of TasksSubtasks are large enough (and it’s worthwhile to spend effort to distribute them wisely)Primary concerns are distributed control, achieving reliability, avoiding bottlenecks
Limitations
Other stages of distributed problem solvingare non-trivial:
Problem DecompositionSolution Synthesis
Computational overheadMessagesTime – deliberation, analyze offer/bid, wait for decisions
Ideas for the practical exercise (I)
Emergent cooperationDifferent firemen could be made to fight together a certain fire, without explicit coordination mechanisms
E.g. each fire truck goes to the nearest fireReactive agents, rule-based behaviour of each firemanDifferent police cars could be working on the same road blocking
Ideas for the practical exercise (II)
Explicit cooperationExchange of messages between firemen/policemen/ambulances (or higher-level coordinators) to distribute fires, form teams, coordinate joint movements, etc.Some kind of global planning
Ideas for the practical exercise (III)
Negotiation Different firemen could “negotiate” who fights a certain fire
Negotiation via auctions or via Contract NetIt could even be a multi-attribute auction, depending on their distance to the fire, whether they are already fighting other fires or on their way to other fires, ...
Readings for this week
Chapter 8 of the book An introduction to MultiAgent Systems (M. Wooldridge), 2nd ed.Chapter 4 of the book Agentes Software y Sistemas Multi-Agente (A. Mas)Paper on cooperation hierarchy (Doran et al.)Paper on Partial Global Planning (Durfee, Lesser)Paper on Contract Net (Smith)