Upload
molly-foster
View
216
Download
0
Embed Size (px)
Citation preview
Collective Intelligence
23418 2
Outline
bull What is Swarm Intelligence (SI)
bull Multi-Agents System (MAS)
bull Simulate SI for Search
ndash Ant Colony Optimization (ACO)
ndash Particle Swarm Optimization (PSO)
23418 3
bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals
bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals
bull These systems provide important techniques that can be used in the development of artificial intelligent systems
The Computational Beauty of Nature
23418 4
Examples of Collective Behavior in Nature and Society
bull Many agents (individualpart)Many agents (individualpart)
bull Local and simple interactionsLocal and simple interactions
bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip
Which can be treated as Multi-Agent System
23418 5
Emergencebull Goldstein ldquoThe arising of novel and
coherent structures patterns and properties during the process of self-organization in complex systems
bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo
bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow
orders from a leader
A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 2
Outline
bull What is Swarm Intelligence (SI)
bull Multi-Agents System (MAS)
bull Simulate SI for Search
ndash Ant Colony Optimization (ACO)
ndash Particle Swarm Optimization (PSO)
23418 3
bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals
bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals
bull These systems provide important techniques that can be used in the development of artificial intelligent systems
The Computational Beauty of Nature
23418 4
Examples of Collective Behavior in Nature and Society
bull Many agents (individualpart)Many agents (individualpart)
bull Local and simple interactionsLocal and simple interactions
bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip
Which can be treated as Multi-Agent System
23418 5
Emergencebull Goldstein ldquoThe arising of novel and
coherent structures patterns and properties during the process of self-organization in complex systems
bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo
bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow
orders from a leader
A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 3
bull Some social systems in Nature can present an intelligent collective behavior although they are composed by simple individuals
bull The intelligent solutions to problems naturally emerge from the self-organization and communication of these individuals
bull These systems provide important techniques that can be used in the development of artificial intelligent systems
The Computational Beauty of Nature
23418 4
Examples of Collective Behavior in Nature and Society
bull Many agents (individualpart)Many agents (individualpart)
bull Local and simple interactionsLocal and simple interactions
bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip
Which can be treated as Multi-Agent System
23418 5
Emergencebull Goldstein ldquoThe arising of novel and
coherent structures patterns and properties during the process of self-organization in complex systems
bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo
bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow
orders from a leader
A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 4
Examples of Collective Behavior in Nature and Society
bull Many agents (individualpart)Many agents (individualpart)
bull Local and simple interactionsLocal and simple interactions
bull New properties New properties emergeemerge bull phase transition pattern formation group movement hellipphase transition pattern formation group movement hellip
Which can be treated as Multi-Agent System
23418 5
Emergencebull Goldstein ldquoThe arising of novel and
coherent structures patterns and properties during the process of self-organization in complex systems
bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo
bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow
orders from a leader
A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 5
Emergencebull Goldstein ldquoThe arising of novel and
coherent structures patterns and properties during the process of self-organization in complex systems
bull Murray Gell-Mann ldquoSuperficial complexity that arises from a deep simplicityrdquo
bull Bottom-up behavior Simple agents following simple rules generate complex structuresbehaviors Agents donrsquot follow
orders from a leader
A termite cathedral A termite cathedral mound produced by a mound produced by a termite colony a classic termite colony a classic example of emergence in example of emergence in nature nature
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 6
bull Colonies of social insects can achieve flexible reliable intelligent complex system level performance from insect elements which are stereotyped unreliable unintelligent and simple
bull Insects follow simple rules use simple local communication (scent trails sound touch) with low computational demands
bull Global structure (eg nest) reliably emerges from the unreliable actions of many
Biological motivation Insect Societies
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 7
bull Collective systems capable of accomplishing difficult tasks in dynamic and varied environments without any external guidance or control and with no central coordination
bull Achieving a collective performance which could not normally be achieved by any individual acting alone
bull The colony as a whole is the seat of a stable and self-regulated organization of individual behavior which adapts itself very easily to the unpredictable characteristics of the environment within which it evolved
Insect Societies
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 8
Self Organizationbull Insect societies have developed systems of collective
decision making operating without symbolic representations exploiting the physical constraints of the environment in which they evolved and using communications between individuals either directly when in contact or indirectly (stigmergy) using the environment as a channel of communication
bull Through these direct and indirect interactions the society self organizes and faced with a problem finds a solution with a complexity far greater than that of the insects of which it is composed
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 9
Stigmergy
bull Indirect communication via interaction with environment [Gasseacute 59]ndash Sematonic [Wilson 75] stigmergy
bull action of agent directly related to problem solving and affects behavior of other agents
ndash Sign-based stigmergybull action of agent affects environment not directly
related to problem solving activity
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 10
Flocks Herds and Schoolsbull In the late 80rsquos Craig Reynolds created a simple model of animal motion that he called Boids
bull Itrsquos generates very realistic motion for movement from three simple rules which define a boidrsquos steering behaviour
bull This model and its variations has been used to drive animations of birds insects people fish antelope etc in films (eg Batman Returns Lion King)
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 11
Boid rulesSeparation steer to avoid crowding local flockmates
- A fundamental rule that has priority over the others- Also useful in avoiding collisions with other objects in the environment
Alignment steer towards the average heading and speed of local flockmates
- Enforces cohesion to keep the flock togetherHelps with collision avoidance too
Cohesion steer to move toward the average position of local flockmates
- Agents at edge of the herd more vulnerable to predators
- Helps to keep the flock together
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 12
bull Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull The express lsquoSwarm Intelligencersquo was originally used by Beni Hackwood and Wang in 1989 in the context of cellular robotic systems to describe the self-organization of simple mechanical agents
bull It was later extended to include ldquoany attempt to design algorithms or distributed problem-solving devices inspired by the collective behavior of social insect colonies and other animal societiesrdquo [Bonabeau Dorigo and Theraulaz 1999]
Swarm Intelligence
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 13
Swarm Intelligence (Contrsquod)
bull SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment
bull Although there is normally no centralized control structure dictating how individual agents should behave local interactions between such agents often lead to the emergence of global behavior
bull Sometimes called lsquoCollective Intelligencersquo
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 14
Components of SIbull Agents
ndash Interact with the world and with each other (either directly or indirectly)
bull Simple behavioursndash eg ants termites bees wasps
bull Communicationndash How agents interact with each otherndash eg pheromones of ants
Simple behaviours of individual agents
+ Communication between a group of agents
= Emergent complex behaviour of the group of agents
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 15
Characteristics of SI
bull Distributed no central control or data source
bull Limited communication
bull No (explicit) model of the environment
bull Perception of environment (sensing)
bull Ability to react to environment changes
Is SI relevant to people
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 16
The Web becomes a Giant Brain
Some see the Web evolving
intoa collective
brain for humankind
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 17
What is Multi-agent Systemsbull A set of agents which interact in a common environmentbull Focus on the collaborative resolution of global
problems by a set of distributed entitiesbull Agent attempt to satisfy their own local goals as well as
the collaborative global goals bull To successfully interact they will require the ability to
cooperate coordinate and negotiate with each other much as people do
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 18
What is MAS(Contrsquod)
bull MAS as seen from Distributed AIndash a loosely coupled network of entities that work
together to find answers to problems that are beyond the individual capabilities or knowledge of each entity
bull A more general meaningndash systems composed of autonomous components
that exhibit the following characteristicsbull each agent has incomplete capabilities to solve a problembull there is no global system controlbull data is decentralizedbull computation is asynchronous
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 19
bull Traditionalndash Client-serverndash Low-level
messagesndash Synchronousndash Can not do the job
bull Agent breakthroughsndash Peer-to-peer
topologyndash Blackboard
coordination modelndash Encapsulated
messagingndash High-level message
protocols
Client ServerFunction(Parameters)
Return(Parameters)
Traditional Software
IntelligentAgents
IntelligentAgents
IntelligentAgents
Blackboard
MessageReply
Agents IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
IntelligentAgentsIntelligent
Agents
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 20
Communication models
bull Theoretical models Speech act theory
bull Practical modelsndash shared languages like KIF KQML DAMLndash service models like DAML-Sndash social convention protocols
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 21
Working together
bull Benevolent Agentsndash assume agents are benevolent our best
interest is their best interest
bull Self-Interested Agentsndash Agents will be assumed to act to further their
own interests possibly at expense of othersndash Potential for conflict
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 22
Example mechanism Contract Net Protocol (CNP)
bull Negotiation as a collaboration mechanism
bull Negotiation on how tasks should be shared
ndash A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)
ndash An agent may subcontract another agent to perform a (sub)task
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
Contract
Bid
agent agent
CNP
Task announceme
nt
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 24
CNP (Contrsquod)
Contractor
Potential candidate agents
Task announcement (broadcast)
Contractor
Candidate Candidate
Bid
Bid
Phase 1 Task Announcement
- The contractor agent publicly announces a task
- Potential candidates evaluate the task according to their won skills and availability
Phase 2 Submission of Bids Proposals
- Agents that satisfy the requiremenst ie are able to perform the task send their bid proposal to the contractor
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 25
CNP (Contrsquod)
Contractor
Selected candidate
Contractor
Contracted agent
Contract
Phase 3 Selection
- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates
Phase 4 Contract awarding
- A contract is established between the contractor and the selected candidate
- A privileged bilateral communication channel is established between the two agents
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 26
Attributes of Multi-agent Systems
Apply MAS when some of the following features show up in a problem
bull Decentralizationbull Complex components often best described at
the knowledge levelbull Adaptive behaviorbull Complex interactionsbull Coordinationbull Emergent aggregate behaviors
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 27
Applications of MAS
Advanced Manufacturing Management Systemsminus Agents as representatives of machines users business
processes etc
Intelligent Information Search on Internetminus Some agents may show learning capabilities (learn the
preferences of their users )
Intelligent security enforcement on Internetminus Agents are representative of sensors or IDSs
Shopping Agents in Electronic Commerceminus With search price comparison and bargaining capabilities
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 28
Applications of MAS (Contrsquod)
Multi-agent auction in E-commerce
Distributed Surveillanceminus For information search or to look for special events informing
their users of relevant news
Distributed Signal Processingminus For problem diagnosis situation assessment etc in the
network
Distributed Problem Solvingminus Collaborative design scheduling and planning
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 29
How to simulate SI for search
Example1 Ant --gt Ant Colony Optimization (ACO)
Example2 Bird Flocking --gt Particle Swarm Optimization (PSO)
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 30
Part Ant Colony Optimization Ⅱ(ACO)
First proposed by M Dorigo 1992
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 31
Natural Antsbull Individual ants are simple insects with
limited memory and capable of performing simple actions
bull However an ant colony expresses a complex collective behavior providing intelligent solutions to problems such asndash carrying large itemsndash forming bridgesndash finding the shortest routes from the nest
to a food source prioritizing food sources based on their distance and ease of access
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 32
Natural Antsbull Moreover in a colony each ant has its prescribed task but
the ants can switch tasks if the collective needs it ndash Outside the nest ants can have 4 different tasks
bull Foraging searching for and retrieving food
bull Patrolling looking for food supply
bull Midden work Sorting the colony refuse pile
bull Nest maintenance work construction and clearing of chambers
ndash An antrsquos decision whether to perform a task depends onbull The Phisical State of the environment
ndash If part of the nest is damaged more ants do nest maintenance work to repair it
bull Social Interactions with other ants
Communication (direct or indirect) is necessary
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 33
bull They establish indirect communication system based on the deposition of pheromone over the path they followndash An isolated ant moves at random but when it finds a pheromone
trail there is a high probability that this ant will decide to follow the trail
ndash An ant foraging for food deposits pheromone over its route When it finds a food source it returns to the nest reinforcing its trail
ndash So other ants have greater probability to start following this trail and thereby laying more pheromone on it
ndash This process works as a positive feedback loop system because the higher the intensity of the pheromone over a trail the higher the probability of an ant start traveling through it
How can the natural ants find the shortest path
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 34
bull Since the route B is shorter the ants on this path will complete the travel more times and thereby lay more pheromone over it
bull The pheromone concentration on trail B will increase at a higher rate than on A and soon the ants on route A will choose to follow route B
bull Since most ants will no longer travel on route A and since the pheromone is volatile trail A will start evaporating
bull Only the shortest route will remain
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 35
Problems of ASbull Ant System tends to converge quickly
ndash This means that its exploitation of the best solution found is too high it should be exploring solution space more
ndash Pheromone evaporationupdate rule (better rule may exist)
bull Led to extensions of the ant systemndash Elitist Strategy for Ant Systems (EAS)ndash Rank based Ant Systems (AKRANK)ndash MAX-MIN Ant system (MMAS)ndash Ant Colony System (ACS)
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 36
Part Ⅲ Particle Swarm Optimization (PSO)
1048713Firstly Proposed by Kennedy and Eberhart 1995
ldquoOnce again nature has provided us with a technique for processing information that is at once elegant and versatilerdquo
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 37
bull In PSO a ldquoswarmrdquo is defined as an apparently disorganized collection (population) of moving individuals that tend to cluster together while each individual seems to be moving in a random direction
Bird flocking is one of the best example of PSO in nature
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 38
Modeling bird flocking
bull The synchrony of flocking behavior is thought to be a function of birdrsquos efforts to maintain an optimal distance between themselves and their neighborsndash Birds and fish adjust their physical movement to avoid
predators seek for food and mates ndash Humans tend to adjust our beliefs and attitudes to
conform with those of our social peers Humans change in abstract multidimensional space collision-free
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 39
Modeling bird flocking (Contrsquod)
bull Definitions ndash Flock is a group of objects that exhibit the general
class of polarized (aligned) non-colliding aggregate motion
ndash Boid is a simulated bird-like object ie it exhibits this type of behavior It can be a fish bee dinosaur etc
bull Rules for flockingndash Cohesion Each boid fly towards the centroid of its
local flock mates (that is boid in its local neighborhood)ndash Separation Each boid keep a certain distance away
from local flock mates to avoid collisionsndash Alignment Each boid align its velocity vector and keep
velocity magnitude similar with that of the local flock
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 40
bull Imagine a birdrsquos flock in an area where there is a single food source
bull A bird donrsquot know where the food is but it knows its distance to the food
bull The best strategy is to follow the bird that is closer to the food
bull Particles save and communicate the best solution they have found
From Bird to Particle
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 41
Features of PSO
bull Population initialized by assigning random positions and velocities potential solutions are then flown through hyperspace
bull Each particle keeps track of its ldquobestrdquo (highest fitness) position in hyperspacendash This is called ldquopBestrdquo for an individual particlendash It is called ldquogBestrdquo for the best in the populationndash It is called ldquolBestrdquo for the best in a defined neighborhood
bull At each time step each particle stochastically accelerates toward its pBest and gBest (or lBest)
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 42
Particle Swarm Optimization Process
Step1 Initialize population in hyperspace
Step2 Evaluate fitness of individual particles
Step3 Modify velocities based on previous best and global (or neighborhood) best
Step4 Terminate on some condition
Step5 Go to step 2
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 43
How do particles fly
bull Combination of gBest and the pBest (lBest)ndash need a compromise
bull lBest can bendash Social the particles around are always the same no matter
where they are in spacendash Geographical the particles around are those whose distance is
the shortest
bull Global PSO vs Local PSOndash the global version converges quickly to a solution but it gets
more easily stuck in local minima
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 44
Illustrating the velocity update schema of global PSO
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 45
PSO Related issues
bull Controlling velocities (determining the best value for Vmax)ndash Usually set maximum velocity to dynamic range of variable
bull Usually set c1 and c2 to 2
bull Inertia weight influences tradeoff between global and local explorationndash Good approach is to reduce inertia weight during run (ie from
09 to 04 over 1000 generations)
bull Swarm Size and Neighborhood Size
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 46
Advantages of PSO
bull Adaptation operates on velocitiesndash Most other methods operate on positionsndash Effect PSO has a builtin momentumndash Particles tend to hurdle past optima ndash an advantage
since the best positions are remembered anyway
bull Simple in conceptbull Easy to implementbull Computationally efficientbull Effective on a variety of problems
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 47
Summarybull Swarm Intelligence (SI)
ndash an artificial intelligence technique based around the study of collective behavior in decentralized self-organized systems
bull Multi-Agent Systems (MAS)ndash A system that consists of a number of agents
which interact with one-anotherndash Communication Coordination
Collaboration
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update
23418 48
bull Ant Colony Optimization (ACO)ndash Inspired by ant colony foragingndash pheromone as heuristic information
(stigmergy)ndash Iteration between ConstructAntSolutions and
UpdatePheromones
bull Particle Swarm Optimization (PSO)ndash Inspired by bird flockingndash Heuristic information results from partnersndash Particle Velocity Update