View
234
Download
0
Category
Preview:
Citation preview
8/4/2019 Biologically Inspired Algorithm
1/29
Biologically InspiredBiologically InspiredComputationComputation
Mohamed A ElMohamed A El-- SharkawiSharkawiThe CIA labThe CIA lab
Department of ElectricalDepartment of ElectricalEngineeringEngineering
University of WashingtonUniversity of WashingtonSeattle, WA 98195Seattle, WA 98195-- 25002500
elsharkawi@ee.washington.eduelsharkawi@ee.washington.eduhttp://http://cialab. ee.washington. educialab. ee.washington. edu
M. A. El-Sharkawi, BIA 3
... the Babbling of Old Fools ?... the Babbling of Old Fools ?
M. A. El-Sharkawi, BIA 4
!! To an uninspired scientist with a hammer,To an uninspired scientist with a hammer,
everything looks like a naileverything looks like a nail
!! If at first, the idea is not absurd, then there is noIf at first, the idea is not absurd, then there is no
hope for ithope for it
Albert Einstein
Seek new ideas even if out of theSeek new ideas even if out of the
ordinaryordinary
M. A. El-Sharkawi, BIA 5
Seek new ideas even if you areSeek new ideas even if you are
unsure of their appealunsure of their appeal
!! If we knew what it was we were doing, itIf we knew what it was we were doing, it
would not be called research, would it?would not be called research, would it?
M. A. El-Sharkawi, BIA 6
Wild Idea is a Sign of brillianceWild Idea is a Sign of brilliance
!! Imagination is more importantImagination is more important
than knowledge.than knowledge.
For knowledge is limited to all weFor knowledge is limited to all we
now know and understand,now know and understand,
while imagination embraces thewhile imagination embraces the
entire world, and all there ever willentire world, and all there ever will
be to know and understandbe to know and understand
!! Logic will get you from A to B.Logic will get you from A to B.
Imagination will take youImagination will take you
everywhereeverywhere
8/4/2019 Biologically Inspired Algorithm
2/29
History of Computation
M. A. El-Sharkawi, BIA 8
5K YR Ago
RHIND
PAPYRUS
RHIND
PAPYRUS
RHIND
PAPYRUS
M. A. El-Sharkawi, BIA 12
Driven by necessity we have progressed from
using fingers
8/4/2019 Biologically Inspired Algorithm
3/29
to tables
M. A. El-Sharkawi, BIA 14
to abacus
to mechanical machines
M. A. El-Sharkawi, BIA 16
to slide rules
to more mechanical machines
M. A. El-Sharkawi, BIA 18
to calculators
8/4/2019 Biologically Inspired Algorithm
4/29
M. A. El-Sharkawi, BIA 19
to computers
M. A. El-Sharkawi, BIA 20
Impact of Modern ComputingPower
! Staggering impact on our ability to
Compute through iterations
Create amazingly complex algorithms
M. A. El-Sharkawi, BIA 21
New Wave of Computing
! Moores law is still true computers double their power every 18
month.
! Computationally intensive techniqueshard to implement a few years agoare now feasible. Among these techniques are the so
called biologically inspired algorithms(BIA).
M. A. El-Sharkawi, BIA 22
Biologically Inspired AlgorithmsBiologically Inspired Algorithms
M. A. El-Sharkawi, BIA 23
Biologically Inspired SystemsBiologically Inspired Systems
! Philosophers argued that with allhuman achievements in science andengineering, nature still providesthe best systems that can ever befashioned.
M. A. El-Sharkawi, BIA 24
Biologically Inspired SystemsBiologically Inspired Systems
This is trueeven if wecompare the
most complexmachine withthe simplestform of abiological cell.
8/4/2019 Biologically Inspired Algorithm
5/29
M. A. El-Sharkawi, BIA 25
AndAndheeeeereheeeeeressAlbertAlbert
!! When the solution isWhen the solution is
simple, God is answering.simple, God is answering.
Albert Einstein
M. A. El-Sharkawi, BIA 26
BIA or Biocomputation
! The use ofbiological processesorbehavior as metaphor, inspiration, orenabler in developing new computingtechnologies
! The field is highly multidisciplinary,Engineers, computer scientists, molecularbiologists, geneticists, mathematicians,physicists, and others.
M. A. El-Sharkawi, BIA 27
Main steps of BiologicallyMain steps of BiologicallyInspired SystemsInspired Systems
! Observe and study: Observeanimal and human behaviors
and Study biological structures
! Mimic: Acquired knowledge may help us mimic
nature and develop better engineeringsystems and machines.
M. A. El-Sharkawi, BIA 28
The ABCThe ABCs of BIAs of BIA(Bezdek)(Bezdek)
AArtificialrtificial BBiologicaliological CComputationalomputational
M. A. El-Sharkawi, BIA 29
BIA SystemsBIA Systems!! Neural NetworksNeural Networks!! Evolutionary AlgorithmsEvolutionary Algorithms!! Fuzzy SystemsFuzzy Systems
!! Swarm IntelligenceSwarm Intelligence!! BoidsBoids!! Particle SwarmParticle Swarm!! DNA ComputingDNA Computing!! Artificial LifeArtificial Life!! Intelligent AgentsIntelligent Agents
!!
M. A. El-Sharkawi, BIA 30
Nature is a Powerful Paradigm
! Brain""""Neural Networks
! Evolution theory""""Evolutionary Algorithms
! Flocking birds""""Particle Swarm Optimization,Boids
! Insects""""Swarm Intelligence
!
!
8/4/2019 Biologically Inspired Algorithm
6/29
M. A. El-Sharkawi, BIA 31
Why BIA?Why BIA?
# May require little or no knowledge of the physicalsystem they emulate.
# System is developed by observations and intuition.
# Can find substance in data bases throughstochastic search.
# Can be self- tuned using raw experiential data.
# Noise tolerant and robust.
# Objectives no longer need to be in restrictivemathematical forms.
# Nonlinearity is no longer a disabling constraint.
M. A. El-Sharkawi, BIA 32
Key ImprovementsKey Improvements
$The performance of BIAs is betterunderstood and appreciated.#NN can have an explanation facilities
and is no longer a black boxes.#Stability criteria for fuzzy control, long
lacking, can be established# Convergence is improved and global
optimization within a predeterminedspace can be achieved.
M. A. El-Sharkawi, BIA 33
Conventional Model basedConventional Model based
SystemSystem
Sensors
-Model
Desired
Parameteror StateEstimation
DecisionModel
M. A. El-Sharkawi, BIA 34
NonNon--Model based SystemModel based System
Process
Process
Engine
Engine
Data
from
Sen
sors
Data
from
Sen
sors
DesiredDesired
DecisionDecision
M. A. El-Sharkawi, BIA 35
Classical Control: Design
System
inputs
Control
Inputs
Constraints
M. A. El-Sharkawi, BIA 36
Classical Control: Operation
System
inputs
Control
Inputs
Constraints
8/4/2019 Biologically Inspired Algorithm
7/29
M. A. El-Sharkawi, BIA 37
Intelligent Control
System
inputs
Control
Inputs
Constraints
M. A. El-Sharkawi, BIA 38
Intelligent Control
System
inputs
Control
Inputs
Constraints
M. A. El-Sharkawi, BIA 39
Neural Networks
M. A. El-Sharkawi, BIA 40
M. A. El-Sharkawi, BIA 41
M. A. El-Sharkawi, BIA 42
InputPat
terns
OutputDe
cision
Connections
Input layer
Hidden layer
Output layer
8/4/2019 Biologically Inspired Algorithm
8/29
M. A. El-Sharkawi, BIA 43
InputPattern
s
OutputDecisio
n
Connections adjustment
Input layer
Hidden layer
Output layer
M. A. El-Sharkawi, BIA 44
InputPatterns
OutputDecision
Connections (Fixed)
Input layer
Hidden layer
Output layer
M. A. El-Sharkawi, BIA 45
ElEl--Sharkaw
i
Sharkaw
i
NN TrainingNN Training
M. A. El-Sharkawi, BIA 46
NN Training:NN Training:El-Sharkawis Quintuplets
NotEl
NotEl--Sharkawi
Sharkawi
M. A. El-Sharkawi, BIA 47
ElEl--Sharkawi
Sharkawi
Testing of Trained NNTesting of Trained NN
M. A. El-Sharkawi, BIA 48
NotEl
NotEl--Shar
kawi
Shar
kawi
Testing of Trained NNTesting of Trained NN
8/4/2019 Biologically Inspired Algorithm
9/29
M. A. El-Sharkawi, BIA 49
Almost !Almost !
Evolutionary AlgorithmsEvolutionary Algorithms
M. A. El-Sharkawi, BIA 51
Evolutionary Algorithm
! Single Search Technique (SST) could be trapped
in undesirable minima
! Convergence of SST depends on the selected
initial condition
M. A. El-Sharkawi, BIA 52
Evolutionary Algorithm vs SST
Single Search Evolutionary Search
M. A. El-Sharkawi, BIA 53
Population Pool
1 0 0 1 11 1 1 11 0 0 000 0 11 1 1 0 00 0
...
Byte 1 Byte 2 Byte n
1
00 1 11 1 1 11 0 0 0 00 0 11 110 0 0 0
...
1 0 0 1 11 1 111 0 0 000 0 11 1 1 0 0
00
...
1 0 01
1
1 1 1 11 0 00 00 0 1
1
1 10 000
...
0
individual
#1
#2
#3
#K
2 n
M. A. El-Sharkawi, BIA 54
Fitness Evaluation
#1
#2
#3
Individuals
1 0 0 111 0
00 111 0
1 0 0 11 1 0
1 0 01 11 0
0
#n
FitnessComputations
f(.) Normalize
Ranked Individuals
#q
#p0 0 11 1 0
1 0 0 11 1 0
#p
#q
0
0 0 11 1 00
1 0 0 11 1 0
#1 1 0 0 111 0
1 0 01 11 0#3
#n 1 0 0 11 1 0
#2 00 111 00
8/4/2019 Biologically Inspired Algorithm
10/29
M. A. El-Sharkawi, BIA 55
Crossover
#p
#q
0 0 11 1 00
1 0 0 11 1 0
Crossover0
0 1
1 1
0
0
1 0
0 1
1 1
0#p
#q
Crossover point
M. A. El-Sharkawi, BIA 56
Success of Crossover
11111111 00000001 00000001 11111111
00000001 00000001 11111111 11111111
Global OptimalityGlobal Optimality
Smartness Beauty
M. A. El-Sharkawi, BIA 57
Experiment
M. A. El-Sharkawi, BIA 58
Fuzzy Systems
M. A. El-Sharkawi, BIA 59
AndAndheeeeereheeeeeressAlbertAlbert
!! The only real valuableThe only real valuable
thing is intuition.thing is intuition.
Albert Einstein
M. A. El-Sharkawi, BIA 60
Universe (X)
Subset A
Subset B
Subset C
Crisp Set/SubsetCrisp Set/Subset
1=A1=B
1=C
0=A
8/4/2019 Biologically Inspired Algorithm
11/29
M. A. El-Sharkawi, BIA 61
9
8
10
On a scale of
one to 10, how
goodwas the
dive?
! close, heavy, light, big, small, smart, fast, slow,
hot, cold, tallandshort.
9.5
M. A. El-Sharkawi, BIA 62
! Fuzzy Probability
! Example #1 Billy has 9 toes. The
probability Billy has 10
toes is zero.
The fuzzy membership
of Billy in the set of
people with 10 toes is
nonzero.
M. A. El-Sharkawi, BIA 63
Example #2 A bottle of liquid has a probability of
of being rat poison and of being pure
water. A second bottles contents, in the fuzzy
set of liquids containing lots of rat
poison, is .
The meaning of for the two bottles
clearly differs significantly and would
impact your choice should you be
dying of thirst.
(cite: Bezdek)
#1
#2
M. A. El-Sharkawi, BIA 64
Fuzzy ConflictsFuzzy Conflicts
Tall Short
M. A. El-Sharkawi, BIA 65
Fuzzy ConflictsFuzzy Conflicts
Tall ShortTall or Short?Tall or Short?
M. A. El-Sharkawi, BIA 66
Fuzzy AssociationFuzzy Association
4
5
6
7
Very Tall
Tall
Medium
Short
Very Short
8/4/2019 Biologically Inspired Algorithm
12/29
M. A. El-Sharkawi, BIA 67
M. A. El-Sharkawi, BIA 68
! Based on intuition and judgment
! No need for a mathematical model
! Provides a smooth transition betweenmembers and nonmembers
! Relat ively simple, fast and adaptive
! Less sensitive to system fluctuations
! Can implement design objectives, dif ficult toexpress mathematically, in linguistic ordescriptive rules.
M. A. El-Sharkawi, BIA 69
Applications DomainApplications Domain
! Fuzzy Logic
! Fuzzy Modeling
Neuro-Fuzzy System! Fuzzy Control
Intelligent Control
Hybrid Control
! Fuzzy Pattern Recognition
M. A. El-Sharkawi, BIA 70
SomeSomeInteresting ApplicationsInteresting Applications
! Smooth ride control
! Camcorder auto-focus and jiggle control
! Braking systems
! Copier quality control
! Rice cooker temperature control
! High performance drives
! Air-conditioning systems
M. A. El-Sharkawi, BIA 71
Swarm Intelligence
Swarm IntelligenceSwarm Intelligence
8/4/2019 Biologically Inspired Algorithm
13/29
M. A. El-Sharkawi, BIA 73
Swarm IntelligenceSwarm Intelligence
=CoordinationCoordination withoutwithout
Direct CommunicationDirect Communication
M. A. El-Sharkawi, BIA 74
Swarm Intelligence
! Appears in swarms of certain insectspecies
! Interactions is indirect (stigmergy)
! Complex forms of social behaviorcan achieve a number of tasks
M. A. El-Sharkawi, BIA 75
SWIN - Stigmergy
! Stigmergy means communicationthrough the environment
Pheromone laying : pheromone attractsants who lay even more pheromone
Task- relatedstigmergy: e. g. termite nestbuilding
M. A. El-Sharkawi, BIA 76
Pheromone Trails
M. A. El-Sharkawi, BIA 77
Features
! Distributed andMulti- agentsearch
! Uses stigmergy (communicationthrough the environment) for agentinteraction
! Adaptive
! Uses reinforcement learning
! Randomness """" robustness
M. A. El-Sharkawi, BIA 78
Example of Swarm IntelligenceExample of Swarm Intelligence
Application to Complex SystemsApplication to Complex Systems(Routing in Data Networks)(Routing in Data Networks)
8/4/2019 Biologically Inspired Algorithm
14/29
M. A. El-Sharkawi, BIA 79
Routing in Data NetworksRouting in Data Networks
!Why is it so complex? Distributed problem, requires coordination of a
number of nodes, which might not have direct
communication
Must cope with node failures
Must re-adjust routes that get congested
M. A. El-Sharkawi, BIA 80
Existing Schemes
! Wired
Distance vector (Bellman-Ford)
Linkstate (Dijkstras)
OSPF (Internet Standard linkstate-based)
! Wireless
Table-driven (all nodes have routing tables to all
destinations)
On-demand (routing tables are created on demand)
M. A. El-Sharkawi, BIA 81
Swarm Advantages in Routing
Yes for distributed BF,
but not for link-state.
Yes, packets can be
re-routed if links fail
Robustness
Slow adaptationYes, adapts to
congestionpatterns
Load balancing
Some, but may cause
oscillations. Can be slow.
Yes, can be very fastAdaptive
Some of them(e.g
Bellman Ford)
YesDistributed
Non-Swarm-basedSwarm- based
M. A. El-Sharkawi, BIA 82
AntNetAntNet
! Introduced by Di Caro and Dorigo
! Designed for packet switching networks
! Uses ants who affect routing based on
trip times to destination
! This is not done directly, but through a
processed quantity
M. A. El-Sharkawi, BIA 83
ANTNET: Routing TablesANTNET: Routing Tables
G
D
0.720.28
0.850.15
FB
Next Hop
Destination
Sum of row=1
Probability thatnext-hop is B ifdestination is D
F D
n
BK
Current
M. A. El-Sharkawi, BIA 84
A
B
C
D
G
E
F
AB 0.23
BC 0.11
AB 0.23
CD 0.14
BC 0.11AB 0.23
DE 0.15
CD 0.14
BC 0.11
AB 0.23
Forward AntsForward Ants
8/4/2019 Biologically Inspired Algorithm
15/29
M. A. El-Sharkawi, BIA 85
A
B
C
D
G
E
F
AB 0.23
BC 0.11
AB 0.23
CD 0.14
BC 0.11
AB 0.23
DE 0.15
CD 0.14
BC 0.11
AB 0.23
Backward AntsBackward Ants
M. A. El-Sharkawi, BIA 86
Information Carried byInformation Carried by
Backward AntBackward Ant
! Trip-time table
Contains trip-time estimate to every destination
G
D
0.010.18
0.020.24
VarianceMean
Time
Destination
M. A. El-Sharkawi, BIA 87
Why Forward and Backward
! Routing tables are updated only by ants
who were successful in reaching
destination
! If a ant is lost, it will not generate a
backward ant
next-hops that were not successful will not
be updated
M. A. El-Sharkawi, BIA 88
The Algorithm-Essential steps
! Launch forward ant
! Find path randomly, but based onrouting table probabilities
! Remember the path and the arrival timesfor forward ant
! Backward ants trace path in reverse
! While at that, they update routing tables
M. A. El-Sharkawi, BIA 89
AntNet: Table updates
Pdfis theprobability of choosing nodefas next-hop, when the destinationis d
N is the set of neighbors of the current node
fis the next-hop node
n are all the other neighbor nodes
+ and - are the positive and negative reinforcement, respectively
r= weighting factor (step size)
Nn,fn,P)'r(
)P)('r(
PP
PP
dn
df
dndn
dfdf
=
=
+=
+=
+
+
1
11
M. A. El-Sharkawi, BIA 90
Weighting Factor
T= trip time from current node to the destination
= average ofT
c= scaling factor (usually 2)
8/4/2019 Biologically Inspired Algorithm
16/29
M. A. El-Sharkawi, BIA 91
AntNet
! Equation are based on negative feedback:
Positive reinforcement (+) should besmaller as probabilities get larger and viceversa
Negative reinforcement (-) should be largeras probabilities get larger and vice versa
M. A. El-Sharkawi, BIA 92
AntNet
! Problem: Tcould be unreliable (high
variance)
! Solution: Use over ratio to adjust r
! Why? Because high / means Tis
unreliable and vice versa
M. A. El-Sharkawi, BIA 93
r = T/ > 0.5r = T/ < 0.5
Adjusting r
a
e
a
e
+
Good time Bad time
! Add nonlinearity: r"(r )h
! Why? To manipulate the learning rate r
! Choice of h: usually 2 (heuristic)
T unreliable
T reliable
Monotonicallydecreasingfunction
M. A. El-Sharkawi, BIA 94
Wireless Issues
! Broadcast advantage
When node transmits, it can be heard by all nodes in itsrange
! Energy constraints
Routing must take into account energy
Energy can also be adjusted by adjusting range
! Noise andbandwidth requirements also affectenergy and range
! Node mobility : nodes can move in and out ofrange
M. A. El-Sharkawi, BIA 95
Swarm ControlSwarm Control
M. A. El-Sharkawi, BIA 96
ObjectiveObjective
! To move a group of vehicles (rovers,
robots, underwater vehicles)
independently to achieve common or
distributed objectives.
8/4/2019 Biologically Inspired Algorithm
17/29
M. A. El-Sharkawi, BIA 97
Artificial Potential Field (APF)Artificial Potential Field (APF)
! The motion of the vehicle is governed by a
minimum of two fields: attractive andrepulsive fields.
! Attractive field: a force that moves thevehicle toward the target position.
! Repulsive field: a force that moves thevehicle away from obstacles or areas that isbeing covered by other vehicles.
M. A. El-Sharkawi, BIA 98
Target
destination
Obstacle
Vr
VaV
Vr
Va
V
Vr
Va
V
M. A. El-Sharkawi, BIA 99
Attraction VelocityAttraction Velocity
Cartesian distance between the vehicle and the target
Da: distance between the vehicle and the target
: angle between the vehicle and the target
=
sin
cosDV aa
M. A. El-Sharkawi, BIA 100
Repulsion VelocityRepulsion Velocity
Dr:distance between the vehicle and the obstacle.
: angle between the vehicle and the obstacle.
=
sincos
DV
r
r 1
M. A. El-Sharkawi, BIA 101
Total VelocityTotal Velocity
! andare modulation parameters (can bedynamically changing)
For example: if the vehicle is to achieve astand-still state at the target point, is set tozero whenDa approaches zero.
! is the effect of other influences (otherrobots to avoid)
++= ra VVV
M. A. El-Sharkawi, BIA 102
BoidsBoids
8/4/2019 Biologically Inspired Algorithm
18/29
M. A. El-Sharkawi, BIA 103
History ofHistory ofBoidsBoids (Flocking)(Flocking)
! Pioneered by Craig Reynolds
! Coherent flocking behavior using simple
rules was first demonstrated in the late
1980s .
To develop realistic motions of groups of
actors in computer animation.
Each actor (boid) has a scripted path
predetermined by the animator.
M. A. El-Sharkawi, BIA 104
Flocking rules
! Basic rules (necessary and sufficient)
Cohesion: fly towards the center of your local flock mates.
Separation: keep a certain distance away from nearest flockmates.
Alignment: align your velocity vector with that of the localflock
! additional rules (for better flocking dynamics)
Evasion: avoid occupying the same local airspace as yournearest flockmate. Evasion is a localized form of separation.
Migration: fly towards a pre-specified location
M. A. El-Sharkawi, BIA 105
Cohesion
! Move toward the
center of local
flockmates
M. A. El-Sharkawi, BIA 106
Separation
! Steer to avoid collision
with local flockmates
M. A. El-Sharkawi, BIA 107
Alignment
Steer towards the
average heading of
local flockmates
M. A. El-Sharkawi, BIA 108
Evasion
Avoid occupying the
same local space as
your nearest
flockmate.
8/4/2019 Biologically Inspired Algorithm
19/29
M. A. El-Sharkawi, BIA 109
Migration
Fly towards a pre-
specified location
M. A. El-Sharkawi, BIA 110
Final motion vector
Original velocity
Cohesion
Mitigation
Net velocity
Alignment
Evasion
Separation
M. A. El-Sharkawi, BIA 111
Control Applications
! Simple behaviors for individuals and pairs: Seekand Flee
Pursue andEvade
Obstacle Avoidance
Path or wall Following
Flow Field Following
! Combined behaviors and groups: Crowd Path Following
Leader Following
Unaligned Collision Avoidance
Queuing(at a doorway)
Flocking(combining:separation
M. A. El-Sharkawi, BIA 112
Tracking
Particle Swarm OptimizationParticle Swarm Optimization
M. A. El-Sharkawi, BIA 114
Particle Swarm OptimizationParticle Swarm Optimization
=Coordination withCoordination withDirectDirectCommunicationCommunication
8/4/2019 Biologically Inspired Algorithm
20/29
Particle Swarm Optimization
! Inventors: James Kennedy and RussellEberhart
! An Algorithm originally developed toimitate the motion of a Flock of Birds,or insects
! Assumes Information Exchange (SocialInteractions) among the search agents
! Basic Idea: Keep track of Global Best
Self Best
How does it work?
! Problem:Find X which minimizes f(X)
! Particle Swarm: Start:Start: Random set of solution vectors
Experiment:Experiment: Include randomness in the choiceof new states.
Remember:Remember: Encode the information aboutgood solutions.
Improvise:Improvise: Use the experience informationto initiate search in a new regions
PersonalBest at previous step
Currentmotion
Component in thedirection of personal best
Component in thedirection of previous motion
Component in thedirection of global best
New Motion
Global best
8/4/2019 Biologically Inspired Algorithm
21/29
M. A. El-Sharkawi, BIA 121
PSO Modeling
! Each solution vector is modeled as The coordinatesof a bird or a particle in a
swarm flying through the search space
All the particles have a non- zero velocity andthus never stop flying and are alwayssampling new regions.
! Each particle remembers Where the global best and where the local
best are.
M. A. El-Sharkawi, BIA 122
! The search is guided by The collective consciousness of the
swarm
Introducing randomness into thedynamics in a controlled manner
PSO Modeling
M. A. El-Sharkawi, BIA 123
InertiaInertia
non- zero velocityPS never stop flying
Controlled randomness
Particle Swarm Dynamics
))()().(,0(
))()().(,0()(.)1(
2
1
kxkxar
kxkxarkvwkv
GroupBest
SelfBest
rr
rrrr
+
+=+
)()()1( kvkxkxrrr
+=+
The collective consciousnessof the swarm
Self consciousnessof the swarm
M. A. El-Sharkawi, BIA 124
PSO
x is a solution vector particle and v is the velocityof this particle
a1
and a2 are two scalars,
w is the inertia
r(0,1) is a uniform random number generatorbetween 0 and 1
vgbvsb
vold
vnew
M. A. El-Sharkawi, BIA 125
Design Parameters
! a1
and a2
!w: Should be between [0.9 and 1.2]
High values of w gives a global search
Low values of w gives a local search
! vmax: To be designed according to the
nature of the search surface.
M. A. El-Sharkawi, BIA 126
The Art of Fitness Function! To move robots to boundary points
Metric: |f(x)- boundary value|
8/4/2019 Biologically Inspired Algorithm
22/29
Results - Case 1
M. A. El-Sharkawi, BIA 128
The Art of Fitness Function! Distribute robots uniformly on the
boundary
Metric:|f(x)- boundary value| -Distance to closest neighbor
(to penalize proximity to neighbors)
Results - Case 2
M. A. El-Sharkawi, BIA 130
The Art of Fitness Function! Distribute robots uniformly on the
boundary close to current state
Metric:|f(x)- boundary value| - Distance to
closest neighbor + Distance tocurrent state
(penalize proximity to neighbors, penalize distance from currentstate)
Results - Case 3
M. A. El-Sharkawi, BIA 132
PSO ChallengesPSO Challenges
! Like any search technique, PSO could be
unsuccessful at distinguishing between global and
local minima
Local minimum is easier to find
If fitness function cannot amplify the difference
between global and local minima, PSO is likely
to stay in the local minima
8/4/2019 Biologically Inspired Algorithm
23/29
M. A. El-Sharkawi, BIA 133
Modified PSOModified PSO
! Two-step PSO (or gradient-approximation)
! Cluster PSO
M. A. El-Sharkawi, BIA 134
TwoTwo--Step PSOStep PSO
! Each particle takes two steps: short and long
Then decide on optimal step based on steeper
negative gradient
x0
xS
xL
M. A. El-Sharkawi, BIA 135
TwoTwo--Step PSOStep PSO
! Better method at not overflying narrow
valleys
!Problems:
Particles may take the short step more often
than the long step, resulting in slower
convergence
Can still get trapped in local minima
M. A. El-Sharkawi, BIA 136
Cluster PSOCluster PSO
! It is a hierarchical version of PSO:
PSO are arranged in clusters
Each cluster contains multiple agents
Each cluster has a centroid that acts,effectively, as a standard PSO agent
Each agent within the cluster is attractedto itspersonal best, the cluster best, andthe cluster centroid
M. A. El-Sharkawi, BIA 137
Cluster PSOCluster PSO
vc
vgbvcbvabva
vcb
vcc
M. A. El-Sharkawi, BIA 138
Cluster PSOCluster PSO
vc=wcvc+ac1(xcb-xcc) +ac2 (xgb-xcc)
va=wa va+a1(xab-xa)+a2 (xcb-xa) +a3 (xcc-xa)
xcc=xcc+vc
xa=xa+vc+va
8/4/2019 Biologically Inspired Algorithm
24/29
M. A. El-Sharkawi, BIA 139
Cluster PSOCluster PSO
! Cluster PSO combines globally superior
ability of standard PSO in avoiding localminima with the locally efficient search
which can find narrow global minima.
MAYBE!
MultiobjectivesMultiobjectives Optimization &Optimization &
Pareto FrontsPareto Fronts
M. A. El-Sharkawi, BIA 141
Conventional OptimizationConventional Optimization
! Any optimization problem can be framed as
finding the parameter minimizing a given
objective function
! findx* that minimizes y =f(x)
! Typically, one solution
x
f(x)
x*
f(x*)
M. A. El-Sharkawi, BIA 142
Conventional OptimizationConventional Optimization
! Most optimization problems have several
(possibly conflicting) objectives.
! These problems are frequently treated as
Single-objective optimization problemsby
transforming all but one objective into
constraints.
Single cost function where all objectives are
weighted according to their importance.
M. A. El-Sharkawi, BIA 143
Weighted Aggregation for MultiWeighted Aggregation for Multi--
Objective OptimizationObjective Optimization
! Objective: minimize the following functions
( ) ( ) ( )xf...,...,xf,xf n21
( ) ( ) ( ) ( )xfw......xfwxfwxJ nn+++= 2211
! Formulation: aggregate all functions in a weighted
fashion to form one cost index
M. A. El-Sharkawi, BIA 144
Limitation of AggregatedLimitation of Aggregated
Optimization functionOptimization function
! Single aggregated function leads to onlyone solution.
! The relative importance must be selectedbefore hand and the final solution isdependent on the selection.
! Trade offs can not be easily evaluated.
! Solution is not possible fornonconvexsearch spaces.
8/4/2019 Biologically Inspired Algorithm
25/29
M. A. El-Sharkawi, BIA 145
TradeTrade--off Analysis (Example of Conflictingoff Analysis (Example of Conflicting
Objectives)Objectives)
! Designing ofdistributed controllers while
reducing the cost.
! Place functional blocks on a chip such that chip
area is minimized and power dissipation is also
minimized.
! Find the vehicle that covers the most distance in a
day while requiring the least energy.
! Placing enough sensors to monitor a system while
reducing the cost.
M. A. El-Sharkawi, BIA 146
Simple Problem, Single solutionSimple Problem, Single solution
Find x* such that ( ) ( )xfxf i*
i for all i
f1(x*)
f2(x*)
f1
f2
M. A. El-Sharkawi, BIA 147
Simple Problem, Multiple SolutionsSimple Problem, Multiple Solutions
Example: Operational AmplifierExample: Operational Amplifier
DC gain [dB]
transitfreq.[M
Hz]
bette
r
better
M. A. El-Sharkawi, BIA 148
F
1f
2F
1f
2f
MultiMulti--Objective Optimization: Pareto FrontObjective Optimization: Pareto Front
better
better
minimize 21, ff
better
better
maximize 21, ff
Pareto FrontPareto Front
M. A. El-Sharkawi, BIA 149
Goal of MultiGoal of Multi--ObjectiveObjective
Optimization (MOO)Optimization (MOO)
!Finding a set of decision variables that
satisfies constraints and
optimizes a vector of objectives.
! The term optimize means finding a
solution which would give values of all the
objective functions acceptable to the
designer
M. A. El-Sharkawi, BIA 150
feasible region
Mathematical Formulation of MOOMathematical Formulation of MOO
! Find the vectorx* =[x1*, x2*, ...,xn*]T
which satisfies
m inequality constraints:
gi(x)>= 0, i=1,2,...,m
and p equality constraints:
hi(x) = 0, i=1,2,...,p
and optimizes the vector function;
f(x)=[f1(x), f2(x), ..., fk(x)]T
8/4/2019 Biologically Inspired Algorithm
26/29
M. A. El-Sharkawi, BIA 151
Terminologies used in MOOTerminologies used in MOO
!! The solution is dominant ifThe solution is dominant if
( ) ( )( ) ( ) ( )k...,......,,ixfxf
ixfxf
i
*
i
i
*
i
21oneleastatfor
and,allfor
Recommended