October 26, Optimization

Multi-Robot Systems

CSCI 7000-006Monday, October 26, 2009

Nikolaus Correll

So far

• Probabilistic models for multi-robot systems– Extract probabilistic behavior of sub-systems– Small state space: rate equations– Large state space: DES simulation

Today

• System optimization using probabilistic models– Find optimal control parameters– Explore new capabilities using models– Find optimal control and system parameters

Comparison of Coordination Schemes

• Metrics for comparison– Speed– System cost– System Size– System Reliability– Benefits to the User

System-design is a constraint optimization problemSolution: Appropriate Models

Speed

Cost

Size

Reliability

Benefits

Mapping

No Mapping

Too large

Too slow

Too expensive

Model-based design

Real System Model Controller Design

Size, Cost, … Speed, Reliability, … Control parameters

Model-based optimization

• Physical simulator– Simulate controllers and

robot designs• DES simulator– Simulate controllers and

available information• Optimize using– Systematic experiments– Learning/optimization

Communication

Navigation accuracy

Optimization using analytical models

• Probabilistic state machine is derived from the robot controller

• One difference equation per state

111

)(1

)()1(

)(1

0

kNkNNkN

kNT

kNpkNkN

TkNpkNpkNkN

rcs

cc

sccc

rsrsrrr

Search Collision1/Tc

pc

Ns Nc

Restpr

Tr

Optimization using analytical models

• Probabilistic state machine is derived from the robot controller

• One difference equation per state

111

)(1

)()1(

)(1

0

kNkNNkN

kNT

kNpkNkN

TkNpkNpkNkN

rcs

cc

sccc

rsrsrrr

Search Collision1/Tc

pc

Ns Nc

Restpr

Tr

System parameters

Optimization using analytical models

• Probabilistic state machine is derived from the robot controller

• One difference equation per state

111

)(1

)()1(

)(1

0

kNkNNkN

kNT

kNpkNkN

TkNpkNpkNkN

rcs

cc

sccc

rsrsrrr

Search Collision1/Tc

pc

Ns Nc

Restpr

Tr

Control parameters

Optimal Control: Brief Intro

• Find optimal control inputs for a dynamical system to optimize a metric of interest

• Example: Tank reactor, maximize quantity B by tuning inflow and outflow

• Known: system dynamics

Ainflow

outflow

A->B->C

A->C

Static Optimization

• Find optimal control inputs (constant)

• Example: inflow 50l/min, outflow 10l/min

• Constraint: Volume of the tank at final time

inflow

A

outflow

flow volume

time

Dynamic Optimization• Find optimal control

input profiles (time-varying)

• Example: max inflow for 10s, outflow off, after 50s and outflow max

• Constraint: Volume of the tank during the whole process

flow volume

time

inflow

A

outflow

Optimal Control

• Capture terminal and stage cost as well as constraints using a single cost function

• The optimization problem is then solved by minimizing this cost function

Example: Coverage

• Collaboration policy:– Robots wait at tip for Ts – Waiting robots inform other

robots to abandon coverage • Trade-Off between additional

exploration versus decreased redundancy

• Communication introduces coupling among the robots (non-linear dynamics)

N. Correll and A. Martinoli. Modeling and Analysis of Beaconless and Beacon-Based Policies for a Swarm-Intelligent Inspection System. In IEEE International Conference on Robotics and Automation (ICRA), pages 2488-2493, Barcelona, Spain, 2005.

Optimal Control Problem

• A static beacon policy does not reduce completion time but only energy consumption

• Is there a dynamic policy which improves coverage performance?

• Find the optimal profile for the parameter Ts minimizing time to completion

Model

N. Correll and A. Martinoli. Towards Optimal Control of Self-Organized Robotic Inspection Systems. In 8th International IFAC Symposium on Robot Control (SYROCO), Bologna, Italy, 2006.

Optimization Problem

• Terminal cost: time to completion• Stage cost: energy consumption• Constraints: number of virgin blades zero

u=

Possible optimization method: Extremum Seeking Control

• Necessary condition of optimality:

• Optimization as a feedback control problem:

• Gradient Estimate by sinusoidal perturbation:

Optimal Marker PolicyStationary Marker

Optimal policy“Turn marker onafter around 180s,mark for 5s and goOn.”

Methodfmincon using the macroscopic model and optimal parameters based on base-line experiment.

Results/Discussion

• Optimal results when beacon behavior is turned on toward the end of the experiment

• Intuition: Exploration more important in the beginning

• An optimal beacon policy only exists if there are more robots than blades

Randomized Coverage with Mobile Marker-based Collaboration

Search Inspect MobileMarker

Avoid Obstacle

Wall | Robot Obstacle clear

Blade pt

1-pt | Marker

Tt expired

Translate Inspect Inspect

g=0 no collaborationg=1 full collaboration

No Collaboration vs. Mobile Markers

20 Real Robots Agent-based simulation

No Markers

Mobile Markers

Model-based design: Pitfalls

• Model-based controller design depends on – accurate parameters– Ideal model

• Optimization problem(s)– Find optimal control parameters– Find optimal model parameters

e

m

l

e

m

l

Estimate both model and control parameters simultaneously

M

M

Model

Model

“Optimal control under uncertainty of measurements”

Simultaneous optimization of model and control parameters

• How to select pi when Ti are unknown?• Optimization algorithm– Initial guess for model and control parameters– Run the system and collect data– Find optimal fit for model parameters– Find optimal control parameters– Repeat until error between experiment and model

vanishesN. Correll. Parameter Estimation and Optimal Control of Swarm-Robotic Systems: A Case Study in Distributed Task Allocation. In IEEE International Conference on Robotics and Automation, pages 3302-3307, Pasadena, USA, 2008.

Optimal Control of System and Control Parameters

Control Parameters

System Parameters

All experimentsNext experiment

Case Study: Task Allocation

• Finite number of tasks• Robots select task i with

probability1. pi =const.

(Independent robots)2. pi (k) function of Nj (k)

• Task i takes time Ti in average

K. Lerman, C. Jones, A. Galstyan, and M. Matarić, “Analysis ofdynamic task allocation in multi-robot systems,” Int. J. of RoboticsResearch, 2006.

1. Independent Robots

• Model(Number of robots in state i)

• Parameters– probability to do a task– System parameters

• Analytical optimization–

n ,,1

2. Threshold-based Task Allocation

• Probability to do a task– Stimulus– Threshold

• Stimulus: Number of robots doing the task already

• Model (non-linear)

• Optimization: numerical

Experiment

• Step 1: Estimate model parameters– Ti are unknown– Take random control parameters– Measure steady state– Find Ti given known control parameters

• Step 2: Find optimal control parameters

System dynamicsIn

depe

nden

t Rob

ots

(Lin

ear M

odel

)Th

resh

old-

base

d (N

on-L

inea

r Mod

el)

Difference Equations DES Simulation

100 robots 1000 robots

Results

Linear Model Non-Linear Model

25 robots 25 robots 100 robots

Summary

• System models can be used for finding optimal control policies and parameters

• Models can be physical simulation, DES, or analytical

• More abstract model allow for more efficient search, even analytical

• System parameters can be optimized simultaneously with system in the loop

Upcoming

• Multi-Robot Navigation (M. Otte)• Learning and adaptation in swarm systems• 3 weeks lectures, 1 week fall break• November 29: reports due• 2 weeks project presentations, random order