Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Spatio-Temporal Case-Based Reasoning

for Behavioral Selection

Maxim Likhachev and Ronald ArkinMobile Robot Laboratory

Georgia Tech

Maxim Likhachev and Ronald Arkin

Broad Picture of the Work

• Part of Multi-Level Learning in Hybrid Deliberative/Reactive Mobile Robot Architectural Software Systems project at Georgia Tech

• Sponsored by the DARPA MARS program

Spatio-Temporal Case-Based Reasoning for Behavioral Selection


Motivation• Constant parameterization of robotic behavior

results in inefficient robot performance• Manual selection of “right” parameters is difficult

and tedious work



Motivation (cont’d)• Use of Case-Based Reasoning methodology for an

automatic selection of optimal parameters in run-time



Evaluated on:• Simulations

• Real robot– ATRV-JR in outdoor environment

– Nomad 150 in indoor environment



Related Work• ACBARR, SINS and KINS systems

– use of case-based reasoning and reinforcement learning for the optimization of behavioral parameters

– contribute to some ideas behind the present algorithm

• Automatic optimization of parameters – genetic programming

– reinforcement learning



Behavioral Control and CBR Module

CBR Module controls: Weights for each behavior BiasMove Vector

Noise Persistence Obstacle Sphere



Input Features for Case Selection• Vector of spatial characteristics of environment

– D - distance to the goal– <σ, r> - degree of obstruction and distance to the most obstructing cluster of obstacles for each of K angular regions around the robot



Input Features for Case Selection• Vector of temporal characteristics of environment

– Rs - short term robot movement

– Rl - long term robot movement



Computation of Traversability Vector F

F: – represents traversability of each region

– approximates obstacle density function around the robot

– independent of goal distance

– smoothed over time:



Input Features: ExampleSpatio-Temporal Case-Based Reasoning for Behavioral Selection

f0=0.92

f1=0.58

f2=1.0

f3=0.68

f0=0.02

f1=0.22

f2=0.63

f3=0.02

Vtemporal

ShortTerm: Rs=1.0LongTerm: Rl=0.7

Vtemporal

ShortTerm: Rs=0.01LongTerm: Rl=1.0


High Level Structure of CBR Module Spatio-Temporal Case-Based Reasoning for Behavioral Selection

Currentenvironment

FeatureIdentification

Spatial Features &Temporal Features

vectors

Spatial Features Vector Matching

(1st stage of Case Selection)

Temporal Features Vector Matching

(2nd stage of Case Selection)

Set ofSpatiallyMatching

cases

Set of Spatially and Temporally

Matching cases

Case switching

Decision tree

CaseAdaptation

Case Library

All the casesin the library

Best Matching orcurrently used case

CaseApplication

Case ready for application

Case Output Parameters(Behavioral Assemblage

Parameters)

Random Selection Process

(3rd stage of Case Selection)

Best Matchingcase


Case Example I Spatio-Temporal Case-Based Reasoning for Behavioral Selection

CLEARGOALSpatial Vector:D (goal distance) = 5 density distance Region 0: σ0 = 0.00; r0 = 0.00Region 1: σ1 = 0.00; r1 = 0.00Region 2: σ2 = 0.00; r2 = 0.00Region 3: σ3 = 0.00; r3 = 0.00Temporal Vector:(0 - min, 1 - max) ShortTerm_Motion Rs = 1.000 LongTerm_Motion Rl = 0.700Case Output Parameters:MoveToGoal_Gain = 2.00Noise_Gain = 0.00Noise_Persistence = 10Obstacle_Gain = 2.00Obstacle_Sphere = 0.50Bias_Vector_X = 0.00Bias_Vector_Y = 0.00Bias_Vector_Gain = 0.00CaseTime = 3.0


Case Example II Spatio-Temporal Case-Based Reasoning for Behavioral Selection

FRONTOBSTRUCTED_SHORTTERMSpatial Vector:D (goal distance) = 5 density distance Region 0: σ0 = 1.00; r0 = 1.00Region 1: σ1 = 0.80; r1 = 1.00Region 2: σ2 = 0.00; r2 = 1.00Region 3: σ3 = 0.80; r3 = 1.00Temporal Vector:(0 - min, 1 - max) ShortTerm_Motion Rs = 0.000 LongTerm_Motion Rl = 0.600Case Output Parameters:MoveToGoal_Gain = 0.10Noise_Gain = 0.02Noise_Persistence = 10Obstacle_Gain = 0.80Obstacle_Sphere = 1.50Bias_Vector_X = -1.00Bias_Vector_Y = 0.70Bias_Vector_Gain = 0.70CaseTime = 2.0


Results Spatio-Temporal Case-Based Reasoning for Behavioral Selection

0

100

200

300

400

500

600

700

800

Heterogeneous Homogenous,15% density

Homogeneous,20% density

Environment type

Nonadaptive system CBR system

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

Heterogeneous Homogenous,15% density

Homogeneous,20% density

Environment type

Nonadaptive system CBR system

Average travel distance Mission success rate

Simulations:

ATRV-JR: 12% average performance improvement in time steps ( based on 10 runs for each system in outdoor environment)


Simulations & real robot experiments: Performance improvement as a function of

obstacle density


Performance improvement using CBR over nonadaptive system

0.00

5.00

10.00

15.00

20.00

25.00

0.00 5.00 10.00 15.00 20.00 25.00

Obstacle density

Per

cent

impr

ovem

ent

Traveled Distance Improvement Time Steps Improvement

Simulations Nomad 150

Based on 10 runs for each systemin indoor environment


Real Robot Run with CBRSpatio-Temporal Case-Based Reasoning for Behavioral Selection


Real Robot Run without CBRSpatio-Temporal Case-Based Reasoning for Behavioral Selection


Trajectories of the robotSpatio-Temporal Case-Based Reasoning for Behavioral Selection

Robot with CBR module Robot without CBR module

11% less travel distance


Conclusions• Automatic selection of optimal behavioral

parameters results in robot performance improvement (based on simulations and real robot experiments)

• Careful manual selection of behavioral parameters is no longer required from a user

• Future Work– Automatic learning of cases:

• identifying when to create a new case• applying reinforcement learning techniques in finding optimal

parameters for existing cases

– Integration with other adaptation & learning methods (e.g., Learning Momentum)


Documents

Spatio-Temporal Case-Based Reasoning for Behavioral Selection