MindRACES, First Review Meeting, Lund, 11/01/2006

1

Anticipatory Behavior for Object Recognition and Robot Arm

Control

Modular and Hierarchical Systems, &Anticipatory Behavior and Control

Department of Cognitive PsychologyUniversity of Würzburg, Germany

Martin V. Butz, Oliver Herbort, Joachim Hoffmann, Andrea Kiesel

MindRACES, First Review Meeting, Lund, 11/01/2006 2

Related Publications

Date Journal/conference

Title Author

09/2005 CogWiss 2005

Towards the Advantages of Hierarchical Anticipatory Behavioral Control

Oliver Herbort, Martin V. Butz, & Joachim Hoffmann

11/2005 AAAI Fall Symposium

Towards an Adaptive Hierarchical Anticipatory Behavioral Control System

Oliver Herbort, Martin V. Butz, & Joachim Hoffmann

09/2005 In Book: Foundations of Learning Classifier Systems

Computational Complexity of the XCS ClassifierSystem

Matin V. Butz, David E. Goldberg, & Pier Luca Lanzi

(in press) Evolutionary Computation Journal (ECJ)

Automated Global Structure ExtractionFor Effective Local Building BlockProcessing in XCS

Matin V. Butz, Martin Pelikan, Xavier Llorà, & David E. Goldberg

Date Journal/conference

Title Author

11/2005 IEEE Transactions on Evolutionary Computation

Gradient Descent Methods in Learning Classifier Systems: Improving XCS Performance in Multistep Problems

Martin V. Butz, David E. Goldberg, & Pier Luca Lanzi

07/2005 GECCO 2005 (best paper nomination)

Extracted Global Structure Makes Local Building Block Processing Effective in XCS

Martin V. Butz, Martin Pelikan, Xavier Llora, David E. Goldberg

07/2005 GECCO 2005 (best paper nomination)

Kernel-based, Ellipsoidal Conditions in the Real-Valued XCS Classifier System

Martin V. Butz

11/2005 Book Rule-based Evolutionary OnlineLearning Systems:A Principled Approach to LCS Analysis and Design

Martin V. Butz

MindRACES, First Review Meeting, Lund, 11/01/2006 3

Overview

• Anticipatory Behavioral Control• Scenario involvement• Modular systems• Targeted system integrations

Learning of environment dynamics Object recognition, symbol grounding Hierarchical anticipatory arm control

MindRACES, First Review Meeting, Lund, 11/01/2006 4

Anticipatory Behavior Control

(Hoffmann, 1993, 2003)

effect A

effect Baction

effect C

situation

• Actions are selected, initiated and controlled by anticipating the desired sensory effects.

Goal

MindRACES, First Review Meeting, Lund, 11/01/2006 5

The Big Challenge

C la ss

Musc le -c ontro l

J o ints

Lo c a l fe a ture s

G lo b a l fe a ture s

Pe rc e p tio n

M o tiva tio nsEp site m ic va ria b le s

MindRACES, First Review Meeting, Lund, 11/01/2006 6

Scenario Involvement

• Watching a scene, learning existence and behavior of objects (Scenario 2) Continuous movement Blocking of movement Object permanence

• Control and manipulation of objects (Scenario 1) Cognitive, anticipatory arm control Interactive object manipulation

• Finding objects (Scenario 1) Search of particular objects (with certain properties) Search in room or house

• Behavior triggered by motivations (and possibly emotions) (Scenario 1)

MindRACES, First Review Meeting, Lund, 11/01/2006 7

Simple Object Recognition

• Scenario 2: Watching a scene Predicting object behavior /

movement Tracking multiple objects Learning object

permanence• Scenario 1:

Manipulating objects (with robot arm or directly)

Anticipatory control with inverse models (IM)

MindRACES, First Review Meeting, Lund, 11/01/2006 8

Multiple Objects

• Scenario 1: Searching objects Searching objects of

certain properties Partial observability

(fovea, multiple rooms) Multiple motivations

for multiple objects

F

F

FF

F

F

F

W

W

W

WWW

MindRACES, First Review Meeting, Lund, 11/01/2006 9

Learning Modules

• XCS predictive modules State prediction RL prediction

• The ALCS framework ACS2 & XACS

Predictive module RL module

• AIS for rule-linkage (OFAI)• Neural network modules

Hebbian-learning LSTM units (IDSIA) Rao-Ballard networks

• Kalman filtering techniques• Context processing (LUCS)

MindRACES, First Review Meeting, Lund, 11/01/2006 10

Integration of Modules

• Learning environment dynamics AIS-based sequences (OFAI) Context information for sequences (LUCS) Top-down, bottom-up (Kalman filtering-based)

combination of information• Combination with LSTM-based mechanisms

(IDSIA) For object permanence Object location out of sight (fovea region)

MindRACES, First Review Meeting, Lund, 11/01/2006 11

A Hierarchical Control Model

Body / Environment

interneurons

processing(visual, …)

motorsignals proprioception

IM

desired effects

IM

IM

IM IM IM

IM

descending signals exteroception

hand coordinates

joint angles

muscle lengthmuscle tension

MindRACES, First Review Meeting, Lund, 11/01/2006 12

11,

1q

2q

22 , elbowyxyx ,,,,

IM IM

motor torque

joint angle

arm configuration

hand coordinates

IM

IM IM

Current Cognitive Arm Model

MindRACES, First Review Meeting, Lund, 11/01/2006 13

Results: Arm

11,

1q

2q

22 , elbowyxyx ,,,,

IM IM

IM

IM IM

MindRACES, First Review Meeting, Lund, 11/01/2006 14

Summary

• Simple simulations For object recognition Object manipulation Development of interactive control structures

• Modular system combinations LSTM integration into XCS / ACS Context processing integration into XCS / ACS Integration of Kalman filtering techniques Rule-linkage with AIS principles Hierarchical combinations

• Anticipatory, developmental arm control models Learning to control an arm Learning the existence of objects

Object recognition Object behavior Object persistence