Teaching an Agent by Playing a Multimodal Memory Game: Challenges for Machine Learners and Human Teachers AAAI 2009 Spring Symposium: Agents that Learn

Teaching an Agent by Playing a Multimodal Memory Game:

Challenges for Machine Learners and Human Teachers

AAAI 2009 Spring Symposium: Agents that Learn from Human TeachersMarch 23-25, 2009, Stanford University

Byoung-Tak Zhang

Biointelligence LaboratorySchool of Computer Science and Engineering

Cognitive Science, Brain Science, and BioinformaticsSeoul National University, Seoul 151-744, Korea

[email protected]://bi.snu.ac.kr/

http://bi.snu.ac.kr/

© 2009, SNU Biointelligence Lab, http://bi.snu.ac.kr/

2

Talk Outline

Multimodal Memory Game (MMG)

Challenges for Machine Learners

Challenges for Human Teachers

Toward Self-teaching Cognitive Agents


ImageImage SoundSound TextText

But, I'm getting married tomorrowWell, maybe I am...I keep thinking about you.And I'm wondering if we made a mistake giving up so fast.Are you thinking about me?But if you are, call me tonight.


Image-to-Text Generator(I2T)

Image-to-Text Generator(I2T)

Text-to-Image Generator(T2I)

Text-to-Image Generator(T2I)

Text Text HintHint









Hint Hint ImageImage

Machine LearnerMachine Learner

Toward Human-Level Machine Learn-ing: Multimodal Memory Game

(MMG)

ImageImage SoundSound


4

Text Generation Game (from Im-age)

TextText

I2TI2T

Learningby Viewing

Learningby Viewing

T2IT2IGameManager

GameManager

Text HintT

TextTextImageImage SoundSound


5

Image Generation Game (from Text)

I2TI2T

Learningby Viewing

Learningby Viewing

T2IT2IGameManager

GameManager

Hint ImageI

Characteristics of MMG Game

Interactive Multimodal Long-lasting Hard to learn Scalable data Humans as teachers Difficulty controllable Learning by imitation (viewing and watching)


6

Three Approaches

Learning Architecture¨ Model

Learning Strategies¨ Algorithms

Teaching Strategies¨ Humans


7

Methods of Machine Learning

Symbolic Learning¨ Version Space Learning¨ Case-Based Learning

Neural (Connectionist) Learning¨ Multilayer Perceptrons¨ Self-Organizing Maps ¨ Hopfield Networks

Evolutionary Learning¨ Evolution Strategies¨ Evolutionary Programming¨ Genetic Algorithms¨ Genetic Programming

Probabilistic Learning¨ Bayesian Networks¨ Boltzmann Machines¨ Hidden Markov Models¨ Deep Belief Networks¨ Hypernetworks

Other Machine Learning Methods¨ Reinforcement Learning ¨ Decision Trees¨ Boosting Algorithms¨ Kernel Methods (SVM)¨ PCA, ICA, LDA etc.

Learning with Hypernetworks


9

x1x2

x3

x4

x5

x6

x7

x8 x9

x10

x11

x12

x13

x14

x15

)( 2121...21

21

21...21

321

321321

21

2121

321

321321

21

2121

2 ,...,,

)()()()(

2 ,...,,

)()()()(

,,

)()()()3(

,

)()()2(

)()(

,,

)()()()3(

,

)()()2()(

...)(

1exp)Z(

isfunction partition thewhere

,...)(

1exp

)Z(

1

...6

1

2

1exp

)Z(

1

)];(exp[)Z(

1 )|(

ondistributiy probabilit The

...6

1

2

1 );(

rkhypernetwo theofenergy The

m kkiiikiii

k

kiiikiii

iiiiiiiiii

iiiiiiiiii

K

k iii

mmmk

K

k iii

nnnk

iii

nnn

ii

nn

nn

iii

nnn

ii

nnn

xxxwkc

W

xxxwkcW

xxxwxxwW

WEW

WP

xxxwxxwWE

x

xx

x

Nn

K

iii

i

D

WWWW

SkXSSS

xxxX

WSXH

I

1)(

)()3()2(

,...,,

}{

:set Training

),...,,(

|| , ,

)(

),,(

as defined isrk hypernetwo The

21

x

[Zhang, DNA12-2006]


10

How to Learn from Image-Text Pairs


11

How to Generate Image from Text

Image-to-Text Translation Re-sults


12

AnswerQuery

I don't know what happened

There's a kitty in my guitar case

Maybe there's something I can do to make sure I get pregnant

Maybe there's something there's something I … I get pregnant

There's a a kitty in … in my guitar case

I don't know don't know what know what happened

Matching & Completion

Text-to-Image Translation Re-sults


13

Query Matching & Completion

I don't know what happened

Take a look at this

There's a kitty in my guitar case

Maybe there's something I can do to make sure I get pregnant

Answer

Further Challenges

Challenges for Machine Learners

Incremental learning Online learning Fast update One-shot learning Predictive learning Memory capacity Selective attention Active learning Context-awareness

Persistency Concept drift Multisensory integration


15

Challenges for Human Teachers

Getting feedback Sequencing examples Identifying the weak points Choosing problems Controlling parameters Evaluating progress Estimating difficulty Generating new queries Modeling the effect of

learning parameters

Catching environmental change Minimal interactions Multimodal interaction


16

Conclusion Multimodal memory game (MMG)

¨ Highly-interactive lifelong learning scenario¨ Challenges current machine learning techniques

Challenges for machine learners¨ More attentive, active behavior¨ Rather than parameter fitting, passive adaptation

Human partners¨ More active role in interacting with the agents

The future: Self-teaching cognitive agents¨ Cognitive learning agents that teach themselves

= Active learning agents + cognitively-aware human teachers¨ Design new queries and test their answers by interacting with humans


17

Documents

Teaching an Agent by Playing a Multimodal Memory Game: Challenges for Machine Learners and Human Teachers AAAI 2009 Spring Symposium: Agents that Learn