1

人間の動作のモデル化とロボットへの応用  Part  2

中村　仁彦

機械情報工学科(機械B) 東京大学

情報技術論 2011.04.25

index  

symbol

icon

Evolution / Development

C.S. Peirce (1839-1914)

index

Proto  Symbol  Space  (a  set  of  HMM)

Natural  Language  Processing

Dic?onary  Corpus

Statistical System

Technological Realization

association through

luxurious linguistic

knowledge

Linguistics

Phonology　音韻論 Morphology　形態論

Syntax　統語論 Semantics　意味論 Pragmatics　語用論

Machine Translation

Natural Language Processing

“Machine Translation of Languages “ (MIT Press 1955 translation) Warren Weaver 1949 applying statistical and cryptanalytic techniques

‘’The Mathematics of Statistical Machine Translation: Parameter Estimation. ’’ P.F. Brown, S.S Della Pietra, V.J.Della Pietra, R.L. Mercer (IBM T.J. Watson Research Center ) Computer Linguistics, Volume 19, No.2, 263-311, 1993.

The IBM Translation Model

2

Landauer, T. K. & Dumais, S. T. (1997). A solution to Plato's problem: The Latent Semantic Analysis theory of acquisition, induction and representation of knowledge. Psychological Review, 104, 211-240.

"Latent Semantic Analysis (LSA)” a high dimensional linear associative model to analyze a large corpus of natural text and generate a representation that captures the similarity of words and text passages.

"Plato's Problem" the term given by Noam Chomsky. It presents the question of how we account for our knowledge when environmental conditions seem to be an insufficient source of information. Noam Chomsky, "Barriers", MIT Press, 1986.

The Proto-Symbol Space

Motion recognition / abstract

Motion G

eneration 6

4

1

Discrete HMM 3

Proto-symbol

Motion elements

joint angles

Motion

2 5

Mathematical Model of Mirror Neuron

Continuous HMM

Walk Run

[Inamura et al. 02] Hidden Markov Model

1q 2q q NqN−1

11a

12a

22a

NNa 1−

}{ 21 ΜoooO =

)(1 ob )(2 ob

)}(,{ oba iij=λParameter of HMM ：state transition probabilities

),; ()(1

jjij

M

jiji ocob Σ=∑

=

µN

)(obi ：output probabilities

ijaProto-symbol

Continuous

3

Relationship between proto-symbols

Kullback-Leibler information as pseudo-distance

Definition of distance between proto-symbols

Similarity measure between probability density functions

( ) ( )[ ]∑ −=i

TT

n

ii ypypTn

D 211121 |log|log11),( λλλλ

Kullback-Leibler information

{ }),(),(21

1221 λλλλ DDDs +=

Run Walk Transition in the proto-symbol space

Experiment 1 : Behavior Recognition

walk kick

Deforming Proto-Symbol Space

W. Takano, H. Tanie, and Y. Nakamura, “Key Feature Extraction for Probabilistic Categorization of Human Motion Patterns,” Proc. of IEEE International Conference on Advanced Robotics, pp.424-430, 2005.

16 Motion Categories

walk, raise, wave, kick, bow, sit, crawl1,crawl2 stand, primitive walk, rise, run, applause, lie,

open and close legs, standing up right

Each motion category includes 100 motion patterns

1600 motion patterns are used and proto symbols of the samenumber can be acquired.

Motion patterns applied with proto symbol space

4

Proto Symbol Space Based on Joint Angles

Sit，Lie Standing

4 DOF in each arm, 6 DOF in each leg, position (x,y,z) of a body, attitude matrix Sequence of 32 dimensional data

Proto Symbol Space Based on Cartesian Coordinates

x,y,z coordinates of 3 joints in each arm, 3 joints in each leg, and body

Sequence of 39 dimensional data

Clusters of proto symbols seem to be located distributed

Proto Symbol Space in Joint-Angle and Cartesian Coordinates

Cluster analysis is implemented using probability density functions that approximate the distribution of proto symbols

Joint Angles

Cartesian Coordinates

Optimization of Weights

page 16

central point of the cluster is defined as the target point for each proto symbol corresponding to the cluster

Optimization of weights by the steepest descent method such that each proto symbol is placed at the target point in the space

5

Result of Cluster Analysis

Initial State Optimized state

Clusters are distributed by optimizing the weights

Extraction of Key Features

Raising right hand

Waving left hand

Kicking with right leg

Applauding

Ａ　body Ｂ　left shoulder Ｃ　left elbow Ｄ　left hand Ｅ　right shoulder Ｆ　right elbow Ｇ　right hand Ｈ　left hip Ｉ left knee Ｊ　left ankle Ｋ　right hip Ｌ　right knee Ｍ　right ankle The weights indicate the parts

that characterize the motion patterns

Extracting motion modifier of whole body motion patterns based on information geometric representation Hideharu Suzuki MS thesis, March 2008 Department of Mechano-Informatics University of Tokyo Advisor: Y. Nakamura

Tangential Property of Proto-Symbol Space

Information Geometry

abstracted （punch）

modifier （large pumch）

Tangent vector

geodedic

Parametric space

（kick）

（walk）

Proto-symbol space

Mapping to tangent spaces

large(punch） large(kick）

・modifier = local tangent ・conceptual modifier = common properties from many local tangents

6

Conceptual Modifier (1)

Basic Motion

Modified by modifiers

Conceptual Modifier (2)

Basic Motion

Modified by modifiers

Conceptual Modifier (3)

Basic Motion

Modified by modifiers

W. Takano, K. Yamane, and Y. Nakamura, “Capture Database through Symbolization, Recognition and Generation of Motion Patterns,” Proc. of IEEE International Conference on Robotics and Automation, pp.3092-3097, 2007. W. Takano, D. Kulic, and Y. Nakamura, “Interactive Topology Formation of Linguistic Space and Motion Space,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1416 -- 1422, 2007.

Deformation of Proto-Symbol Space by Word Labeling

7

The hierarchical structure of motion symbols and words

Machine Translation

Natural Language Processing

“Machine Translation of Languages “ (MIT Press 1955 translation) Warren Weaver 1949 applying statistical and cryptanalytic techniques

‘’The Mathematics of Statistical Machine Translation: Parameter Estimation. ’’ P.F. Brown, S.S Della Pietra, V.J.Della Pietra, R.L. Mercer (IBM T.J. Watson Research Center ) Computer Linguistics, Volume 19, No.2, 263-311, 1993.

The IBM Translation Model

The number of word labels ： 64

Motion capture corpus related to baseball

Capture sampling ： 33ms The number of motion data：537

The number of markers : 34

The number of symbols : 50 Total length of data ： 4088s

Motion Database Specification

“diving stand_up right_throw_pose standing”

Constructed word space

8

Reconstructed symbol space

Distances among symbols become larger. This means that discrimination can be improved.

“left_swing run”

Experimental result (motion search)

W. Takano and Y. Nakamura, “Integrating Whole Body Motion Primitives and Natural language for Humanoid Robots”, Proc. of IEEE-RAS International Conference on Humanoid Robots, pp.708-713, 2008. W. Takano and Y. Nakamura, “Statistically Integrated Semiotics that Enables Mutual Inference Between Linguistic and Behavioral Symbols for Humanoid Robots,” Proc. of IEEE International Conference on Robotics and Automation, 2009. W.Takano and Y. Nakamura, “Incremental Learning of Integrated Semiotics Based on Linguistic and Behavioral Symbols,” Proc. of IEEE/RSJ International Conference on Intelligent Robots and Systems, pp.2545-2550, 2009.

Where motion meets language

Research Purpose

Natural language model for Semantics and syntaxs based on text corpus

Robot can make inferences by using rich linguistic knowledge

9

Research Purpose Linguistic processing by integrating behavioral symbolization model and natural language models for humanoid robots.

・Behavioral symbolization based on HMMs is stochastic graphical framework. ・Natural language model for morphological analysis is also similar stochastic structured. ex. ・ CHASEN [K. Takeuchi, Y. Matsumoto 1995] ：HMM. ・ MECAB [T. Kudo, K. Yamamoto Y. Matsumoto 2004] ：Conditional Random Field.

High-Integrity between behavioral symbolization model and natural language model

Integration of Behavioral Symbol and Natural Language Model

play

0.1

run throw

player ball

0.2 0.05

0.0 0.0

…

player throw a

player throw a ball a

player run a

…

0.5

0.45

0.3

evaluation

Morphological Analysis

"Rosette Morphological Analysis System" google, amazon, MSN, Rakuten Dictionary for Japanese Morphological Analysis "UniDic" National Institute of Japanese Language "Juman" Morphological Analysis System for Japanese Language Language Media Laboratory Kyoto University (based on HMM) "ChaSen" Computational Linguistics Lab Nara Institute of Science and Technology Yuji Matsumoto IPA word-class system "MeCab" Taku Kudou Google, Mac OS X, iPhone Open Source IPA word-class system IPADIC , 3-4 times faster than ChaSen

Natural Language Processing Experimental Result: Interpretation of motion primitive as sentences

The number of latent state in motion language model : 50 The number of proto symbol : 10 The number of words : 15 The number of word class : 5

10

Experimental Result: Interpretation of motion primitive as sentences

Experimental Result: Motion Generation from Sentences

A player runs. A pitcher crouches on the mound..

Pedestrian scramble Shibuya, Tokyo

Humanoid Robotics = Science of Anthropomorphism

11

many thanks to my dear colleagues

W. Takano, K. Yamane, T. Inamura, T. Sugihara, D. Kulic K. Yamamoto, H. Tanie, H. Suzuki

and many more at YNL @UT

Category S of Grant-in-Aid for Scientific Research (20220001) FY2008-2012 Japan Society for the Promotion of Science. "Establishing Man-Machine Communication in Unifying Body Motion and Language"

Category S of Grant-in-Aid for Scientific Research (15100002) FY2003-2007 Japan Society for the Promotion of Science. "Development of Dynamic Information Processing Model of Intelligence"

Acknowledgements