Implicit Probabilistic Models of Implicit Probabilistic Models of Human Motion for Synthesis and Human Motion for Synthesis and

TrackingTracking

Hedvig Sidenbladh, KTH, Sweden (now FOI, Sweden)

Michael J. Black, Brown University, USA

Leonid Sigal, Brown University, USA

Articulated 3D trackingArticulated 3D tracking

= model parameters

I = image

1111 )|()|()|()|( ttttttttt dIppIκpIp Recursive Bayesian formulation:

Extreme case

• Non-linear motion, strong dependencies

• Model dependencies analytically– e.g. [Hogg, Rohr] for walking

• Dynamical models– e.g. [Wren&Pentland , Bruderlin&Calvert]

• Learn from Mocap examples– e.g. [Bowden, Brand, Molina&Hilton]

Modeling Human MotionModeling Human Motion

Texture SynthesisTexture Synthesis

Efros & Freeman’01

“Database”Synthetic Texture

– e.g. [De Bonnet&Viola, Efros&Leung, Efros&Freeman, Paztor&Freeman, Hertzmann et al]

– Image(s) as an implicit probabilistic model.

Motion TextureMotion Texture

– Motion examples as an implicit probabilistic model.

Probabilistic FormulationProbabilistic Formulation

Generative model: )( ti

),()|( tti Np

Problem: Model for all motions i in the database!)|( tip No learning required, all the variability in the data captured.

)|( 1ttp

Database example i

)|( 11 tipSampling from

Sampling from , taking t = i

Probabilistic Database SearchProbabilistic Database Search

• Sort database in some way to enable search in sublinear time

• Linear search infeasible for large database!

)|( tip • No need to visit all database examples - only

need to sample from distribution

time

joint angles

c =[c1, c2, c3, c4]

• Sort into tree-structure according to PCA coefficients

Probabilistic Database SearchProbabilistic Database Search

ii cEach level in the tree corresponds to one coefficient l.

Sort examples i into tree: Left subtree for negative value of cl,i, right for positive value.

Probabilistic Database SearchProbabilistic Database Search

)|()|( titi

tt

pp cc

c

Approximated by sampling from tree iteratively:

)|0(

)|0(

,,,

,,,

tlilleftl

tlilrightl

ccpp

ccpp

SynthesisSynthesis

Running

Walking

Small database with running, walking, skipping, dance etc. Changing color indicates new example sequence.

Future work: Add editing possibility, gravity model, goal function.

Goal: Generate smooth and plausible-looking motion.

TrackingTrackingGoal: Efficiently generate samples (image data will sort out which are good).

Temperature parameter controls randomness of tree search.

Arm Tracking ExampleArm Tracking Example

Constant velocity model

1000 samples~1 min/frame

Image likelihood model from

[Sidenbladh & Black, ICCV 01]

Example based model

””Mocap Soup” [Cohen] at Mocap Soup” [Cohen] at SIGGRAPH 02SIGGRAPH 02

– Arikan & Forsyth. Interactive motion generation from examples

– Li et al. Motion textures: A two-level statistical model for character motion synthesis

– Lee et al. Interactive control of avatars animated with human motion data

– Kovar et al. Motion graphs

– Pullen & Bregler. Motion capture assisted animation: Texturing and synthesis

Here we formulate a probabilistic model suitable for stochastic search search and Bayesian tracking.

ConclusionsConclusions

• Implicit motion model - replace learning with search

• Analog to example based texture synthesis

• Larger database - sub-linear time search

• Tree structure sorted with PCA coefficients• Probabilistic tree search - sampling from the

tree approximates sampling from )|( tip