Design and Implementation of Motion Synthesis System Based on Support Phase

Quan Yu

OutlineBackground and Related WorkMy WorkSystem Design and Implementation

• Design Idea and System Framework

• Torso Coordinate and BVH File

• Support Phase Detecting

• Dynamic Time Warping

• Motion Blending

• Skinning

ResultConclusion and Future Work

BackgroundMotion Capture System Two Defects

–Only available data that can be used.–The data can only be applied to the character who shares

the same skeleton structure with the actor.

Solution: motion synthesis and retargeting

motion transition

Related WorkMotion Graph ：Make a transition between two frames with

the most similar postures.

Support Phase:Make a transition between two support

phases

computation consuming can be smoother

My Work

Implement an interactive motion synthesis system based on support phase detecting, dynamic time warping and motion blending

Skinning Mesh Animation

System Design and Implementation

• Design Idea and System Framework

• Torso Coordinate and BVH File

• Support Phase Detecting

• Dynamic Time Warping

• Motion Blending

• Skinning

Design Idea

Support Phases Detecting: —Find the idealist pair of support phases—Synchronize the motion and narrow the searching scope

Dynamic Time Warping—Find the start frame and end frame for blending—The smaller distance, the smoother transition

Motion Blending—Generate the transition with linear or spherical linear

interpolation

Design Idea(Cont.)

a general motion transition

a transition based on support phases

System Framework

System Flowchart

Torso Coordinate and BVH File• Torso Coordinate ： a tree structure• BVH File ： skeleton structure and motion

data HIERACHYROOT hip{ OFFSET 0.00 0.00 0.00 CHANNELS 6 Xpositon Ypos. Zpos. Xrotation Zrot. Yrot. JOINT chest { OFFSET 0.000000 5.018153 -1.882228 CHANNELS Xrotation Zrotation Yrotation …… End Site { OFFSET 0.000000 -6.699570 0.000000 } }}MOTIONFrames: 100Frame Time: 0.033333

0.000000 42.014908 0.240040 14.929800 0.146550 ……

Torso Coordinate and BVH File(Cont.)Transformation Matrix

—Compute joints positions in the world coordinate

parent world offset Rz Rx RyM M M M M M

E.g. : transformation matrix for left ankle

LeftAnkle Hip LeftHip LeftFemur LeftTibia offset Rz Rx RyM M M M M M M M M

The position of left ankle: [0,0,0,1]TLeftAnkle LeftAnkleP M

Torso Coordinate and BVH File(Cont.)

void calJointsPositionAtFrame(Joint* joint,int frame_no){ GLfloat M[16]; glPushMatrix(); glTranslatef(x_offset,y_offset,z_offset); glRotatef(radX, 1.0f, 0.0f, 0.0f ); glRotatef(radY, 0.0f, 1.0f, 0.0f ); glRotatef(radZ, 0.0f, 0.0f, 1.0f );

glGetFloatv(GL_MODELVIEW_MATRIX,M); Vector3 pos=Vector3(M[12],M[13],M[14]); mapJointsPositionAtFrame[frame_no]->push_back(pos);  

for (int i=0; i<joint->numOfChildren(); i++ )calJointsPositionAtFrame( joint->getChild(i), frame_no );  // recursion

glPopMatrix();}

Recursive Program by utilizing matrix stack of OpenGL

Support Phase Detecting• Find the idealist pair of support phases• Support Phase

—The constrain between the feet and the ground— Four Categories

• no support(NS), right leg support(RS), left leg support(LS), double leg support(DS)

• Classification criteria– The height and velocity of feet

NS> *> *

RS> *

&< >

< * < <

LS< *

&< <

> * < >

DS< <

&< >

< < < >

Support Phase Category

The height of left leg

The velocity of left leg

The height of right leg

The velocity of right leg

“ > ” ： larger than the threshold“ < ” ： smaller than the threshold” * ” ： unrelated

Support Phase Detecting(Cont.)

Choose the candidate pairs

“ O ” : possible“ X “ : impossible

e.g.Rules:

Support Phase Detecting(Cont.)Choose the idealist pair of support phasesSelection Criteria

—The lengths of A_sp and B_sp are nearly the same.

—Both the lengths of A_sp and B_sp are close to the ideal length

—Final criterion

—Choose the pair with minimal w.

11 1

2

nw

n

1 22 1

2 _

n nw

IDEAL LENGTH

1 2w w w

Dynamic Time WarpingFind the start frame for A_sp and end frame for B_spFeature Vector

—Position, velocity and acceleration of Joints in WCS.—Align the root joint of frame B_sp_j with that of frame A_sp_i.

1 2 1 2 1 2,align x x z z rady radyM M M

Velocity: 1 1

2 interval

k kk i ii

pos posvel

Acceleration:1 1

2 interval

k kk i ii

vel velacc

Alignment Matrix:

Frame distance— Measure the similarity of a pair of frames(postures)

Choose the pair with minimal frame distance

position distance:k k

NA B

pos i ii

L pos pos

velocity distance: k k

NA B

vel i ii

L vel vel

acceleration distance ： k k

NA B

acc i ii

L acc acc

frame distance:pos vel accL L L L

Dynamic Time Warping(Cont.)

Dynamic Time Warping(Cont.)Criteria for blending length(1) Use the ideal length (threshold) if possible;

(2) Or else, use the length as long as possible;

(3) If the length is smaller than minimal threshold, use minimal length, and

adjust the start fame and the end frame.

Motion BlendingMotion Alignment

—Align the position of the root joint of B_sp_i with that of A_sp_i—Align the velocity direction of the root joint of B_sp_i with that

of A_sp_i

Motion Blending (Cont.)Interpolation①root position: linear interpolation②root orientation: linear interpolation with small angle③other channels: spherical linear interpolation

1( ) [1 ( )]

startBlend k endBlend BlendLength kk A BP k P k P

1( ) [1 ( )]

startBlend k endBlend BlendLength k

X X Xk A BR k R k R

Also and

YkR

ZkR

1

1

( , , ( ))

sin ( ) sin(1 ( ))

sin sin

startBlend k endBlend BlendLength k

startBlend k endBlend BlendLength k

i i ik A B

i iA B

q slerp q q k

i iq q

Skinning Linear blend skinning Skin: obj file Weight : xml file Binding: 3dmax skin modifier The position of a vertex in WCS

1

1

ˆ ˆb

i i ii

v wM M v

1

1b

ii

w

vertex position in WCS at binding pose

Inverse transformation matrix of bone i at the binding pose

transformation matrix of bone i at a frame

weight of vertex related to bone i

the number of bones influencing this vertex

vertex position in WCS at a frame

v̂1ˆ

iM i

iM

iw

b

v

ResultUser interface

— VC2005 、 OpenGL and Qt

Result(Cont.)parameter dialog

Result(Cont.)Support phase detecting resultAdjust the threshold manually if there is a need

Result(Cont.)

a transition from walking to running

Result (Cont.)

a transition from walking to jumping

Result (Cont.)

skinning result

Conclusion and Future Work

Design and implement a motion synthesis system based on support phases, dynamic time warping and motion blending.

Future Work– motion path synthesis– improve skinning algorithm and result

Thank you!