Motion CaptureMotion Capture

Motion Capture

Motion capture Motion Tracking or Mocap is a technique of digitally recording movements for entertainment sports medical applicationshellip

Way of using a physical device to control animation

Motion Capture Technologyminusminusmagnetic

bull Electromechanical transducers

-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20

Motion Capture Technologyminusminus Mechanical Capture Systems

bull Any environmentbull Measures joint anglesbull Restricts the motion

Motion Capture Technologyminusminus Optical Capture Systems

bull Place markers on the object or on the actor

Passive reflection Active light sources

bull Cameras determine marker positions (Raw Point Data)

bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture

Motion capture Motion Tracking or Mocap is a technique of digitally recording movements for entertainment sports medical applicationshellip

Way of using a physical device to control animation

Motion Capture Technologyminusminusmagnetic

bull Electromechanical transducers

-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20

Motion Capture Technologyminusminus Mechanical Capture Systems

bull Any environmentbull Measures joint anglesbull Restricts the motion

Motion Capture Technologyminusminus Optical Capture Systems

bull Place markers on the object or on the actor

Passive reflection Active light sources

bull Cameras determine marker positions (Raw Point Data)

bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture Technologyminusminusmagnetic

bull Electromechanical transducers

-Polhemus Fastrakbull Limited rangeresolutionbull Tetheredbull Sensitive to metalbull No identification problembull 6 dof informationbull Realtimebull Low frequency 30 to 120 Hzbull Few markers 10minus20

Motion Capture Technologyminusminus Mechanical Capture Systems

bull Any environmentbull Measures joint anglesbull Restricts the motion

Motion Capture Technologyminusminus Optical Capture Systems

bull Place markers on the object or on the actor

Passive reflection Active light sources

bull Cameras determine marker positions (Raw Point Data)

bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture Technologyminusminus Mechanical Capture Systems

bull Any environmentbull Measures joint anglesbull Restricts the motion

Motion Capture Technologyminusminus Optical Capture Systems

bull Place markers on the object or on the actor

Passive reflection Active light sources

bull Cameras determine marker positions (Raw Point Data)

bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture Technologyminusminus Optical Capture Systems

bull Place markers on the object or on the actor

Passive reflection Active light sources

bull Cameras determine marker positions (Raw Point Data)

bull 8 or more camerasbull Restricted volumebull High Frequency (240Hz)bull Occlusions

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture

Technology Issuesbull Resolutionrange of motionbull Calibrationbull Occlusioncorrespondencebull Accuracy Marker movementplacement Sensor noise Skew in measurement time

bull Occlusioncorrespondence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

2 Perform a motion trial3 Compute marker positions from

camera images4 Identify and uniquely label markers5 Calculate joint angles from markers

paths

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

1 Find the skeleton dimensions and exact marker positions on the body

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Automatic Calibration

bull Design Goalsbull 1048707 Fully

automaticbull 1048707 Any skeletonbull 1048707 Accurate

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Inputbull Generic

Skeletonbull Actorrsquos

kinematics

structure and

rough handle

positions

Input

bull Calibration Data

bull Initial path data

that exercises

all of the subjectrsquos

DOFs

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Independent Variables

bull DOFsbull Bone lengthsbull Handle offsetsbull Global scale

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Marker Identification

At each frame motion capture gives us a set of points

We would like something more intuitive

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Marker Identification

Making sense of raw datahellip

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture -- Optical motion capture process

Calculate joint angles from marker paths Inverse Kinematics problem (IK) 1 Create a handle on body- position or orientation2 Pull on the handle3 IK figures out how joint angles should changeInputsAn articulated skeleton with handles Desired positions for handlesOutputsJoint angles that move handles to desired positions

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture

Inverse Kinematics problem can be solved using a non-linear optimization technique which seeks to minimize the deviation between the recorded data and the hierarchical model

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Motion Capture

The minimized fitness function can be defined as

F() = p (Pj ndash Pj)2 + O0 (O0j ndash O0j)2 +

O1 (O1j ndash O1j)2 + c Cj2

where is the set of joint angles for the set of joints J Pj is the position of the jth joint given and Pj is the recorded joint position from the capture phase O0j and O1j are two vectors defining the orientation of the joint withO0j and O1j being the recorded orientations Two vectors are used because together with their cross product they will form a right-handed coordinate system The quantities p O0 and O1 are scalar weights that can be tuned to achieve better results Additionally a joint angle constraint term Cj in the form of a penalty function is employed Joint angle constraints for humans have been measured and can be found in the biomechanical literature

Jj

Jj

Jj

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

BIOMECHANICAL MOVEMENTS

There is a huge amount literature devoted

simulating human movements See for instance the seminal paper by A Witkin

and M Kass Spacetime Constraints in proceedings of SIGGRAPH 1988 159-168[1]

and D C Brogan K P Granata and P N Sheth Spacetime constraints for biomechanical movements in proceedings Applied Modelling and Simulation 2002 [2] [2]

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

BIOMECHANICAL MOVEMENTS

The authors of the paper [1] use a variant of sequential quadratic programming to perform an iterative gradient-based optimization

The user provides the algorithm with an initial configuration a final configuration and initial guesses to populate the remaining n-2 character configurations and n joint torques

We can represent the algorithmrsquos elements symbolically by using S to represent the state vector C to represent the vector of spatial and dynamical constraint violations and R( S) to represent the objective function

We must solve for S such that C =0 and R( S) is minimized Provided an initial S three elements are computed the

constraint violations C the Jacobian of the constraint functions and the Hessian of the objective function

S

CJ

2

2

S

R

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

BIOMECHANICAL MOVEMENTS

bull A two-step process first solves for a local change in the state vector Sα that minimizes the objective function without consideration of constraint violations

bull To minimize the objective function the deriviate R(S) = 0 is used in [2]

bull A second-order Taylor series expansion is used to approximate R at S

bull Letting Sα = X ndash S we can solve

bull Although we know R(S) is minimized at S + Sα spatial and dynamical constraints may be violated

)(0)(2

2

SS

R

S

RSR

SS

R

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

BIOMECHANICAL MOVEMENTS

We must find a second change in the state vector Sβ that preserves the minimization of R while eliminating any constraint violations at S + Sα We pursue a second step that projects Sα to the null space of the constraint Jacobian

J (Sα + Sβ) + C = 0 Note both the constraint vector C and the Jacobian J are

evaluated at S + Sα Solving for Sβ in

- C = J (Sα + Sβ)

drives C to zero The new value of S is incremented by (Sα + Sβ)

The algorithm computes new state vectors until the reduction

in the objective requires violating constraints

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Biomechanics To better understand human

movements biomechanical models must be developed that accurately describe human physiology and control strategies

The standardized definition of the spacetime constraints state vector constraints vector and objective function facilitates the reconfiguration and reuse of biomechanical models

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Biomechanics

Experiment To implement a simulation that

determines the movement trajectory simultaneously with the optimum activation torques

The simulated walker is placed on a downward-sloped plane such that if it were passively simulated (with no internal torque sources) it would settle into a steady gate

The movement trajectories of the passive-dynamic walker require zero-torque activation and are known to be the optimal trajectories for actively powered walkers

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Biomechanics

Experiment

bull The (a passive-dynamic) model a planar knee-less walker included two legs of mass mL joined by a revolute joint located at the hip with point mass mH

bull The walker moves along a plane of slope γ with respect to horizontal A time-dependent vector θ = [θS θN]T represents the walker configuration where θS and θN are the angles of the stance-leg and nonstance-leg versus ground normal

During walking only one foot is in contact with the ground at any time ie single-stance Ground clearance of the swing-leg is ignored

A bipedal walker Lumped masses are positioned at the hip (mH) and on each leg (mL) The spacetime-constraints walker applies torques at the stance foot (A) and at the hip (B)

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Biomechanics

Experiment The masses and limb lengths of the spacetime-constraints

walker are modeled exactly as the passive-dynamic walker However the spacetime constraints algorithms permit non-zero torques about the hip and stance leg contact point and the classical homogeneous forward integration techniques are used to compute constraint violations not to compute movement explicitly

The state vector S contains the entire movement trajectory and is composed of a scalar dt and angle vector θ = [θS θN]T for every time increment t = 1hellipn By including the time increment dt as a variable the swing period is permitted to approach an optimum

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Biomechanics

Experiment The duration of a walk cycle can vary and resides under

the control of the optimization algorithm Such an extension was unnecessary for computer animators who prefer to specify when events start and end but human movement certainly capitalizes on efficiencies obtained by changing a movements pace

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime ConstraintsAndrew Witkin Michael Kass SIGGRAPH 1988

Early Computer Animations (Pixarrsquos Luxo Jr 1986)

Can we hope to generate such motions automatically

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Some aspects of animation are best left to the animator (personality appeal etc)

But perhaps we can relieve the animator from the burden of making the motion look physically plausible

Forward Simulation Solves an initial value problembull Given initial state forces along the way and

equations describing the systemhellip

bull Integrate the equations

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

But What Abouthellipbull What if the animator wants to specify

both initial and final conditionsbull Two-point boundary problemsbull And what about specifying some

conditions describing what happens during the motion

bull And what if some of the forces are unknown

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

An example We have a physical simulation that can be driven

as an initial-value problembull Given a sequence of torques that are applied by a

bicyclist we can simulate its final path We donrsquot have the inversebull Given a sequence of landmarks compute the

torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

An example Given a sequence of landmarks

compute the torques required to make a bicyclist ride through themhellip make the torques ldquooptimalrdquo

bull One could propose trajectories that go through landmarks but require unreasonable amounts of energy

bull One could propose low-energy trajectories that donrsquot go through landmarks

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Constrained optimization problem 1048707 Physical systems tend to minimize

energy 1048707 Spatial and physical constraints

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Example of Boundary Value Problem- moving particle

bull Particle position x(t)bull Force on particle f(t)

bull Equation of motion m1048707minus fminus mg = 0bull Boundary conditions x(t0) = a x(t1) = b

bull Objective function

x

dttfRt

t1

0

2)(

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Boundary Value Problembull Find f(t) such that the resulting x(t)

satisfies the boundary conditions hellipbull 1048707 hellip subject to the requirement that R is

minimalbull Possible approach solve for f(t) and x(t)

simultaneously using the equation of motion as a constraint between them

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Boundary Value Problem Discretizationbull Discretize the time interval [t0t1] with step h

bull The variables are x1 x2 hellipxi hellip xn and f1 f2 hellipfi hellip fn

bull Solve a constrained optimization problem in these unknowns

bull Enforcing the equation of motion results in n constraintspi = m(xi-1 -2xi + xi+1)h2 ndash fi - mg i 1 i n

bull Boundary constraints

ca =x1minusa = 0 cb =xn minusb= 0

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Constrained Optimizationbull Minimize an objective function

R(s1hellip s1)

in n variables s1hellip sn subject to m constraints

Ci(s1hellip sn ) = 0

Sequential Quadratic Programmingbull In order to minimize R we must zero its gradientbull At the same time we must also make sure that the

constraints are satisfiedbull Idea iterate alternating between zeroing the gradient

of R and zeroing C

Spacetime Constraints

Iterative Convergence

Spacetime Constraints

Iterative Convergence