Upload
alberta-davidson
View
219
Download
1
Tags:
Embed Size (px)
Citation preview
Modeling the Shape of People from 3D Range Scans
Dragomir AnguelovAI Lab
Stanford University
Joint work with
Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun, James Davis
The Dataset
Cyberware Scans 4 views, ~125k polygons ~65k points each
Problems Missing surface Drastic pose changes
70 scans
48 scans
Modeling Human ShapePose
vari
ati
on
Body-shape variation
Space of Human Shapes Movie
[scape movie]
Talk outline Data processing pipeline
Registration Recovering an articulated skeleton
Modeling the space of human deformations Pose deformations Body shape deformations
Shape completion Partial view completion Animating motion capture sequences
Talk outline Data processing pipeline
Registration Recovering an articulated skeleton
Modeling the space of human deformations Pose deformations Body shape deformations
Shape completion Partial view completion Animating motion capture sequences
Data Processing Pipeline
Registration - CC Algorithm [Anguelov et al. 2004]
Input:Pair of scans
Output: Correspondences
Correlated Correspondence Algorithm
ZX
ZX
Computes an embedding of mesh Z into mesh X
1. Defines a discrete Markov Net M over correspondence variables of mesh Z
Markov Net potentials enforce• Minimal surface deformation• Similar local surface appearance• Preservation of geodesic distance
2. Embedding of Z into X is computed by performing Loopy Belief Propagation on M
Related work: [Huttenlocher et al 00] [Coughlan 02]
Results: Human poses datasetModel
• 4 markers were used on each scan to avoid the need for multiple initializations of Loopy-BP
Cyberware scans
Registrations
Recovering articulation
Input: models, correspondences
Output: rigid parts, skeleton
Recovering articulation [Anguelov et al’04]
Stages of the process1. Register meshes using Correlated Correspondences
algorithm2. Initialize
1. break template surface into N arbitrary components
3. Cluster surface into rigid parts4. Estimate joints
Related work: [Cheung et al’03]
1 2 3 4
Probabilistic Generative Model
Transformations
T
TransformedModel
y1 yN…
Modela1
x1
aN
xN
…
…
Part labels
Points
zK
bK
…z1
b1Instanc
e
Point corrs
Points
Contiguity Prior Parts are preferably contiguous regions
Adjacent points on the surface should have similar labels
Enforce this with a Markov network:
a1 a2
a3
Clustering algorithm
Algorithm Given transformations , perform min-cut* inference
to get
Given labels , solve for rigid transformations
Objective
Contiguity prior
Data Likelihood
* [Greig et al. 89], [Kolmogorov & Zabih 02]
Results: Puppet articulation
Results: Arm articulation
Results: 50 scans of a human
Tree-shaped skeletonfound
Rigid parts found
Talk outline Data processing pipeline
Registration Recovering an articulated skeleton
Modeling the space of human deformations Pose deformations Body shape deformations
Shape completion Partial view completion Animating motion capture sequences
Modeling Pose Deformationinput
Deformations
output
Regression function
Joint angles
Modeling Pose Deformation
template
Predict independently for each triangle
Reconstructcomplete shape
Related work: [Allen et al ‘03][Sumner+Popovic 2004]
Representation of pose deformation
Pose deformation
Rigid articulateddeformation
Given estimates of R, Q, synthesizing the shape is straightforward :
Learning pose deformation
For each polygon, predict entries of from rotations of nearest 2 joints (represented as twists ).
Linear regression parameters :
Obtaining values of in the first place:
Twists and exponential maps
Twist
From twist to rotation matrix
Joint angles
Pose deformation space
Learning body-shape deformation Include also change in shape due to different
people:
Do PCA over body-shape matrices :
Getting estimates of :
PCA over body shape
Combining pose and body shape spaces
Talk outline Data processing pipeline
Registration Recovering an articulated skeleton
Modeling the space of human deformations Pose deformations Body shape deformations
Shape completion Partial view completion Animating motion capture sequences
Shape completion Find surface Y from our space which matches a set
of markers Z
Y[Z] : completed mesh deforms out of space spanned by , R to match Z
Y’[Z]: predicted mesh constrained to be in space spanned by , R
Target optimized by iteratively solving for , R orY while holding the others fixed
Partial view completion
Process: Add a few markers (~6-8) Run CC algorithm to get >
100 markers Optimize to find Y[Z]
Shape completion from motion capture data
Conclusions Presented a data-driven method of modeling human
deformations induced by Pose Body shape
Extending the model Nonlinear prediction of pose deformation (ongoing)
Shape complete original scans using current model Acquire and learn from the entire pose-bodyshape matrix Prior on likely joint angles, e.g. [Popovic + et al ’04] Enforce temporal consistency in tracking applications
Extending the possible applications Markerless motion capture (shape completion in shape-from-
silhouette data) Modeling other beasts