Rishabh BattulwarResearch and Development at Digital Domain
Data-Driven Future in Visual Effects
VFx pipelines
Fluids
FireSmoke Rigid Body
Hair
What’s important?
Facial Animation
Natural Facial Shapes Realistic Appearance
Facial Motion
Highly Non-Linear
Organic
Person Specific
Facial Animation Goal
Traditional Model
Ekman and Friesen (1978)
Facial Action Coding System (FACS)
50-60 Primary Shapes
Empirical Interpretations
GENERIC Facial Shape Basis*
Codify Facial Expressions
Inspired by Facial Musculature
FACS Model
FACS shapesAnimation Rig Setup
50-60 Primary Shapes
More than 1000Combination Shapes
FACS Model
50-60 Primary Shapes
More than 1000Combination Shapes
Primary Shape Primary Shape Hand-CORRECTED Combo Shape
An example of combination shape in FACS model
[A] [B] [A+B]*
FACS Model
FACS shapesAnimation Rig Setup
50-60 Primary Shapes
More than 1000Combination Shapes
FACS Model
FACS based process
Geometric Approach
Low-Resolution Capture
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
Geometric Approach
Correspondence Mapping
Performance Transfer using Differential Geometry
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
Quick Result !
Geometric Approach
MASQUERADE
Geometric Mesh Deformer Output
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Quick Result !
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
NO ! Medium-scale &
Hi-resolution detail !
NOT !True to the Person
Data Preparation
Raw Data Capture Rigs
Data Preparation
High-Resolution Captures
Capturing Facial Range of Motion
Offline Processing
Data Preparation
More High-Resolution Captures
Capturing Mesoscopic Detail
Texture Maps
MASQUERADE
Data-driven Shape ModelTaking sparse facial capture to high-resolution data
Moser et. al. ‘17 - Masquerade
Medium-scale &Hi-frequency detail ! True to the Person !
Medium-scale &Hi-frequency detail ! True to the Person !
Medium-scale &Hi-frequency detail ! True to the Person !
Issues in Geometric Approach
Specific features (lip shapes)Collapsing Geometry
Synthetic Data-driven Corrections on CG Creatures
Sculpted Corrections for CG Creatures
No FACS-based BlendShape-Rig
Data-driven Corrections on CG Creatures
Pre-Correction Post-CorrectionSource Hendler et. al. ‘17 - Direct Drive
More examples - Wrinkle Map Regression
Comparison
Takes Several Months
Shape-space modeled iteratively
Takes 1-2 Weeks
Shape-space built from physical data
Final Result
Markeredface input Large-Scale
Geomteric Deformation using low-resolution capture
Training Data!!
Final High-Resolution Output Mesh
Overview of Data-Driven Facial Animation
Final Result
Acknowledgements
Thanks to~
David Corral, David Mclean, Rickey Cloudsdale, Lucio Moser, Geoff Wedig, Mark Williams, Ron Griswold, Nafees Bin Zafar, Doug Roble,
Jeremy Buttell, Derek Crosby, Ron Miller, Darren Hendlerand the entire Digital Domain Team!
~
Recommended
