Click here to load reader

Data-Driven Future in Visual Effects - NVIDIAon- · PDF file Data-Driven Future in Visual Effects. VFx pipelines Fluids Smoke Fire Rigid Body Hair. What’s important? Facial Animation

  • View
    8

  • Download
    0

Embed Size (px)

Text of Data-Driven Future in Visual Effects - NVIDIAon- · PDF file Data-Driven Future in Visual...

  • Rishabh Battulwar Research and Development at Digital Domain

    Data-Driven Future in Visual Effects

  • VFx pipelines

    Fluids

    FireSmoke Rigid Body

    Hair

  • What’s important?

    Facial Animation

    Natural Facial Shapes Realistic Appearance

  • Facial Motion

    Highly Non-Linear

    Organic

    Person Specific

  • Facial Animation Goal

  • Traditional Model

    Ekman and Friesen (1978)

    Facial Action Coding System (FACS)

    50-60 Primary Shapes

    Empirical Interpretations

    GENERIC Facial Shape Basis*

    Codify Facial Expressions

    Inspired by Facial Musculature

  • FACS Model

    FACS shapesAnimation Rig Setup

    50-60 Primary Shapes

    More than 1000 Combination Shapes

  • FACS Model

    50-60 Primary Shapes

    More than 1000 Combination Shapes

    Primary Shape Primary Shape Hand-CORRECTED Combo Shape

    An example of combination shape in FACS model

    [A] [B] [A+B]*

  • FACS Model

    FACS shapesAnimation Rig Setup

    50-60 Primary Shapes

    More than 1000 Combination Shapes

  • FACS Model

    FACS based process

  • Geometric Approach

    Low-Resolution Capture

  • Geometric Approach

    MASQUERADE

    Geometric Mesh Deformer Output

  • Geometric Approach

    Correspondence Mapping

    Performance Transfer using Differential Geometry

  • Geometric Approach

    MASQUERADE

    Geometric Mesh Deformer Output

    Quick Result !

  • Geometric Approach

    MASQUERADE

    Geometric Mesh Deformer Output

    NO ! Medium-scale &

    Hi-resolution detail !

    NOT ! True to the Person

    Quick Result !

  • NO ! Medium-scale &

    Hi-resolution detail !

    NOT ! True to the Person

  • NO ! Medium-scale &

    Hi-resolution detail !

    NOT ! True to the Person

  • NO ! Medium-scale &

    Hi-resolution detail !

    NOT ! True to the Person

  • Data Preparation

    Raw Data Capture Rigs

  • Data Preparation

    High-Resolution Captures

    Capturing Facial Range of Motion

    Offline Processing

    https://drive.google.com/file/d/1xliMb6yCs3lVTcP1skbBSWfU40kYjDDr/view https://drive.google.com/file/d/1xliMb6yCs3lVTcP1skbBSWfU40kYjDDr/view

  • Data Preparation

    More High-Resolution Captures

    Capturing Mesoscopic Detail

    Texture Maps

  • MASQUERADE

    Data-driven Shape Model Taking sparse facial capture to high-resolution data

    Moser et. al. ‘17 - Masquerade

    https://drive.google.com/file/d/1aWUo3lE7nFkz2rPsU4bg2VQqLQ23-gjl/view https://drive.google.com/file/d/1aWUo3lE7nFkz2rPsU4bg2VQqLQ23-gjl/view

  • Medium-scale & Hi-frequency detail ! True to the Person !

  • Medium-scale & Hi-frequency detail ! True to the Person !

  • Medium-scale & Hi-frequency detail ! True to the Person !

  • Issues in Geometric Approach

    Specific features (lip shapes)Collapsing Geometry

  • Synthetic Data-driven Corrections on CG Creatures

    Sculpted Corrections for CG Creatures

    No FACS-based BlendShape-Rig

  • Data-driven Corrections on CG Creatures

    Pre-Correction Post-CorrectionSource Hendler et. al. ‘17 - Direct Drive

  • More examples - Wrinkle Map Regression

  • Comparison

    Takes Several Months

    Shape-space modeled iteratively

    Takes 1-2 Weeks

    Shape-space built from physical data

  • Final Result

    Markered face input Large-Scale

    Geomteric Deformation using low-resolution capture

    Training Data!!

    Final High-Resolution Output Mesh

    Overview of Data-Driven Facial Animation

  • Final Result

  • Acknowledgements

    Thanks to ~

    David Corral, David Mclean, Rickey Cloudsdale, Lucio Moser, Geoff Wedig, Mark Williams, Ron Griswold, Nafees Bin Zafar, Doug Roble,

    Jeremy Buttell, Derek Crosby, Ron Miller, Darren Hendler and the entire Digital Domain Team!

    ~

Search related