Towards Human-Realistic Animation by Observing Real Human Dynamics Darren Cosker Towards...

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human Dynamics

Towards Human-Realistic Animation by Observing Real Human Dynamics

Darren CoskerDepartment of Computer Science,University of Bath

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

MotivationMotivation

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

OverviewOverview

• Analysis and Re-Synthesis of Facial Dynamics– Dynamics > CG Puppet– Dynamics > us– Dynamics > Computer Vision

• Intuitive/Automatic methods for animation– Voice Driven Animation/Analysis– Reactive Avatars

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Time

Par

amet

er M

agni

tude

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing/Resynthesising DynamicsAnalysing/Resynthesising Dynamics

• Study effect of manipulating temporal characteristics of emotions, e.g. smiles

QuickTime™ and a decompressor

are needed to see this picture.

[Emotion 2007]

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing/ResynthesisingDynamicsAnalysing/ResynthesisingDynamics

• What did we learn?– ‘Genuine’ Dynamics = More

likeable/Trustworthy/Employable– ‘Fake’ Dynamics = Less

Likeable/Trustworthy/Employable

• Relationship between Attractiveness and Facial Dynamics

• To Do! - Mock Video Game Scenario

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

1. Define facial regions local

to sub-facial actions

2. Construct new training

set of artificial images

Analysing Multiple ActionsAnalysing Multiple Actions

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Analysing Multiple ActionsAnalysing Multiple Actions

3. Create a corresponding artificial training set of shape vectors

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Performance AnalysisPerformance Analysis

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Image Based TransferImage Based Transfer

QuickTime™ and aH.264 decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Performance AnalysisPerformance Analysis

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

3D Model Transfer3D Model Transfer

• Animation of a 3D facial model

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

3D Model Transfer3D Model Transfer

QuickTime™ and aMPEG-4 Video decompressor

are needed to see this picture.

Dynamic knowledge easily applied to existing pipelines

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

3D Facial Modelling3D Facial Modelling

• 4D Camera– Continuous 3D Scanning (Cardiff/Surrey)

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

3D Facial Modelling3D Facial Modelling

(Link to 3D Example Expression)

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

3D Facial Modelling3D Facial Modelling

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Voice Driven AnimationVoice Driven AnimationVisual Speech Synthesis

‘Non-Verbal’ Action Synthesis

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

QuickTime™ and aCinepak decompressor

are needed to see this picture.

QuickTime™ and aNone decompressor

are needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Perceptual EvaluationPerceptual Evaluation

• Motivation: How best to evaluate facial animation?– Perceptual test based on McGurk effect

QuickTime™ and aSorenson Video decompressorare needed to see this picture.

[ACM TAP 2005]

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Perceptual EvaluationPerceptual Evaluation

• Record participant responses

• Compare real and synthetic

• Subtle - participants blind to real motivation

• Isolate strengths and weaknesses

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Reactive Full Body Avatar AnimationReactive Full Body Avatar Animation

• Motivation:– Animate based on

an input performance

– Key avatar drives other avatar behaviour

QuickTime™ and aSorenson Video 3 decompressorare needed to see this picture.

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Conclusions and Future DirectionsConclusions and Future Directions• Humans are very sensitive to different dynamics

• Different interpretation depending on timing

• Human realism will depend on understanding dynamic behaviours

• Realism will also depend on the accurate modelling and resynthesis of dynamics - e.g. McGurk effect

• Technology now allows study of 3D dynamics - our work has only just started!

Towards Human-Realistic Animation by Observing Real Human DynamicsDarren Cosker

QuickTime™ and aTIFF (Uncompressed) decompressor

are needed to see this picture.

Thanks for listening!

Acknowledgements:• Eva Krumhuber, Antony Manstead, Lanthao Benedikt,

David Marshall, Paul Rosin, Alistair Calvert, Cathy Holt, Daniel Mason, Gemma Whatling, Simon Rushton, Susan Paddock

• RAEng, EPSRC.