Proxemics Recognition Yi Yang 1, Simon Baker, Anitha Kannan,
Deva Ramanan 1 1 Department of Computer Science, UC Irvine
Slide 2
Proxemics Proxemics: the study of spatial arrangement of people
as they interact - anthropologist Edward T. Hall in 1963Edward T.
Hall
Slide 3
Slide 4
Slide 5
Slide 6
Slide 7
Slide 8
Brother and Sister Holding Hands Friends Walking Side by Side
Husband Hugging and Holding Wifes Hand Mom Holding Baby
Slide 9
Touch Code Hand Touch Hand Hand Touch Shoulder Shoulder Touch
Shoulder Arm Touch Torso
Slide 10
Applications Personal Photo Search: Find a specific interesting
photo Analysis of TV shows and movies Kinect Web Search
Auto-Movie/Auto-Slideshow Locate interesting scenes
Slide 11
Proxemics DataSet 200 training images 150 testing images
Collected From Simon, Bing, Google, Gettyimage, Flickr No video
data No Kinect 3d depth data
Slide 12
Proxemics DataSet Number of People Complexity
Slide 13
A Nave Approach Input Image
Slide 14
A Nave Approach Input ImageImage Feature
Slide 15
A Nave Approach Input Image Pose Estimation i.e. Find skeletons
Image Feature
Slide 16
A Nave Approach Input Image Interaction Recognition Hand touch
Hand Pose Estimation i.e. Find skeletons Image Feature
Slide 17
Nave Approach Results Hand touch Hand Hand touch Shoulder
Slide 18
Human Pose Estimation Not bad when no real interaction between
people - Y. Yang & D. Ramanan, CVPR 2011
Slide 19
Interactions Hurt Pose Estimation Occlusion + Ambiguous
Parts
Slide 20
Our Approach Direct Proxemics Recognition Input ImageImage
Feature
Slide 21
Our Approach Direct Proxemics Recognition Input Image
Interaction Recognition Image Feature Hand touch Hand
Slide 22
Pictorial Structure Model - P. Felzenszwalb etc., PAMI
2009
Slide 23
Pictorial Structure Model
Slide 24
Slide 25
Two Head Monster Model i.e. Hand-Touching-Hand
Slide 26
Two Head Monster Model i.e. Hand-Touching-Hand
Slide 27
The models Hand touch HandHand touch Shoulder Shoulder touch
ShoulderArm touch Torso
Slide 28
Match Model to Image
Slide 29
Refinements + Extensions Sub-categories Because of symmetry, 4
models for hand-hand etc R. Hand L. Hand L. Hand L. Hand R. Hand R.
Hand
Slide 30
Refinements + Extensions Sub-categories Because of symmetry, 4
models for hand-hand etc Co-occurrence of proxemics: Reduce
redundancy, map Multi-Label -> Multi-Class R. Hand L. Hand L.
Hand L. Hand R. Hand R. Hand
Slide 31
Nave Approach Results Hand touch Hand Hand touch Shoulder
Slide 32
Direct Approach Results Hand touch Hand Hand touch
Shoulder
Slide 33
Quantitative Results [1] Y. Yang & D. Ramanan, CVPR
2011
Slide 34
Improves Pose Estimation Y & D CVPR 2011 Our Model
Slide 35
Improves Pose Estimation Y & D CVPR 2011 Our Model
Slide 36
Improves Pose Estimation Y & D CVPR 2011 Our Model
Slide 37
Conclusion Proxemics and touch codes for human interaction
Slide 38
Conclusion Proxemics and touch codes for human interaction
Directly recognizing proxemics significantly outperforms
Slide 39
Conclusion Proxemics and touch codes for human interaction
Directly recognizing proxemics significantly outperforms
Recognizing proxemics helps pose estimation
Slide 40
Acknowledgements Thank Simon and MSR for internship
Slide 41
Acknowledgements Thank Anitha for a lot of suggestions
Slide 42
Acknowledgements Thank Anarb for gettyimages
Slide 43
Acknowledgements Thank Eletee for her beautiful smiling
Slide 44
Acknowledgements Thank everybody for not falling asleep
Slide 45
Thank you
Slide 46
Articulated Pose Estimation - Yi Yang & Deva Ramanan, CVPR
2011