16
Temporal Segmentation of Egocentric Videos VIVEK PRADHAN

Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Temporal Segmentation of Egocentric VideosVIVEK PRADHAN

Page 2: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Problem Statement

Page 3: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

DataSet

Page 4: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

DataSetShow External Video.

Page 5: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Problem Statement

Page 6: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Problems with Short Term motion detectors

Page 7: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Problems with Short Term motion detectors

Page 8: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Cumulative displacement curvesDivide Image into 10x5 cells and compute displacement of cells.

Make it cumulative over time to cancel out head motions.

Page 9: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Properties of CD curves

Page 10: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Properties of CD curves

Page 11: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Properties of CD curves

Page 12: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

SVM features from CD curves.Radial Projection Response

Motion Clusters

FOE estimation

Page 13: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Gaze Fixation through CD curvesCauses head motion to stop.

This can be exploited to find the gaze fixation.

Page 14: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Experiments and Results

Page 15: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

Experiments and Results

Page 16: Temporal Segmentation of Egocentric Videosvision.cs.utexas.edu/381V-fall2016/slides/pradhan_paper.pdf · 2016-11-29 · Class 1 853K 58K 453K 999K 1053K 126K 94% 73% 72% 98% 86% 96%

CD curves for other classes and tasksMight have very different results for chest mounted cameras.

Authors have performed the same experiment with 3D CNNs.

Activity recognition.