Embed Size (px)
Citation preview
Greg Rehm, Shahbaz Rezaei
Look, Listen and LearnRelja Arandjelović, Andrew Zisserman
February 20th, 2018
●●
●●
Misra, “Shuffle and Learn: Unsupervised Learning Using Temporal Order Verification.” 2016. Vol. 9905
Doersch “Unsupervised Visual Representation Learning by Context Prediction.” (2015):
● >4 young males in row
● Silly hair
● Dumb look on face
.01 sec. window
FFT
Half window overlap
FFT
●●●●
●●
●○○○
●○
■■■
○■■■
Train the entire network
FC:512*128 FC:128*34
Class Label
Class Label
Scalar product
Freeze weights
Train FCs
1 sec
1 sec......
Max pool
One-vs-Al
lSVM4*3*512=6144
Freeze weights
Train SVM
Pretrained on Flickr
mean over the class scores
Max pool
One-vs-Al
lSVM
4*3*512=6144
Max pool
One-vs-Al
lSVM
4*3*512=6144
... ...
1 sec
Max pool
One-vs-Al
lSVM4*3*512=6144
Randomly initialize weights
Train SVM
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
Max pool FC
4*4*512=8192
Freeze weights
Train FC/Softmax
Soft max
1000
Pretrained on Flickr
Max pool FC
4*4*512=8192
Randomly initialize weights
Train FC/Softmax
Soft max
1000
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
Frequency
Time
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
[Arandjelovic, Relja, and Andrew Zisserman. "Look, listen and learn." ICCV 2017]
●○○
●○○
○○○○
