• Home

  • Documents

  • 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in...
12
5/15/17 | 1 Since 1614 Scale selection in Convolutional Neural Networks using Multi - Scale Filters and Dimensional MinMax Pooling Gereon Vienken Sander Bohte Max Losch Steven Scholte

1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17 | 1

Since 1614

Scale selection in Convolutional Neural Networks using Multi-Scale Filters and Dimensional MinMax Pooling

Gereon Vienken Sander Bohte Max Losch Steven Scholte

Page 2: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Overview

›Project description›Results›Lessons learned

Page 3: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

CNN

https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/

Page 4: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Pooling

›Filters

http://vaaaaaanquish.hatenablog.com/entry/2015/01/26/060622

Page 5: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

MinMax

›Select optimal filter scale

›Smallest Filter with relevant information

›Threshold for value distribution

Page 6: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

MinMax

›Original Implementation by Bohte and Gehreab›Shows Min approximates optimum›Normalized Input›Pre-implemented absolute Gabor Filters

Page 7: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Implementation

Page 8: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Results

Page 9: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Results

Page 10: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Conclusion

›Slight overfitting of Min in the end›Min performs slightly better then Max›Threshold not as important

Page 11: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Lessons learned

›Start on Dataset with smaller images› Runtime: 3Days -> 5 Hour

›Make sure you understand restrictions of ground research

› Normalised images›If you use different machines make sure you understand the influence

› Overfitting accoutred on GPU cluster and weak GPU but not on TitanX

›Be aware: Drucker’s Law!› "If one thing goes wrong, everything else will, and at the

same time."

Page 12: 1 Scale selection in Convolutional Neural Networks using ...€¦ · Scale selection in Convolutional Neural Networks using Multi-Scale Filters and ... not on TitanX ›Be aware:

5/15/17

Questions

http://www.consciousentities.com/2014/02/the-hard-problem-problem/


Convolutional Neural Networks (CNN)

Convolutional Neural Networks (CNN)

Data & Analytics
Doubly Convolutional Neural Networks - papers.nips.ccpapers.nips.cc/paper/6340-doubly-convolutional-neural-networks.pdf · convolutional neural network (DCNN). In Figure 3 we have

Doubly Convolutional Neural Networks - papers.nips.ccpapers.nips.cc/paper/6340-doubly-convolutional-neural-networks.pdf · convolutional neural network (DCNN). In Figure 3 we have

Documents
Neural Predictive Coding using Convolutional Neural ...Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics Arindam

Neural Predictive Coding using Convolutional Neural ...Neural Predictive Coding using Convolutional Neural Networks towards Unsupervised Learning of Speaker Characteristics Arindam

Documents
Shepard Convolutional Neural Networks - Neural Information …papers.nips.cc/paper/5774-shepard-convolutional-neural... · 2016-02-16 · Shepard Convolutional Neural Networks (ShCNN)

Shepard Convolutional Neural Networks - Neural Information …papers.nips.cc/paper/5774-shepard-convolutional-neural... · 2016-02-16 · Shepard Convolutional Neural Networks (ShCNN)

Documents
Constrained Convolutional Neural Networks for …vgg/rg/slides/ccnn1.pdf · Constrained Convolutional Neural Networks for Weakly Supervised Segmentation ... [CCNN] Convolutional Neural

Constrained Convolutional Neural Networks for …vgg/rg/slides/ccnn1.pdf · Constrained Convolutional Neural Networks for Weakly Supervised Segmentation ... [CCNN] Convolutional Neural

Documents
Convolutional neural networks for direct inference of ...eprints.whiterose.ac.uk/142937/1/Convolutional Neural Networks for... · This is a repository copy of Convolutional neural

Convolutional neural networks for direct inference of ...eprints.whiterose.ac.uk/142937/1/Convolutional Neural Networks for... · This is a repository copy of Convolutional neural

Documents
Convolutional Neural Network · Convolutional Neural Networks (CNN) •A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital

Convolutional Neural Network · Convolutional Neural Networks (CNN) •A.k.a. CNN or ConvNet Adit Deshpande, A Beginner's Guide To Understanding Convolutional Neural Networks. Digital

Documents
Scale-Free Convolutional Neural Network for Remote Sensing

Scale-Free Convolutional Neural Network for Remote Sensing

Documents
Convolutional neural networks deepa

Convolutional neural networks deepa

Technology
Exploring the Design Space of Deep Convolutional Neural Networks at Large · PDF file · 2016-12-21Deep Convolutional Neural Networks at Large Scale by ... Exploring the Design Space

Exploring the Design Space of Deep Convolutional Neural Networks at Large · PDF file · 2016-12-21Deep Convolutional Neural Networks at Large Scale by ... Exploring the Design Space

Documents
Convolutional Neural Networks for Large-Scale Bird Song …ceur-ws.org/Vol-1609/16090560.pdf · 2016-07-15 · Convolutional Neural Networks for Large-Scale Bird Song Classification

Convolutional Neural Networks for Large-Scale Bird Song …ceur-ws.org/Vol-1609/16090560.pdf · 2016-07-15 · Convolutional Neural Networks for Large-Scale Bird Song Classification

Documents
Convolutional Neural Networks for No-Reference Image Quality Assessmentopenaccess.thecvf.com/...Convolutional_Neural_Networks_2014_CVPR_paper.pdf · Convolutional Neural Networks

Convolutional Neural Networks for No-Reference Image Quality Assessmentopenaccess.thecvf.com/...Convolutional_Neural_Networks_2014_CVPR_paper.pdf · Convolutional Neural Networks

Documents
Skin Lesions Classification Using Convolutional Neural ... · Skin Lesions Classification Using Convolutional Neural Networks in Clinical Images Danilo Barros Mendes * ... scale,

Skin Lesions Classification Using Convolutional Neural ... · Skin Lesions Classification Using Convolutional Neural Networks in Clinical Images Danilo Barros Mendes * ... scale,

Documents
Graph Convolutional Neural Networks for Web-Scale … · 2018. 6. 7. · Graph Convolutional Neural Networks for Web-Scale Recommender Systems Rex Ying∗†, Ruining He∗, Kaifeng

Graph Convolutional Neural Networks for Web-Scale … · 2018. 6. 7. · Graph Convolutional Neural Networks for Web-Scale Recommender Systems Rex Ying∗†, Ruining He∗, Kaifeng

Documents
Convolutional Neural Networks (Part II)lvelho.impa.br/ip17/proj/slides/1110GoingDeeperConv.pdf · Convolutional Neural Networks 1 Convolutional Neural Networks (Part II) 08, 10 &

Convolutional Neural Networks (Part II)lvelho.impa.br/ip17/proj/slides/1110GoingDeeperConv.pdf · Convolutional Neural Networks 1 Convolutional Neural Networks (Part II) 08, 10 &

Documents
Application of Convolutional Neural Networks to Reconstruct GeV … · 2020. 6. 12. · Application of Convolutional Neural Networks to Reconstruct GeV-Scale IceCube Neutrino Events

Application of Convolutional Neural Networks to Reconstruct GeV … · 2020. 6. 12. · Application of Convolutional Neural Networks to Reconstruct GeV-Scale IceCube Neutrino Events

Documents
Visualizing and Understanding Convolutional Neural ... - arXiv and Understanding Convolutional Neural ... - arXiv

Visualizing and Understanding Convolutional Neural ... - arXiv and Understanding Convolutional Neural ... - arXiv

Documents
ImageNet Classification with Deep Convolutional Neural ... · Convolutional neural networks Output Hidden Data Here's a one-dimensional convolutional neural network Each hidden neuron

ImageNet Classification with Deep Convolutional Neural ... · Convolutional neural networks Output Hidden Data Here's a one-dimensional convolutional neural network Each hidden neuron

Documents
Convolutional Neural Networks at scale in Spark MLlib

Convolutional Neural Networks at scale in Spark MLlib

Technology
Using Support Vector Machines, Convolutional Neural ... · Using Support Vector Machines, Convolutional Neural Networks and ... Using Support Vector Machines, Convolutional Neural

Using Support Vector Machines, Convolutional Neural ... · Using Support Vector Machines, Convolutional Neural Networks and ... Using Support Vector Machines, Convolutional Neural

Documents
Convolutional Neural Networks Arise From Ising Models …sunilpai/convolutional-neural-networks.pdf · Convolutional Neural Networks Arise From Ising Models and Restricted Boltzmann

Convolutional Neural Networks Arise From Ising Models …sunilpai/convolutional-neural-networks.pdf · Convolutional Neural Networks Arise From Ising Models and Restricted Boltzmann

Documents
O-CNN: Octree-based Convolutional Neural Networks for … · O-CNN: Octree-based Convolutional Neural Networks for 3D ... object classification, ... Octree-based Convolutional Neural

O-CNN: Octree-based Convolutional Neural Networks for … · O-CNN: Octree-based Convolutional Neural Networks for 3D ... object classification, ... Octree-based Convolutional Neural

Documents
Convolutional Neural Network

Convolutional Neural Network

Documents
Convolutional Neural Networks Analyzed via Convolutional ...yromano/posters/ml_csc_poster.pdf · Convolutional Neural Networks ... Recently, convolutional sparse coding (CSC)

Convolutional Neural Networks Analyzed via Convolutional ...yromano/posters/ml_csc_poster.pdf · Convolutional Neural Networks ... Recently, convolutional sparse coding (CSC)

Documents
Convolutional Neural Networks for Deep Learning: An Introduction · 2017-10-05 · Introduction: Convolutional Neural Networks (CNN) Convolutional Neural Networks: A deep learning

Convolutional Neural Networks for Deep Learning: An Introduction · 2017-10-05 · Introduction: Convolutional Neural Networks (CNN) Convolutional Neural Networks: A deep learning

Documents
Convolutional Neural Fabrics - arXiv

Convolutional Neural Fabrics - arXiv

Documents
Lecture 10: Convolutional Neural Networks · CS231n: Convolutional Neural Networks K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large -Scale Image Recognition,

Lecture 10: Convolutional Neural Networks · CS231n: Convolutional Neural Networks K. Simonyan and A. Zisserman, Very Deep Convolutional Networks for Large -Scale Image Recognition,

Documents
Neural Network Part 3: Convolutional Neural Networkspages.cs.wisc.edu/.../lecture12-neural-networks-3.pdf · Convolutional neural networks •Strong empirical application performance

Neural Network Part 3: Convolutional Neural Networkspages.cs.wisc.edu/.../lecture12-neural-networks-3.pdf · Convolutional neural networks •Strong empirical application performance

Documents
Hyperbolic Graph Convolutional Neural Networkspapers.nips.cc › paper › 8733-hyperbolic-graph-convolutional-neural-networks.pdfGraph Convolutional Neural Networks (GCNs) are state-of-the-art

Hyperbolic Graph Convolutional Neural Networkspapers.nips.cc › paper › 8733-hyperbolic-graph-convolutional-neural-networks.pdfGraph Convolutional Neural Networks (GCNs) are state-of-the-art

Documents
CONVOLUTIONAL NEURAL NETWORKS: MOTIVATION, …

CONVOLUTIONAL NEURAL NETWORKS: MOTIVATION, …

Documents