20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner...

Visual Computing – Data Analysis – Consulting rene@radiology-explorer.com

René Donner

Deep Learning

René Donner Deep Learning

Overview

The (amazing) things Deep Learning can do!

How does it work?

How can you start with DL?!

Roughly …

Deep learning finds patterns in data corresponding to high-level, abstract concepts

!… just like we do, be we are better …

!… currently. In most tasks, not all, any more.

What it can be used for

Image recognition

Text understanding, translation

Voice recognition

Playing video games

Driving cars

Image recognition

Google Research Blog

Image recognition

Google Image Search

Scene labeling

http://www.purdue.edu/newsroom/releases/2014/Q1/smartphone-to-become-smarter-with-deep-learning-innovation.html

Text recognition

http://www.pyimagesearch.com/2014/09/22/!getting-started-deep-learning-python/

Large-Scale Deep Learning for Intelligent Computer Systems, !Jeff Dean, Google, BayLearn 2015

Text understanding

2013 Glove: Global Vectors for Word Representation, Jeffrey Pennington, Richard Socher and Christopher D. Manning

Word embeddings

Information extraction / Reasoning

MetaMind

Information extraction / Reasoning

MetaMind

It is not perfect

http://www.news.cornell.edu/stories/2015/03/images-fool-computer-vision-raise-security-concernsderStandard.at

Some well know research groups

Stanford / BaiduAndrew Ng

NYU / FacebookYann LeCun

UToronto / GoogleGeoffrey Hinton

NVIDIA

brand new: M40(same as Geforce GTX Titan X)

Images: NVIDIA website

How does it work?

Difference to classic ML

18http://rinuboney.github.io/2015/10/18/theoretical-motivations-deep-learning.html

Deep learning

http://theanalyticsstore.ie/deep-learning/

Visualization

1. Layer

higher Layers

Emergence of Object-Selective Features in Unsupervised Feature Learning, Adam Coates, NIPS 2012

Deep learning

How does it work?

http://theanalyticsstore.ie/deep-learning/ http://stats.stackexchange.com/questions/114385/!what-is-the-difference-between-convolutional-neural-networks-restricted-boltzma

Optimization

Stochastic gradient descent

Automatic differentiation

blog.datumbox.com

Local minima

Less problematic than thought - saddle points

https://ganguli-gang.stanford.edu/figures/14.Saddlepoint.jpg

Deep learning

Low level features of color images

https://www.coursera.org/course/neuralnets

Deep learning

25http://www.pamitc.org/cvpr15/files/lecun-20150610-cvpr-keynote.pdf

ImageNet topologies

ImageNet Classification with Deep Convolutional Neural Networks", Alex Krizhevsky

“Inception” deep neural network architecture. Source: Christian Szegedy et. al. Going deeper with convolutions. CVPR 2015

MNIST - Demo

http://deeplearning4j.org/rbm-mnist-tutorial.html

Deep learning - why does it work?

Can cope with huge amounts of data

Learns small invariances

Overcomplete, sparse, representations

Learn Embedding

Lots of data

Recent advance: it is actually computable!

Deep learning - pros

Not-domain specific

Supervised / Semi-supervised / Unsupervised

Classification / regression in last layer

Simple math

Deep learning - cons

Lots of meta-parameters

Needs a lot of data

Very compute intensive

Getting started with DL

Frameworks

Many different DL toolboxes

Efficiency important (GPU)

Attention to numerical issues

Frameworks

Caffehttp://caffe.berkeleyvision.org/Plain text filesFastest CNN, GPU

!Keras

https://github.com/fchollet/kerasPython, on top of Theano

!TensorFlow

http://tensorflow.org/Python, by Google

!MXNet

https://github.com/dmlc/mxnetPython, R, Julia

Tensorflow

General gradient descent library

Tutorials

Stanford tutorial:https://deeplearning.stanford.edu/wiki/index.php/UFLDL_TutorialMatlab code snippets

videolectures.nethttp://videolectures.net/deeplearning2015_montreal/

courserahttps://www.coursera.org/course/neuralnets

Practical hints

Bengio ArxivPractical Recommendations for Gradient-Based Training of Deep Architectures, http://arxiv.org/abs/1206.5533http://rinuboney.github.io/2015/10/18/theoretical-motivations-deep-learning.html!

Kaggle http://www.kaggle.com/c/galaxy-zoo-the-galaxy-challengehttp://benanne.github.io/2014/04/05/galaxy-zoo.html

Relevant conferences NIPS (https://sites.google.com/site/deeplearningworkshopnips2013/accepted-papers)CVPR, ICMLMany interesting papers on arxiv.org

Current research topics

Parallelization

What is deep learning, actually?

Alternative, faster, simpler methods

Multi-domain, transfer learning

Visual Computing – Data Analysis – Consulting rene@radiology-explorer.com

René Donner

Deep Learning

20151216 Computer Vision Meetup Deep Learning Deep Learning - Rene...Deep Learning . René Donner...

Documents

TygerBurger Kuilsrivier 20151216

20151216 - Booosting - Carlijn vd Werf Bout

Deep Learning for Depth Learning CS 229 Course Project ...cs229.stanford.edu/...DepthLearning_FinalReport.pdf · Deep Learning for Depth Learning CS 229 Course Project, ... Deep Learning

fast.ai - Learning Deep Learning

Deep Learning for Laser Based Odometry Estimation - …juxi.net/workshop/deep-learning-rss-2016/papers/Nicolai - Deep... · Deep Learning for Laser Based Odometry Estimation ... deep

Executing Deep Learning Strategies Masterclass Preview - Enterprise Deep Learning

Deep Learning for Reinforcement Learning in Pacman · Deep Learning for Reinforcement Learning in Pacman Deep Learning für Reinforcement Learning in Pacman Vorgelegte Bachelor-Thesis

Tutorial: Deep Reinforcement Learning - Machine Learning ...hunch.net/~beygel/deep_rl_tutorial.pdfTutorial: Deep Reinforcement Learning - Machine Learning

Introduction to Deep Learning - Joyce Hojoyceho.github.io/cs534_s17/slide/25-deep-learning.pdf · Deep Learning: Overview ... Review: Backpropogation https: ... • Deep Learning

TygerBurger Eersterivier/Blue Downs 20151216

From Reinforcement Learning to Deep Reinforcement …fagostin/assets/files/...Keywords: Machine learning · Reinforcement learning Deep learning · Deep reinforcement learning 1 Introduction

Essf 20151216 essf full

Introducing Deep Learning with MATLAB - MathWorks · Introducing Deep Learning with MATLAB7 How A Deep Neural Network ... relevant features of an image. With deep learning, ... Deep

TygerBurger Goodwood 20151216

Zuger presse 20151216

Deep Learning Lab Course 2017 (Deep Learning …ais.informatik.uni-freiburg.de/teaching/ws17/deep...Deep Learning Lab Course 2017 (Deep Learning Practical) Labs: (Computer Vision)

An Introduction to Deep Learning · Deep Learning An Introduction to ... •Deep Learning •Feed Forward Network •Problems of Deep Learning •AutoEncoders •Restricted Boltzmann

Deploying Enterprise Deep Learning Masterclass Preview - Enterprise Deep Learning

Deep Learning - South Dakota State UniversityDeep Learning Deep learning (also known as deep structured learning, hierarchical learning or deep machine learning) is a branch of machine

Tutorial: Deep Reinforcement Learning · Outline Introduction to Deep Learning Introduction to Reinforcement Learning Value-Based Deep RL Policy-Based Deep RL Model-Based Deep RL