Embed Size (px)
Citation preview
Quoc V. Le
Deep Learning: Overview and Latest Results
joint work with many Google & Stanford collaborators
Coates et al 2009
Quoc V. Le
Feature Engineering
Handcrafted Features
Classifier
X X’
“Face”Y
Quoc V. Le
Why Deep Learning?
x
Corner Detectors
…
Edge Detectors
Raw Image
Cat, Face Detectors
Quoc V. Le
Model Partitioning
Machine 4
Machine 1 Machine 2
Machine 3
Quoc V. Le
GoogleBrain infrastructure
ParameterServer
ModelWorkers
DataShards
p’ = p + ∆p
∆p ∆p ∆p
Le et al, 2011, Dean et al 2011
Quoc V. Le
Why scaling up?
Amount of data
Accuracy
Deep Learning
Traditional Machine Learning
Quoc V. Le
GoogleBrain teaches itself
Watch YouTube for 3 days (10M frames)
Run on 16,000 machines1 billion synapses
Le et al 2011
Quoc V. Le
GoogleBrain teaches itself
Watch YouTube for 3 days (10M frames)
Run on 16,000 machines1 billion synapses
“Cat” Le et al 2011
Quoc V. Le
Quoc V. Le
Applications
- Understanding Images
- Understanding Speech
- Understanding Texts
Quoc V. Le
Image Recognition
28% vs. 5%Error rate:
Achieves superhuman performance
Now used in Google+ Photosearch and StreetView
LeCun et al 1998, Krizhevsky et al 2012
Quoc V. Le
Speech Recognition
Voice Search
23% vs. 8%Error rate:
Hinton et al, 2011
Quoc V. Le
Applications
- Understanding Images
- Understanding Speech
- Understanding Texts
Quoc V. Le
Mapping sequence to sequence
A
W X Y Z <EOS>
B C <EOS> W X Y
W
Z
WX XWY YXWZ YXW<EOS>
Quoc V. Le
Sample ConversationsDescribe your problem: i am having issues accessingvpn
Machine: hiHuman: helloMachine: could you please let me know what are the operating systems you are using ?Human: i am using linuxMachine: ok it was in the ssl vpn on which you stated it was brokenMachine: please check if you are logged in to corp networkMachine: what is the error that you are running pleaseHuman: i am seeing an error related to vpnMachine: what is the error message that you are getting when connecting to vpn using network connect ?Human: connection refused or something like thatMachine: may i know the version of network connect you connect ?
Quoc V. Le
Sample ConversationsHuman: i am not sure i know thatMachine: is the network connect prompting for any user certificateMachine: ?Human: yesMachine: are you at home or at work at the moment ?Machine: office ?Human: i am at homeMachine: try thisMachine: goto <URL> page and enter your name and passwordHuman: doneMachine: try logging in with <NAME> and check if its working fine nowHuman: yes , now it works !Machine: greatMachine: anything else that i can helpHuman: no , that was very usefulMachine: you are other than <NAME> : )Machine: have a nice night
Quoc V. Le
Sample Conversations
Human: what is the purpose of life ?Machine: to serve the greater good .Human: what is the purpose of living ?Machine: to live forever .Human: what is the purpose of existence ?Machine: to find out what happens when we get to the planet earth .Human: where are you now ?Machine: i ’m in the middle of nowhere .Human: what is the purpose of dying ?Machine: to have a life .Human: what is the purpose of being intelligent ?Machine: to find out what it is .Human: what is the purpose of emotions ?Machine: i don ’t know .
Quoc V. Le
Applications
- Understanding Images
- Understanding Speech
- Understanding TextsImage to Texts
Quoc V. Le
Generating captions for images
A person riding a motorcycle on a dirt road
A group of young people playing a game of frisbee
Two dogs play in the grass
Vinyals, et al, 2015
Quoc V. Le
Deep Learning tutorials & software
Deep Learning tutorials are available on my website:
Part 1 (Basic): http://ai.stanford.edu/~quocle/tutorial1.pdf
Part 2 (Advanced): http://ai.stanford.edu/~quocle/tutorial2.pdf
Popular open source software for Deep Learning: CAFFE https://github.com/BLVC/caffe
Quoc V. Le
Thanks
Jeff Dean Oriol VinyalsIlya SutskeverGreg Corrado
Matthieu DevinAndrew Ng Rajat MongaNavdeep Jaitly
+GoogleBrain team, Tomas Mikolov, Thang Luong, William Chan, Wojciech Zaremba
