Embed Size (px)
Citation preview
TensorflowMultithreading to construct neural networks
Maxence QueyrelJune 15, 2016
Quinten / UPMC
Table of contents
1. Introduction : Neural networks hyperparameters
2. Tensorflow with Spark
3. Distributed Tensorflow
4. Conclusion
1
Introduction : Neural networkshyperparameters
Introduction : Neural networks hyperparameters
• Neural networks are complex models : They have a lot ofhyperparameters
• It is hard to define a neural network structure/graph• Need to test several structures : execution time could be veryimportant
2
Introduction : Example of hyperparameters
- Number of layers- Number of neurons for each layer- Activation function- Learning rate- Number of iterations- dropout probability
- Number of filters- Filter’s dimensionality- Convolution step
Only for Convolutional neural networks
3
Tensorflow with Spark
Tensorflow with Spark : The tools you need ?
TensorFlow
Spark
A cluster
4
Tensorflow with Spark : How does it work ?
5
Tensorflow with Spark : How does it work ?
6
Tensorflow with Spark : How does it work ?
7
Tensorflow with Spark : How does it work ?
8
Tensorflow with Spark : How does it work ?
9
Tensorflow with Spark : How does it work ?
10
Tensorflow with Spark : Example
11
Tensorflow with Spark : efficiency
Input : Iris DataSet from scikit learn.This benchmark has been made on 45 NN for each execution.
12
Distributed Tensorflow
Distributed Tensorflow : What is it ?
This is a new option in TensorFlow 0.8.0, it allows to :
• Run a TensorFlow graph on a cluster• Split the graph in several jobs• Jobs can contain several tasks
13
Distributed Tensorflow : How does it work ?
14
Distributed Tensorflow : How does it work ?
15
Distributed Tensorflow : How does it work ?
16
Distributed Tensorflow : How does it work ?
17
Distributed Tensorflow : How does it work ?
18
Distributed Tensorflow : How does it work ?
19
Distributed Tensorflow : How to use it ?
• We have to launch our program by command line with thecorrect arguments
• To get the arguments, we need to define some TensorFlow Flagsin the code
20
Distributed Tensorflow : Example
Python code
Shell code
21
Conclusion
Conclusion
• Importance of including distributed programming• Spark : It exists other libraries running on spark like Theano orCaffe
• Distributed TensorFlow : Development of libraries to replicatemodels
22
Thank you for the intention
22
Questions ?
22
