An Introduction Deep Learning with TensorFlow

• How Am I?

• Machine Learning & Deep Networks

• What is ML ?

• Neural Network Example

• What is Deep Learning?

• TensorFlow (v1.0)

• What is TF?

• TF Architecture

• TF evolution

• TF language features through an example

• The Computational Graph

• TensorBoard

• TF in Google Cloud ML

• TF High Level API

• TF Serving

• TensorFlow Hands-On

• TensorFlow Examples

• Linear Model

• Shallow MNIST

• Deep MNIST

• ConvNet MNIST

Agenda

1

How Am I ?

• Ndjido Ardo BAR : Data Scientist @ Davidson Consulting UAE

• Background : Research In Mathematics

• Now: Working in @DU Telecom as a Consultant

• Past:

• Worked @AXA (Paris): Develop A Credit Scoring Model with H2O

• Worked @BearingPoint Hypercube: Did consulting for different companies:

• Hypercube (Paris): A web-based Machine Learning Platform

• Jonson & Jonson (Florida): Lens Design Process Optimisation (Vistakon)

• Keolis: Setup a Big Data Platform and Developed a Fraud Detection Model

• And many more …

• Co-Founder of a StartUp (MLouma)

• Worked @Pasteur Institute: Involved in BioStatistical Research

@ndjido2

Machine Learning&

Deep Networks

3

What Machine Learning?

- Machine Learning is a field of Computer Science that aims at giving computers the ability to learn complex patterns without being explicitly programmed.

- Compared to Statistical models, ML does not make strong hypothesis on the distribution of the variables in a model.

Random Forest Neurale Network K-means

Few examples of ML Algorithm

4

What Machine Learning?Detailed Example a ML Algorithm: Neural Network

Y(real)

Y(estimated)Error

BACK PROPAGATION

FORWARD PROPAGATION1

2

activation function

5

What Machine Learning?Example a ML Algorithm: Neural Network

Activation Functions

FORWARD PROPAGATION

Neurone

Error Output

loss function

6

What Machine Learning?Example a ML Algorithm: Neural Network

Error = Output

BACK PROPAGATION

Layer (L)Layer (L-1)

Chain Rule

Update (Gradient Descent)

learning rate

1

2

7

What is Deep Learning ?

Deep Learning is one of the field of AI (Artificial Intelligence) using sophisticated techniques to train multi-layers Neural Networks with complex data format without the need to perform Feature Engineering.

8

What is Deep Learning ?Why does it even matter ?

- New techniques to easier build better Neural Networks: - solutions to vanishing or exploding gradient - Representational learning

(source: UBC)

9

What is Deep Learning ?Why does it even matter ?

Perf

orm

ance

Amount of Data

Classical Algorithms

Deep Learning

- Hardware Performances with GPU and distributed computation

- Overwhelming and more complex amount of data (Big Data): - sensors data - Video, - Voice

- Outperforms classical methods with complex data

10

What is Deep Learning ?Basic Deep Learning Architectures

Convolutional Network

Recurrent Neural Network Recursive Neural Network

Deep Belief Network AutoEncoder

11

Deep Learning Library Zoo

12

TensorFlow v1.0

13

TensorFlow What is TensorFlow ?

- An Open Source Machine Learning tool started by the Google Brain Team in 2015

- TF was create to build complex Neural Networks but is general enough to build a wide variety of ML algorithm or computation systems

- A library for numerical computation using Data Flow Graphs

- TF is highly scalable. It can be used on desktop, mobile devices, and Data centres with both CPU and GPU

- TF can be used by Researchers, Data Scientists and Developers.

Data Flow Graph

This is a Tensor

14

TensorFlow TensorFlow Architecture

TensorFlow Core Execution System

CPU GPU Android IOS …

TF supports a lot of front-end API : c++, Python, …

15

TensorFlow TensorFlow, the most active ML Open Source tool !

(source: Google Next)

~1000 commits / month

16

TensorFlow Features Anatomy of a TF Model: The Babylonian Algorithm

1

2

3

4 5

6

7

8

1 Constant value

2 Type casting -> float32

3 Variable in the Graph

4 Updating a Variable

5 Adding 2 Variables

6 Session Object

7 Initialise All Variables

8 Running the Model

17

TensorFlow Features TF Placeholder and Feed Dictionary

1 Placeholder for input value A

2 Feeding the graph with inputs A & B

1

2

Notice there no need for initialisation

18

TensorFlow Features TF Computational Graph

A TF program performs 2 main steps: 1) Assembling a ”lazy” Graph 2) Running the computation

ResultAfter Running in a tf.Session()

Com

puta

tiona

l G

raph

19

TensorFlow Features TF Variable Scope & TensorBoard

Massive deep neural network are notoriously hard to debug and optimise. TensorBoard just make it easier! You can use TensorBoard to visualise your TensorFlow graph, plot quantitative metrics about the execution of your graph, and show additional data like images that pass through it. When TensorBoard is fully configured, it looks like this:

Demo with a Linear Regression

Decorate TF programs with tf.name_scope

20

TensorFlow Features Run TF in Cloud ML

Cloud StorageCloud ML

setup.py

submit to Cloud ML with gcloud ml-engine package uploaded1 2

3

44

data fetched

train + model stored

Demo on Cloud ML21

TensorFlow Features TensorFlow High Level API

• Different Level of abstraction for Researchers, Data Scientist and Developers

• With ML Toolkit TF is now a serious competition to Scikit-Learn

(source: Google Next) 22

TensorFlow Features TensorFlow Serving : Serving Pipeline

Data

TF Cluster

TF Model Storage

Training+

StorageModel #1Model #2

Model #n

…

TensorFlow ServingClient

ClientClient gR

PC

23

Hands-on TensorFlow

24

https://github.com/ndjido/tensorflow-dubai-meetup-hands-on

TensorFlow Hands-On

. . .784 = 28 x 28 pixels

(.8, .2, 0, 0, 0, 0, 0, 0, 0, 0)

(1, 0, 0, 0, 0, 0, 0, 0, 0, 0)

Predicted

One-hot encoding

0 1 2 3 4 5 6 7 8 9

flattened

Shallow MNIST

. . .

25

784 = 28 x 28 pixels

0 1 2 3 4 5 6 7 8 9

. . .

. . .

. . .

. . .

TensorFlow Hands-On Deep MNIST

200

100

60

30. . .

10

26

TensorFlow Hands-On ConvNet MNIST 28

28

28 x 28 x 6

14 x 14 x 12

7 x 7 x 24

200

0 1 2 3 4 5 6 7 8 9

27

Thank You!

Questions ?

@ndjido28