Download pptx - Modern frameworks for machine learning

Sergii Nechuiviter. Samsung R&D Institute Ukraine

Machine Learning in practice

Tools and frameworks

Sergii [email protected]

Samsung R&D Institute Ukraine


Machine Learning Workflow We need to go deeper

First generation of NN frameworks Modern NN framework

Speculations and summary

Content

2A sample title

A sample title 3

Machine LearningWorkflow


Good beginning

4Machine Learning Workflow

Good Architecture

Machine Learning Workflow 5

– Experiments– Datasets– Trained models– Notebooks

– REST API– Other

cognitiveservices

A bit more advanced options

Machine Learning Workflow 6

Wolfram Data Science platform (Expected soon)

AmazoneMachine Learning

Sergii Nechuiviter. Samsung R&D Institute Ukraine Machine Learning Workflow 7

ML Framework Implementation Pros

scikit-learn Python The most popular, standard de-facto

Accord .Net Framework .Net C# Best for C#

mlpack C++ Fastest on CPU

MLlib from Spark Scala Distributed out of the box

RapidMiner Java Vast pack of algorithmsGUI workflow designer

Generic machine learning tools


Too much data to analyze?

8A sample title


First generation ofNN frameworks

A sample title 9


CaffeBerkeley Vision and Learning Center

Specialized on convolutional NN

Configs, Python

No cluster support

GPU support

10A sample title


Torch7Facebook

Imperative

LuaJIT + Python

No cluster support

GPU support

Embeddable, with ports to iOS, Android and FPGA backends

11A sample title


TheanoInitiated by LISA lab, now a vast community

Computational graph optimizations: generic and for RNNPython

No cluster support

GPU support

Pros: Easy NN with Keras, Lasagne and Blocks; RNN optimizations

Cons: Low-level, Error messages can be unhelpful

12A sample title

http://www.iro.umontreal.ca/rubrique.php3?id_rubrique=27

https://github.com/fchollet/keras

https://lasagne.readthedocs.org/en/latest/

https://github.com/mila-udem/blocks


ModernNN frameworks

Modern NN frameworks 13

DL4J


Deeplearning4JSkymind

NN specific ? Fast tensor backend

Java, Scala, Clojure

Spark, Hadoop, Mesos, Akka

Multi-GPU

Embeddable due to Java

Pros: Enterprise support

DL4J

14Modern NN frameworks

http://www.skymind.io/


TheanoInitiated by LISA lab, now a vast community

Computational graph optimizations: generic and for RNNPython

No cluster support

GPU support

Pros: Easy NN with Keras, Lasagne and Blocks; RNN optimizations

Cons: Low-level, Error messages can be unhelpful


http://www.iro.umontreal.ca/rubrique.php3?id_rubrique=27

https://github.com/fchollet/keras

https://lasagne.readthedocs.org/en/latest/

https://github.com/mila-udem/blocks


What is computational graph?


• Automatic differentiation

• Automatic parallelization

• Merging of similar subgraphs

• Arithmetic simplification ()

• Improvements to numerical stability


Computational graph optimizations


TensorFlowGoogle

Computational graph optimizations: generic and for RNNC++, Python

From V0.7 cluster support

From V0.7 multi-GPU

Linux and ARM deployment; No Windows

Google Cloud Machine Learning, Rescale

Pros: TensorBoard for visualization

Cons: slow and bulk



CNTKMicrosoft

Computational graph optimizations: generic and for RNNNetwork Description Language + C/C++

Cluster support

Multi-GPU

?

Azure Machine Learning

Pro: fastest at ImageNet LSVR 2015


http://image-net.org/challenges/LSVRC/2015/results




ImageNet LSVR 2015



NeonNervana™

?

Python

Cluster

Multi-GPU

?

Nervana™ Cloud

Pros: fastest convolutions



convnet-benchmarks

GoogleNet V1 - Input 128x3x224x2246-core Intel Core i7-5930K CPU @ 3.50GHz + NVIDIA Titan X + Ubuntu 14.04 x86_64https://github.com/soumith/convnet-benchmarks

Neon TensorFlow Torch Chainer Caffe0

500

1000

1500

2000

72 135 112 189

786

157310 349

497

1148

forward (ms) backward (ms)

Exec

ution

tim

e, m

s


http://research.google.com/pubs/pub43022.html

http://research.google.com/pubs/pub43022.html

https://github.com/soumith/convnet-benchmarks


MXNetDMLC

Computational graph optimizations: generic and for RNN + ImperativePython/R/Julia/Go/C++

Cluster support

Multi-GPU support

Windows, Linux, Mobile, Apple

AWS recommended

Pros: versatile, fast as Torch and Theano



Theano DL4J TensorFlow CNTK Neon MXNet

Creator – Skymind Google Microsoft Nervana™ DMLCComputational Graph V ? V V ? V

Cluster support – V V V V V

GPUSupport 1 Multi Multi Multi Multi Multi

Mobile – may be V ? ? VCloud – V V V V VPerformance Ok ? Slow Fast Fast Ok

Modern NN frameworks

A sample title 24


Speculations and summary

Speculations and summary 25


Analogcomputers

Specialized digital computers

Generic computers

Generic computers with coprocessors

Generic computer with coprocessor

GPU computations

Why deep learning happened?

Rise of single CPU performance Distributed

computing

Single core reach plateau

of performancePrice of computations

is dropping

26Speculations and summary


Analogcomputers

Specialized digital computers

Generic computers

Generic computers with coPU

Generic computer with coPU

GPU computa-tions

2016:NN on chip

Quantum PU?

Why deep learning happened?

Rise of single CPU performance Distributed

computing

Single core reach plateau

of performancePrice of computations is dropping

?

?

27Speculations and summary

Human-Machine cooperation

?


Try Microsoft Azure Machine Learning FreeTrialTry scikit-learnKnow about computational graphsTry MXNet

Be ready for changes!

Summary

Speculations and summary 28

A sample title 29