Embed Size (px)
Citation preview
Machine Learning
Narong Intiruk Data Scientist, T2P Co,.Ltd.
Begin with
Agenda• What is Machine Learning?
• How is Machine Learning different from traditional programming?
• How does it work?
• How to use it?
• Tools
• Resources
What is Machine Learning?
“Field of study that gives computers the ability to learn without being explicitly programmed”
- Arthur Samuel, 1959
What is Machine Learning?
Learning Algorithm
"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P if its performance at
tasks in T, as measured by P, improves with experience E."
- Tom M. Mitchell
Cat / Not Cat
this is a catE
T
Cat -> Cat : 80%
P
How is Machine Learning different from traditional programming?
How is Machine Learning different from traditional programming?
if (eyes == 2) & (legs == 4) & (tail == 1)… then print “Cat”
“ Cat ”
Traditional programming
InputProgram
Computer
Output
- eyes - legs - tail - ….. - …..
How is Machine Learning different from traditional programming?
if (eyes == 2) & (legs == 4) & (tail == 1)… then print “Cat”
“ Cat ”
Traditional programming
InputProgram
Computer
Output
- eyes - legs - tail - ….. - …..
How is Machine Learning different from traditional programming?
“ Cat ”
Machine Learning
Input
Program
Computer
Output
Cat Recognition
- eyes - legs - tail - ….. - …..
How does it work?
Feature Extraction
Learning Algorithm Evaluation
Optimization
Data
Model
Error
Predicted Value = f(X)We need f that Predicted Value close to True Value
(X)
(f)
Learning Process
Parameters
f( )=“ Cat ”
f( )=“ Cat ”
What is f()?
How does it work?
DataNumber/Categorical
Age Height Weight Gender
19 160 55 male
25 170 55 male
30 175 60 femal
ImageText
Sound
How does it work?
Feature Extraction
How does it work?
What are the things that should be considered?
Feature Extraction
Good BMI Bad BMI
Weight (X1)
Height (X2)
Good BMI
Bad BMI
Height Weight good/bad160 55 good
170 55 bad
175 60 good
.. .. .. Features Space
How does it work?
Feature Extraction
How does it work?
Multiple Dimension Feature
Feature Extraction
How does it work?
Feature Extraction
How does it work?
Feature Extraction
Word Importance
the 0.03
property 0.32
viceroy 0.56
….. …..
How does it work?
Feature Extraction
Principle Component
Analysis
Dimensionality Reduction
How does it work?
Feature Extraction
Automatic Feature Extraction Using Deep Learning
http://magizbox.com/training/deep_learning/site/introduction/
How does it work?
Hand-Crafted Feature Extraction
Feature Extraction
Automatic Feature Extraction by using DNN-Deep Learning
http://magizbox.com/training/deep_learning/site/introduction/
How does it work?
Automatic Feature Extraction
Feature Extraction
Automatic Feature Extraction by using CNN-Deep Learning
Image
https://devblogs.nvidia.com/parallelforall/deep-learning-nutshell-core-concepts/
How does it work?
Feature Extraction
Automatic Feature Extraction by using RNN-Deep Learning
Text
How does it work?
How does it work?
Learning Algorithm
Car for achieve the goal!
Framework for learning the target function
How does it work?
Learning Algorithm
Type of Learning
How does it work?
Learning Algorithm
Supervised Learning > Classification
( , “Cat”) LearningAlgorithm
Cat Model “Cat”Is cat or not?
How does it work?
Learning Algorithm
Supervised Learning > Regression
( , 4.5) LearningAlgorithm
House PriceModel 4.3 M.
What is the price of this house?
How does it work?
Learning Algorithm
Unsupervised Learning
( ) LearningAlgorithm
House Grouping
Model
What is the group of this house? Group 1
How does it work?
Learning Algorithm
Unsupervised Learning
Group1
Group2
Group3
How does it work?
Learning Algorithm
Reinforcement Learning
( ) LearningAlgorithm
Flappy BirdModel
How should I move?“ Down ”
Reward(+,-)
How does it work?
Learning Algorithm
Machine Learning Algorithms
https://en.wikipedia.org/wiki/List_of_machine_learning_concepts
How does it work?
Learning Algorithm
Tree Based
How does it work?
Learning Algorithm
Tree Based : Random Forest
How does it work?
Learning Algorithm
Support Vector Machine
3D Features2D Features
How does it work?
Learning Algorithm
Neural Network
How does it work?
Learning Algorithm
Neural Network : Deep Learning
Deep Neural Network (DNN)
How does it work?
Learning Algorithm
Neural Network : Deep Learning
Convolutional Neural Network (CNN)
How does it work?
Learning Algorithm
Neural Network : Deep Learning
Recurrent Neural Network(RNN)
How does it work?
Evaluation
Learning Algorithm
Cat / Not Cat Model
this is a cat
Cat -> Cat : 200Cat -> Not Cat : 10
How many is loss?
How does it work?Optimization
Tuning the car’s engine
How does it work?Optimization
Parameters Space (W1,W2)
f(X) = X1W1 + X2W2 +b
J is the cost function of Learning Algorithm
How does it work?Optimization
http://playground.tensorflow.org/
Find the best parameters for the functionbad fair good very good
How to use it?
test
trainX
Y
How to use it?
Feature matrix and Label vector
1 0.2 0.32 -0.93 0.32
2 -0.17 0.32 0.52 -0.93
3 -0.93 -0.17 0.52 -0.17
4 0.32 0.52 -0.17 -0.93
1
0
0
1
XY
}
Input Output
How to use it?
Feature Extraction : Number/Categorical
no x1 x2 x3 x4
1 0.2 0.32 -0.93 0.32
2 -0.17 0.32 0.52 -0.93
3 -0.93 -0.17 0.52 -0.17
X
}Age Height Weigh
tGender Abnormal
19 160 55 male No
25 170 55 male No
30 120 60 femal Yes
0
0
1
Y
Normal(0)/ Abnormal(1)
Covert categorical to number and normalize data
How to use it?
Feature Extraction : Text -> Binary Vectors
doc1 : You are so cool doc2 : You are very bad doc3 : You are very good
word indexyou 0are 1so 2
cool 3very 4bad 5good 6
corpus dictionary
doc1 : 0 1 2 3 doc2 : 0 1 4 5 doc3 : 0 1 4 6Feature index
no x0 x1 x2 x3 x4 x5 x61 1 1 1 1 0 0 02 1 1 0 0 1 1 03 1 1 0 0 1 0 1Binary Feature Vectors
How to use it?Feature Extraction : Text -> TF-IDF Vectors
doc1 : You are so cool doc2 : You are very bad doc3 : You are very good
word indexyou 0are 1so 2
cool 3very 4bad 5good 6
corpus dictionary
doc1 : 0 1 2 3 doc2 : 0 1 4 5 doc3 : 0 1 4 6Feature index
no x0 x1 x2 x3 x4 x5 x61 0.1 0.1 0.3 0.3 0 0 02 0.1 0.1 0 0 0.2 0.3 03 0.1 0.1 0 0 0.2 0 0.3TF-IDF Feature Vectors
word indexyou 0.1are 0.1so 0.3
cool 0.3very 0.2bad 0.3good 0.3
corpus TF-IDF
https://en.wikipedia.org/wiki/Tf%E2%80%93idf
How to use it?Feature Extraction : Text -> Word2Vec
word2Vec model
How to use it?Feature Extraction : Text -> Word2Vec -> Visualize
How to use it?Feature Extraction : Image -> Pixel Vector
gray scale image (MxN) normalized image matrix (MxN)
[0,0,0,..,0.6,0.5,0,0,0,0,0,0,0,0,0……..0]
pixel1 pixelMxN
Pixel Vector
How to use it?Feature Extraction : Image -> Pixel Vector
[0,0,0,..,0.6,0.5,0,0,0,0,0,0,0,0,0……..0]image1
Deep Neural Network
How to use it?Feature Extraction : Image -> raw / gray scale image
raw image
12345
Convolutional Neural Network
http://cs.stanford.edu/people/karpathy/convnetjs/
How to use it?
Model Evaluation
Confusion Matrix
How to use it?
Model Evaluation
Confusion Matrix Example
Normal Abnormal
Predict Normal 90 10
Predict Abnormal 10 90
Precision = 90/(90 + 10) = 0.9Recall = 90/(90 + 10) = 0.9
Acc = (90+90)/(90+10+90+10) = 0.9
How to use it?
Model Evaluation : Underfitting and Overfitting
Good on training set but bad on test set
Bad on training set and test set
Good on training set and test set
How to use it?
cluster number
Unsupervised Learning Workflow
How to use it?
Evaluation
Sum of squared erros
ToolsData Preparation
Machine Learning
Deep Learning
Keras
Data Visualization
scikit-learn
Resources
http://scikit-learn.org/stable/
https://keras.io/
https://www.tensorflow.org/
http://cs.stanford.edu/people/karpathy/convnetjs/
https://www.coursera.org/learn/machine-learning
[DeepLearning.TV] https://www.youtube.com/channel/UC9OeZkIwhzfv-_Cb7fCikLQ
http://machinelearningmastery.com/introduction-python-deep-learning-library-keras/
[Deep Learning Summer School 2016] https://www.youtube.com/playlist?list=PLWtzrfzH7gsfxTs8neTRJDXuqAn7qeV4E
http://courses.washington.edu/css490/2012.Winter/lecture_slides/02_math_essentials.pdf
[Deep learning at Oxford 2015] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu
