Upload
vinaya-muralidharan
View
84
Download
2
Embed Size (px)
Citation preview
@vinaya1980 @hrishikarekar
TESTING IN THE AGE OF MACHINE LEARNING
March 2016
https://www.youtube.com/watch?v=DCgHsxISE0Q
W H E R E A R E W E H E A D E D
httpsweforum.org/agenda/2017/01/worried-about-ai-taking-your-job-its-already-happening-in-japan?utm_content=buffera14af&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
THE WORLD OF ARTIFICIAL INTELLIGENCE
NLP
Neural Networks
MACHINE LEARNING
DEEP LEARNING
…..
THE FUTURE IS ALREADY HERE
Google RankBrain
Assistants
HOW DOES IT WORK
Supervised
• Labelled data• Given new
data, predict outcome
• Classification
Unsupervised
• No labels• Find hidden
structures• Clustering
Reinforcement
• Decision process
• Actions are rewarded or punished
• Learns to optimize rewards
http://scikit-learn.org/
LEARNING FROM DATA
SUPERVISED LEARNING - CLASSIFCATION
http://www.mikedeff.in/MLIntro.PNG
INTO THE REALM OF PROBABILITIES
Y = f ( x ) Y ≈ f ( x )
What is scrum ?
{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.0001860455668065697, "characteristics": 0.00006915141420904547, "compare": 0.00017757616296876222, "definition": 0.9970965385437012, "disadvantages": 0.0000534967657586094, } }}
Can you tell me about scrum ?
{ "Prediction": { "details": { "Algorithm": "SGD", "PredictiveModelType": "MULTICLASS" }, "predictedLabel": "definition", "predictedScores": { "advantages": 0.01977257989346981, "characteristics": 0.022757112979888916, "compare": 0.008386141620576382, "definition": 0.21092116832733154, "disadvantages": 0.04002799838781357 } }}
TOLERANCE LEVELS
Y ≈ f ( x )
Know the probability that is within acceptable limits
EVALUATE WITH DIFFERENT MODELS
Evaluate against a set of algorithms to iterate towards a model that’s closest representation and for further tuning
https://s3.amazonaws.com/MLMastery/MachineLearningAlgorithms.png?__s=h4reg8jqwyg4sz3bzdqf
EVALUATION – DATA SET APPROACHES
Random split• 70% train, 30% test
K-fold cross validation
Split into 3 datasets• #1 Train on 1 and 2, test on 3• #2 Train on 2 and 3, test on 1• #3 Train on 1 and 3, test on 2
Never use the same dataset for training and evaluating
ITERATIVE – LEARNING PROCESS
Be prepared to throw the model and start again
MODEL ACCURACY - CONFUSION MATRIX
Strive for better models, not 100% accurate.
OVERFITTING
MODEL IS AS GOOD AS THE TRAINING DATA
If all of the algorithms perform poorly,• it maybe worth considering if there is a lack of learning
structure in the data set
• some transformation needed to make the structure more learnable • remove unnecessary noise - stop words are typically
removed because they cause unnecessary noise)
SUMMARY
Machine Learning applications demand a shift in testing approach
• Use objective acceptance levels to evaluate the application
• Express test outcomes in statistical terms
• Have a high level understanding of the underlying working of the application
@vinaya1980 @hrishikarekar
THANK YOU