Machine Learning Introduction, seminar 1

Grigorios Chrysos21/12/2016

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Introduction

Machine Learning: Discipline that enables the computers to “learn” without being explicitly programmed.

IntroductionBased on the definition, machine learning methods should:

● extract info from data automatically (learn patterns),

● generalise beyond the learning data,● (task-dependent) make predictions.

Image credit: https://xkcd.com/894/

IntroductionMultidisciplinary field:

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Real-world applicationsIn the previous decades:

● Pathfinder system● Digit recognition● AESOP system● COMPASS system

Additional examples and links in http://bit.ly/2hDyr01.

Real-world applications● Robotics

○ Self-driving cars, home appliances

● Personalised services○ “Personalised assistants”, e.g. SIRI, Cortana○ Reccomender systems, e.g. sites for movies/music

● Healthcare○ personalisation of treatment

● Text-processing○ machine translation

● Security applications○ cybersecurity○ bank fraud detection

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Supervised Learning● Definition: Supervised Learning is the subfield of inferring a function

from labelled training data.

● Explanation: The method learns a function g that maps the input to the output. Then, g should be able to infer the label of unseen data.

Supervised LearningFormal definition: Given N training pairs of (xi, yi) with xi the input feature, yi the respective label, i.e. D = {(x1, y1), (x2, y2), …, (xN, yN)},

learn a function g → X : Y .

At test time for a new sample x*, infer the y*estimate = g(x*, D).

Supervised LearningVisual example:

Image credit: http://wiki.cs.princeton.edu/

Supervised Learning● Classification: The function g accepts an (input) observation x* and

assigns it to a set of discrete classes (labels).○ Example: Given images of hand gestures (previous slide), recognise the gesture of

an unseen hand.

Supervised Learning● Classification: The function g accepts an (input) observation x* and

assigns it to a set of discrete classes (labels).○ Example: Given images of hand gestures (previous slide), recognise the gesture of

an unseen hand.

● Regression: The function g accepts an (input) observation x* and estimates the continuous output variable.

○ Example: Given info about {a product, product competitors, market state, the past performance}, predict the future market-share of the product.

Supervised LearningLinear vs Non-linear modelling

Image credit: https://docs.microsoft.com/en-us/azure/

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Machine Learning process

Machine Learning process1. Data acquisition:

a. Define the requirements.b. Search for related (academic) datasets.

Machine Learning process1. Data acquisition:

a. Define the requirements.b. Search for related (academic) datasets.c. Download if the license is appropriate for your application.d. Ensure it includes real-world use-cases for the task:

i. Accuracy of labels in extreme cases.ii. Balanced representation across classes/values.

Machine Learning process1. Data acquisition:

a. Define the requirements.b. Search for related (academic) datasets.c. Download if the license is appropriate for your application.d. Ensure it includes real-world use-cases for the task.e. (Randomly) divide into training/validation/test sets.

Machine Learning process1. Data acquisition2. Model selection:

a. Decide your approach to the task.b. Decide the learning method,

■ e.g. SVM/KNN for classification.c. “No free lunch” theorem.d. Choose an off-the-shelf implementation.

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing:

a. Clean the data.b. (Optionally) scale/transform the data.

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing4. Feature extraction

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing4. Feature extraction5. Model learning

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing4. Feature extraction5. Model learning6. Model evaluation

a. Proximity to the optimisation criterion.

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing4. Feature extraction5. Model learning6. Model evaluation

a. Proximity to the optimisation criterion.b. Task-dependent performance evaluation, e.g.:

■ classification: error rate, confusion matrix, sensitivity.■ regression: RMSE, correlation.

Machine Learning process1. Data acquisition2. Model selection3. Pre-processing4. Feature extraction5. Model learning6. Model evaluation

a. Proximity to the optimisation criterion.b. Task-dependent performance evaluation.c. Statistical tests.

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Practical tips

The materials mentioned in the next slides are just few indicative sources, there are many more sources available:

● https://www.quora.com/How-do-I-learn-machine-learning-1 ● https://github.com/josephmisiti/awesome-machine-learning

Practical tips● books, books and more books for learning:

○ Bayesian Reasoning and Machine Learning, Barber: http://web4.cs.ucl.ac.uk/staff/D.Barber/pmwiki/pmwiki.php

○ Machine Learning, Mitchell: http://bit.ly/2aHXJHd ○ Probabilistic programming and Bayesian methods for hackers, http://bit.ly/1ta2E3y

Practical tips● Online courses:

○ ‘Machine Learning’ by Andrew Ng, Coursera, https://www.coursera.org/learn/machine-learning

○ ‘Intro to Machine Learning’, Udacity, https://www.udacity.com/course/intro-to-machine-learning--ud120

○ Other videos, e.g. from a Summer School: http://bit.ly/2ifibPV

Practical tips● Sources of information/papers:

○ Conferences: NIPS, CVPR, ICML, etc.○ Journals: PAMI, IJCV, JMLR.

● Sites to bookmark:○ https://github.com/ : Open source implementations○ https://www.kaggle.com/ : Data science competitions○ http://www.datakind.org/ Data science for humanity

Agenda● Introduction to Machine Learning● Real-world applications● Supervised Learning - Definition/Introduction

○ Classification○ Regression

● Problem tackling - Machine Learning process● Practical tips● Practical example

Practical example

Practical example

Practical example

Complete code along with links and requirements in the link:

http://bit.ly/2hrZwAd

Thank you!

Questions?

Contact details:

https://github.com/grigorisg9gr

g.chrysos@imperial.ac.uk