Upload
machine-learning-valencia
View
561
Download
0
Embed Size (px)
Citation preview
The Future of Machine Learning
IDAL; Intelligent Data Analysis Laboratory Universitat de València
http://idal.uv.es
José D. Martín Guerrero
4
Machine Learning"A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its
performance at tasks in T, as measured by P, improves with experience E”
5
Machine learning project
PROBLEM(Analysis)
Data extraction Data process
Feature engineering MODELS Validation
All is connected!; feedback is always necessary for the success of the project
6
Feature EngineeringPROBLEM(Analysis)
Data extraction Data process
Feature engineering MODELS Validation
Curse of dimensionalityTypical problem in bioinformatics: O(103) features & O(10) samples
The correct use of inputs is key for a successful
ML application
7
Feature EngineeringFEATURES
FeatureSelection
Feature ExtractionManifolds
Models
We can select a subset (selection); transform (extraction) or "attack" the model directly (deep
learning).
8
ModelsPROBLEM(Analysis)
Data extraction Data process
Feature engineering MODELS Validation
No free lunch!
9
Models
Any machine learning model has a certain structure and we have to choose this (for example, the architecture of a neural
network).
First we have to choose the model that we will use in a given problem.
Parameters are obtained by search procedures usually controlled by other parameters we have to choose.
Parameters
Search Algorithm
StructureMODEL
10
Example: Deep LearningPromising models without feature engineering; apparently,
they perform pretty well but …
How many layers, how many neurons per layer,
which activation function?Inputs
Outputs
Hidden Layers
The most widely used algorithm is the backpropagation after initialization using RBM (Restricted Boltzmann Machines); what adaptation constant must one use?; if we use regularization, how do we weigh that factor?; if
we use dropout (to avoid overfitting), what % must we remove?; if we inject noise what is the best value for its energy?
Hectic tuning
12
Automatic Workflows Automatic Model Selection
Automatic Tuning Automatic Representation
Automatic Prediction Strategies
It would be very nice to have a formal apparatus that gives us some ‘optimal’ way of recognizing unusual phenomena and inventing new
classes of hypotheses that are most likely to contain the true one; but this remains an art for the creative human mind.” E. T. Jaynes 1985
Future: Automatic
Future: Just around the corner … Reinforcement Learning
Supervised Learning Unsupervised Learning
Reinforcement Learning
• It does not need a teacher to learn a desired signal
• There is a goal (objective function) to be maximized
• The outcome is a sequence of actions rather than a static model
• It can deal with long-term objectives, not only a certain steps ahead in the future!!
• Similar to some stages of human learning
Reinforcement Learning
EXPERIENCE INTERACTION
ARTIFICIAL LEARNING
MAXIMIZATION OF A CERTAIN OBJECTIVE FUNCTION
POLICY
A priori knowledge Environment adaptation
Reinforcement Learning
AGENT
ENVIRONMENT
at
st+1 (after action at)
rt+1
st (before action at)
Long term reward
Action-value function
Optimal policy
st: State (at time t) at: Action (at time t) rt+1: Immediate reward
Discount rate: a reward received k time steps in the future is worth only k−1 times what it would be worth if it were received immediately
Values of the discount rate close to 1 avoids the agent to be myopic (maximization of rt+1)
Reinforcement Learning: Applicability
- Traditionally, RL has been theoretically studied but until very recently, practical applications were restricted to well-known synthetic problems and/or Robotics.
!!- Any dynamic problem that can be defined in a state-
space, in which certain actions can be taken, and an objective function has to be maximized, is susceptible to be tackled using RL.
!!
- Some practical applications on Marketing or Medicine (individualization of campaigns or treatments). !
!!
Reinforcement Learning: An example (drug prescription)
States: evaluation of the state of the patient ! Actions: possible actions that can taken by doctors wrt to drug prescription ! Reward: the action involves a change in the state. Depending on this
resulting state, a reward can be assigned !
The aim is to maximize the long-term reward
It is possible to know the dosage (actions) that should be administered to maintain patients within a given state.
! Other factors can also be included in the
computation of the reward (e.g., expenses).
21
Conclusions
Two ways have been mentioned:!1. Automatic election of the parameters in a machine learning project 2. Reinforcement Learning
Predicting the future is too challenging to talk about it but it is so exciting that one must talk about it
There’s plenty of room to come up with new ideas … already present!
1. Validation in Bayesian nets 2. Quantum Machine Learning