1

Introduction to Machine Learning

Active Learning

Barnabás Póczos

2

Credits

Some of the slides are taken from Nina Balcan.

3

Modern applications: massive amounts of raw data.

Only a tiny fraction can be annotated by human experts.

Billions of webpages Images

Classic Supervised Learning Paradigm is Insufficient Nowadays

Sensor measurements

4

Modern applications: massive amounts of raw data.

Expert

We need techniques that minimize need for expert/human

intervention => Active Learning

Modern ML: New Learning Approaches

The Large Synoptic

Survey Telescope

15 Terabytes of data …

every night

5

Active Learning Intro

▪ Batch Active Learning vs Selective Sampling Active Learning

▪ Exponential Improvement on # of labels

▪ Sampling bias: Active Learning can hurt performance

Active Learning with SVM

Gaussian Processes▪ Regression

▪ Properties of Multivariate Gaussian distributions

▪ Ridge regression

▪ GP = Bayesian Ridge Regression + Kernel trick

Active Learning with Gaussian Processes

Contents

6

• Two faces of active learning. Sanjoy Dasgupta. 2011.

• Active Learning. Bur Settles. 2012.

• Active Learning. Balcan-Urner. Encyclopedia of Algorithms. 2015

Additional resources

7

A Label for that Example

Request for the Label of an Example

A Label for that Example

Request for the Label of an Example

Data Source

A set of

Unlabeled

examples

. . .

Algorithm outputs a classifier w.r.t D

Learning

Algorithm

Expert

• Learner can choose specific examples to be labeled.

• Goal: use fewer labeled examples

[pick informative examples to be labeled].

Underlying data distr. D.

Batch Active Learning

8

Unlabeled

example 𝑥3

Unlabeled

example 𝑥1

Unlabeled

example 𝑥2

Request for

label or let it

go?

Data Source

Learning

Algorithm

Expert

Request label

A label 𝑦1for example 𝑥1

Let it

go

Algorithm outputs a classifier w.r.t D

Request label

A label 𝑦3for example 𝑥3

• Selective sampling AL (Online AL): stream of unlabeled examples,

when each arrives make a decision to ask for label or not.

• Goal: use fewer labeled examples[pick informative examples to be labeled].

Underlying data distr. D.

Selective Sampling Active Learning

9

• Need to choose the label requests carefully, to get

informative labels.

• Guaranteed to output a relatively good classifier for

most learning problems.

• Doesn’t make too many label requests.

Hopefully a lot less than passive learning.

What Makes a Good Active Learning Algorithm?

10

• YES! (sometimes)

• We often need far fewer labels for active learning

than for passive.

• This is predicted by theory and has been observed

in practice.

Can adaptive querying really do better than passive/random sampling?

11

• Threshold fns on the real line:

w

+-

Exponential improvement.

hw(x) = 1(x ¸ w), C = {hw: w 2 R}

• How can we recover the correct labels with ≪ N queries?

-

• Do binary search (query at half)!

Active: only O(log 1/ϵ) labels.

Passive supervised: Ω(1/ϵ) labels to find an -accurate threshold.

+-

Active Algorithm

Just need O(log N) labels!

• N = O(1/ϵ) we are guaranteed to get a classifier of error ≤ ϵ.

• Get N unlabeled examples

• Output a classifier consistent with the N inferred labels.

Can adaptive querying help? [CAL92, Dasgupta04]

12

Uncertainty sampling in SVMs common and quite useful in

practice.

• At any time during the alg., we have a “current guess” wt

of the separator: the max-margin separator of all labeled

points so far.

• Request the label of the example closest to the current

separator.

E.g., [Tong & Koller, ICML 2000; Jain, Vijayanarasimhan & Grauman, NIPS 2010;

Schohon Cohn, ICML 2000]

Active SVM Algorithm

Active SVM

13

Active SVM seems to be quite useful in practice.

• Find 𝑤𝑡 the max-margin

separator of all labeled

points so far.

• Request the label of the example

closest to the current separator:

minimizing 𝑥𝑖 ⋅ 𝑤𝑡 .

[Tong & Koller, ICML 2000; Jain, Vijayanarasimhan & Grauman, NIPS 2010]

Algorithm (batch version)

Input Su={x1, …,xmu} drawn i.i.d from the underlying source D

Start: query for the labels of a few random 𝑥𝑖s.

For 𝒕 = 𝟏, ….,

(highest uncertainty)

Active SVM

14

• Uncertainty sampling works sometimes….

• However, we need to be very very very careful!!!

• Myopic, greedy techniques can suffer from sampling bias.

(The active learning algorithm samples from a different (x,y)

distribution than the true data)

• A bias created because of the querying strategy; as time goes

on the sample is less and less representative of the true data

source.[Dasgupta10]

DANGER!!!

15

DANGER!!!

• Main tension: want to choose informative points, but also want to guarantee that the classifier we output does well on true random examples from the underlying distribution.

• Observed in practice too!!!!

16

Interesting open question to analyze under

what conditions they are successful.

Other Interesting Active Learning Techniques used in Practice

17

Centroid of largest unsampled cluster

[Jaime G. Carbonell]

Density-Based Sampling

18

Closest to decision boundary (Active SVM)

[Jaime G. Carbonell]

Uncertainty Sampling

19

Maximally distant from labeled x’s

[Jaime G. Carbonell]

Maximal Diversity Sampling

20

Uncertainty + Diversity criteria

Density + uncertainty criteria

[Jaime G. Carbonell]

Ensemble-Based Possibilities

21

Active learning could be really helpful, could provide

exponential improvements in label complexity (both

theoretically and practically)!

Need to be very careful due to sampling bias.

Common heuristics

• (e.g., those based on uncertainty sampling).

What You Should Know so far

22

Gaussian Processes for Regression

23

http://www.gaussianprocess.org/

Some of these slides are taken from D. Lizotte, R. Parr, C. Guesterin

Additional resources

24

• Nonmyopic Active Learning of Gaussian Processes: An

Exploration–Exploitation Approach. A.Krause and C. Guestrin,

ICML 2007

• Near-Optimal Sensor Placements in Gaussian Processes: Theory,

Efficient Algorithms and Empirical Studies. A.Krause, A. Singh,

and C. Guestrin, Journal of Machine Learning Research 9 (2008)

• Bayesian Active Learning for Posterior Estimation, Kandasamy,

K., Schneider, J., and Poczos, B, International Joint Conference on

Artificial Intelligence (IJCAI), 2015

Additional resources

25

Why GPs for Regression?

Motivation:

All the above regression method give point estimates. We would like a method that could also provide confidence during the estimation.

Regression methods:

Linear regression, multilayer precpetron, ridge regression, support vector regression, kNN regression, etc…

Application in Active Learning:

This method can be used for active learning: query the next point and its label where the uncertainty is the highest

26

• Here’s where the function will most likely be.(expected function)

• Here are some examples of what it might look like. (sampling from the posterior distribution [blue, red, green functions)

• Here is a prediction of what you’ll see if you evaluate your function at x’, with confidence

GPs can answer the following questions:

Why GPs for Regression?

27

Properties of Multivariate Gaussian

Distributions

28

1D Gaussian DistributionParameters

• Mean,

• Variance, 2

29

Multivariate Gaussian

30

A 2-dimensional Gaussian is defined by

• a mean vector = [ 1, 2 ]

• a covariance matrix:

where i,j2 = E[ (xi – i) (xj – j) ]

is (co)variance

Note: is symmetric,

“positive semi-definite”: x: xT x 0

2

2,2

2

2,1

2

1,2

2

1,1

Multivariate Gaussian

31

Multivariate Gaussian examples

= (0,0)

18.0

8.01

32

Multivariate Gaussian examples

= (0,0)

18.0

8.01

33

Marginal distributions of Gaussians are Gaussian

Given:

The marginal distribution is:

bbba

abaa

baba xxx ),(),,(

Useful Properties of Gaussians

34

Marginal distributions of Gaussians are Gaussian

35

Block Matrix Inversion

Theorem

Definition: Schur complements

36

Conditional distributions of Gaussians are Gaussian

Notation:

Conditional Distribution:

bbba

abaa1

bbba

abaa

Useful Properties of Gaussians

37

Higher Dimensions Visualizing > 3 dimensional Gaussian random

variables is… difficult

Means and variances of marginal variable s are practical, but then we don’t see correlations between those variables

Marginals are Gaussian, e.g., f(6) ~ N(µ(6), σ2(6))

Visualizing an 8-dimensional Gaussian variable f:

µ(6)σ2(6)

654321 7 8

38

Yet Higher DimensionsWhy stop there?

39

Getting Ridiculous

Why stop there?

40

Gaussian Process

Probability distribution indexed by an arbitrary set (integer, real, finite dimensional vector, etc)

Each element (indexed by x) is a Gaussian distribution over the reals with mean µ(x)

These distributions are dependent/correlated as defined by k(x,z)

Any finite subset of indices defines a multivariate Gaussian distribution

Definition of GP:

41

Gaussian Process

Distribution over functions….

If our regression model is a GP, then it won’t be a point estimate anymore! It can provide regression estimates with confidence

Domain (index set) of the functions can be pretty much whatever

• Reals

• Real vectors

• Graphs

• Strings

• Sets

• …

42

Bayesian Updates for GPs

• How can we do regression and learn the GP from data?

• We will be Bayesians today:• Start with GP prior

• Get some data

• Compute a posterior

43

Samples from the prior distribution

Picture is taken from Rasmussen and Williams

44

Samples from the posterior distribution

Picture is taken from Rasmussen and Williams

45

PriorZero mean Gaussians with covariance k(x,z)

46

Data

47

Posterior

48

Ridge Regression

Linear regression:

Ridge regression:

The Gaussian Process is a Bayesian Generalization

of the kernelized ridge regression

49

Weight Space View

GP = Bayesian ridge regression in feature space

+ Kernel trick to carry out computations

The training data

50

Bayesian Analysis of Linear Regression with Gaussian noise

Linear regression:

Linear regression

with noise:

51

Bayesian Analysis of Linear Regression with Gaussian noise

The likelihood:

52

Bayesian Analysis of Linear Regression with Gaussian noise

The prior:

Now, we can calculate the posterior:

53

Bayesian Analysis of Linear Regression with Gaussian noise

After “completing the square”

MAP estimation

Ridge Regression

54

Bayesian Analysis of Linear Regression with Gaussian noise

This posterior covariance matrix doesn’t depend on the observations y,

A strange property of Gaussian Processes

55

Projections of Inputs into Feature Space

The Bayesian linear regression suffers from

limited expressiveness

To overcome the problem )

go to a feature space and do linear regression there

a., explicit features

b., implicit features (kernels)

56

Explicit Features

Linear regression in the feature space

57

Explicit Features

The predictive distribution after feature map:

Reminder: This is what we had without feature maps:

58

Explicit Features

Shorthands:

The predictive distribution after feature map:

59

Explicit FeaturesThe predictive distribution after feature map:

A problem with (*) is that it needs an NxN matrix inversion...

(*)

(*) can be rewritten:

Theorem:

60

Proofs• Mean expression. We need:

• Variance expression. We need:

Lemma:

Matrix inversion Lemma:

61

From Explicit to Implicit Features

62

From Explicit to Implicit Features

The feature space always appears in the form of:

Lemma:

No need to know the explicit N dimensional features.

Their inner product is enough.

63

GP pseudo code Inputs:

64

GP pseudo code (continued)

Outputs:

65

Results

66

Results using Netlab , Sin function

67

Results using Netlab, Sin function

Increased # of training points

68

Results using Netlab, Sin function

Increased noise

69

Results using Netlab, Sinc function

70

Applications: Sensor placement

Temperature modeling with GP

Near-Optimal Sensor Placements in Gaussian

Processes: Theory, Efficient Algorithms and

Empirical Studies. A.Krause, A. Singh, and C.

Guestrin, Journal of Machine Learning Research (2008)

71

An example of placements chosen using entropy and mutual information

criteria on temperature data. Diamonds indicate the positions chosen

using entropy; squares the positions chosen using MI.

Applications: Sensor placement

Entropy criterion:

Mutual information criterion:

Properties of Multivariate Gaussian distribution

Gaussian process = Bayesian Ridge Regression

GP Algorithm

GP application in active learning

What You Should Know

73

Thanks for the Attention! ☺