Daphne’sApproximateGroup ofStudents

Outline

Linear Regression Unregularized L2 Regularized

What is a GP? Prediction with a GP Relationship to SVM Implications

What does this mean?

Linear Regression Predicting Y given X Y = wtx + n

w_ml = argmax y[m+1] = w_mltx[m+1]

L2 Regularized Lin Reg L2 Regularized (Gaussian Prior on w)

Y = wtx + n w ~ N(0,S) w_map = argmax blah + ||w||^2

What is a random process?

It’s a prior over functions

What is a Gaussian Process?

It’s a prior over functions that generalized a Gaussian Random Vector

Prior over Y(x) ~ N(0,I)

Alternate Definition The thing with Euler’s equation

This is weird Not used to thinking of prior over Ys Or are we?

We ARE used to thining about prior over w What prior over y does this induce

Math P(w) -> P(Y) Wow! This became a Gaussian Process!

Prediction with a GP Predict y*[m+1] given y[1]…y[m]

We get a covariance = error bars Wow! This prediction is the same as w_map

but we get error bars!

Generalize that shit - Covariance Functions

Note that we have a thing here that is defined by C(x1,x2) which can be kernelized

Has to be pos semidefinite Is a kernel function

Relationship to SVM

Example

How do we reconcile these views?

Does this change anything?