Course Calendar Class DATE Contents

1 Sep. 26 Course information & Course overview

2 Oct. 4 Bayes Estimation

3 〃 11 Classical Bayes Estimation - Kalman Filter -

4 〃 18 Simulation-based Bayesian Methods

5 〃 25 Modern Bayesian Estimation Particle Filter

6 Nov. 1 HMM(Hidden Markov Model)

Nov. 8 No Class

7 〃 15 Supervised Learning

8 〃 29 Bayesian Decision

9 Dec. 6 PCA(Principal Component Analysis)

10 〃 13 ICA(Independent Component Analysis)

11 〃 20 Applications of PCA and ICA

12 〃 27 Clustering, k-means et al.

13 Jan. 17 Other Topics 1 Kernel machine.

14 〃 22(Tue) Other Topics 2

Lecture 2 Plan

• Maximum a posteriori (MAP) Estimation

• Maximum Likelihood (ML) Estimation

• Minimum Variance (MV) Estimation

-- Batch (Non sequential) processing --

Notations and Facts on Probability

2

2

1

2 2

Probability desity(mass) function: discrete case, continuous case

Probability distribution function

Normal (Gaussian) density function

1

2

Random variable

X i i

X

x

p x Pr X x , Pr X

F x Pr X x

N , e

X

X Pr X

2 2

2 2

means distributed according to the desity

Mean (Average):

Variance

Covariance

XX

X Pr X

X E X X Pr X dX

E X X Pr X dX

R E X X X X Pr X dX

Independent: and are independent

Random vector (N-D) :

Mean vector:

Covariance Matrix

matrix

Conditional density:

Conditional Mean:

Varianc

T

XX

X

X Y

Pr X ,Y Pr X Pr Y

E

E N N

Pr X ,YPr X Y

Pr Y

E X Y X p X Y dX

X

X X

R X X X X

1

2

1

1

e (Used in error evaluation)

(scalar)

T

NT

N i i i i

i

N

ˆ ˆJ E

X

ˆE X X E X X X X

X

X X X X X