Discriminant Functions for the 3 Normal Density CASE I ...cseweb.ucsd.edu/classes/wi10/cse190/lec5.pdf · Pattern Classification, Chapter 2 (Part 3) 7 The hyperplane separating R

1

CSE190a, Winter 2010

Pattern classification & Image Formation

Biometrics CSE 190 Lecture 5

CSE190a, Winter 2010

Announcements

•  Projects: Any questions? Discussions

Pattern Classification, Chapter 2 (Part 3)

3 Discriminant Functions for the Normal Density

• We saw that the minimum error-rate classification can be achieved by the discriminant function

gi(x) = ln P(x | ωi) + ln P(ωi)

•  Case of multivariate normal

€

gi(x) = −12(x − µi)

tΣi−1(x − µi) −

d2ln2π − 1

2lnΣi + lnP(ω i)


4

CASE I

Case Σi = σ2I (I stands for the identity matrix)

€

gi(x) = wit x + wi0 (linear discriminant function)

where :

wi =µi

σ 2 ; wi0 = −1

2σ 2 µitµi + lnP(ω i)

(ω i0 is called the threshold for the ith category!)


5

•  The decision surfaces for a linear machine are pieces of hyperplanes defined by:

gi(x) = gj(x)


6

2


7

The hyperplane separating Ri and Rj are given by

always orthogonal to the line linking the means!


8


9 Case II

Case Σi = Σ (covariance of all classes are identical but arbitrary!)

Now consider case where P(ωi)=P(ωj),

€

gi(x) = −12(x − µi)

tΣi−1(x − µi) −

d2ln2π − 1

2lnΣi + lnP(ω i)

= −12(x − µi)

tΣ−1(x − µi) −12lnΣ + lnP(ω i)

= −12(x − µi)

tΣ−1(x − µi) + lnP(ω i)

€

gi(x) = −(x − µi)tΣ−1(x − µi)

€

(x − µi)tΣ−1(x − µi) is sometimes called the Mahalanobis distance


10 Case II

Case Σi = Σ (covariance of all classes are identical but arbitrary!)

Hyperplane separating Ri and Rj

Here the hyperplane separating Ri and Rj is generally not orthogonal to the line between the means!


11


12 CASE III

Case Σi = arbitrary

The covariance matrices are different for each category

Here the separating surfaces are Hyperquadrics which are: hyperplanes, pairs of hyperplanes, hyperspheres, hyperellipsoids, hyperparaboloids, hyperhyperboloids)

3


13


14

CSE190, Winter 2010 Biometrics

Image Formation: Outline •  Factors in producing images •  Projection •  Perspective •  Vanishing points •  Orthographic •  Lenses •  Sensors •  Quantization/Resolution •  Illumination •  Reflectance


Earliest Surviving Photograph

•  First photograph on record, “la table service” by Nicephore Niepce in 1822. •  Note: First photograph by Niepce was in 1816.


Images are two-dimensional patterns of brightness values.

They are formed by the projection of 3D objects.

Figure from US Navy Manual of Basic Optics and Optical Instruments, prepared by Bureau of Naval Personnel. Reprinted by Dover Publications, Inc., 1969.


Effect of Lighting: Monet

4


Change of Viewpoint: Monet

Haystack at Chailly at Sunrise (1865) CSE190, Winter 2010 Biometrics

Pinhole Camera: Perspective projection

• Abstract camera model - box with a small hole in it

Forsyth&Ponce


Geometric properties of projection

•  Points go to points •  Lines go to lines •  Planes go to whole image

or half-plane •  Polygons go to polygons •  Angles & distances not preserved

•  Degenerate cases: –  line through focal point yields point –  plane through focal point yields line


Parallel lines meet in the image •  vanishing point

Image plane


The equation of projection

Cartesian coordinates: •  We have, by similar triangles, that

(x, y, z) -> (f x/z, f y/z, -f) •  Ignore the third coordinate, and get


A Digression

Homogenous Coordinates and

Camera Matrices

5


Homogenous coordinates •  Our usual coordinate system is called a Euclidean

or affine coordinate system

•  Rotations, translations and projection in Homogenous coordinates can be expressed linearly as matrix multiplies

Euclidean World

3D

Homogenous World

3D

Homogenous Image

2D

Euclidean World

2D

Convert

Convert

Projection


Homogenous coordinates A way to represent points in a projective space 1.  Add an extra coordinate

e.g., (x,y) -> (x,y,1)=(u,v,w)

2. Impose equivalence relation such that (λ not 0)

(u,v,w) ≈ λ*(u,v,w) i.e., (x,y,1) ≈ (λx, λy, λ)

3. “Point at infinity” – zero for last coordinate e.g., (x,y,0)

•  Why do this? –  Possible to represent

points “at infinity” •  Where parallel lines

intersect •  Where parallel planes

intersect

–  Possible to write the action of a perspective camera as a matrix


Euclidean -> Homogenous-> Euclidean

In 2-D •  Euclidean -> Homogenous: (x, y) -> k (x,y,1) •  Homogenous -> Euclidean: (u,v,w) -> (u/w, v/w)

In 3-D •  Euclidean -> Homogenous: (x, y, z) -> k (x,y,z,1) •  Homogenous -> Euclidean: (x, y, z, w) -> (x/w, y/w, z/w)


The camera matrix

Turn this expression into homogenous coordinates – HC’s for 3D point are

(X,Y,Z,T) – HC’s for point in image

are (U,V,W) Perspective

Camera Matrix A 3x4 matrix


End of the Digression


Affine Camera Model

•  Take Perspective projection equation, and perform Taylor Series Expansion about (some point (x0,y0,z0).

•  Drop terms of higher order than linear. •  Resulting expression is affine camera model

Appropriate in Neighborhood About (x0,y0,z0)

6


•  Perspective

•  Assume that f=1, and perform a Taylor series expansion about (x0, y0, z0)

•  Dropping higher order terms and regrouping.


Rewrite Affine camera model in terms of Homogenous Coordinates


Orthographic projection Starting with Affine camera mode

Take Taylor series about (0, 0, z0) – a point on optical axis


The projection matrix for orthographic projection

Parallel lines project to parallel lines Ratios of distances are preserved under orthographic


Other camera models •  Generalized camera – maps points lying on rays

and maps them to points on the image plane.

Omnicam (hemispherical) Light Probe (spherical)


Some Alternative “Cameras”

7


What if camera coordinate system differs from object coordinate system

{c}

P {W}


Euclidean Coordinate Systems


Coordinate Changes: Rigid Transformations

Rotation Matrix Translation vector CSE190, Winter 2010 Biometrics

A rotation matrix R has the following properties:

•  Its inverse is equal to its transpose R-1 = RT

•  its determinant is equal to 1: det(R)=1.

Or equivalently:

•  Rows (or columns) of R form a right-handed orthonormal coordinate system.


Rotation: Homogenous Coordinates

•  About z axis

x' y' z' 1

=

x y z 1

cos θ sin θ

0 0

-sin θ cos θ

0 0

0 0 1 0

0 0 0 1

rot(z,θ) x

y

z

p

p'

θ


Roll-Pitch-Yaw

Euler Angles

8


Rotation •  About (kx, ky, kz), a unit

vector on an arbitrary axis (Rodrigues Formula)

x' y' z' 1

=

x y z 1

kxkx(1-c)+c kykx(1-c)+kzs kzkx(1-c)-kys

0

0 0 0 1

kzkx(1-c)-kzs kzkx(1-c)+c kzkx(1-c)-kxs

0

kxkz(1-c)+kys kykz(1-c)-kxs kzkz(1-c)+c

0

where c = cos θ & s = sin θ

Rotate(k, θ)

x

y

z

θ

k


Block Matrix Multiplication

What is AB ?

Homogeneous Representation of Rigid Transformations

Transformation represented by 4 by 4 Matrix


Camera parameters •  Issue

–  camera may not be at the origin, looking down the z-axis

•  extrinsic parameters (Rigid Transformation) –  one unit in camera coordinates may not be the

same as one unit in world coordinates •  intrinsic parameters - focal length, principal point,

aspect ratio, angle between axes, etc.

3 x 3 4 x 4

Documents

Discriminant Functions for the 3 Normal Density CASE I ...cseweb.ucsd.edu/classes/wi10/cse190/lec5.pdf · Pattern Classification, Chapter 2 (Part 3) 7 The hyperplane separating R