Agenda of Week VII

Review of Week VI

Multiple regression

Canonical correlation

Review of Week VI

Inference on MV Mean Vector Two populations

Inference on MV Mean Vector Multi-populations: MANOVA

Multiple Regression

Basic assumptions Linearity

X is an n by K matrix with rank K

Regression

Homoscedasticity

y X

[ | ] 0E X

2[ ] [ ' | ]Var E X I

Multiple Regression

Hypotheses

0 : 0 . : 0aH vs H

0 : 0 . : 0i a iH vs H

By ANOVA

When the null hypothesis is rejected,

For all i.

Multiple Regression

' ( ) '( )

' ' ' ' ' '

' 2 ' ' '

Min y X y X

y y X y y X X X

y y X y X X

1

'2 ' 2 ' 0

( ' ) '

X y X X

X X X y

OLS model

OLS solution

Multiple RegressionANOVA table

Source Sum of squares d.f. Mean square F

Regression 2' 'b X y ny k- 1 2' '

1

b X y ny

k

Residual 'e e n- k 'e e

n k

Total 2'y y ny n- 1 2'

1

y y ny

n

2' '1

'

b X y nyk

e en k

Coefficient of

determination

22

'1

'

e eR

y y ny

Multiple Regression

2 2 1'. [ ] ( ' )

e es Est V s X X

n k

2

kk kkt

s S

t-statistics (d.f. = n-k)

Where Skk is the kth diagonal element of (X’X)-1

Standard error of the regression

Multiple Regression

2

2

kkt s S

Standard error of coefficient

Canonical Correlation

Correlation structure between a group of independent variables and a group of dependent variables A kind of multiple regression with more than two

dependent variables Example: Physical size group vs. Exercise group How much explained the variation of a group of

variables by other group of variables?

Canonical Correlation

Basic model

1

.

.i

p

X

X X

X

Group of independent variables

1

.

.i

q

Y

Y Y

Y

Group of dependent variables

( ) ( ), ( ) ( )

( , ) ( )XX XX YY YY

XY XY

Cov X S Cov Y S

Cov X Y S

Covariance matrixes

Canonical Correlation

Basic model

1 1

1 1

' ......

' ......

i i ip p

i i iq q

V X X X

W Y Y Y

LC of two groups of variables

' '( , ) ( ( , ) )

' ' ' 'XY XY

XX YY XX YY

SV W r V W

S S

Correlation between V and W

Canonical Correlation

Correlation testing

0 : 0 : 0xy a xyH vs H Hypotheses

22

1

3

1, 12 3

2 2

2 2

1

1

( 1)2

4( )1.5 ,2 5

p qp q ms

Wilks r

pqF

pqms

p qp qm n sp q

Correlation

between V and W

Canonical Correlation

,( , ) ( , )

. .

( ) ( ) 0,

( ) ( ) 1

Max V W or r V W

s t

E V E W

V V V W

Objective: To find α and β maximizing ρ or r.

Canonical Correlation

1 1' '2 2

1 1 1 1,XX YYV e S X W f S Y

1 1' '2 21 1' , 'XX YYe S f S

1st canonical variable

kth canonical variable

1 1' '2 2,k k XX k k YYV e S X W f S Y

Canonical Correlation

1 1 1( , )r r V W

1st canonical correlation coefficient

kth canonical coefficient coefficient

( , )k k kr r V W

Canonical Correlation

1 112 2

1 112 2

,

k

XX XY YY YX XX

k

YY YX XX XY YY

e eigen vector of kth eigen value of

S S S S S

f eigen vector of kth eigen value of

S S S S S

Canonical Correlation

( , ) , ( , )XX X YY Yr V X AS D r W Y BS D

Canonical loading matrix

Canonical cross loading matrix

( , ) , ( , )XY Y YX Xr V Y AS D r W X BS D