02 Regresi Linier Sederhana

7/30/2019 02 Regresi Linier Sederhana

1/42

presented by:

Regresi Linier Sederhana

(RLS)

Dudi Barmana, M.Si.

2


2/42

Agenda

Persamaan RLS & asumsi yg mendasarimodel

Pendugaan (titik & interval) parameter model

Pengujian parameter model dg Uji-t dan Uji-F(Anova), serta penafsirannya

Korelasi dalam RLS: Koefisien korelasi linier

() Ukuran penilaian kemampuan/kesesuaian

model

Prediksi menggunakan model


3/42

Today Quote

Jika seseorang merasa bahwa mereka tidak pernah

melakukan kesalahan selama hidupnya, makasebenarnya mereka tidak pernah mencoba hal-hal baru

dalam hidupnya

---Einstein---


4/42

Persamaan RLS dan

Asumsi yg mendasari model


5/42

Simple Linear Regression Model

yi = 0 + 1 xi + i xi : regressor variable

yi : response variable 0: the intercept, unknown

1: the slope, unknown

i : error with E(i) = 0 and Var(i) = 2

(unknown)

The errors are uncorrelated sehingga cov(i,j)= 0; i j

5


6/42

Given x,

E(y|x) = E(0 + 1 x + ) = 0 + 1 x

Var(y|x) = Var(0 + 1 x + ) = 2 Responses are also uncorrelated.

Regression coefficients: 0, 1

1: the change of E(y|x) by a unit change inx

0: E(y|x=0)

6


7/42

Pendugaan parameter model

(titik & interval)


8/42

Pendugaan Titik

Least-squares Estimation of the Parameters

Estimation of0 and 1

n pairs: (yi, xi), i = 1, , n

Method of least squares: Minimize

8

n

i

ii xyS1

2

1010 )]([),(


9/42

Least-squares normal equations:

9


10/42

10

The least-squares estimator:


11/42

11

The fitted simple regression model:

A point estimate of the mean of y for a

particular x

Residual:

An important role in investigating the

adequacy of the fitted regression model and

in detecting departures from the underlying


12/42

12

Properties of the Least-Squares Estimators

and the Fitted Regression Model

are linear combinations of yi

are unbiased estimators.

01and

xxii

n

i

ii Sxxcyc /)(,

1

1

xy 10

01

and


13/42

13

011010

110

1

1

)()(

)(

)()()(

xxxyEE

xc

yEcycEE

i

ii

i

ii

n

i

ii

i xxi

i

xx

i

i

ii

i

ii

S

xx

S

c

yVarcycVarVar

22

2

222

2

1

)(

)()()(

)1

()(2

2

0

xxS

x

nVar


14/42

14

Some useful properties:

The sum of the residuals in any regression

model that contains an intercept 0 is always 0,i.e.

Regression line always passes through the

centroid point of data,

i

ii

i

ii

i

i xxyyyye 0))(()( 1

i i

ii yy

),( yx

iiii

i

ii xxyyxex 0))(

( 1

i i

iiiii xxyyxxyey 0))()))(((( 11


15/42

15

Estimator of2 Residual sum of squares:

xyT

xy

i

i

ii i

i

ii

i

i

SSS

Syy

xxyy

yyeSS

1

1

2

2

1

22

sRe

)(

))((

)(


16/42

16

Since ,

the unbiased estimator of2 is

MSE is called the residual mean square.

This estimate is model-dependent.

2)2()( nSSE E

EE MS

nSSs

2

22


17/42

Pendugaan Interval

Assume that i are normally andindependently distributed

2000

~se

nt

2

1

11 ~

se

nt


18/42

Parameter0

Pendugaan interval sebesar (1-) 100 % 0 :

XY 10

s

XXn

X

tn

i

i

n

i

i

n

21

1

2

1

2

21,20

n

i

i

n

i

i

XXn

X

s

1

2

1

2

0se


19/42

Parameter1

Pendugaan interval sebesar (1-) 100 % 1 :

21

XX

YYXX

i

ii

n

i

iXX

s

1

21)(se

n

i

i

,n

XX

s

1

22121

t


20/42

20

Confidence interval for2:


21/42

Pengujian parameter model

dg Uji-t dan Uji-F (Anova), serta

penafsirannya


22/42

Hypothesis Testing on the Slope and

Intercept

22

Assume i are normally distributed

yi ~ N(0 + 1 xi , 2 )

Use of t-Tests Test on slope:

H0: 1 = 10 v.s. H1: 110

)/,(~ 2

11 xxSN


23/42

23

If2 is known, under null hypothesis,

(n-2) MSE/ 2 follows a 2n-2

If

2

is unknown,

Reject H0 if |t0| > t/2, n-2

)1,0(~/

21010 N

SZ

xx

2

1

1011010 ~

)(

/

n

xxE

tseSMS

t


24/42

24

Test on intercept:

H0: 0 = 00 v.s. H1: 000 If2 is unknown


2

0

000

2

0000 ~

)(

)//1(

n

xxE

tseSxnMS

t


25/42

25

Testing Significance of Regression

H0

: 1

= 0 v.s. H1

: 1

0

Accept H0: there is no linear relationship

between x and y.


26/42

26

Reject H0: x is of value in explaining the

variability in y.


21

1

0

~)(

n

tse

t


27/42

27

The Analysis of Variance (ANOVA)

Use an analysis of variance approach to test

significance of regression


28/42

28

SST: the corrected sum of squares of the

observations. It measures the total variability in

the observations.

SSRes

: the residual or error sum of squares

The residual variation left unexplained by the

regression line.

SSR: the regression or model sum of

squares The amount of variability in the observations

accounted for by the regression line

SST = SSR + SSRes

i

ii

i

ii yyyyyy222 )()()(


29/42

29

The degree-of-freedom:

dfT = n-1 dfR = 1

dfRes = n-2

dfT = dfR + dfRes

Test significance regression by ANOVA SSRes = (n-2) MSRes ~

2n-2

SSR = MSR ~ 21

SSR

and SSRes

are independent

xyR SSS 1

2,1

ReRe

0 ~)2/(

1/

ns

R

s

R FMS

MS

nSS

SSF


30/42

30

E(MSRes) = 2

E(MSR) = 2 + 1

2 Sxx

Reject H0 if F0 > F/2,1, n-2

If 1 0, F0 follows a noncentral F with 1 and n-

2 degree of freedom and a noncentrality

parameter

2

2

1

xx

S


31/42

31

More About the t Test

The square of a t random variable with f degree

of freedom is a F random variable with 1 and f

degree of freedom.

xxs SMSset

/

)(

Re

1

1

10

0ReRe

1

Re

2

12

0

FMS

MS

MS

S

MS

S

ts

R

s

xy

s

xx


32/42

Korelasi dalam RLS:

Koefisien korelasi linier ()


33/42

33

The estimator of


34/42

34

Test on

100(1-)% C.I. for


35/42

Ukuran penilaian

kemampuan/kesesuaian model


36/42

Coefficient of Determination

36

The coefficient of determination:

The proportion of variation explained by theregressor x

0 R2 1

Example, R2 = 0.9018. It means that 90.18%

of the variability in strength is accounted for

by the regression model.

TT

R

SS

SS

SS

SSR sRe2 1


37/42

37

R2 can be increased by adding terms to the

model.

For a simple regression model,

E(R2) increases (decreases) as Sxx increases

(decreases)

R2

does not measure the magnitude of theslope of the regression line. A large value of

R2 imply a steep slope.

R2 does not measure the appropriateness of

the linear model.

22

1

2

12

)(

xx

xx

S

SRE


38/42

Prediksi menggunakan model


39/42

Prediction of New Observations

39

is the point estimate of the newvalue of the response

follows a normal distribution with mean 0

and variance:

0100

xy

0y

00yy

])(1

1[)()( 0200xxS

xx

n

yyVarVar


40/42

40

The 100(1-)% confidence interval on a future

observation at x0 (a prediction interval for

the future observation y0)


41/42

41


42/42

pertanyaan

Documents

02 Regresi Linier Sederhana