1 Part 5 Response of Linear Systems 6.Linear Filtering of a Random Signals 7.Power Spectrum Analysis 8.Linear Estimation and Prediction Filters 9.Mean-Square

1

Part 5Response of Linear Systems

6. Linear Filtering of a Random Signals

7. Power Spectrum Analysis

8. Linear Estimation and Prediction Filters

9. Mean-Square Estimation

2

6. Linear Filtering of a Random Signal

Linear System

Our goal is to study the output process statistics in terms of the input process statistics and the system function.

)}({)( tXLtY

)}.({)}({)}()({ 22112211 tXLatXLatXatXaL

][L )(tX )(tY

t t

),( istX),( istY

3

Deterministic System

Deterministic Systems

Systems with MemoryMemoryless Systems)]([)( tXgtY

Linear-Time Invariant (LTI) systems

Time-Invariant systems

Linear systems)]([)( tXLtY

Time-varying systems

)()(.)()(

)()()(

thtXdtXh

dXthtY

( )h t( )X t

LTI system

4

Memoryless SystemsThe output Y(t) in this case depends only on the present value of the input X(t). i.e., .)}({)( tXgtY

Memorylesssystem

Strict-sense stationary input

Strict-sense stationary output.

Memorylesssystem

Wide-sense stationary input

Need not bestationary in any sense.

Memorylesssystem

X(t) stationary Gaussian with

)(XX

R

Y(t) stationary,butnot Gaussian with

).()( XXXY

RR

5

Linear Time-Invariant SystemsTime-Invariant SystemShift in the input results in the same shift in the output.

Linear Time-Invariant SystemA linear system with time-invariant property.

)()}({)}({)( 00 ttYttXLtXLtY

LTI)(t )(th

Impulse

Impulseresponse ofthe system

t

)(th

Impulseresponse

Fig. 14.5

)}({)(1)( tLth dtt

6

Linear Filtering of a Random Signal

LTI

)()(

)()()(

dtXh

dXthtYarbitrary input

t

)(tX

t

)(tY

)(tX )(tY

)()()( dtXtX

.)()()()(

)}({)(

})()({

})()({)}({)(

dtXhdthX

dtLX

dtXL

dtXLtXLtY

By Linearity

By Time-invariance

7

Theorem 6.1

Pf ：

)()]([

)]([)()()()]([

thtXE

dtXEhdtXhEtYE

8

Theorem 6.2

dvduvuRvhuhR XY )()()()(

If the input to an LTI filter with impulse response h(t) is a

wide sense stationary process X(t), the output Y(t) has the

following properties:(a) Y(t) is a WSS process with expected value

autocorrelation function

(b) X(t) and Y(t) are jointly WSS and have I/O cross- correlation by

(c) The output autocorrelation is related to the I/O cross-correlation by

dhtYE XY )()]([

duuRuhR XXY )()()(

dwwRwhR XYY )()()(

)()( hRX

)()( hRXY

9

Theorem 6.2 (Cont’d)Pf:

10

Example 6.1X(t), a WSS stochastic process with expected value X = 10 volts, is the input to an LTI filter with

What is the expected value of the filter output process Y(t) ?Sol：

.otherwise0

sec, 1.00)(

5 teth

t

Ans: 2(e0.51) V

11

Example 6.2A white Gaussian noise process X(t) with autocorrelation function RW ( ) = 0 ( ) is passed through the moving-average filter

For the output Y(t), find the expected value E[Y(t)], the I/O cross-correlation RWY ( ) and the autocorrelation RY ( ). Sol：

.otherwise0

,0/1)(

TtTth

otherwise.0

,/)()(

otherwise.0

,0/)(:

200 TTT

RTT

RAns YWY

12

Theorem 6.3If a stationary Gaussian process X(t) is the input to an LTI Filter h(t) , the output Y(t) is a stationary Gaussian process with expected value and autocorrelation given by Theorem 6.2.

Pf ： Omit it.

13

Example 6.3For the white noise moving-average process Y(t) in Example 6.2, let 0 = 1015 W/Hz and T = 103 s. For an arbitrary time t0, find P[Y(t0) > 4106].Sol：

Ans: Q(4) = 3.17105

14

Theorem 6.4The random sequence Xn is obtained by sampling the continuous-time process X(t) at a rate of 1/Ts samples per second. If X(t) is a WSS process with expected valueE[X(t)] = X and autocorrelation RX ( ), then Xn is a WSS random sequence with expected value E[Xn] = X and autocorrelation function RX [k] = RX (kTs).

Pf：

15

Example 6.4Continuing Example 6.3, the random sequence Yn is obtained by sampling the white noise moving-average process Y(t) at a rate of fs = 104 samples per second. Derive the autocorrelation function RY [n] of Yn.Sol：

otherwise.0

,10)1.01(10][ :

6 nnnRAns Y

16

Theorem 6.5If the input to a discrete-time LTI filter with impulse response hn is a WSS random sequence, Xn, the output Yn

has the following properties. (a) Yn is a WSS random sequence with expected value and autocorrelation function (b) Yn and Xn are jointly WSS with I/O cross-correlation

(c) The output autocorrelation is related to the I/O cross- correlation by

n

nXnY hYE .][

i j

XjiY jinRhhnR ].[][

.][][

i

XiXY inRhnR

iXYiY inRhnR ].[][

17

Example 6.5A WSS random sequence, Xn, with X = 1 and auto-correlation function RX[n] is the input to the order M1 discrete-time moving-average filter hn where

For the case M = 2, find the following properties of the output random sequence Yn : the expected value Y, the autocorrelation RY[n], and the variance Var[Yn].Sol：

.20

,12

,04

][ and ,otherwise0

,1,,1,0/1

n

n

n

nRMnM

h Xn

otherwise.0

,22/1

,12

,03

][:n

n

n

nRAns Y

18

Example 6.6

.otherwise0

,1,,1,0/1 MnMhn

A WSS random sequence, Xn, with X = 0 and auto-correlation function RX[n] = 2n is passed through the orderM1 discrete-time moving-average filter hn where

Find the output autocorrelation RY[n].Sol：

otherwise.0

),1(/)(][ :

22 MnMnMnRAns Y

19

Example 6.7A first-order discrete-time integrator with WSS inputsequence Xn has output Yn = Xn + 0.8Yn-1 . What is theimpulse response hn ?Sol：

otherwise.0

,2,1,08.0][ :

nnRAns

n

Y

20

Example 6.8Continuing Example 6.7, suppose the WSS input Xn with expected value X = 0 and autocorrelation function

is the input to the first-order integrator hn . Find the second moment, E[Yn

2] , of the output.Sol：

.20

,15.0

,01

][

n

n

n

nRX

21

Theorem 6.6

.

1

1

][ ,

]0[]1[]1[

]1[

]0[]1[

]1[]1[]0[

n

XXX

X

XX

XXX

X XE

RRMR

R

RR

MRRR

Rn

If Xn is a WSS process with expected value and auto-correlation function RX[k], then the vector has correlationmatrix and expected value given by

nX

nXR

][ nXE

TnnX

TnMnMnn

XXER

MXXXX

n

. vectorldimensiona- theis ] [ where 21

22

Example 6.9The WSS sequence Xn has autocorrelation function RX[n] asgiven in Example 6.5. Find the correlation matrix of

Sol：

. 3332313033 XXXXX

.20

,12

,04

][

n

n

n

nRX

23

Example 6.10

. 1111 T

Mh

The order M1 averaging filter hn given in Example 6.6 can be represented by the M element vector

The input is

The output vector , then .

. 110T

LXXXX

TMLYYYY 210

XHY

X

L

ML

M

H

M

M

M

Y

ML

L

M

X

X

X

X

h

hh

hh

h

Y

Y

Y

Y

1

1

0

1

01

01

0

2

1

1

0

.otherwise0

,1,,1,0/1 MnMhn

24





25

7. Power Spectrum Analysis Definition: Fourier Transform

Definition: Power Spectral Density

dfefGtgdtetgfG

fGtg

ftjftj 22 )()( ,)()(

ifpair ansformFourier tr a are )( and )( Functions

.)(

2

1lim

)( 2

1lim)( is )( process

stochastic WSS theoffunction density spectralpower The

22

2

T

T

ftj

T

TT

X

dtetXET

fXET

fStX

26

Theorem 7.1

dfefSRdeRfS

fSRtX

fjXX

fjXX

XX

22 )()( ,)()(

pair ansformFourier tr

theare )( and )( process, stochastic WSSa is )( If

Pf：

27

Theorem 7.2

)()( (c)

)0()()( (b)

allfor 0)( (a)

:properties following ith thefunction w value-real a is )(

density spectralpower the,)( process stochastic WSSaFor

2

fSfS

RtXEdffS

ffS

fS

tX

XX

XX

X

X

Pf：

28

Example 7.1

.)(power average thecalculate and )(function

density spectralpower theDerive .0 where)(

function ation autocorrel has )( process stochastic A WSS

2 tXEfS

bAeR

tX

X

bX

Sol：

AtXEfb

AbfSAns X

)(,

)2(

2)(: 2

22

29

Example 7.2A white Gaussian noise process X(t) with autocorrelation function RW ( ) = 0 ( ) is passed through the moving-average filter

For the output Y(t), find the power spectral density SY (f ).

Sol：

.otherwise0

,0/1)(

TtTth

2

0

)2(

)2cos(12)(:

fT

fTfSAns Y

30

Discrete-Time Fourier Transform (DTFT)

Definition :

2/1

2/1

22

21012

)( ,)(

ifpair (DTFT) ansformFourier tr time-discrete a

are )(function theand },,,,,,{ sequence The

deXxexX

Xxxxxx

njn

n

njn

Example 7.3 : Calculate the DTFT H() of the order M1 moving-average filter hn of Example 6.6. Sol：

.otherwise0

1,,0/1 MnMhn

2

2

1

11)(:

j

Mj

e

e

MHAns

31

Power Spectral Density of a Random Sequence

Definition :

. 12

1lim)( is

sequence random WSS theoffunction density spectralpower The2

2

L

Ln

njn

LXn eXE

LSX

Theorem 7.3 : Discrete-Time Winer-Khintchine

2/1

2/1

22 )(][ ,][)(

:pair ansformFourier tr time-discrete a

are )( and ][ process, stochastic WSSa is If

deSkRekRS

SkRX

kjXX

k

kjXX

XXn

32

Theorem 7.4

).()( ,integer any for (d)

),()( (c)

. [0])( (b)

, allfor 0)( (a)

:properties following the

has )(density spectralpower the, sequence random WSSaFor

221

21

XX

XX

Xn

/

/- X

X

Xn

SnSn

SS

RXEdS

S

SX

33

Example 7.4

.otherwise0

,1,0,14/)2(][

. offunction density spectral

power theDerive follows. as ][function ation autocorrel

and valueexpected zero has sequence random WSSThe

2 nnkR

X

kR

X

X

n

X

n

Sol：

)2cos(12

)(:2

XSAns

34

Example 7.5

.2/10 where),(2

1)(

2

1)(

].[function

ation autocorrel theis What follows. as )(density spectral

power andmean zero has sequence random WSSThe

000

X

X

X

n

S

kR

S

X

Sol：

)2cos(][: 0kkRAns X

35

Cross Spectral DensityDefinition :

.][)(

theyieldsn correlatio-cross theof transform

Fourier the, and sequences random Sjointly WSFor

.)()(

theyieldsn correlatio-cross theof transform

Fourier the,)( and )( processes random Sjointly WSFor

2

2

k

kjXYXY

nn

fjXYXY

ekRS

tytral densicross spec

YX

deRfS

tytral densicross spec

tYtX

36

Example 7.6

).(output theofdensity spectralpower theFind

).()()()()(

thatfound weS,jointly WS are )( and )( when case, In this

0. with process noise WSSa is )( where)()()(Let

tY

RRRRR

tNtX

tNtNtXtY

NNXXNXY

N

Sol：

)()()()()(: fSfSfSfSfSAns NNXXNXY

37

Example 7.7

).(n observatio theofdensity spectralpower and

ation autocorrel thefind t,independen are )( and )( that Suppose

).()()()()(

thatfound weS,jointly WS are )( and )( when case, In this

0. with process noise WSSa is )( where)()()(Let

tY

tNtX

RRRRR

tNtX

tNtNtXtY

NNXXNXY

N

Sol：

)()()(: fSfSfSAns NXY

38

Frequency Domain Filter Relationships

Time Domain ： Y(t) = X(t)h(t)Frequency Domain : W(f) = V(f)H(f)

where V(f) = F{X(t)}, W(f) = F{Y(t)}, and H(f) = F{h(t)}.

( )h t

LTI system

x(t)

)()(.)()(

)()()(

thtxdtxh

dxthty

39

Theorem 7.5

).()()( is output the

ofdensity spectralpower the),(function sfer with tran

filter LTI a input to theis sequence random WSSaWhen

).()()( is )(output the

ofdensity spectralpower the),(function sfer with tran

filter LTI a input to theis )( process stochastic WSSaWhen

2

2

XYn

n

XY

SHSY

H

X

fSfHfStY

fH

tX

Pf：

40

Example 7.8

process? stochasticoutput theofpower average theisWhat

).(output filter theofation autocorrel anddensity spectral

power the),( and )( find ,/1 and 0 Assume

,otherwise.0

,0)1()(

response impulseh filter wit RCan input to theis )(

function ation autocorrel with )( process stochastic A WSS

tY

RfSRCbb

te/RCth

eR

tX

YY

-t/RC

bX

Sol：

RCb

RCtYE

bfRCf

RCbfSAns Y /1

/1)]([ ,

])2][()/1()2[(

)/1(2)(: 2

2222

2

41

Example 7.9

]?[ is What . sequence

output theoffunction density spectralpower the, )( Derive

otherwise.0

,1 ,11

,01

response impulseh filter wit a input to theis sequence This

).2cos(22)(density sepctralpower has sequence random The

2nn

Y

n

Xn

YEY

S

n

n

h

SX

Sol：

2][ ),6cos(22)(: 2 nYESAns Y

42

Example 7.10

.output filter average

-moving time-discrete for the )(density spectralpower theFind

otherwise.0

,10/1

filter average-moving time-discrete 1order the

throughpassed is ][function ation autocorrel and

0 valueexpected with sequence random WSSThe2

n

Y

n

nX

Xn

Y

S

MnMh

M

δnR

X

Sol：

)2cos(1

)2cos(1)(:

2

2

M

MSAns Y

43

Theorem 7.6

).()()( ),()()(

arefunction density spectralpower output theandfunction density spectral

power crossoutput -input theoutput,filter theis and ),(function

h transferfilter wit LTI a input to theis process stochastic WSS theIf

).()()( ),()()(

arefunction density spectralpower output theandfunction density spectral

power crossoutput -input theoutput,filter theis )( and ),(function

h transferfilter wit LTIan input to theis )( process stochastic WSS theIf

*

*

XYYXXY

n

n

XYYXXY

SHSSHS

YH

X

fSfHfSfSfHfS

tYfH

tX

Pf：

44

I/O Correlation and Spectral Density Functions

h(t)hn

h(-t)h-n

RX()RX[k]

RXY()RXY[k]

RY()RY[k]

H(f)H()

H*(f)H*()

SX(f)SX()

SXY(f)SXY()

SY(f)SY()

Time Domain

Frequency Domain

45





46

8. Linear Estimation and Prediction Filters Linear Predictor1. Used in cellular telephones as part of a speech

compression algorithm.2. A speech waveform is considered to be a sample

function of WSS process X(t).3. The waveform is sampled with rate 8000 samples/sec

to produce the random sequence Xn = X(nT).4. The prediction problem is to estimate a future

speech sample, Xn+k using N previous samples Xn-

M+1 , Xn-M+2 , …, Xn.5. Need to minimize the cost, complexity, and power

consumption of the predictor.

47

Linear Prediction Filters

TnMnn XXXY

1Use

to estimate a future sample X=Xn+k.

We wish to construct an LTI FIR filter hn with input Xn such that the desired filter output at time n, is the linear minimum mean square error estimate

Then we have

The predictor can be implemented by choosing .

. where, )( 1XYYn

TnL RRaXaXX

. where

, )(

01T

M

nT

nL

hhh

XhXX

B

B

ahB

48

Theorem 8.1

h

][

]1[

]1[

1

1

kR

kR

kMR

X

X

X

X

ERR

X

X

X

kn

n

n

Mn

XXXY knn

Let Xn be a WSS random process with expected value E[Xn] = 0 and autocorrelation function RX[k]. The minimum mean square error linear filter of order M1, for predicting Xn+k at time n is the filter such that

where

is called as the cross-correlation matrix.

.1

knnn XXX RRh

B

49

Example 8.1 Xn be a WSS random sequence with E[Xn] = 0 and autocorrelation function RX[k]= ( 0.9)|k|. For M = 2 samples, find , the coefficients of the optimum linear predictor for X = Xn+1, given . What is the optimum linear predictor of Xn+1 , given Xn 1 and Xn.What is the mean square error of the optimal predictor?Sol：

Thhh 10

Tnn XXY 1

50

Theorem 8.2

, , , 11 nMnMn XXX

If the random sequence Xn has a autocorrelation function RX[n]= b|k| RX[0], the optimum linear predictor of Xn+k, given the M previous samples is

and the minimum mean square error is .Pf：

nk

kn XbX ˆ

)1](0[ 2* kXL bRe

51

Linear Estimation Filters

TnMnn YYYY

1Estimate X=Xn based on the noisy observations Yn=Xn+Wn.We use the vector of the M most recent observations.Our estimates will be the output resulting from passing the sequence Yn through the LTI FIR filter hn. Xn and Wn are assumed independent WSS with E[Xn]=E[Wn]=0 and autocorrelation function RX[n] and RW[n]. The linear minimum mean square error estimate of X given the observation Yn is

Vector From :

. where, )( 1XYYn

TnL nn

RRaYaYX

.

,

where,

1

1

TnMnn

TnMnn

nnn

WWW

XXX

WXY

52

Theorem 8.3

h

Let Xn and Wn be independent WSS random processes with E[Xn]=E[Wn]=0 and autocorrelation function RX[k] and RW[k]. Let Yn=Xn+Wn. The minimum mean square error linear estimation filter of Xn of order M1 given the input Yn is given by such that

by given is where

)( 101

nn

nnnn

XX

XXWX

TM

R

RRRhhh

B

]0[

]1[

]1[

1

1

X

X

X

n

n

n

Mn

XXXY

R

R

MR

X

X

X

X

ERRnnn

53

Example 8.2 The independent random sequences Xn and Wn have expected zero value and autocorrelation function RX[k]= (0.9)|k| and RW[k]= (0.2)k. Use M = 2 samples of the noisy observation sequence Yn = Xn+Wn to estimate Xn. Find the linear minimum mean square error prediction filter

Sol：

Thhh 10

54





55

9. Mean Square Estimation

]))()([( 2tXtXE

Linear Estimation：Observe a sample function of a WSS random process Y(t) and design a linear filter to estimate a sample function of another WSS process X(t), where Y(t) = X(t) + N(t).

Wiener Filter：The linear filter that minimizes the mean square error.

Mean Square Error：

( )h t

LTI system

Y(t) )(tX

56

Theorem 9.1 ： Linear Estimation

.

is minimum The

otherwise.,0

,0),)(

)()(

is ]))()([(

theminimizeshat function t transfer The ).(

function transfer and )(input h filter witlinear a ofoutput theis )(

).(function density spectral cross and ),( and )(function

density spectralpower with process stochastic WSSare )( and )(

2

2

X

-Y

XYX

*L

YY

XY

XYY

df(f)S

(f)S(f)Se

re error mean squa

fSfS

fSfH

tXtXE

e errormean squarfH

tYtX

fSfSfS

tYtX

57

Example 9.1

filter? estimation optimum theoferror squaremean theis What (b)

?)(given )(

estimatingfor filter linear optimum theoffunction transfer theis What (a)

t.independenmutually are )( and )( .10)(function

density spectralpower with WSSa is )( where),()()( Observe

.50002

)50002sin()(

function ation autocorrel and 0 with process stochastic WSSa is )(

5

tYtX

tNtXfS

tNtNtXtY

R

tX

N

X

X

Sol：

Documents

1 Part 5 Response of Linear Systems 6.Linear Filtering of a Random Signals 7.Power Spectrum Analysis 8.Linear Estimation and Prediction Filters 9.Mean-Square