AN ADAPTIVE RECURSIVE FILTER

Soon-Ju Ko

NAVAL POSTGRADUATE SCHOOL

Monterey, California

THESISAN ADAPTIVE RE-CURSIVE FILTER

by

Soon-Ju Ko

December 1977

Thesis Advisor: Sydney R. Parker

Approved for public release; distribution unlimited

T18256

1JNCMSSTFTFDSECURITY CLASSIFICATION OF THIS PAGE (Whan Data Hnt.r.d,

REPORT DOCUMENTATION PAGE READ INSTRUCTIONSBEFORE COMPLETING FORM

1 REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

4. TITLE (and Sub.tltta) _ .

An Adaptive Recursive Filter5. TYPE OF REPORT 4 PERIOD COVERED

Electrical EngineerDecember, 197/

8. PERFORMING ORG. REPORT NUMBER

7. AUTHORf*;

Soon-Ju Ko

8. CONTRACT OR GRANT NLMBERf«J

9. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT. PROJECT. TASKAREA 8 WORK UNIT NUMBERS

II. CONTROLLING OFFICE NAME AND ADDRESS

Naval Postgraduate SchoolMonterey, California 93940

12. REPORT DATE

December 197713. NUMBER OF PAGES109

14. MONITORING AGENCY NAME » AOORESSf/f dtttarmnt from Controlling O/ltct)

Naval Postgraduate SchoolMonterey, California 93940

IS. SECURITY CLASS, (ol thla riport)

Unclassified

15«. DECLASSIFI CATION/ DOWN GRADINGSCHEDULE

16. DISTRIBUTION STATEMENT (ol thia Report)

Approved for public release; distribution unlimited

17. DISTRIBUTION STATEMENT (ol ihm abatract antarad In Block 30, II dlllarant /root Rapott)

IS. SUPPLEMENTARY NOTES

1$. KEY WORDS (Continue on ravaraa mid* II nacaaaary and Idantlty by block numbar)

20. ABSTRACT (Continue on tavaraa alda II nacaaaary and Idantlty by block number;

An adaptive recursive digital filter is presented in whichfeedback and feedforward gains are adjusted adaptively to mini-mize a least square performance function on a sliding windowaveraging process. A two-dimensional version of the adaptivefilter is developed and its performance compared with the optimalWiener filter. The filter is shown to be effective in separatingthree diagonal trajectory streaks from a background of correlated

DD FORM1 JAN 73 1473 EDITION OF 1 NOV 68 IS OBSOLETE

S/N 0102-014-6601|

UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE (Whan Data Kntarad)

UNCLASSIFIEDfliCUKITY CLASSIFICATION OF THIS P»GEf*h«i Dots Enfr*d

noise added to white noise. Although the recursive adaptive filterapproaches the optimal Wiener filter in performance, it does notrequire a priori statistical knowledge as does the Wiener filterto which it is compared. The results indicate that the recursiveadaptive filter "learns" the statistics and adapts.

DD Form 1473. 1 Jan 73

S/N 0102-014-6601UNCLASSIFIED

SECURITY CLASSIFICATION OF THIS PAGE(T*>»n Dmtm Entmrmd)

Approved for public release; distribution unlimited

AN ADAPTIVE RECURSIVE FILTER

by

Soon-Ju KoLieutenant, Republic or" Korea Navy

B.S.E.E., Seoul National University, 1975

Submitted in partial fulfillment of therequirements for the degree of

ELECTRICAL ENGINEER

from theNAVAL POSTGPADUATE SCHOOL

December 1977

ABSTRACT

An adaptive recursive digital filter is presented in

which feedback and feedforward gains are adjusted adaptively

to minimize a least square performance function on a sliding win

dow averaging process. A two-dimensional version of the adap-

tive filter is developed and its performance compared with

the optimal Wiener filter. The filter is shown to be effec-

tive in separating three diagonal trajectory streaks from a

background of correlated noise added to ivhite noise. Although

the recursive adaptive filter approaches the optimal Wiener

filter in performance, it does not require a priori statistical

knowledge as does the Wiener filter to which it is compared.

The results indicate that the recursive adaptive filter "learns"

the statistics and adapts.

TABLE OF CONTENTS

I. INTRODUCTION 7

II. LINEAR STOCHASTIC PROCESSES 10

A. INTRODUCTION 10

B. DEFINITION AND PROPERTIES OF LINEAR

STOCHASTIC PROCESSES 11

C. MODELING OF LINEAR STOCHASTIC PROCESS -22

1. Introduction 22

2. Method of Least Squares 25

3. Method of Filter Response 34

III. ADAPTIVE FILTERS 43

A. NONRECURSIVE FILTER 43

1. Introduction 43

2. Performance Surface, Gradient and

Wiener Solution 45

3. LMS Algorithm for One-Dimensional Filter 48

4. Two-Dimensional Adaptive Filter 52

a. Adaptive Filter Structure 52

b. Wiener Solution 54

c. LMS Algorithm 56

B

.

RECURSIVE FILTER 58

1. Introduction 58

2. One-Dimensional Adaptive Recursive

Filter 59

a. Structure 59

b. Problem in Wiener Solution 61

c. LMS Algorithm 64

3. Two-Dimensional Recursive Adaptive Filter

IV. ADAPTIVE NOISE CANCELLER 77

A. THE CONCEPT OF ADAPTIVE NOISE CANCELLING 7 7

B. NOISE CANCELLING WITHOUT AN EXTERNAL

REFERENCE INPUT 82

v - EXPERIMENT AND RESULTS 85

V. CONCLUSIONS 105

BIBLIOGRAPHY 108

INITIAL DISTRIBUTION LIST 109

I. INTRODUCTION

The term "filter" is often applied to any device or

system that processes incoming signals or other data in such a

way as to eliminate noise, to smooth signals, to identify each

signal as belonging to a particular class, or to predict the

input signal from instant to instant. There is an abundance

of literature covering the theories involved under the head-

ings of estimation, identification, modeling, prediction, etc.

The usual method of estimating a signal corrupted by noise is

to pass it through a filter that tends to suppress the noise

while leaving the signal relatively unchanged. The design of

such filters falls in the domain of optimal filtering, which

originated with the pioneering work of Wiener [8] and was ex-

tended and enhanced by the work of Kalman-Bucy [9] and others.

Filters used for the above purpose can be fixed or adap-

tive. The design of a fixed filter is based on a priori know-

ledge of both signal and noise statistics. On the other hand,

adaptive filters have the ability to adjust their own para-

meters automatically, and their design requires little or

no a priori knowledge of signal or noise characteristics.

This work presents an approach to signal filtering using an

adaptive filter that is in some sense self-designing.

The adaptive filter described here bases its own "design" (its

adjustment of internal parameters) upon estimated (measured) sta-

tistical characteristics of input and output signals.

The statistics are not measured explicitly and then used

to design the filter; rather, the filter design is accomplished

in a single process by a recursive algorithm that automatically

updates the system parameters with the arrival of each new data

sample. It is assumed that the input and the output at the

sampling instants are the only measurable quantities of the

system. It is also assumed that the unknown filter coefficients

parameters) to be designed enter linearly in the difference

equations which describes the self-designing process.

The steepest descent method is employed in which the pre-

vailing filter parameter vectors are perturbed at each itera-

tion in a manner so as to decrease a prescribed functional

(error criterion or cost function) to be minimized. The steep-

est descent method is one of the well known gradient based

algorithms.

For the case where the functional being minimized is the

mean square error, where the error is the difference between

filter output signal and the desired signal, the filter is

called the least mean square filter (LMS filter) . Various

adaptive algorithms are currently available depending upon

the cost function and the method used to minimize cost function.

The popularly used performance criteria are the least mean

square criterion, the maximum likelihood ratio (MLR) criterion,

and the maximum signal to noise ratio (SNR) criterion. Here

the LMS criterion only is studied and the steepest descent

method is employed. Inevitable errors in the estimation of

the statistics prevent the adaptive filter from delivering

optimal performance since the adaptive filter is not based on

the a priori knowledge of statistics. In Chapter II, the con-

cept of linear stochastic processes is reviewed as a preliminary

study for this thesis, and the modeling of stochastic processes

is studied. These can be considered as background material

for the following chapters.

In Chapter III, the concept of adaptive filters is intro-

duced and the structure of the signal and the mathematical

model of the processor is delivered. The algorithm for the non-

recursive adaptive filter by Widrow [1] is reviewed and the new

algorithm for the recursive adaptive filter is developed as is

the two-dimensional adaptive filtering process.

In Chapter IV, the adaptive noise cancelling concept is

analyzed rather qualitatively and its application to the special

case in which no desired signal is available is analyzed. In

Chapter V an experiment is performed through computer simulation

to check the feasibility of algorithms developed in the pre-

vious chapter and a comparison with the optimal Wiener solution

is made. In Chapter VI, the conclusions are presented together

with a summary of the experimental results and suggestions for

further research.

II. LINEAR STOCHASTIC PROCESSES

A. INTRODUCTION

The problem of defining a random process is of considerable

importance in the analysis of systems subject to noise distur-

bance. Often a partial definition of the process will suffice

as in the case of linear least mean square error filtering,

where only a knowledge of the correlation function is required.

For other problems, such as those involving nonlinear filtering,

more complete information will generally be needed. A complete

description of a random process requires a knowledge of the

distribution functions of all orders. But in practice few

processes apart from the normal and Markov are defined in this

manner. For the purpose of analysis, a model to generate the

random process is desirable and for a model to give a complete

description of the process, the distribution functions should be

derivable from the model. While both continuous and discrete-

time linear process may be defined, only the discrete -time case

will be considered here. The discrete -time linear process

can be considered to be the result of the digital filtering of

a sequence of independent and identically distributed (IID)

random variables.

The linear processes are important since they are inherently

simple in terms of physical considerations and form a class

which includes many discrete time normal random processes.

In the following section the definition and properties of the

linear processes are summarized.

B. DEFINITION AND PROPERTIES OF LINEAR STOCHASTIC PROCESSES

It has been found useful in the theory of stochastic processes

to divide stochastic processes into two broad classes: sta-

tionary and nonstationary . Intuitively, a stationary process

is one whose distribution remains the same as time progresses,

because the random mechanism producing the process is not chang-

ing as time progresses. A nonstationary process is one which

is not stationary.

Let {x(i), ieT} be a stochastic process with finite second

moments. Its mean value sequence , denoted by m(i) , is defined

for all i in T by

m(i) = E [x(i)] (2-1)

and its covariance kernel, denoted by K (j.i), is defined

for all j and i in T by

K (j.i) = Cov [ x (j), x (i)] (2-2)

An index set T is said to be a linear index set if it has the

property that the sum i+-j of any two numbers i and j of T also

belongs to T. Examples of such index sets are T= {1,2 , . . .},

T = {o,±l, ±2, ....}, T={i;i>o} and T={i:-°° < i < » }

A stochastic process (x(i), ieT }, whose index set T is linear,

is said to be

i) strictly stationary of order k, wherek is a given positive

integer, if any k points i, i+1, . . . i + k in T+ , where

T+ - (x (i) , i>_o}, and any j in T+ , the k dimensional random

vectors (x(i) , x (i+1). . x (i+k) }and{ x(i+j),. . . x(i+j+k)}

are identically distributed.

1

1

ii) strictly stationary if for any integer k, it is

strictly stationary of order k.

iii) widesense stationary (covariance stationary) if it

possesses finite second moments, if its index set T is linear,

and if its covariance kernel K (j , i) is a function only of the

absolute difference |j-i|, in the sense that there exists a

function R(n) such that for all j and i in T

K(j,i) = Cov[x(j), x(i)] = Rxx (j-i) (2-3)

or more precisely, Rj,,,^) has the property for every i and j in Z+

Cov [x(i) ,x (i+m) ] = E [x (i) x (i+m)

]

= Rxx (m ) (2-4)

We call RxX (m ) the covariance function (autocorrelation function)

of wide sense stationary time series (x(i), ieT+ } .

The second problem concerns the concepts of ergodicity

and the strong law of large numbers in terms of linear processes

To present a complete discussion of this question is not rea-

sonable for review purposes, but it is interesting to con-

sider certain aspects of it. For strict sense stationary pro-

cesses, the ergodic theorem is the strong law of large numbers

and states that

if{x(i),ieT } is a strict sense stationary, ergodic

random process and E[ |x (o)

|] <°°,

mthen l^m

_J_Z x (i) =E [x (o) ] with probability 1. (2-5)

m i=l

12

In general, a stochastic process is said to be ergodic if it

has the property that sample (or time) averages formed from

an observed sequence of the process may be used as an approxima-

tion to the corresponding ensemble average.

Stationarity and ergodicity concepts are readily extended to

two-dimensional random fields (for two dimensional signal, the

term random field is preferred to random process)

.

The assumption that the field is stationary means that

the statistics of a point in the field are not dependent on the

location of the point. Then, a stationary, two-dimensional

field has an autocorrelation function defined as:

Rxx (m,n)A E{x(k,£)x(k+m,£+n)

}

(2-6)

and it is also said to be ergodic if the statistical (ensemble)

average of random field x(k,l) is equal to the spatial averag-

ing of all points. That means

E[x(k,i)] = <x> (2-7)

where <x> by definition represents spatial averaging

<x >=lim -i- _LMiMi .

Mi-00 m m ^ 5 ' (2-8)M Mi M 2 o o2-M»

Now, consider a stationary sequence of random variables

{x(i) , ieT}. The correlation function of the sequence may

be written in the form

I^x(n^ e

1WndF(w) (2-9)

L13

where F (w) is a nondecreasing function, called the spectral

distribution. If F(w) is absolutely continuous with F (w) =

f (w) almost everywhere, we may write

it iwn

-7T

Rxx (n) =| e f(w) dw7 (2-10)

Under certain conditions, (e.g. £ |R(n)|<<», finite), then=o

correlation function may be inverted to yield the spectral

density as

f (w)= _1_ i Rxx (n)e~inw

(2-11)2tt -°°

A sequence of random variables(x (i) , ieT} is said to be a

process of moving average if it admits the mean square repre-

sentation

x (i) =_E h(i-j) u(j) (2-12)

where(u (j), jeT} is a collection of orthornomal random

variables (sequence of white noise) . If the sequence (h(i)

,

ieT} is one sided (ie, h(i) = 0, i<0), then {x(i), ieT} is

called a one sided moving average process.

A process is said to be regular if the error for prediction

one time unit ahead is nonzero. It is known that a process is

one of moving averages if and only if its spectral distribution

function is absolutely continuous. Furthermore, a process with

an absolutely continuous spectral distribution function is

regular if its moving average representation is one sided. These

facts serve to motivate the following definition of a linear

process

.

l A

DEFINITION [5]

A linear process {x(i) / ieT} is one having the structure

j

x(j) = Eh(i)u(j-i) = Zh(j-i)u(i) , (2-13)o -°°

where {u(i), ieT} is a sequence of independent and identically

distributed (IID) random variables. The set of real constants00

{h(i) , ieT+} is such that E|h(i)| <<», and the functiono

00

H(z) = Z h(i) z~L where z is a complex variable, is analytic

o

and has no poles outside the unit circle in the z plane. The

correlation function of this process is given by

5£n) = g h(j)h(j+n) (2-14)

and the corresponding spectral density f(w) is

f(w) = - |? h(j)e^ w|

2 = i|H(e iw )

|

2 (2-15)2

TT ° 2 *

It is assumed further that the process (2-13) has a rational

spectral density, that is

f(w) = iI H(elw )

I

2 = l B(e )

A(e lw )

2, (2-16)

where both A(e ) and B(elw ) are polynomials in elw of

finite order with all their poles inside the unit circle.

The process (2-13) may be generated by passing the sequence

{u(i), ieT} through the digital filter H(z). By the assumption

in the definition of the linear process and by the restrictions

on the spectrum, there exists an inversion D(z) where

D (z) = -MeL = _1

—

B(Z) H(z)

In general, D(z) and H(z) may be infinite polynomial in z.

D(z) may be written as the one-sided sequence

D(z) = g d^" 1 (2-17)

Passing the x(i) sequences through the digital filter will

recover the generating sequence u(i), that is

J

u(j) =_E dtj-j)x(ij= § dli)x(j-i) (2-18)

This is called an operation inversion. In general, we will have,

assuming a = 1

n _iEb . z oo

H(z) = -2-=- r- = E h(i)z_1

(2-19)1 +

y a . z o

and the process{x (i) , ieT} may be represented in two ways

i) x(j) = E h(i)u(j-i)(2-20)

m . nn) x(j) = i biu(j-i)- laix(j-i)

o 1

The second representation indicates that if H(z) is an all-

pole function, then we have

miii) E a^x(j-i) = u(j). (2-21)

o

In this case, since inversion uses only a finite number of past

samples, the process is called "finitely invertable." It is

clear that finitely invertable linear processes form a subclass

of autoregressive schemes for which case the set {u(i)} in

(2-21) would be orthonomal rather than independent. The con-

cepts and definitions above can be readily extended to a

two-dimensional linear process. A two-dimensional linear

process {x(m,n), TTiezj, nez2) could have the structure

x(m,n) = ZZh(k,g )u(k-m,£-n)oo

m n= ZZ h(k-m,ft-n)u(k,H) , (2-22)

where (u(k,§L), kez-]_, £.£Z2} is a two-dimensional sequence of

IID random variables with zero mean and unit variance. The set

of real constant {h(k.SL), kez, +, isz2+ } is such that

0O0O , n0000

1 n 1 (1 — lc — V

ZZ I h(k,j)) |<°° , and the function H(Z,,Z ) = EZh(k,X)Z, Z S00 J. z 00 ± z

where Z, and Z2 are complex variables, is analytic. The correla-

tion function of this process is given by

Rw<m ' n ) = ??h(k,Jl)h(k+m,!+n) (2-23)•*•* 00

and the corresponding spectral density f(w,,W2) is

f(w1#w2 ) =(^)

2I? h(k,l)eiwlk e

iw2^

I ^ 2

2TT.H(eiwl, e

iw2) (2-24)

With the same reasoning as in the one-dimensional case, we have

i n general

,

H(ZX , Z

2 )

Nl

N2

Z Z bij 1 2

Z-1

Z ^

M, M 91+Z

1Z aij Z,

1Z2_:i

1=0 j=o(i, j)^(o,o)

00 00 n —V —

u

5 g h(k.J)z1

kz2

l. (2-25)

and the process (x(k,l), keZ,, leZ^Jmay be represented in two

ways

i) x(k,H) = ??h(i.j)u(k-i,l-j)oo

N1N2 Mi M 2

ii) x(k,£) = ZIb. .u(k-i,j2.-j) - E I a, ,x (k-i, jL-j) (2-26)00 1 J 00 J-j

(i rj)^(0,0)

It should be noted that the moving average scheme would be ac-

complished by passing the orthonomal IID random variables through

the nonrecursive filter and the autoregressive schemes through

the recursive filter for both one-dimensional and two-dimensional

random process.

In the study of systems subject to the random signals, the

concept of power spectral density function is of importance.

For a given transfer function of a linear filter, the cross power

spectrum between input and output of the filter, and the output

power spectrum is of primary concern.

Consider a (continuous) system subjected to a random input

signal. Given a linear system with a transfer function

H(jw), the input to the filter a stationary process x.(t)

with an autocorrelation function R (z) and a power spectral

density function G (w) , then the following relationships

are obtained

Gxy (w) - Gxx (w)H*(jw)

G„„(w) = G (w)H(jw) (2-27)yy *\Y

Combining above two equations,

Gyy

(w) = Gxx (w)|H(jw)| 2, (2-28)

18

where G (w) is the cross power spec trum and G (w) is the

output power spectrum, the cross correlation would be cal-

culated by using the inverse Fourier transform of G (w)

.

For the continuous two-dimensional linear system of

H (Jwi/Jw2)/ subjected to a stationary two-dimensional input

signal of power spectrum Gxx (w, , w2

)

.

Bxy (wl' w2*

= Gxx (wl'

W2)H ^W1'^ W2^

Gyy

(wl'

w2 )

= Gxy (wl'

w2

} H ( jwl'

jw2

) (2-29)

Again combining above two relationships, the output power

spectral density function is

G (w1# w2

) = Gxx (w1

, w2)|H(jw1# jw

2 ) | (2-30)

Consider a discrete linear system to which a random input se-

quence is applied with a transfer function H(z). The input

sequence is a stationary process x(i) with an autocorrelation

function Rvv (m) and its?-transform G (z), where G (z) is equi-xx J xx ' xx ^

valent to the" power spectral density in the continuous case,i.e

Gxx< 2>

4 3»Wm>] (2-3D

Then,

G (z) = G (z) H(z)xy xx K'

v

Gw (z) = G (z) H(z)H(z_1

) (2-32)

For the two-dimensional discrete case,

Gxy

(zl'

z2

}= G

xx(z

l'Z2)H^1' z

2}

and Gy^ 1

^2

)= G

xx(z

l'z2

} H (zl'

z2

) H ^l^ Z ~2~] ( 2_33 )

19

As a summary of this section, a discrete-time linear process can be

considered to be the result of digital filtering an indepen-

dent identical random sequence having zeromean and unit variance,

The moving average scheme is the result of filtering through the

nonrecursive filter and the autoregressive scheme is that of

filtering through the recursive filter.

And for a linear system, the relations between transfer

function, power spectrum, and auto correlation are given by

G = G |h|2

yy xx ''

G = G Hxy xx

R (autocorrelation function) - 7[ transform of power

spectral density function.

The Figure 2-1 shows the block diagram which describes

the various relations and concepts.

20

ORTHONORMALI . I . D . RANDOMSEQUENCE

WHITE NOISE \

OF ZERO MEAN\AND UNIT'VARIANCE ;

LINEAR SYSTEM

H

LINEARCORRELATEDRANDOMPROCESS.

G (Z)xx

x(n)RANDOM INPUT

G (Z)=G (Z)H(Z)H(Z1

)VV XX

y(n)

GxxV Z2

)

x(m,n)

H(Z1

, Z2

)

h (m,n)

JGyy(Z lf Z

2)=GXX (Z

1,Z

2)H(Z

1,Z

2)

H(Zjlz2l)

WHITE NOISE

G (Z,, Z )=lxx 1 ' 2

G (Z, ,

yy iz2

) H(Z 1/ Z2)H(Z

1

" 1/ Z

2

~ 1)

FIGURE 2 -1STATISTICAL

PROPERTIES OF LINEAR SYSTEM

21

C. MODELING OF LINEAR STOCHASTIC PROCESS

1. Introduction

A more active concern at this time is that of system

modeling. It has been shown in a previous section that a linear

stochastic process (or field) can be generated by filtering

white noise through a linear filter. The problem can be stated

as follows. What is the filter equation (difference equation)

that produces a typical random process with a specified autocorrelation

function? That is, with the knowledge of second -order statistics,

determine the filter coefficient a's and b's in equations (2-20)

and (2-26) . It is clear that if one is successful in developing

a parametric model for the behavior of some random process,

then the model can be used for different applications, such as

prediction, estimation, smoothing, etc.. As far as the general

modeling problem goes, one of the most powerful models currently

in use is that where a signal x(n) is considered to be the out-

put of some system (filter) with unknown input u(n) such that

the following relationship holdsP q

x(n) = - I clkx(n-k) + gZ

Qb£u(n-£), b =l (2-34)

JC*- J.

where a^, 1<_ k<_ p , b£ 1 <_5,<_q

andthe gain G are the parameters of the hypothesized system.

Equation (2-23) says that the "output" x(n) is a linear function

of past outputs and present and past inputs. That is, the

signal x(n) is predictable from linear combinations of past

22

outputs and inputs . For 1he two-dimensional case, the difference

equation corresponding to equation (2-34) would be

Mx

M2

Lj L 2

x(k,£) = - Z Z ai-x(k-i / £-j)+G Z Z b ,

2u (k-^, £-£

2 )

i=o j=o -1 £,=u £

2=u '

where (i,j)^(0, 0) (2-35)

Equations (2-34)ard(2-35) can also be specified in the frequency

domain by taking the z transform of both sides of Eq (2-34) and Eq

(2-35) .

q -£

H(z) = jL<f> = G l+ *i,

1'*' (2 "36)U(Z) p ,

1+ Z a. ZK

k=lk

oo

where X(Z) = Z x(n)Z~n is the Z transform of x(n), and u(Z)n= —°°

is the z transform of u(n)

.

For the two-dimensional case,

L, L 9-£.. -£

s i \ *2

zi

z2

(2 -37)

X(Z,,Z~) £,=0£ 9=0

U<W Z

1Z2

a Z"1

Z"3

x A1 +.

L L aij

zl

z2i=o j=o J

(i. j)^(o.o)

where X(Z1,Z

2) = 3[x(k,£)]

U(Z1,Z

2) = 3[u(k,£)]

H(Z)an<}I(Z lf Z2 ) in Eqs (2-36)arri (2-37) are the general pole zero

models.

23

The roots of the numerator and denominator polynomials are the

zeros and poles of the model, respectively. There are two

special cases of the model that are of interest,

i) all- zero model ak = 0, a-ij =

ii) all- pole model b = 0, bJU£2

=

1 £ l 1 3

<_ £1

<_ L1

<_ l2< L

2

But (£1

, £2)^ (0,0)

As mentioned in section II-B, the all- zero model is known

in the statistical literature as the moving average (MA) model

and the all -pole model is then known as the autoregressive (AR)

model. The pole-zero model is then known as the autoregressive

moving average (ARMA) model. It should be recalled here that we

are interested in the case where u(n) or u(kTi) is white noise, and

this will be treated as a special case in the following. The

modeling problem can be stated as"given signal x(n) or x(k,2j,

find the filter coefficients (a's and b's)and the gain, G, in

Equation (2-34) in some manner." Two approaches will be given for

a solution of the above problem. The first is the method of least

squares which is based on the optimal estimation concept, and the

second isthefilter response method in which linear system pro-

perties are used. The one-dimensional case will be treated

first, then two-dimensional case including some examples. For

example, a lowpass correlated random process (field) and band

24

limited random process (field) are chosen since they represent

practical examples.

2. The Method of Least Squares

Athoughthe following can be applied to the deterministic

signal and stochastic processes, stationary or nonstationary

,

it is emphasized for only a stationary random process and only the

all-pole model is considered [6] . In the all- pole model, we

assume that the signal x(n) is given by as a linear combination

of past values and some input u(n)

:

Px(n) = - Z A vx(n-k) + Gu(n) (2-38)

k=l K

where G is a gain factor.

Here, it is assumed that the input u(n) is totally unknown,

which is the case in many applications. Therefore, the

signal x(n) can be predicted only approximately from a linearly

weighted summation of past samples.

Let this approximation of x(n) be x(n), where

x(n) = - I Avx ( n"k ) / (2-39)k=l

K

then the error between the actual value x(n) and the predicted

value x(n) is given by

AP

e(n) = x(n) - x(n) = x(n) + E A vx(n-k). (2-40)k=l

K

e(n) is also known as the residual. In the method of least

squares, the parameter ^'s are obtained as a result of the

minimization of the mean square error with respect to all of

the parameters

.

25

If the signal x(n) is assumed to be a sample of a random

process, then the error in equation (2-40) is also a sample

of a random process. In the least square method, we minimize

the expected value of the square of the error.

2P

E[e^(n)] =E{[x(n) + E Avx(n-k)]z

} (2-41)k=l

K

2E[e (n) ] is minimized by setting

8E[e2(n)l . ^ 1

< . <(2 _ 42)

3A£

From (2-41) and (2-42) we obtain the set of equations

PI Ak BKx(n-k)x(n-i)] = -E [x (n)x (n-i) ] 1-i-P (2-43)

k=l

Then the minimum average error is given by

E = E[x 2(n)] + F AkE(x(n)x(n-k) ) (2-44)

k=l

For a stationary process x(n), we have

E[x(n-k)x(n-i)] = Rxx (i~k)

where P^x(i) is the autocorrelation of the process.

Note that equations (2-42) and (2-44) lead to the well known

orthogonality principle [7] . Since in the least squares

method, we assumed that the input is unknown, it doesn't make

much sense to determine a value for the gain G. However,

there are certain interesting points that can be made.

26

Equation (2-39) can be written as

x(n) = - ? A,x(n-k) + e(n) (2-45)k=l

K

Comparing (2-45) and (2-38) , it is seen that the only input

signal u(n) that will result in the signal x(n) as output is

that where Gu(n) = e(n). (2-46)

That is, the input signal is proportional to the error signal.

e(n) can be also considered as the modeling error. The error

variance can be calculated by Equation (2-41) and the filter

coefficient A, (k=l . . . p) would be calculated by equation

(2-43) if the correlation function of process x(n) is available

At this moment, it should be recalled that a linear random

process is generated by linear filtering of white noise. Therefore,

we are interested in white noise inputs for the purpose of

modeling a given linear random process. That is, the

input u(n) is assumed to be a sequence of uncorrelated samples

(white noise) with zero mean and unit variance .

E[u(n)] = 0, E[u(n)u(m)] = <5mm

Then the output x(n) forms a stationary random process

x(n) = - ? Avx(n-k) + Gu(n) (2-47)k=l

K

Multiplying equation (2-34) by x(n-i) and taking the expectation,

E[x(n)x(n-j) ] =- I AVE [x (n-k) x (n-i)

]

k=lK

+E[Gu(n)x(n-i)

]

(2-48)

Noting that u(n) and x(n-i) are uncorrelated for i>0 and re-

calling that for stationary process, E [x (n) x (n-i) ] =R (i)

,

s

Equation (2-35) turns out to be

PRxx

(i) = - 2 AkRxx (i-k)for p > i> o (2-49)

k=l

and R(o) can be obtained by plugging x(n) of Equation

(2-3 8) into E quation (2-48)

p2R

xx (o) = ~ l A&ik)+G (2-50)k=l

Therefore , the gain can be given by

°2 =R- <0)+ V A

k* xx <k> (2-51)

k=l

It is noted that through Equation (2-46), that is Gu(n)=e(n) /

the white noise input of zero mean and unit variance generates

the random process e(n) , which is again white with zero mean

2and variance of G . Therefore, from Equation (2-4 9) , the re-

cursive filter coefficients A, , (k=l, ....p) can be calculated

and using these values the gain G would be calculated by Equation

(2-51) with the knowledge of autocorrelation function of a given

class of linear random process. So far, modeling of one-

dimensional stochastic process has been considered. Similar

reasoning can readily be extended to the modeling of two-

dimensional random fields. Again, the two-dimensional all- pole

model is considered.

Mi M2x(k,&)=-£ z A..x(k-i,£-j)+Gu(k / £) (2-52)

i=o j =o J

(i,j)^(o,o)

28

Let's define the set ft(k,£) such that for all i,j

(k-i, PL-j ) en(k,£),

all the values of x(k,£) in Q,(k,i) will be used to estimate

the point x (k, £)

.

K-Ml

^2a » •

V

•

• 4 *

•

#

• » •

i• x(k,£-M )

4 •

• #

•J»

fi(k,£)

x(k,£)# #

• • • •

FIGURE 2-2

DEFINITION OF fi(k,£)

29

Again the coefficients A- • will be determined such that the

mean squared error is minimized. The estimate of x(k,&)

,

Ax(k«£), is given by a linear combination of the previous values

Ax(k,M = - Z

nEx(k-i f Jl-j) (2-53)

The mean squared error is

E[e 2 (k,£)]=E{[x(k,£)- x(k,£)]2

} (2-54)

If E[e (k r £)] is minimized, x(k,£) is the "linear

least squares estimate" of x(k,£).

Going through the same procedure as in the one-dimensional

case, that is, substituting (2-53) in (2-54) and differentiating

with respect to each A. ., setting derivatives equal to zero, we

obtain the following set of simultaneous equations for the

unknown A.

.

E([x(k; t)4(k,Jl)]x(i,j)} = o for allfij)efi, (2-55)

which says that the coefficients A. . must be such that the

estimation error [x (k,£) -X (k,£) ] is statistically orthogonal to

each x(i,j) that is used to form the linear estimate. This

is known as the orthogonality principle in linear least square

estimation.

The modeling error is the difference between the true

value x(k,£) and the estimate x(k,£). By definition,A

e(k,£)=x(k,£)-x(k,£)=Gu(k,£) (2-56)

from the equation (2-52) and (2-53)

.

Again, we are interested jntbevhite noise field of zero

mean andunit variance. Then the modeling error is also a

30

random field.

Rewriting the Equation (2-52) in terms of the error e(k,£) gives

x(k,£)=-rfiZ A

j_.x(k-i / £-j)+e(k,£) (2-57)

To calculate the error variance, multiply (2-57) by x(k-m,£-n)

and take the expectations

E[x(k,£)x(k-m,£-n) ]=- ZfiZA

i-E [x (k-i 7 £-j ) x (k-m f £-n)

]

+E[e(k,£)x(k-m,£-n) ] (2-58)

For the stationary process,

E[x(k-i,£-j)x(k-m,£-n) ] -F^lm-in-j ] , then (2-58) will be

RxxCm1n)=-E^ZA

i.Rxxfrn-if n-j) for allfm, n)efi. (2-59)

The second term on the rLghtside of (2-58) is zero, because of the

orthogonality principle and R(0,0) can be obtained by the

following equation:

Fj) 7 Q)=-Z QL A

ij Rxx(i,j)+G2

(2-60)

Therefore the modeling error variance,

E [e2 (k,£) ]=E{ [ (Gu(k,£)

]

2} = G 2 is given by the equatiom

G2=E[e 2 (k,£) ]=R^(D,0)+E I A

i-j^x ^'^ and the mean of error

e(k,£) is E[e(k,£) ]=E[Gu(k,£) ]=0

Again with the knowledge of autocorrelation function of the

two-dimensional stationary random field, the filter coefficient

A. . can be obtained from the Equation (2-59) and using these

values of A. • , the gain G in Equation (2-52) can be calculated

by Equation (2-60)

.

Example 1 Consider a one-dimensional stationary band limited

random process forwhichtheautocorrelation function is given by

31

R^x (m) = />,m| cos(w m)

Find a model which describes this process,

all-pole model is chosen such that

An

x(n) = - i A,x(n-k)+Gu(n)k=l

K

where E [e(n) ] =

E [u(n)u(m) ] = <Smn

One has to choose the order of the difference equation; here,asecond

order model (p=2) is chosen. Then

x(n)=-A1x(n-l) -A

2x (n-2) +Gu (n)

The problem shrinks down to that of finding A,, A2 / G with the

given R (m)

.

3 xx

From Equation (2-4 9)

k=l

R^l) = - A-^o) - A2^-l)

W ] " " Al^ 1} " A2%0)

Putting this in matrix form

-%D - m 0)

From Rxx(m) = Piml cos (w m)

o '

-1 - p COS w

- p COS w -1) \

A2 J

cos w<

P^ cos 2 w,

Therefore At n ,-i cO <> \1 =__ £ cox w„ (1-/°^ cox 2 wq)

n 2 21 - P COS Wo

32

A2

= P2 sin 2w

1 - p2cos 2 w

From Equation (2- 51)

G = R (0) +xx

? A,

k=l XX (k)

= 1 + A,R (1)+ A 9 P (2)1 XX ^ XX

= 1 + A,p cos w + A2P

cos 2 Wo

, _ p2cos 2wo (l-ft2cos 2 w°) p 4 sin 2w cos 2 w

1 -/

>2cos 2w 1 - p2cos 2 w

Example 2 Given a stationary two-dimensional random field with

Rxx(m,n) = p|m|p|n| ,

find the autoregressive model of this random field (most mono-

chromatic images can be assumed to have this form of auto-

correlation function). A first-order model (M = 1, M2

= 1) is

chosen. Then equation (2-52) can be written as follows:

x(k,A) = -7 A10x(k-1,£)-A11x(k-1 / £-l)-A01x(k,£" 1

) +Gu(k,£)

Where u(k,£) is white noise with zero mean and unit variance.-

Then, using Equation (2-59)

1 1

iym,n)=-i^Q £ Q

AijR

y|m-i,n-j) for all(m / n

) £ fi .

(i f j)^(o.O)

Rxx(l„0) = - A1Q

Rxx (0,0) - A

11Rxx

(0 -1) -A^Rxx (1 ,-1)

Rxx(l^l) = - A.. _R (0,1) - Ain R (0.0) -An1 Rxx(l# 0)10 xx 11 xx 01

Rxx(0 v l) = - A,.R (-1,1)- A...R (-l,0)-A n ,R (0, 0).10 xx 11 xx 01 xx

Putting thisin matrix form gives

-Rxx.(0,0) -Rxx(0,-1) -Rxx(l,-1)

-Rxx(0,-1) -Rxx(0, 0) -Rxx(l f 0)

-Rxx(-1 T 1) -Rxx(-l t 0) -Rxx(0 f 0)

N

10

11

L

0lJ

Rxx(l r 0);

Rxx(l T l)

Rxx(0»l)^

33

For thegiven auto correlation function above

-1-f> -f

z

-P -1 -7°

fi2 _

;/

o _! /

10

11

l

01

which yields

L

01

L

ll

L10

= ~P

= P:

The modeling error variance or the square of the gainG can be cal-

culated by Equation (2-60)

21 X

G =Rxx(°'°>

+i=

Z

Q ^Aij R

xx(i '^

(ir j)^(0,0

= 1 + A01

Rxx(0 ' 1)+ All

Rxx (1^ + ^O^x (1 ' 0)

= (1-p2

)

2

D I ml pt n I

Therefore, the complete model for Rxx = r f is

x(k,£) = px(k,£-l) + /3 x(k, £) + px(k-l,£) + (l-:f )u(k,£)

where E (u(k.£) ) =

Eiu(k.A) u(k-p,£-q)] = O.pq.

3 . Method of Filter Response

Another method of modeling a linear stationary random

process is based on the concept thatalinear random process is a

result of filtering white noise through a linear filter. In

Section B. of this chapter, the properties of linear systems

have been discussed.

34

For the discrete system, it is known that the filter output

power spectral density is the Z transformation of the output correla-

tion function. That is,

Gyy(Z) = 3 [R.yy

(m)]

Gyy(Zi, Z 2 ) = 3 [R yy(m,n)]

and also it is noted that

Gyy (Z) = Gxx(Z) H (Z) H (Z~ )

Gyy(Z , Z ) = Gxx(Z ,Z)H(Z,Z)H(Z , Z ),12 12 12 1 2'

where Gxx(Z), Gxx(Zi, Z2)is the input power spectral density

function and H(Z), H (Zi, Z2) is the transfer function of the

filter. Denoting the white noise input as u(n)oru(k,£) and

the output of filter, which is a linear stationary random pro-

cess we are going to model as x (n)or X(k,£)r

the problem

can be stated as follows: For a given autocorrelation function

ofalinear random process (field) R(m) , (R (m,n)), find a linear

system such that when the input is white noise, the output of the

filter gives a given autocorrelation function R(m) , (R(m,n)).

u(n)

u(k,*)

H(Z)

H(Z X , z 2 )

x(n)—+-

FIGURE 2-3

THE MODELING PROBLEM IN THE DISCRETE CASE

x(k,4)

Given ^(m), Rxx (m ' n)

35

If the input is white noise, then

GUU (Z) = Const,

Guu (Zl'

Z2 }

= Const.

Therefore, the solution for the required filter is to find a

function H(Z) or H(Z-,, Zo) which satisfies

G (Z) = Const-H(Z)H(Z_1

)X^V

GXX (Z1# Z2

) = Const-H^, Z2)H (Z£

!

, Z2

_1) (2-62)

where G„„(Z), G (Z, , Z ) is known through the relationXX XX X ^

Gxx< Z) = 3 I Rxx<m >l

Gxx< Zl'

Z 2> =3[Rxx<m ' n 'l

since R (m) , R (m,n) is given.xx xx 3

Butforthe two-dimensional case, there is an inherent difficulty

in factorization of G (Zj, Z2

) to H(Z-,, Z2)H(z7 , Z

2~

).

Therefore, only separable functions can be modeled by this

technique.

Example 1 For a given stationary linear random process with

R (n) = S 2p,mI cos(w m) , m=o, 1, 2 . . .

Find a difference equation which will give a random process

with autocorrelation above.

G (Z) = MR (m) ]xx ^ xx

_ g* 2 (l-|°2

) . [-Z /°cos w +(l4-^2 )-Z~

1cox w Q ] (2-51)

(l-2f>Zcos w +p2Z2

) (l-2/

PZ~ 1cos w +f2z~

2)

The second step is to find a factored expression for G (Z) ,i.e.

Gxx Z) = G1(Z)

* Gl^ 2" 1

)

36

Assuming that G (Z) has the form

GXX (Z) = 3 2 (l-p2

) az + b az + b (2-52)

l-2/°Z cos w +/^2z~

2l-2pz cos w +p

2z

Comparing (2-52) and (2-51) , a and b can be obtained as

a = j ( 1-p cos w +p2 + l+y° cos w +p

1 2b = j 1-p cos Wo+p 1+ P COS W +jO

From equation (2-50), if we choose H(Z)= az 1 + b

l-2pz'1cos w +p

2z~

2

2 2then the term & (l-j° ) in Equation (2-52) can be considered

as G (Z)uu

Therefore

(1-p2

) d2

if m=0

Ruu (m)

if m^O

and the complete model is drawn block diagram form in Figure 2-4.

u(n)

Hm _ az +b

x(n) has givenautocorrelation

White Noisewithvariance= (1-/32) 2

l-2pz cos wo +p

2 z-2

x(z)

FIGURE 2-4

FILTER FOR ONE-DIMENSIONAL BAND PASS PROCESS

37

lb put the system's input-output relation into state equation

form, it is defined for convenience as

u(z) = z~ w (z)

where w(n) is also white noise with same autocorrelation

function

R(m) = (1-P2)tf

2if m=0

WW '

if m^O

then the transfer function is

X(Z)H(Z) =

D ef ining

Xx(z)

az 1 + bU(Z)

-p2z

1X2

(z)

-1 2 -2l-2pZ cos w +p z

X2(z) U(z)

-1 2 -21-2'P cos woz +p z

X(Z)-1

= (az +b)X2(z)

then

and

Xx(n) = -p x

2(n-l)

X2(n) = x

1(n-l) + 2p cos w x

2(n-l) + w(n-l)

X(n) = ax2(n-l)+bx

2(n) = -P ax(n)+bx

2(n)

Putting the state and output equations in matrix form gives

( i \\ ( „ ~2 V . . .> f - \x-^n)

x9 (n)

- p

1 2 cosw

-2

x1(n-l)

>

i x2(n-1

w(n-l)

x(n) = (-p a b) x-^n)

x (n)

Example 2 The two-dimensional "band limited" discrete

Markov process is defined by the autocorrelation function

t, / \ ,2 r , (m) , \ir^(m ) / \i m=0.1.2...R^On,!!) -<*•

[frv cos (w

fm)] [p 2

cos(w2n)] nmQ

'

fl'

t2mmm then

the discrete power spectral density

Gxx (zl'

Z 2> =6J2 (1-Pi2)d-P2

2)

A(Z1 / 2

2}

B(ZX

, Z2 )

where

2-1 2-1A(Z

X, Z

2) = [-Z

1Cosw

1+(l+y0

1)-Z

1CosW-jJ [-Z

2CosW

2+(l+j?

2)-Z

2Cos w

2]

B(?l'

z2

)= [(1" 2 r

3

izicos w

i+/

]

izi

) d-2p1 zj[1cos w

1+p2z^

2)]

[(l-2^2z2cos w2+^

2z2

) (l-^z^cos w2+(G2 z

2

2)]

Putting A(Z,, Z2

) in the following form

A(ZX

, Z2

) = [(alfz^

1+b1

) (a2z2+b

1) ] [(a

2z2

1+b

2) (a

2z2+b

2) ]

and comparing this equation and above A(Z,Z~)t a,, a2 , b, , b

2

obtains

al j( Jl-p^os w

1+

p1+ V 1 + p-|_cos v^+p^)

a2=7 ( 7 3 "P2cos w

2+Pi

+ V : +f°2

cos w2+l°2

}

|( Jl-p^cos wx

+p x

2- yi +

(

o1cos w1+px

2)

b2=

* (n/

1^003 w2

+,

p 2

2- v/

X +f3

2cos W

2+P2

2)

Let H(Z1

, Z2

) = (a1z~1+b

J) (a

2z~ 1+b

2)

(l-2^1z~ 1cos w

1+/°2z

1

2) d-2/0

2z2

1cos W2+P2

Z2

2

39

then from (2-50)

R (m,n) =uu

f

O)2(l-P1

2) d-P2

2) if n=o, m=0

o if n^O, m^O

u(z 1# z2

)

White Noiseof variance=^2 (l-p1

2)(l-f>

H(Z1;Z2 ) —>-

FL<(m,n)

^'"cos^mXpJwSTiJ.n)

FIGURE 2-5

FILTER TO GENERATE BAND PASS RANDOM FIELD

It is convenient to define

.

U(ZX , Z

2) = Z

1

-1Z2

" 1W(z 1/ z2 )

W(k,£) is also white noise with the same statistics as w(k,£)

Rww (n 'm)=\ ^(l-f^

2) d-f2

2) if n=0 and m=0

if n^O or m^O

Then, the filter has the form

X(Z lf Z2

) = (a1z~ 1+b) ( a

2z2

' 1+ k>2

) Z1

~ 1Z2

~ 1

w(z lf z2 )

(l-2piz~ 1cos

-2 -1Wl+Pl

Zl ^

^ 1 ~ 2P2

Z2

COS W ->"1"P-> Z '> ^

2-2;2T2 Z

2

The following definitions are made:

40

N1(Zlf Z

2)

N2(Z ir Z

2)

-piWv z2

'

Zo-lw(Z , Z )

2 l 2

cos w, +2—^2"

1-2^.2:, """"cos w, + 3, ~,

(a^ 1 + b1)N

2(Z

1 , Z2 )1*1

From the last three definitions one can write the set of

difference equations

\N-^k,*.)

N 9 (k,£)

/

\

\

-Pi'

N1(k /

£-l)

1 If.cos w1

N2(k,£-1)

/SN

1

w(k,£-l)

x-^k,^ = -^>, a1N1 (k,£)+b1N2 (k,il)

Now, additional definitions are made.

M1(Z

1' V = -PIZ2~\ {Z

1'Z2

}

M2(Z

1 , Z2

) =Zl

Xl

(Z 1' Z2 }

l-2^2z2

1 cos w2+/>

2

2z2

-2

From these definitions it follows that

-1X(Zlf Z

2) = (a

2z2

+ b2

) M2

(Z 1# Z2

)

(2 _ 55)

Then

rMx(k,£)

, M (k,£)

(-f/

1 2f> COS w2 J

M^k'-l,*)

M (k-l,Jl)

' (P

x1(k-l,£)

(2-56)X(k,£) = -/

o2^a

2M1(k / I) + b

2M2(k,£)

Combining these all together , the following form is obtained

41

^(k+1,4)^

M2 (k+1,*)

N1(k,£+1)

N2(k / £+l)J

1

lo

"Pi

2p2cos w2

-p,a, b.

-Pi

2plCos^

M^kffcf

M2(k,£)

N1(k,£)

N2(k,£)

f

'"1

UJ

w(k,£)

and

(k,fc) = [-p2

a2

b ]

r

M1(k,4)

M2(k,£)

N^kfJl)

N2 (k,£)J

TVo methodsof modeling have been discussed. Some of the above

examples will be used in later chapters. It should be noted

again that the filter response method cannot be used for the

case where the autocorrelation function is nonseparable.

42

III. ADAPTIVE FILTERS

A. NONRECURSIVE FILTER

1. Introduction

Many forms of adaptive filters have been described in

the literature, some of which have been shown to be optimal

(or suboptimal) in certain applications. The special form

of an adaptive nonrecursive filter developed by Widrow [1]

is reviewed here to give some insights to the recursive adap-

tive filter developed in next section.

The filter to be considered here consists of a tapped

delay line, variable weights whose input signals are the signals

at the delay line taps, a summer to add the weighted signals,

and machinery to adjust the weights automatically. The impulse

response of such a discrete system is completely controlled

by the weights. The adaptation process automatically seeks an

optimal filter impulse response by adjusting the weights.

Two kinds of processes take place in the adaptive filter:

training and operating. The training (adaptation) process is

concerned with adjusting the weights, and the operating process

consists in forming output signals by weighting the delay line

tap signals, using the weights resulting from the training

process. During the training process, an additional input

signal, "the reference (or desired) response," must be supplied

to the adaptive filter along with the usual input signals.

43

This requirement may in some case restrict the use of

this particular form of adaptive filter. An example illustrat-

ing the use of the desired signal is the case of modelling an

unknown system by a discrete adaptive filter as shown in Figure

3-1. Here a discrete input signal x(n) is applied to an un-

known system to be modeled. The discrete adaptive model is

supplied with an input x(n). The output of the unknown system

d(n) is compared with the output y(n) of the adaptive system.

This system can self-adapt to minimize the mean square error,

(throughout this thesis, the mean square error is chosen

as the performance measure), where the error is defined as the

difference between the output of the adaptive model and the

desired signal (for this problem the desired signal is the output

of the unknown system to be modeled)

.

GHHadjustableweights

y(n) FilterfcOutput^

Input

Unknown System

To be modeled

systemoutput

FIGURE 3-1 MODELLING OF UNKNOWN SYSTEM

44

then Ny(n) =

i| w±x (n-i) (3-1)

e(n) = y(n) - d(n) (3-2)

Noting that equation (3-1) is the convolution summation, the

sequence of weights (wi) can be seen asthe impulse response

of the adaptive system.

2The weights w- are adjusted to minimize E(e ) .

It will be shown that if the input and output signals of the

system being modeled are stationary, the error signal has a

mean square value which is a quadratic function of the weight

settings

.

For the minimization of mean square error, the steepest

descent method is used. Throughout this thesis , the terms

"filter coefficient updating process" and "adaptation process" are

used interchangeably and it is assumed that the input to the

adaptive system and the desired signal are stationary random

processes (or random fields for the two-dimensional case) .

2 . Performance Surface, Gradient and the Wiener Solution .

The input signals are weighted and summed to form an

output signal [Equation (3-1) ] . Introducing the vector notation

such that w( n )

=^w '

wl '

w2 * * * W

N^

and X* = [x(n), x(n-l), x(n-2) . . . x(n-*l)] (3-3)

45

Then E quation (3-1), which describes the linear combination

(operating process), can be written in matrix form.

y(n) = "^x = x% (3-4)

and e(n) = y(n) - d(n)

= WTx - d(n)

T.he square of this error is

e2(n) = TFx"x%~ - 2d(n) t\T

Tx + d 2(n)

(3-5)

(3-6)

the mean square error, the expected value of e (n) , is

2,

_

% , -2,_ x ^T«. ^T w (3-7)E[e (n)] = d*(n) - 2W^

d + W- Rxx

where the vector of cross-correlation between the input signals

and the desired response is defined as

E[d(n) X (n) ] = E|d(n)x(n)

d(n)x(n-l) xd

d(n)x(n-N) (3-8)

and where the correlation matrix of the input signal is defined

as

E[X(n)XT

(n) ] = E X(n)X(n) X(n)X(n-l) . . . X(n)X(n-N)

X(n-l)x(n)X(n-l)X(n-l) . . .X(n-2)X (n-N)

X(n-N)X(n-N)

A= R.xx

(3-9)

46

It may be observed from (3-7) that for stationary input signals,

the mean-square error is precisely a second order function of

the weights. The mean square error performance function may be

visualized as a bowl shaped surface, a quadratic function of the

weight variables. Then the adaptive process has the job of con-

tinually seeking the "bottom of the bowl." A means of accomplish-

ing this by the well-known method of steepest descent is dis-

cussed below.

In the non- stationary case, the bottom of the bowl may be

moving, and the orientation and curvature of the bowl may be

changing. The adaptive process has to track the bottom of the

bowl when inputs are nonstationary . The method of steepest

descent uses the gradient of the performance surface in seeking its

minimum. The gradient at any point on the performance surface

may be obtained by differentiating the mean-square error function

of Equation (3-5) with respect to the weight vector. The

gradient is

V{E(e 2(n))}= "2 Rxd + 2 Rxx W (3-10)

To find the "optimal" weight vector WTMq , .i.e. the one that yields the

least mean square error, set the gradient to zero. Accordingly,

Rxd= Rxx WLMS

W = R_1

R (3_11)LMS Rxx Rxd

Equation (3-11) is known as the Wiener-Hopf equation in

matrix form.

47

Then the minimum mean square error may be obtained by

substituting (3-11) into (3-7)

E [e2(n)] min = a"

2(n) - w£

MS Rxd (3-12)

3. LMS Algorithm

In seeking the minimum mean-square error by the

method of steepest descent , one begins with an initial

guess as to where the minimum point of the mean-square error

surface may be. This means that one begins with a set of initial

conditions for the weights. The gradient vector is then mea-

sured, and the next guess is obtained from the present guess

by making a change in the weight vector in the direction of the

negative of the gradient vector. The method of steepest descent

can thus be described by the following relation

W(n+1) = W(n) + kV[E(e 2(n))] (3-13)

2The expression for V[E(e (n) )] is obtained by using

Equation (3-10)

.

W(n+1) = W(n) + 2kRxxW - 2k Rxd (3-14)

2The gradient vector V[E(e (n))] is the gradient of the expectation

of the squared error function when the weight vector is W(n)

.

When the performance function is quadratic, the gradient

is a linear function of the weights. The advantage of working

with the quadratic performance surface lies both in this linear

relation and in the fact that such a surface has a unique-

minimum .

48

The purpose of the adaptation process is to find an exact or

an approximate solution to the Wiener -Hoff equation (3-11)

.

One way of finding the optimum weight vector is simply to solve

(3-10) . Although this solution is generally straight forward,

it could present serious computational problems when the num-

ber of weights N is large and when input data rates are high.

In addition to the necessity of inverting an H x N matrix, this

method may require as many as j\l(N+l)/2 autocorrelation and cross

correlation measurements to be made to obtain the elements of

^x' Rxd*

No perfect solution of equation (3-11) is possible in prac-

tice to estimate perfectly the elements of the correlation

matrices.

A method for finding approximation solutions to (3-11) is

presented below. The accuracy of this method is limited by

statistical sample size, since weight values are found that

are based on finite-time measurements of input-data signals.

This method does not- require explicit measurements of

correlation functions, nor does it require matrix inversion.

It is the "LMS" algorithm based on the steepest descent method.

This algorithm does not even require squaring, averaging, or

differentiation in order to make use of gradients of mean-

square error functions.

When using the LMS algorithm, changes in the weight vector

are made along the direction of the estimated gradient vector.

49

Accordingly

_ Ar 2

W(n+1) = W(n) + kV{ [E(e^(h) )]

}

(3-13)

Where

W(n) = weight vector before adaptation

W(n+1) = weight vector after adaptation

k = scalar constant controlling rate of convergence

A 2,_,„ ^ ,. c „ r _2

and stability (k<o)

V [E (e^ (n) )] = estimate of gradient of Ete^Cn)] with respect

to W with W = W(n)

One method for obtaining the estimated gradient of the

mean square error function is to take the gradient of a single

time sample of the squared error; that is

V[E(e 2(n)] = V[e 2

(n)] = 2e(n) V[e(n)] (3-14)

From Equation (3-4)

V[e(n)] = V[y(n)-d(n)] =V [WT(n) X (n) -d (n)

]

= X(n) (3-15)

Thus,A 2V[E(e z

(n))] = 2e(n)X(n)

= 2[WT(n)X(n)-d(n)]X(n) (3-16)

The gradient estimate of (3-16) is unbiased, as will be shown

by the following argument: For a given weight vector W(n) , the

expected value of the gradient estimate is

:

E{V [E (e2(n))]}= 2E{ [W

T(n) X(n) -d (n) ] X (n)

}

" 2V" 2Rxd < 3 - 17 >

50

Comparing (3-17) and (3-10) , we see that

E{ V[E(e 2(n))] } = V E [e

2(n)

]

(3-18)

and therefore for a given weight vector, the gradient estimateA 2V[E(e (n) ) ] is unbiased.

Using the gradient estimation formula (3-16) , the weight

iteration rule Equation (3-13) becomes

W(n+1) = W(n) + 2k e(n) X (n) (3-19)

and the next weight vector is obtained by adding to the present

weight vector scaled by the value of error. This is the LMS

algorithm. Looking at Equation (3-19) , the adaptation process

is a simple first order recursion equation which can be realized

as shown below.

X(n)

2kX(n)W(n+1) UNIT

DELAY

"W7-JWEIGHTSETTING

e(n)

FIGURE - 3-2 FILTER COEFFICIENT UPDATING

51

Tothis point, the basic concept of LMS nonrecursive adaptive filter

has been introduced and reviewed. More details can be seen in [1] .

Widrow [1] showed that the weight vector mean converges to the

Wiener solution aidthatthe boundsof the step size k should be in the

region such that

1

Amax< k < o for the stability and convergence,

where X is the maximum eigenvalue of Rmax xx

4 . Two-dimensional adaptive filter .

a. Adaptive filter structure

The input-output relation of the two-dimensional

filter is given by two-dimensional convolution.

P *3

y(k,Jt) l E • wi

. x(k-i, £-j), (3-20)i=o j=o J

where y (k, I) is the filter output

and W. . is the fiiiteimpulse response of filter.

Here, it is assumed that the input. X(k,£) is a stationary random

field.

In Equation (3-20) , a set of two-dimensional

stationary input signals is weighted and summed to form an out-

put signal and the filter output is intended to match a desired

(reference) signal in accordance with the minimization of mean

squared error, where the error is the difference between filter

output and desired signal.

Ae(k,£) = y(k,£ ) - d(k,£). (3-21)

52

Introducing the vector notation such that

and

oo ol

WT

% W10

Wll -• W

lqW20

,W ]

pq

X ± = [x(k,JD, x(k,£-l), . . .x(k,i-q)x(k-l,A)x(k-l f A-l) . . .x(k-l,£-q)

x(k-2,£) x(k-p,£ -q)] (3-22)

then Equation (3-20) can be written in matrix form.

y(k,£) = ^Fx = X^W (3-23)

where W is a weight vector of dimension (p+1) (q+l)xl

X is a input signal vector of dimension (p+1) (q+l)xl

The weight vector of the filter is supposed to be adjusted in

the direction such that performance criterion (mean square error)

is to be minimized. Thus, the linear combinatorial system

in Equation (3-20) will be given with variable weights.

x(k,l)

yOt.DHCOM 3-3 LIHEAB COMBINATORIAI- -SYSTEM itf

TWO-DIMENSIONAL ADAPTIVE flLTER

53

In Figure 3-3, the linear combinatorial structure is given. Then,

the adaptive nonrecursive filter can be drawn as following:

r

x(k,*)

LLINEAR COMBINATORIAL

OF FIGURE 3-3

Input IWITH VARIABLE WEIGHTStationary f

RandomInput 7

y(k,£)filter output

3—

V

e(k,£)

FIGURE 3-4 STRUCTURE OF NONRECURSIVE ADAPTIVEFILTER

b. Wiener solution

From equation (3-22) and (3-23), the error signal

can be written by

e(k,Jt) = W^X - d(k,£)

The square of this error is

(3-24)

e2(k,£) = WTXXTW - 2d(k,£) XTW + d

2(k,£) (3-25)

E [e2(k, £) ] =E [d

2(k, £) ] -SR^W^+W^R^W

The expected value of e (k,£) is

(3-26)

where the vector of cross correlations between the input

signal and desired response is defined as

54

'd(k,£)x(k, £)

d(k, £)x(k, £-1)

E[d(k, £)X] = E

d(k, £)x(k, £-q)

d(k, £)x(k-l, £)

d(k,£)x(k-l,£-q)

V d(k,£)x(k-p, £-q)

=ARxd (3-28)

and where the correlation matrix of the input signals is

defined as

E[X3T] =

/xlxl

x l*2 x f3 xlX (p+q) (q+D

X2X 1X2X2

X2X3

X2X (P+D (q+D

x 3'xl X3X2

X3X3 *3 x(p+D (q+D

>

(3-29)

*P+1 ) q+l)Xtl^p+l) q+2)

x2

X(.p tj) q+])

x(pf]} q+|

=RXX , where x(p4])(q+

^x(k-p / £-q)

55

The gradient at any point on the performance surface can be

obtained by differentiating the mean square error function of

equation (6)

V[E(e 2(k,*) )] = 3 E(e

2(k,t)

3 w

"2Rxd+ 2Rxx™

To find the "optimal" weight vector W MS that yield the least

mean square error, set the gradient to zero. Accordingly,

Rxd " VWLMS Rxx

Rxd (3-30)

Equation (7) is the Wiener Hopf equation in matrix form, again

the minimum mean square error is obtained by substituting

(3-30) into (3-26)

.

E[e 2(k,£)] min

= E[d2(k,£)] - W

LMST R

xd (3-31)

c. LMS Algorithm

Consider a two-dimensional field x(k,£) to be processed

(usually two-dimensional filters are used in process-

ingdiscrete two-dimensional image fields) and assume that the

two-dimensional field consists of NxN discrete points (which may be a

sensed signal by NxN pixel elements of a sensor) . The adaptation

processes is that of adjusting the filter coefficient W- • in

56

accordance of minimization of the mean square error as in the one-

dimensional case. The adaptation scheme may be predetermined as

being columnwise scanning or diagonal, or row-wise scanning.

Here the row scanning process is adopted as shown in Figure 3-5.

p • • § m

li •

o •

k-p •*»

Region for jfc^

adaptation.y(k,A) =

p qI Ew

ia(fc-ii*-j)fko o

e(k,i) -

e(1) \ • •

• • •

• • • •

e • •

• •

Region for(j+l)

adaptation

^n

• < »l »

.th

*^-(j+l) th estimation point

estimation point

* •

FIGURE 3-5 ADAPTATION SCHEME

57

Therefore ,theNxNadaptation processes will be required to complete

the filtering of anNxN two-dimensional field. Using e(k,£)

to denote ' the error at the jth iteration (then e(k+l,£) is

the (j+l)th error), then the filter coefficient updating process

can be described by

W(j+1) = W(j) + uV[E(e2(k,£)]

where W(j+1) = coefficient vector after adaptation

W (j) = coefficient vector before adaptation

u = negative scalar constant controlling rate

of convergence and stability.

The gradient of tie mean square error is to be estimated by

V[E(e 2(k,£)] = V[e 2

(k,£)]

where e(k,£) = y(k,£) - d(k,£)

then W(j+1) = W(j) - ue (k, I) V [e(k, I)

]

= W(j) - 2ue(k,£)X

where X is defined by equation (3-22)

.

Along with y(k,l) = W^and e(k.£) = y(k,£) - d{k,l) ,

the LMS algorithm will be completed.

B. RECURSIVE FILTER

1. Introduction

In the previous section, it is shown that adaptive non-

recursive filters have a finite impulse response; that is, they

can produce only zeros with no poles in the filter transfer

function. This limits the capability of transversal adaptive

58

filters in many applications. To overcome this limitation a

new adaptive filter structure is described which is capable of

producing poles in the transfer function. The basic configura-

tion considered here is quite standard; that is, the present

output sample of the filter y(n) is a linear combination of the

present and past samples of the input x(n), x(n-l), . .x(n-M)

and the past samples of the output, y(n-l), y(n-2). . . y(n-N).

The present output sample of the filter is compared against a

reference sample. The resulting error samples are used to adjust

the filter parameters, feed forward gains and feed back gains to

minimize some error function. TheQie-dimensional recursive filter

is developed first, then it is extended to the two-dimensional

adaptive filter.

Recently Feintuck [2] and White [3] have proposed a technique

for making digital filters with zeros and poles adaptive. This

development may enhance the possiblity of obtaining accurate

models for unknown systems. The new approach is developed into an

algorithm. It employs the steepest-descent criterion for para-

meter adjustment but it differs in the estimation of mean

squared error gradient vector from Feintuck [2] and Widrow [1]

.

2 . One-Dimensional Adaptive Recursive Filter

a. Structure

The recursive filter is described by its transfer

function

59

M

Y(Z) i=oZ

bi

Z

X(Z)1 + I aL Z

i=l (3-32)

In the time domain, the input-output relation of the digital

filter is given by

M Ny(n) = I b

±x(n-i)- I a.y(n-i) (3-33)

i=o i=l

where y(n) = nth sample of the filter output

x(n) = nth sample of the filter input

a. = feedback coefficients i = 1/2 f . N

b. = feed forward coefficients i=0,l,2..N

The output samples of the filter are intended to match those of

a reference (or desired) signal d(n). in accordance with

the minimization of some error criterion, the filter parameters

a., b. will be adjusted at every iteration. Thegsneral scheme of the

adaptive recursive structure is given in Figure 3-6. The two

finite length transversal filters areusedinthe forward path and

feedback path to form the recursive filter of Equation (3-33).

60

d(n)

reference signal

Input Signal

x(n)

FIGURE 3-6 ADAPTIVE RECURSIVE LMS FILTER

USING TWO TRANSVERSAL ADAPTIVE

FILTERS

b. Problem in Wiener Solution

Introducing vector notation for the signals and sets

of filter coefficients we have

*-TA - [ a^, a.^, . . . . aN]

ttTB =

[ bo' b

l bM l

X(n) = [ x(n), x(n-l) . ... x(n-M)]

Y(n) T= [ y(n-l) f y(n-2) . . . y(n-N)]

61

Then Equation (3-33) can be written as

y(n) = B^Xdi) -AT

Y(n) (3-34)

where A is the feedback coefficient vector (nxl)

Bis the feed forward coefficient vector [ (M+l) xl]

X (n)is the iiput signal vector at nth iteration [(M+lxl]

Y (n)isths output signal vector at nth iteration (Nxl)

The performance criterion is again minimum mean squared error,

where tte error is the difference between filter output and desired

signal (reference signal)

.

That is, the filter is used to estimate a desired waveform

d(n) in a minimum mean square error sense. Assume that the

observables are stationary and zero mean and let e(n) denote the

error waveform at nth sample, then

e(n) A y(n)-d(n) (3-35)

= BTX(n)-A

TY(n) -d(n)

and the mean square error is

E[e2(n)] = E[(B r x(n)-ATY(n)-d(n) )

2]

= E [BTX (n) X(n)

TB-2BT X(n) Y^n) A+ATY(n)

Y

TA

-2BTd(n)X(n)+2ATd(n)Y(n)+d2(n)

]

= E [d2(n) ] +B

TRxxB+A

TRyyA-2BrRxyA

" 2lTRdx

+2lTRdy < 3 " 36)

where R^ = E[X(n)XT(n)]

Ryy

= E[Y(n)YT (n)]

Rdx= E td ( n )^( n )l

Rdy = E[d(n)Y(n)]

and ^ = E[X(n)YT(n)]

62

The theory of Wiener filtering employs known second-order input

statistics to dictate the impulse response of the linear filter

that minimizes the mean square error; that is, as in the previous

section, the knowledge of second order statistics R, , R is

assumed to calculate the optimum impulse response

(optimum weight vector in nonrecursive adaptive filter) Wn

But here in the recursive algorithm, it is also required that

the autocorrelation of the output, R , and the cross correlation

of the output and the input, R , and the cross correlation of the

output with the desired waveform, R, , should be assumed known.

Thus, the set of statistics mentioned above is assumed to be known

for a moment, and will be used to determine the weights in

the recursive filter. The statistics for the fixed para-

meter filter are not a function of these statistics, but instead

the weights are a function of these statistics. Therefore, R„..,xy

R, and R are to be considered constant when the differentiationax yy

is made with respect to A and B.

The set of weights (filter coefficient vectors) which minimize

the mean square error can be found by getting the gradient vector

with respect to filter parameters equal to zero.

9E(e2(n)]

9 A= 2R A - 2 R 1 + 2 R, =0

yy xy dy

R~ 1

(R, - R "B) (3-37)yy dy xy

and

9 E[e2(n)] A Vr^/ 2

9 B 3[E(e (n))]

= 2R 1 - 2R A - 2 R,xx xy dx

B *xx-l ( Rdx +*xy

X) (3 " 38)

Thus, one can solve for the filter coefficients if all the

second order statistics are known. But without knowing the

impulse response of the filter, the R , R, and R can not ber e xy dy yy

calculated with only input and reference signal statistics.

Noting that we are looking for the impulse response which

minimizes the mean square error in some way, it is clear that

R , Rd , R are not available and so the wiener approach is

not feasible.

C. LMS ALGORITHM.

An iterative gradient search technique (the method of steepest

descent) is revisited. Here, in the recursive algorithm, it

updates the filter coefficients with steps proportional to the

gradient vector. This updating process is

A(n+1) = A(n) + k AA3.

= A(n) + k_VA[E(e

2(n))] (3-39)

B(n+1) = B(n) + kbAB

= B(n) + kbvB [E(e

2(n)]

where

A(n) , B(n) = filter coefficient vectors before adaptation

A(n+1) B(n+1) = filter coefficient vectois after adaptation

k , k, = scalar constants controlling rate of convergence

and stability (k , k\<o)2 a b

gradie

to A and B respectively.

VA [E(e (n) ) ] , V [E(e (n) ) ] = gradient vectors with respect

64

The updating process (3-39) can be considered as afirst order fil-

tering process with an input proportional to the gradient vector.

2 2But the gradients VA.[E (e (n) ) ] and V B [E(e (n) ) ] should be esti-

mated because the output statistics are not available apriori or

an infinite statistical sample would be required to estimate

perfectly the elements of the correlation matrices in Equations

(3-37) and (3-38) . A method of estimating these gradients will

be presented.

Widrow [1] obtained the estimated gradient of the mean square

error function by taking the gradient of a single time sample of

the squared error (instantaneous estimates) when he discussed

the nonrecursive adaptive filter (see previous section)

.

Here, in this thesis work, a new method of estimating

gradients is proposed. This is to approximate the mean squared

2error E(e (n) ] by an average of a finite number of points at every iteration

and take the gradient of this instead of taking the instantaneous

error square, that is, the approximation used is

9 1 L-lE(eMn))~ — I e (n-£) (3-40)

L£ =

2For E(e (n)), the average of the square error for the previous L points

is taken and then gradient is evaluated for the approximate mean

square error. The estimated gradient of mean square error is

V.E[e2(n)] =V,[

Lf1e2(n-£)]A A

£ =

VRE[e2(n)] =V R [

L: 1 e2(n-£)] (3-41)B B

£^0

65

For convenience,the£l) term in (3-4 0) was dropped,

vector notation for error signal

e(n) =f e(n)

e(n-l)

Introducing the

[ e(n-L+l)

then it is seen that the error signal vector is an (Lxl) vector. The

estimated gradient (Equation (3-41) ) can be put into the matrix form

BLl ~ x "' j v

b

Substituting the estimated gradients in Equation (3-39) , the updating

process for the filter coefficients is:

^aVA

VAE[e2(n)] = VA [ e

T(n) J (n)

]

VRE[e2(n)] = V n [ e

T(n) e(n)]

A(n+1) = A(n) + K a V, [£T (n)e(n)]

and

B(n+1) =B~(n) + KbV B [e

T(n)e (n) ] (3-42)

The function e (n) e(n) is a scalar function of the coefficient

vectors A and B, that is,

TT (n)T(n) = f(A, B)

Therefore , by definition, the gradient of f (A, B) with respect

to A and B is

66

21

3A

3f"5"a.

3f3a.

3f

3f3a

N

3B

3fTO

o

3f3b.

3f3bM

It follows that

5A [E(e

2 (n)H = VA [FT(n)i(n)]

*

d (t^ (n)I (n) )—3~a~:

3(FT (n)e(n))3a.

3(£T(n)e(n) ) I

I3aN J

/

27(5) _L*Mal

T 3e (n)

2£ (n)3a.

27(5)9e(n)^a

(3-43)

N /

and

VB (E(e2(n) )) = V

B(£(n)

T£(n) )

3(£(n)T

£ (n)

3b„

3(£(n)T

£(n)3b,

3(£(n)T£(n)

3 bN

r

n)

o2 F(nT 2£^gl

2 F(n)T 9e(n)

3b-

2e(n)T 3e(n)

y v3bN

(3-44)

67

Consider the terms 3e (n) , 3e (n) in equation (3-43) , (3-44) ,

8ar,

9b~p q

where p=l,2,...N and q = 0,1,2,...M.

Since e (n) and e(n) are defined as

e(n) A y(n) - d(n)

— Te (n) = (e(n), e(n-l), . . . e(n-L+l),

3e(n)7a

/3 e(n)~~51

P

3e(n-l)3a

3e(n-L+l)3~£

3y(n)3a

P

3y(n-l)3a_

3y(n-L+l)3a

p=l, 2 , . . .N (3-45)

and

3gpl) _

3ba

r 3e(n)~T5q

3e(n+l)3 bq

3e(n-L+l)3 bq

3y(n)3bq

3y(n-l)3bq

•

3y(n-L+l)

I 3bq

q=0,l,2,...M (3-46)

Note that3e(n) and 3e;(n) are (Lxl) vectors3ap 3bq

and e (n) is an (lxL) vector.

68

Therefore, V^[E(e 2(n))] and V

B[E(e

2(n) )] in Equation (3-41)

are (Nxl) and [(M+l)xl] vectors respectively.

Equations (3-45) and (3-46) may be considered as sensitivity

vectors which tell how much the change in clp and bq affect

the outputs y(n), y(n-l) . . . y(n-L+l)

.

To calculate the elements ofthe estimated gradients of mean square

error in equation (3-41) , we should calculate the sensitivity

vector of equation (3-45) and (3-46) first.

From the recursive equation (3-33)

,

9y(n)a

, M N,

= — I b.X(n-i) - I A.y(n-i)3ap 32lp l i=o

1i=l

X I

= - y(n-p) - I A.ay(n- i}

(3-47)i=i

13 a P

p = 1/ 2, . . . N

lif- - *<»-<> -A^^q = 0, 1, 2, ... M (3-48)

The sensitivity vector components given by Equations (3-47) and

(3-48) can be interpreted as being the response of a linear system

with transfer function.

1 (3-49)H(Z) =

l+a,zX

+ a_ z"2+ . . . aMz-

N

Henceforth fthis will be called. the "sensitivity filter." Equation

(3-49) is ai all pole filter (recursive filter) with input signals

[-y(n-p)] and [x(n-q)] respectively.

69

Now, what are the initial conditions characterizing the re-

cursive relationships of the sensitivity filter?

x(n-q) is the q time units delayed signal of input x(n) to the

adaptive recursive filter of Equation (3-33) and y(n-p) is the p

time units delayed signal of 1he output y(n) of the recursive

filter.

From x(n) = o

y(n) = for n< o ,

x(n-q) and y(n-p) are sequences with the first q elanents and first p

elements zero respectively.And since changes in the ap and bq coeffici-

ents have no effects on the system's response until n=p and n=q

respectively, it follows that the initial conditions are:

gip - o for n=o, 1 ... p-1

a^n) = o for n=o, 1 ... q-1

A summary of this algorithm is

the following

:

1. Calculate the sensitivity vector components through the

sensitivity recursive filter by equations (3-47) and (3-48)

.

2. Calculate the estimated gradient by equation (3-43) and (3-44).

3. Calculate the filter coefficient vector by equation (3-42).

4. Calculate the filter output by equation (3-34)

5. Form the e(n) vector i~(n) , then go back to the 1st step.

Note that due to the fact that the gradient of finite point square

error average is used for the estimation of the true gradient of

mean square, this filter cannot give an optimal solution, but the

70

more averaging points are used; the better performance is expected

It shoutl be noted that if L=l, that is,

VA [E(e2(n))] = VA [e

2(n)]

VB [E(e

2(n))] = V

B [e2(n)] ,

this corresponds to using the instantaneous error square for

estimatingthe gradient, and this filter reduce s to the adaptive

recursive filter proposed by White [3] . If the further approxima-

tion is made that the sensitivity components of equation

(3-47) , (3-48) are

3y(n) / x-4-^

—

- = -y(n-p)3ap * tr '

3y(n)3bq

= x(n-q)

then the estimated gradient is

VA [E(e (n) )] = - 2 e(n) y(n-l)

y(n-2)

">

and

VB [E(e

A(n))] = 2 e (n)

y(n-N)

x(n)

x(n-l)

x(n-N)J

= -2e(n)Y

(3-50)

= 2e(n)X

(3-51)

71

Then the filter coefficient updating process is

A(n+1) = A(n)-2k e(n)Ya.

B(n+1) = B(n)+2kben)X (3-52)

where k , k< and filter output

y(n) = B(n)TX-AT (n)Y (3-53)

Equations (3-53) , (3-52) , (3-51) , (3-50) are exactly the same as

the algorithm proposed by Feintuck [2] . This Feintuck algorithm

has an advantage in simplicity when compared with the algorithms

proposed by White [3] and proposed here which require additional

recursive filters to generate the estimates of the gradient.

Thus, it may be useful to extend the Feintuck algorithm to the

two-dimensional recursive adaptive filter for simplicity. In

the next section, the algorithms proposed by White and Feintuck

are extended to the two-dimensional algorithm.

3 . Two-dimensional Recursive Adaptive Filter

In this section, a mathematical model of the adaptive

recursive filter for the processing of two-dimensional signals

is proposed. This can be considered as an extension of Fein-

tuck's algorithm to two-dimensional filters.

Two transversal filters having the same structure as the

linear combinatorial system used in the non-recursive two-

dimensional processor, are used in the recursive processor,

one for the feedforward path and one for the feedback path.

The two-dimensional recursive filter is described by

its transfer function.

72

y (v z2 ' ,L ? b

ijzr iz

2

"j

1=0 3=0 j

X (z 1/ z„) .12n

M N -in -nL _ mn 1 Z

m=o n=0(m,n)^(0,0)

In the spatial domain, the input-output relation ofthedigital fil-

ter is given by

p q M Ny(k f £) I E b- .x(k-i,£-j) - E Z a v (k-m,£-n)

i=0 j = J n=0 n=0mn

(3-54)

The following notation is introduced

B = [b b n , ....b b, _ b, , . . . .b n b_ ft b J00 01 oq 10 11 lq 20 pq

XT = [x(k,£) ,x(k,£-l) , . .x(k,£-q)x(k-l,£)x(k-l,£-l) . . .x(k-l,£-q)

x(k-2, £ x(k-p, £-q) ]

ip

and A = [a^ a QT . .a^ a1(a ir . .a

1Na20 ^

YT=Cy(k,£-l) , y(k,£-2) .. .y(k,£-N) y (k-l,£ ) y (k-l,£-l) .. .y(k-l,£-N)

y(k-2, £) y(k-M,£-N) J

The filter coefficient vectors ~A~ and ~B~ are [ (M+l) (N+l) -1] xl

and (p+1) (q+l)xl, respectively, and the input-output signal vectors

are again (p+l)(q+l)xl and [ (M+l) (N+l) -l]xl, respectively

.

Then equation (22) can be written as

y(k f £) = BTX-ATY (3-55)

Here, to obtain an estimate of gradients of the mean square error

function, a single sample of the square error is taken. That is:

V[ECe (k,£))] =Vte (k,£) ] =2e (k,£) V [e (k,£ ) ] , and again the

adaptation scheme (filter coefficient updating process) is used

in the same fashion as in the nonrecursive case [see Figure 3-*5] .

73

Denoting the error at jth iteration as e(k,£) then

A(j+1) = A(j) + 2kae(j)VA (e(j)]

B (j+1) = B(j) + 2kbe(j)VA (e(j)]

where

e(k,£) = e(j) = y(k,£) - d(k,£)

(3-56)

(3-57)

The components cfthevectorsv, [e (k,£) ] andv„ [e (k,£> ] can beB

calculated as following.

From Equation (3-57)

VA[e(k,£)] =V

A[y(k,£)]

3y(k,£)3a

01

3y(k,£) 3y(k,£) .

.

3y(k,£) 3 y(k,£)3a

OM3a

10

3y(k,£)

3a

3aMN

1M

(3-58)

3a20

and VB[e(k,£)] =Vg[y(k,£)]

3y(k,l) ... 3y(k,£) 3y(k,£)

oo oq 10

3y(k,£) 3y(k,£)3b"

3y(k,£)3bpq

iq

(3-59)

TB20

Note thatV. [e (k, £) ] and V_ [e (k,£) ] havethe same dimensions as A" and

B, that is:

[ (M+l) (N+D-l] x 1 and [ (p+1) (q+1) ] /respectively

.

From the recursive relation of Equation (3-54)

3y(k,£) ri n \ r v ~y(k-m,£-n)Ut—' y(k-r.t-s) - I I a 3^

'J

-

rs m=o n=o rs

and 3y(k,£) = x(k-u,y-v) - M N

Tb~3 y(k-m,£-n)

uv m=o n=o mn(m,n)^C0 f 0)

uv

(3-60)

(3-61)

74

The recursive relationship of Equations (3-60) and (3-61) should

be noted again, which can be implemented by additional recur-

sive filters.

Forming the instantaneous error gradient of Equations

(3-54) , (3-55) using the output of additional recursive filters

of Equations (3-60) (3-61) , the filter coefficient adaptation

process of Equations (3-56) (3-57) can be performed. Note that

this algorithm corresponds to the two-dimensional version of the

algorithm proposed by White [3]

.

If we make the approximation

^ (k * £) = - y(k-r,£-s)9ars

lVk,Z)

= x(k-u, y-v) ,TEuv

then it follows that

V A [e(k,£)] = ry(k,£-l)

y(k,£-2)

y(k-l,£)

y(k-l,£-l)

y(k-2,£)

A -= Y

y(k-M,£-N) )

75

and VB[e(k

f £)] =(x(k,£)

x(k,A-l)

x(k,A-q)

x(k-l,£)

^x(k-p,£-q)

A _= X

Therefore, in this case, the complete algorithm is described by

Ae(k T £) = e(j) = y(k,£)-d(k,£)

A(j+1) = A(j)-2k e(j) Yd

B(j+1) = B(j) +2kbe(j) X

and Y(k*£) = BTX - AY

This is the two-dimensional version of the algorithm proposed

by Feintuck [2]

.

76

IV. ADAPTIVE NOISE CANCELLER

A. THE CONCEPT OF ADAPTIVE NOISE CANCELLING

Noise cancelling is a variation of optimal filtering that

is highly advantageous in many applications. Specially in

Wiener filtering or Kalman filtering, which are optimal,

apriori knowledge of both signal and noise statistics are

required. Adaptive filters, on the other hand, have the

ability to adjust their own parameters automatically, and their

design requires little or no apriori knowledge of signal and

noise statistics while the Wiener approach utilizes a fixed

parameter filter based on known statistics.

Figure (4-1) shows the basic problem and the adaptive

noise cancelling solution to it. It makes use of a reference

input derived from one (or more) sensors located at the points

in the noise field where the signal is weak or undetectable.

This input is filtered and substracted from a primary input con-

taining both signal and noise. As a result the primary noise

is attenuated or eliminated by cancellation.

At first glance, subtracting noise from a signal seems to

be a dangerous procedure. If done improperly it could result

in an increase in output noise power. If, however, filtering

and subtraction are controlled by an appropriate adaptive

process, noise reduction can be accomplished with little risk

of distorting the signal or increasing the output noise level.

77

Signal

tAdaptive

Filter

SYSTEM

OUTPUT

i

Filter* Output

e (error)

FIGURE 4-1. THE ADAPTIVE NOISE CANCELLING CONCEPT

x(j)

x(k,Jl)

Adaptive

Filter

x(k-61,£-6

2)

*

7reference input

error

FIGURE 4-2 NOISE CANCELLING WITHOUT AN EXTERNAL

REFERENCE SOURCE

78

The following argument for the above is mainly due to

Wldrow, et al [4] . In Figure (4-1) , a signal S is transmitted

over a channel to a sensor that also receives a noise N.o

A second sensor receives a noise

N, uncorrelated with the signal but correlated in some un-

known way with the noise N . In addition to these noises,

additive random noises M and M, uncorrelated with each othero 1

and with S, N and N,are present. Then the reference input

is

d = s+ n + yi (4-1)o o

and the primary input

x = N1+ M± (4 _ 2)

The noise N.+ M is filtered to produce an output y that is as

close a replica as possible of N + M . This output is

subtracted from the reference input S + N + M to produce the

system output

z = S + N + M -yo o *

In other words, the practical objective of the noise cancelling

system is to produce a system output z = S + N + M - y that is

best fit in the least square sense to the signal S. This ob-

jective is accomplished by feeding the system output back to the

adaptive filter and adjusting the filter through the LMS

adaptive algorithm (described in previous chapter) to minimize

total system output power. Note that the system output serves

as the error signal for the adaptive process.

79

Assume for the moment that the noises M and M, do not exist,o 1

then if one knew the characteristics of the channels over

which the noise N is transmitted to the reference input, it

would be possible theoretically to design a fixed filter capable

of changing N. into N . That is, if the correct model of this

transmission channel, H(z), is obtained, the adaptive filter

would be simply 1 , a fixed filter.HTiT

Assume that S, N , N, , M , M. , and y are statistically

stationary and have zero means. Assume that S is uncorrelated

with N and N, ard that ^andM^ are uncorrelated with each other and

with S, N and N, , and suppose that N is correlated with N .

o i 1 o

The output z is

z=S+N +M -y (4-3)o o

squaring, one obtains

z2= S

2+ (N + M - y)

2+ 2S (N + M - y) (4-4)o o J o o

Taking expectations of both sides and realizing that S is un-

correlated with N / M and y, yieldso o * J

E[z2

] = E[S2

] + E[(N + M - y)2

] + 2E[S( N + M - y) ]o o o o

= E[S2

) + E[(NQ

+ Mq- y J

2] (4-5)

2The signal power E [S ] will be unaffected as the filter is ad-

2justed to minimize E[z ]. Accordingly, the minimum output power

is

min E[z2

] = E[S2

] + min E [ (N + MQ

- y )

2] (4-6)

2Since the filter is adjusted so that E(z ) is minimized, therefore

2E [ (NQ + MQ

- y) ] is minimized. The filter output y is then a

80

best least squares estimate of the noise N + M . Moreover,

2 2when E [ (N + M -y) ] is minimized, E[(z-S) ] is also mini-

mized. Since, from (4-3)

z-S = MQ

+ NQ

- y (4-7)

Adjusting or adapting the filter to minimize the total output

power is thus equivalent to causing the output z to be a best

least square estimate of the signal S for a given structure and

adjustability of the adaptive filter and for the given reference

input. The output z will contain the signal S plus noise.

From (4-3) , the output noise is given by (N + M - y) . Since

2 2minimizing the E[z ] minimizes the E [N + M - y) ] , minimizing

the total output power minimizes the output noise power. Since

the signal in the output remains constant, minimizing the total

output power maximizes the output signal to noise ratio. Note

that if E[(N - y)2

] = can be achieved , then E[z2

] = E(S2),

therefore y = N + M and z = S. In this case, minimizing output

power causes the output signal to be perfectly noise free. Also

note that, on- the other hand, when the reference input is com-

pletely uncorrelated with the primary input, the filter will "turn

itself off" and will not increase output noise.

In this case, the filter output y will be uncorrelated with

the primary input. The output power will be

E[z2

] - E[(S+ MQ

+ NQ )

2] - 2E[y(s+N

Q+ M )] + E[y

2]

= E[(S + M + M )

2] + E[y 2

] (4-7)o o

2Therefore, minimizing output power requires that E(y )

be minimized, which is accomplished by making all weights

2zero, bringing E [y ] to zero.

It should be noted that in applying adaptive techniques

to a practical systems problem, the key step lies in providing an

appropriate desired response signal for the adaptation process,

that is, the reference input should be provided through the ap-

propriate scheme, while the exact knowledge of statistical

characteristics are not required. In adaptive modeling applica-

tions, the desired response is generally available as the output

of the unknown system to be modeled. And also in the noise

cancelling scheme above, the reference input is available by

sensing noise which is correlated with the noise at the primary

input in some manner.

In next section, the signal filtering problem is discussed

when no external reference input free of signal is available.

B. NOISE CANCELLING WITHOUT AN EXTERNAL REFERENCE INPUT

This section is concerned with signal filtering

(estimation) a noise-corrupted signal when no external

reference input is available. Here, it is assumed that only the

noise -corrupted signal is available, that is, referring to the

Figure (4-1) of the previous section, the noise free of .signal N^

82

which is correlated with N that corrupted the signal S is not

available; only S + N is available.

It is proposed to estimate the signal S by cancelling the

noise N in some adaptive way. In the following, it is shown

how a reference input can be obtained for the adaptation process

under certain conditions. Assume that the noise corrupted

signal x = S + N is composed of broad band noise N and a narrow

band signal S, then the autocorrelation function of the signal

is broad and that of the noise is narrow. Also assume that noise

N is uncorrelated with the signal and that the mean values of

both signal and noise are zero.

Consider a signal delayed by 6 units,

x(j-6) = S (j-6) + n (j - 6) (4-9)

where 6 is a sufficient number of time units so that the noise

component is decorrelated, but the signal component still

remains correlated.

Then

E[n(j) n (j-6) ] =

E(S(j) S (j-6) ] ? 0, finite (4-10)

For the two-dimensional signal, a signal delayed by &-., 6?

units in the horizontal and vertical direction respectively,

where 6^ and 62

are sufficient length of spatial units such that

the noise field would be decorrelated but the signal field still

83

remains correlated, then

E[N(k,£) N (k-61 ,A-6 2 )] =

E[S(k,£) S (k-6 lt l-62 )] f 0, finite (4-11)

and again it is assumed the signal field and noise field are not

correlated with each other.

Now if this delayed signal is used as a primary input and

the original input used as a reference input to the adaptive filter,

then referring to Figure (4-1) of previous section,

S(j-6) or S(k-6n,& -^-y) can be considered as N-^in Figure (4-1)

and N(j-6) or n(k-<5-i,fl - 62 ) as M^, and S(j) or S(k,£) can be

considered as N and N(j) or N(k,&) as M in Figure (4-2), re- •

spectively

.

From equation (4-10) and the assumptions that the signal and noise

are uncorrelated, it is seen that the assumptions made in the

last section for the various signals holds here.

Therefore, from the argument in section IV-A, , the

filter output would be a good estimate of the signal S . Figure

4-2 shows the noise cancelling (or signal estimation) scheme

xiscussed above.

84

V. EXPERIMENT AND RESULTS

In this chapter, a computer experiment is performed to

check the feasibility of the algorithms derived in Chapter

HT for certain applications. The signal estimation problem

for a noise corrupted signal is treated here for both one-

dimensional and two-dimensional cases. Nonrecursive adaptive

filtering and recursive filtering have been examined and the perfor-

mances of adaptive filters are compared to that of the optimal

Wiener solution. The adaptive noise cancelling scheme is

used for this application.

First, consider a band limited one-dimensional signal S

corrupted by noise N; it is desired to estimate the signal.

If the statistics of both signal and noise are known apriori,

a fixed optimal filter to estimate the signal can be designed

by the Wiener Hopf solution of equation (3-11)

.

Here it is assumed that these statistics are not known

apriori but only that the signal is narrowband and

the noise is a broad band signal and the signal is entirely

uncorrelated with the noise. Then the signal has a wide cor-

relation function while the noise has a narrow correlation

function. Separation of this broadband noise and narrowband

signal is now required for the estimation of the signal.

It is assumed further that thedesired (or reference) signal

which is needed for the adaptive process is not available, that

is, no other possible reference signal is" available whichmay have some

correlation with the signal we want to estimate .

This problem can be considered as an adaptive noise

cancelling problem without reference input. Assuming that the

noise is white, then from the Figure (4-2) one unit delay is

enough to decorrelate the noise component appearing in the adap-

tive filter input from the noise aonponent in the desired signal. These

components will thus appear in the error but not in the filter

output. The narrowband component, on the other hand, will not

be decorrelated by the delay and will appear in the adaptive

filter output.

The input signal would be

x(j) = S(j) + N(j)

where S(j) bandlimited signal

N(j) white noise

and the reference input would be

d(j) = S(j-l) + N(j-l) .

The form of autocorrelation function of the signal is assumed as

R(m) = p

'

m 'cos w m

For the purpose of computer simulation, the following values

are assigned:

p = 0.95

w = 0.025

and the variance of noise is 0.5.

86

For the optimum filter design using the abovevalues ,a transversal

filter having 10 delays was used. From equation (3-11).

W -1LMS = R x R ,

xx xd

The autocorrelation matrix R was computed as2£X

/' 1.50000

\

0.94970 0.90137 0.85496

.94970 1

.93rJ7—0.

0.

0.

o.

85496

81044

70774

72634

6U7o7

65020

61436

500J0

94970

.90137-

8 5496

,8104-.

76 774

.72o84

.to/67

65 02 J

0.9-.970

1.50 000"

-0.94970-

0.901^7

C.85.V6

0.ei044

.0. 76774

0.72684

0.687o7

0.90137

3. 94970"

. 1.50000-

0.94970

0.90137

0.35496

0.81J44

0.76774

0.72084

C.eiC44 C. 76774 0.72 684 0.68767

0.85496 0.81044 0.76774

(T.90137 0.95496 0.81044

0.94970—0.90137 _ 0.95496

1.50}9J 0.9497) 0.90137

C.S4S70 1.50000 0.94^70

0.9C137 0.94970 1.5OOC0

2.95496— 0.90137 0.94970

C. 91044 0.85496 0.90137

0.76774 0.91044

0.72684

0.76774"

0.91044

0.85496

0.90137

0.94970

1.50000

0.94970

0.65020 0.61436

0.S5496 0.90137

0.697*7

"0.7268 4 ~D

0.76774

_0.8U)44^

0.85496

0.9^137

0.9-.97C

1.5000C C

0.9497C 1

.65020

.68767

.72684

.76774

.81044

.85496

.90137

•9497C

•5033C

and the crosscorrelation matrix R , asxd

Rxd1.00000 0.94970 0.90137 C. 65496 0.81044 0.T6774 0.72684 0.68767 0.65020 0.61436

Then the optimum Wiener Hopf solution gives the filter co-

efficients as

W = Wiener Weight-Vector -/

0.33207

0.21191-

0.1347 5

0.0855b

0.05V27"

_ u»03<.34 -

3. 02169

0.01369—

O.C0370~

_ 0.005<>7

87

For simulation of the bandlimited signal, the state and out-

put equations of example 1) in Section 2 of Chapter II are

used.

For the nonrecursive adaptive filter application, again

10 delays and = -0.005 as a step size in LMS algorithm were

used, and 2 delays for both feedforward and feedback path and

= -0.001 for ka , kb (equation (3-42) ), were used in the recur-

sive filter application of both Feintuck's algorithm and the

algorithm developed here. Eight points were used for error

square averaging for the gradient estimation (L=8, in Equation

(3-40) ) . The experimental results are plotted in the following

along with the descriptions and optimal solution for the purpose

of comparison. The results indicate the adaptive recursive

filter appears to perform as well as the optimal Wiener filter

once it reaches a steady state condition.

88

*

V f\

H4

FIGURE R-l SIGNAL STATISTICS

Autocorrelation =0.96 m cos (0.025 m)

89

I

I 'I

I kl HIP

!l

I i li

U. H

FIGURE R-2 NOISE CORRUPTED SIGNAL

NOISE: WHITE: Zero Mean

: Variance = 0.5

90

FIGURE R-3 WIENER-HOPF FILTERING

10 Delays are used.

91

FIGURE R-4 WIDROW'S NONRECURSIVE ADAPTIVE

FILTERING

Number of delays used: 10

Stepsize Used: -0.005

92

FIGURE R-5 FILTERING BY FEINTUCK'S ALGORITHM

NUMBER OF DELAYS IN FEED FORWARD PATH: 2

IN FEED BACK PATH : 2

STEPSIZE USED: -0.001

93

FIGURE R-6 FILTERING BY THE ALGORITHM USING A

FINITE POINT MOVING SQUARE ERROR AVERAGE

For the estimation of gradient

Number of Delays in Feed Forward Path: 2

in Feed Back Path : 2

Stepsize Used: -0.001

94

As a second application/ consider an image sensed by an image

sensing device of anNxN sensing elements array. It is assumed

that this image is composed of correlated background and a

three diagonal line target trajectory. This image may

be interfered withby the internal noise of device (assumed white) .

Then the output image includes three types of processes:

x(k,£) = S(k,£) + T(k,£) + w(k,£)

where

S(k,JL) = correlated background

T(k,£) = Target strength (three diagonal line)

W(k,£) = noise.

Again it is assumed that no statistics are known apriori

and the correlated background is a narrowband signal. It is

further assumed that thecorrelated background and noise are

uncorrelated with each other. It is proposed to separate

the three diagonal lines from the background noise. Again, the

same argument holds that this problem is a two-dimensional noise-

cancelling problem in which no reference is available. It is

further assumed that the correlated background is a band pass

process for which the autocorrelation function is

RSB (m,n) = p ri p v

>

cos w^n cos w^n

where Pv, Py rePresent horizontal and vertical direction cor-

relation coefficients respectively.

95

Going back to Figure (4-2) in the previous section, the delay

z, z~ will be sufficient to decorrelate the white noise

and diagonal line target. Then as a result of filtering, the

system output (or the residual field) will be the desired sig

nal. It should be noted that this residual signal is com-

posed of an estimate of three diagonal lines and white

noise as well as seme granularity due to the fixed stepsize[l].

The problem of enhancing the target diagonal line which

is subjected to the noise (white noise and adaptation noise)

is another problem of interest. It will not be considered

in this work.

For the purpose of simulation, the following values

were used:

1) p = p, = 0.96 w,= w = 0.143J v K h h v

2) p = p, = 0.99 wu = w = 0.143J v h h v

The variance of correlated background = 1.0

White noise variance =0.1

Target diagonal line intensity = 1.8

For the optimal filter design, using above values for p =

p, = 0.96, the Wiener-Hopf solution of Equation (3-30)is:

ffLMS " R

xx-1

Rxd

96

where WLHS;Rxx , Rxd

are defined by equations (3-22), (3-28),

(3-29)}

respectively.

Using p=3, q = 3 in equation (3-20),

Rxx=

1.1 JwviC

—1.97969-

3.94J29

j. 97989

0.96cl9

-0r9*t-»3-

>.9; .29

J. 92138

.1.86414

vi. 979B9

0.979C9

«.94v29

0.97989

-e.96trl*-

i. 9^133

J.S4j29

ii.92iid

J.9t;29 „. 97939 J.96 19 ^.92138 v. 94.. 29 ,-.92138

—.-97 56 9 r.9*-29- ry*7«89 .:.96"1^ 0.92138" —0.94J?O-

l.UJuJ 0.97989 0.5*029 0. 97939 0.96r i S t. 92138

s.. 97*69 i.A.CJ 1.97C39 ..94 29 '.97989 ;.96.il9

L.9>o29 C. 97989 I. KCO'J C. 97989 0.94C29 c. 97989

-V.-97-989 V.9V-29 -*, 97989 1 .lv'-vv -.7.97989 r,.9-r»29-

••96*19 .97989 -.94 29 G.97°89 l.lCtwO 0.97989

. .92x38 u. 96019 0.97989 U.94C29 0.97989 l.lOOCQ

C.94~29 -.92138" '".«£ 19 n. 97989 } .94 '29 ~ V.97989

414.88

.92? 3«

94', 29

,9213*

,9601

9

,9-7-9?'-

.94029

97989

.1" "'.

< Rxd>T=

J

l.j,.J .. .97989 . .9-V-.-29 .97989 •;.«6' 19 ^.92133 0.94029 0.92138 0.83414

(WLMS)w.3<-079

... 28 jJ 3

- . 1194

t.24o3i

~. -,9 84'i

-J.J2430

J.lie29

- .)i334

-3.04117

97

Only the recursive simulation is performed and compared with

the nonrecursive Wiener filter. The simulation results are

shown on the following pages and indicate that although the

optimal performance of the Wiener filter is not achieved,

the adaptive recursive filter performs well.

98

"triTfltttltUmtt* flit*-<»-*l III tint illll II •*

mvw it at*y.m,

* u/*,-..»,.Nf»M<|ft»» It

Itf*•,/*.-. lit it

fN* /•'//'' '{• tuna It*««« H^ IIIPht itirt •!• III 1

1

Altt , til Lit Ittitt^lt, 'II ill Itat utu. t ti IIlltlttU /// U t lit it

nu >ua

a/ 1 f 1 1 1 ai 1 1

1

aiaa a un *.

/ 1 1 a atma i

nttaiiaititt tat tma i

tatt /atin in*lit I il trit in intit lintat at it t taut 1

1

lift

Z**i .i/intituif tint 1

1

i*'l*\t'iita ta,fii*. -"» tttiiai,

ami */

tuttit-

1 1 in tit tit nit 1 1 1mta *

•a

n

L*'tntitiniti:iii.a t . .. fin 'itittn/i. ,

• it --•**» -// •/// / / ta i a i'»../ •,**>, tii'i'tanun i

•

**tn ta tntumt a art /..«/» "Uiuutmutt <*

• / in at in tan -it t a ;/»•" milium i una i •

tana urn it u tr,ana jitinmiin ?rfm a it i it tit t - nun it 1 1 1

1

••- it tt tn it'ii

t

-...«- . i mi it-tt t

tinnit t--i at a-

tit i ittt itt tt i tt *tnnt/^^.^.m'/'i 'iltiltnuuiu t •

n-. •\—.*t»--*t **itttt tinman

i

iiiittittiiit tttiumuauamtin itmanna aa n ti a inn tuiiiitttitnaantitit-innnata a a unittitt nilnan if iniViittuut • i- »/'itmiittnaa • • .t t- -

ttinnui tati - .'•' •

inam it <•*-*• "•'•

Ititti in n "- 'i.i ****titittntttiiit n-t iliitti titnaniL 't nittttiti'-* > > -tinttiittfttt-'t i—*->tttuiiittitutmtu tanuna 'it- t i titait*t «••- / -*nin

i tit ma i an

t n it *t >iitnitn

•it itmnui

unttt ttttt.• at a im ni-n uiiiat.a 'in n 1 1

n

nitm if i •

' 'ft

trimiim-ni'a i tin it-»mian"•mi nit-in ifmi— t-tttiut r

- unit\it imi im 1

1

i mi n a nmi,nit'antnnri m,: a a raim uniuttnint it t ttniimitimtt

i nit im v ttmuumtutmtuiuiiiummiuuumuiiti1 1 a- a 1 1

'• a tnmi i a 1 1 1 tin t in t a n 1 1 • n a i n t n r f tit 1 1a titan n i in— -I a a ' ama .' u: am 1 1 a a n in art; it n irtmatan nun- umutunimiimuniiuuuuu/uin unitta ta t ta it in imi i tin in it ti a at / ana t t n n t- n iinnnit tittf^ni- tiftttmntt'Liaaiumii tttui 1 1tmtu ti

- - ' intuit ittiii 'nunm tt l nun

•i u titan:t i in i

mft -7

_ _ i inil 't in lilt til

lit

fttttnui-mat tt

tit***

titt*-'* 1

tttttn "til UI ainitial

• *t tu it tt tt t i- •mint lit** it tr it if

i intuit: in t tn

i

tun t fttt iftn ntaint

miunuam t

in tinu tana t 'i- •

tta a tunaItutt ittuttatam u u i

intuit inn4*1.41***, '/lit

tn titt man

u

m tnuutuui 1 1nuni 1 1 1 tttn it tun tin uI it lit uu it in

•»-*-u unnt u ua / tn unuutft*nttti uiumuu

•. . </'-* "inn..- Vv. i-tiuu ItllUU I

. ,. •**-timi i uU iiiii u,( '*--ma uttui uu'»' • •mi tutu tint l t in. i r fta , ' a n at » ; • 1 1 -

• - tun a it : i nun a 1 1 a a a 1 1: 1 1 u mi a c-a——*!!! ium i tt t tu mu * inuu i mti um r

*• *t intuitu t •um i *tu i ru mt'ttm*'•> -in i" -t tmiit u - • unit tt*t tun !•••! uu

n

t '• tt" unut u >t it ui t ti-t t a t it'll 1 1m-

tt / // i 'i ti ttttt ut uuttmtti'imi.'-tnu"-i tit tin i tt intuitu 1 1 itun iuna i unit tit •"•

tuitumuittunuiiuuintmii-niumui"-•nun 1 1 1 'i it uu t u t i it t in t u 1 1 1 1 - /mum •

*t'Uffiti- -tit a tin i t-tiuu tint u* u ui tut —u 'tin it t im t uu m u ' 1 1 tu tin t*ium ut ta uum i tt u~ii v —ttit tttu ui 1 1 • inu if iutf uu u uu unit it il t unu ui 1 1 > t 'it iu -1 1 1 n i n 1 1 1mm i utu in n t in- rmmmtt"*

tintut iuumutt nu mi tutu

tttt i

r t .'tt fit /'•-

i

m

ii 1 1w irmumt ft

1 1 tut 1 1

u

It l tut ui "

UI

tt tu i it tiiauntu

tmti

/itutt un

tittmu tt

l Im it i ui nut- innun in

t it in • — •

tit t nirmum it •

it u itnit ) tu it it

j::,'

-/// /// t

if in tittiuu ttin ti tutu in ittint intuit, uta ittttttuuut*n*4.-if4.<t-'i wl

ttt/tut t uit t tt

nutU *tt nun

*t in in. uttiutut i ut tu ...^ti a a tu uuu ii • : uauu u t u t n a .• 1 1 . u n • 1 1 - -* i i >/•tUUUUUUt t nr-uunun utu uttLuan un i -t t u

FIGURE R-7 CORRELATED BACKGROUND + THREE STREAKS +

WHITE NOISE

Background: R(m,n) = . 9^m

' 0. 96^' cos(0.143 m) cos ( 0. 143n)

Variance =1.0

White noise: zero mean

variance = 0.1

Streaks intensity: 1.8

99

• i.-/»/'^*. >/.... .'»»../ ... /..i.; »/././*.../, //.-..///A/,//...,/.,,/..///.//////// ./ /i.i« » i. *,...*>./*,--/ .1 ...a-./ »*/. -/*/•-/./•* / 1-4 // .//// ./• .///»• .///..../. *..../».. • - -a. f ./...•-/»///*•// W„n/a. »/•/// /. III*.***'. /// .// .//.../. II ill ~*. *t *l I . ./ ././/,.*•»*••.,.'•/•// / . //// // ./. /// ,/WWM»/*/-»a*.//./. ///•-/-/» I'/./ / ./ .// /// '1/11*11. ..Iff-/*. t t '/• I- ' 1 *l* II *1 /. / . . W-WM..;./.-/-/,;, ./• i*l/*/ //.//// ,,/...»*. . .//,/....... -a, .•«. **...!!*•./,•.•/ *.•//,//• u t t-n(..•«/.!•/.;. .*/»»». /.*/ »•!••// // ... , i/. ,i/.., //////,/./.,.//•/* .*/•• --/• •////////» 'i»fit*»hii.,..,./>,„/,/ It t t. I /...I '///.././/. / ..-//./ . ././-/.*,.*../// // /- -.t /• - * */..f* f .r /*.4• */ t*t**ik /•/./// \ t~i.t*t i

.. . i. *i. ... t .. . ,t..*..tt..*i+ .*tt. • /in. t, . /• ,.,,., k.«-^.t.».

<•*/*-.////•» .»*. • It*...t .// 7 /** ../ .. .1 t*i. I !•**.. I. I- ft ••// I'll* tl t. • • / */,. ..//WnM.*. a/. . .-//,.,.//, * ... 1*1 /.*.*..,, /. . / •/ /. 1*1^1 . /../../•/ • /....// // * / ..//...///.,/,«,,t t l*>. Unit %*i *•!** t*r. n i t*rl ,//-|f. - /•/// ., /. It'll, II .«• ./../»/•/• M "/ /•/. 7 -ft lift* I* tttlt>W*///.„,,,, /•!/.. •/, / / ./•» .// // /.//.. 1 ,1*1. . It i, ... I ... !../ /*/••/ • • ././/./,./. ,y ,.-.-.-.i<-Wu,.(W..| .1.1,../. I /./•». /•/ /../. /*h7..:. .//./// /// // //././//// I It I UhJ/7 ...*•%*/*/.,/.* .'tr.ll I. 7* II 7 -• 7 • ,11. 11 II. .1*11111* *t • 7* -../•• ./ ////,.//*/•*•/>..;4///„/..«/...,.. /*.*../... . ./ /.//-».. 7// /•/ ...•/ .../•/ pa/. /.» »// 7/// -/-/ . »» •./...> 7hN*.//..//..»/ .1*1.7**11. I : * -•.!/..... /....,.,,,/ //../..*../ /.t* 7, .. 11*1111. /../•.*/—•»•*••• .**7 /../ ../ ./• -.*... »^«. . »*•/. /. ///• .//••* / .// .-.•/... ./../*/.. ,*t,.t. •/ './ / 7 /• /- ••/ /. 7. 77 /•/..••/«wmksnm/-•/././,,/' /«'»/! •//-•//// .77 ...r ./.. •/ -/./,•/«/.///./// /'-/ - ' I .-I I- ,-/•.'• mmMM-tr**tl «7 ••-*//-//-.-•#*--'/- 1111**1. 7 /-I •*-/•/» / /*• / // .//,//. /.*/ r •/-,/—/- «7«- ••••-•-•- i- •*»/••>'•../*./.* a. a. /*'•>*/'/ •».//* • / • /*» -/*..«. t*.*t .*!.., 11 .Illtt.tt* I* /• 7.. -

*••'•>*« -p**/-/*. H4»/-t../-W/w/r///./ **,. . 7. ... .*•.//-*...// . It .'I t 'I. i*i* I' •> /t 'till I I t.. . •..«7 . /r /..*.. ./ 7, a/. . 7r ... *. -.7' • <7'7. /../ .-: . ,111,1 I ./-//7 -7- / 7. . 7 /!«/../ 7> 7*1 h-..«ktt.h*/-H«i</»/»•! -//,». .-».. 7/«' /••••- 7 7*7 /.-.. 7 /•/. / 7///.1 //.«7I .77 7' 7 '//.->••/ f •«.. *r ••*••» *<M//.. »i» !**».. •../../ 7 */-.• /..I "a! II 7 /*. 7/ 17- /»/• ..(,...; '. t-.r. 7 t «>...«.. •:./. -7/i ../>«h/777hd77. »••• |-«**. •/»•/*/. /••/*! 777/ 7/ 7/77 /.-•«-,. ./. ,,/*/... /////»/. */7. » ../-/•/*/ •* .-.-. /•»•» tH«• «/ ««..•»•*«! »/.(*/*/. ../ //•! . *// 7/7^ .//./.-#.. ./ .../. /.../././ /»/ */«••/ 7. -.. ,»/ ./.-/•/...-.-t*--r--i'il+lfi — >' — •/ -• 1 t -I t* -1 I 1 1 1- .7*frr/i/,tr in-rtv/r»7— /7* - 1 * 1 1 1 -- 1~ t r*-r-—•-—*» •-»-»•»///..* ./.«%........ ..». 7. /•///•» • /• . .//- 77/ ,t //•//•*../*../. 7*7/7/ •/ / »-//-////..**** /I K.U'hM,./«^.,../t|.«.»/ •/ ../ k- .*«»*/ .»/^ / „ /.., /. I. , s«^/. ...i/.v /-•/. ».///.,/• 7. t./ 77/ 7t.Mi/7/»VX«/.../*-/-.//•« •// *7 / >./ /-/•* * r 77 ,;*,.//./,. /,;.*////. / -/*..*/-' »7*<». . 7///77**rl/t.*/»H-hrf7. /././/%%/. .1-™*/-/ *« .•!*./ I. I. 1 1 /*.///(/./..«..»..//..-•.••. -••• /- .'7/..-»**»*lt»-tMk:..«

•»•..*////. ///.;//*!/•/.//. .»./.. »...m: /*...../ }. ,' 1 1 lilt lt\ /k*/•'/•///«/ «*«t "•!•,/..•/ /•/ -7/7/ lit III tl/71 -71 I* <* ll-t J IH*/-.,....-/ / V 7 rt/**». ..-// ...*/*/ /// 7 // 7/. ////-//// /.I 7*4

-/,*../«*/ t ///////••• *mr 1*11' it -,/.,-.-. fr- r , /-/ //.i/l • -I ft / >f r'/t /^Ff ' * P" -•>-*>- 1 t 7 ff f\ -t-.il' '.-»«/•* /Mf/.-././•//////# //*»•• II* t t*lt 7 — . /i . / // 1 *..•! .11111 f*l * -77/7 /. , »/ 7*>-Wt 7Stfiiltt-.tti-tlltllf I •. /./•/.//•/./•I //•///,/"// f-*... //../ 7-7 77 77/7/7/77*™/ 7* *>*•*'•/ 7h*»*7»/*/ tti-tt. I* .1 i .. 7 7/ ./../-/*/ ./ ,7 7- ...7, *•-• 77.7 i 7 -7 -7. * //* * 7 7 > * 7*1 ? . 17 1* . * 7 l*m• -. ..7//7*///77. HI. .** *~t /«.-. s», >.. /a. I* *././ /..«//,-/. /.!/, ill 111 I 1 II. I » 7* n.f«»h»t.-i.r«i«l-.^- tl .1 \tl llt*-ti~l I ». .»-.»./..*././«/**/.. ..7 /« It.ntt'i-- /•.*•../•//*/. ..!• .f*tt/t 17 111 l*>m/U /<I>///| //.»./ ^«• •// /*/«.-/.*-••-. •/.,/-/../.,./»...///-••////./// /.-.// ••- r/M.-.> •/•!,»-***/..•»,//«•,/*/./// •.•7*7/77/ 77/. 777*1*1 .-,./*/. .•-*/*//-• /// */• * 1 II • •/ ,.....-.. 4

..... T.."-,. "/,*!. / }. j*i,

'f/* ..7. ...!'//•»- 7' •/*•/**/*;// 7*"7*//.»/./ /7/- ^/ .* /Z/»"7r..»7/)T/-/... ./™/ »»,./*» »/ . // /*-../*//»>/. ,/./.*/.../.....//./. 7 -/ •// * / //./ / /^. . //.*»»,<»4-.. .•7/%77i*./*/ . ... *7—•//•/•.- ...*/.*•/. a ///////»•*»» "// > lilt I 7 • 7* / «--•-•-•— • v -,»47.1 7/ 7 .»•//*/././•/ // .. *«.*/ /. ///.//*//*! •*,.,*/. ^. i */.. /-I-*/ ./ / / •/77-/-77//77*-/7*k.7*777. ..*/./. .- / 7*7*/ -7 *t •»/.... -7. . /*7/../-//*./ W-/7 /* V / 'I /•//// t.lllll I II 11 * 7» I 7/ 7 ,.* «4/..*/.. ./% W .. /.. //*.. 111*1*- .../.#/ w • ./•«» ..t- 1.11.1*1111 t 7 / /*// 7/ ./•/.. 7-//i /hH/*/-.. ///»/•//..«».»../.../. / .!...•••>•. // *../* /..*./ .*,W/***//. • .*-/ f ./.../. 7. . *^ . *---.. ,r,-7777 .7/1*7 ....-»./•..../, i /• 7*/..f ." I ' • ... 1 1 1 1 .. 7 %t. I,It... I..1 • *./•./•./. *, f 1 '*.**/ 1 1**//W/..W..;!ia/..W«» «,../*. ^7/ -» .//. //.-,..»//.••--.-/•/ /. ^ *•-«(' ****//*l.r4*. 7.,*-*// I. I II. II. 111. 11 1* 7.1 /•/ t/. .../•/« /-».*. t*il 1 1 I 11 ' - ••** *7i• ;/»•,»,• .7*..7*.7/^r /; .7.1.7--1. »t ... 7. /. .. /. 7*7 .//. • ,. t.tinnr 7 7.f*.»7*- .* ..- '.•/. -/*«4-7 .-*t*ft*t../-/*/ ••%/« ••« /..;./,«,. 1. .. / /-/. / „...//•». •//-/•/ 7 / / ».».*. t //, 1*1,1. 1.. . //./,-«77. ..../ .*/ 7 ./ .7/ * .//• . .» ./. .-/ 7/ */•»*. * » t.'i', ... WW**////.-* 7. , / / .. /. /•#«./ ... , 7 /./-.//...77 WMl«-/ // .///«.7/*t 7 7 /// ..--#*•/. 77 .7 .•..*/////.'/.../.. .-**A..t ./ * ,./.... /, ; 1 1 1 1 1 1 1 •* \ I'm*a-//....//-/ / ./ / 7 /-/.' 1*11.. I /./•!../ ./••*•//// W/« I**/../ / /* .//-• - // 7. /* 11111*1* •**>.„> ~~~. ,-.t i , ,-< ,-.y } I , , . 1 1 1 1 1 . ~ I • i - , .,.-../.,/.,;., /,///, •'I'll- ** >t ///-•/*-- /-**-! I Hfttttl**-W*.*/* /•«-.//. ,./.•#•/..* /..••//!./ /•* •/*/... .*/.•- -4/ •*...lt'll till * I** •!•*.. *t*-*i.*l !«*.4- ..j/ .„.// 7//. .///*, 77. ///•/-. 7-7 . ./..#«. ,/,../.....-.*-.- 7 /-.// ;/*• *.•• .-.-.-,.. *-•,

77. 7*7 777,- .**../ 7. .. 7/. . .. - . * ,7 . 7 / ./ •• . //. // , .,., . / . /. . . * 7* 7 -7/ 777.» *«* 7* 7 1 7»< //-.,*1...a/,/W7 »7**7 7. *./..../•• 111*7., t .1 I.I t/.*/.,. ./.... */.///.» Iltllt /7.//-.— I-- — ....... ./.an»..«.,..*./-' .1. *H till 1* I til ./,.*./.*,/* •./-# It II -..I *.*... I: I' I" l * 1 1. .-...!• 1*1 if •*-./.•.,... WW• ••7 *.***!, ...!!»* It II f. ./a* *».. / .4...*..//- ,././. /// •# ./• // />/;/•••.../...» /../km7 7 .1. 1*1*1. ... *..!*!. ., . , ./*..../ *• II I III • -•l.-*7/.7/./-*»*/*l.*'-. /••/-. 7-7. */. .,/M.I7.<./y.-//..--.,...,-/.; I*t t. 1 1 It •* •! -'-, -* / 'i..i*m : w , -.-.ft 1 1, If I* I -!*—• I It- i -l-l—/-

—

.-«.t».«p»/I .//... />*»../•«.././/.// .7 /...»/ 77-.« // /..•-••* '**!..*. 1*1 !*%*••!. I. . I. , .•»..„i>MrMk*•.////// .7*/-7- • / // /// II 1 7 I ll> II . I I I . II t I 7**-"l .,' .

•** I tll'lr* 1***1 it ///-» lt*tl**>4-•/./-/-./•/'/ / 7.7/ /,*.-»//•• ..•/ •/// , /*t .....•••. 7* .-../ ./ II* 17*11*. t '*» /ii'4-.7 a// /••.*/./<•// /• /• / //.///»-// I'll tilt**t .»./ .7*7* Wr-. a/./-*/7 •/,»>./,. .///- .y-. ../////. /*• /-././. ../.../*#. * ./ ///// **/t*rttiiti.*t.>i..,-*i* /-/•/ /// %1'llt«•/ *n*tit.. ri./t. i'/ a * */ /// 1 1 1 t*t* 1 1- •*! i*. ii • •'. ,• ,/ - ./.»./ a, *i*i 7i*>rm*-nkM

<lltfit/t*i.t.'!.*/.*> »/ • /•// 7/7/77 7 //'• 7/4 • i * I tT% I . I •% .

•'...•'- -I • i .•*... /i.. I. f / -ttl.t. 1*1 t* I* . «/,.*/ . .'.tltl'tt til 111*1- It. I III* it.*'* .7* '••»! illllt It I. I 1*1.. -/-»*.-/» ///-//•/ ./rr.i ., /.. .11 tt *.**! 71 II. I It* ».//..w I* ***i 'I* ' It. I I •* . * 1 11 /t / -7*.w/-,// 7*4./ .1/ w tlt.*l..t *-•/ » 7.7// /•/.. 1/ '.tt. I' * •/. »»•//»/•« ...#- . .-/ ....... .,. k »,

///.//. I . ' M-.l . AH/ ..//•./'//<*./. «/»«

»t*.i*»j/i.*/..i/»/»/ • ///// //..•/.. / /,./ •/* ./../i ../......-/-/ , - • /* r*.## ifl.ctf iik.«*/>//. Jit+n*,^*,/*ti* > i - t'ti n't ./ . 7/ tti'li t'titt u..i ."/. !//"•-•. '"".J" /. J. F 7. /*//*.*« /aTt -t *..*!* t *»***! I I * . • // // ./.-.••• / •//.////.././/.-././..•-.• / /. •/ */.|./ • /i/i/./wM.< »•/ •//»./. • .it..*. i i .i*r •/ sit. >t'tt i 1 1 1 it ./a./ .. //. 'ttt-t • / •/.. ../ /•</ /..•!. i.

4

*-//>. 77W ./ .. I. **!..*. I, ' I I 1*11 .11 1**1 1*11*1 .//• . Itt-t'f.t t. ,/, */ 4 .//.r .////. >/7--w/././ /.»..//./...,./..../,» •/ /../..../ I. t**.,l ft* .f* t* .limit //! •/*. . / , -'.4 • .• »7/i'»\-i 1 1, ..t 1 1 1 1 1 i »/ - » .-/.-.••/.,'.' ../ /. . */ ,....» /.// / /-//. I > .i** t *i~*J *t\ * i-|, Hrii//i>mi7 »/. / .11**11

*in*. */.,

/ •/»/*// .i

*»/// »«t

.Ji •,4.,-|

. .1 ... /*

(»/ilf l*fl• #.../////* /' tt II . *

t**.m*.i •!*>!•I*t*t 1*1/•/,/•///-•//. ///t ::,-.: •:::

--. ...-„

./'. 7 ••'..;*

1 •>•)* *t*.. I

:

;r;:::

:

'I *-"

.*,.// /rm.'mi!

FIGURE R- 8 resjdom, SIGNAL AFT£R wie(jer filtering ^^^ ^White Noise + Streaks

100

tuuimuummiiitutuuiumtutumiiutmttmiiutuu tun a i i it an 4 u u u 1 i u :nn tintmif-- - - — -- '.'-,/ n «••,«« n-- -.'/" •

lt--l-—tt*lt—f-*tilll—*--**u

tlf*tii'.::'.:

tit. till! *i litit i '• • 1* It*ii in *tfl It • It,tilt

'

liltlilt'till* It \li • It

i tt ft/rn •* .--•••;//»'i ti / * n ./'//*•----///*

/ • / - *rimrrr*'-***r*rr*-/.'/ '/ / it 1 1 •ii -tti"ti *r ttrrm r • -tt •mi-

tt -1 I III I It II * */' •/ /f^/ ////* //// * Uttl tl

r i it itm ** • •••" trrrr**-**um*rrr*f**tm*ifiu'-a — "*u • •' • • mrrrrtt t1 1 nit itt • "it • -i 'i f i /*/*/ *m t n t i**mmum i "i • - -*u "tm**i****ut{l-. i ...— -/ ntrt •*n%r *i *mm-ru i in in m- fit*itm*t*tii ft*

. -t i n*i it tu*ui 1 1 1; 1 1 1 taut*n i it i .// i intuitu • f i 1 1 1 1 1 < t <• " i t-it u 't*i**ta* • nun 'i* i • "i > ' 1 1 in /«/..!/ -ittt in *t t ti 1 1 ii n i '« •• t*- * • * t*i*ruu*xi it** i*it i - fin tit i in • • f'l-ift mm****- • t t tr ' u • turn ti t r mil -m r-rrtr t*t-nit t «•«; ••//• m*- tit tut- /*•%•• •-•/ '/• tu-i t ti •***t*'tt i* ii n «// itn i titu mi *rr-">rrr- in - i/;,..... .... . , |. . •/ >•/// n am* t in- mu*u t> t i i a - •/// tn*i • ti> 1 1 mi-u *;//;* n**uiiuti****--i***-*iti* -lint u • - f- • rtm -**t >nt *'t it a < -t* n tm i/i«cs. • * r i*' 1 1 a atan 1 1 t» mi ii i i '-m i it ft i ii m n.'t *m i

//*/../- rf' + -i-i>*-i*iuui-iri*iti i in u* at tin* •• rr-wnutriu •/

.

-it+tt*m*tumii-i»-ii-it*iiL'- t •nftuuii*m iiiiiu t tt **i -ttuu-im!/• /•«/ •mmi'mtutit *uim*mu m*ii * tat -• a f i tt* m*i *nt n tu * in a atmi a i • t a i rmrrirn ••tani i u *ia \uuu i * < * i*i- *• 1 1 * ft •- utt mt*u*f t* tt m*t i it tin mttraititnnutttittta f-r-u ii*ii f -nt in t< 1 1*- 1* r 1 1nm i /* / t-itim-ttt /• // m <i t- titm tt i*lat'tauiat i -// * i*utii t-t i ittt***i tun mil -• ami • im tarn taiiii irtirrt*lati 'it > -<• 'ti 1 1 i 'i aiaamtata '-i i • it i / // a **ui it *ut :tt it t iu 1 1 ni ?* rittmuati timn . *•* \

*•-• *-t •*! itiiitu tt im- i ufrr*-ia*iiut'iiaamii i ttm nmnrtnit ti tit in in i n tt'i" i -it* •tit t tit- j"i i a u im? ta i* t in n *m r - • t tir t

- j t"mmnma '•ai- 1' * *< i i»* *• "f-ni'-i'm- i m-- 1* <mnm imntm aim ••' •'!» / •<??' -mm ••nt'-i t • i *'••» "it *i-*> -•• a* 1 1 ii ina tm '•• it ' n • i

•' 'i tm tut ftn i • :;• -inmmimii > i- "/ -//// t* --i •a* n tut i tiw m^ in 'in '.•• it • a ami it am •• i in- t- imitn

~tn*ftiiiiitr.tt < nam taa%i in* .//*-- -mi nma 'aa 'in t in -r • t "/;//« •/ inn

tin **am*itt t i tt i -'tint — tmi •//-«•.-;»- •/ n-mnmm tt i t it • ' it rt^mmtmn it -,».... >ii* '•amunii-<*iT4*iituami<*'>i'- * aa-L t • \n / t vt i limitin a i 'it-taitt * t a i ni ia*tm*a t n • - '*- /-- // / lima i t i* • -if -nm tint-m nt itiiitttaii'ii at •tiitf • •a*nt**:*m if*n- -«••• /• /-// am t t *t r tt rm ''ti--—mm-n-—-**+~+*.i~"~t—inriuit-t*»iti+-"fmm-- — — *t tt t* in •/// tin * i-utt 1 1m--*m-it-um i//<,H4....».i*i / - mi ni ttr-am * 'tt * •'!••///• •/ / ta *i until a tiiaia --mum*t - . .^-...* .. , ti tun iia*i*atiui> •rr-"*~"- ••"imr**i *mat a -

• -t* <*t t >mummmt */*tit • >/ -n in timaiati t ,m timi • n- mt/tr/^ rttttit* r ta *t t in -n *ri t>rr+r imi trim-"//m^.«*» .-.«' / H nati*»n.u it u*tm—-• ' mnm tttim ta • im*tmm n itmm < •*

tat u /,'. , uumtan n i-- t *•• + •-> >t turn * tutu -t ia in i*u nm it t-t ••

ttiutiu*ti*uuaa*'*ti'U'tiiatiuiti--f **•*-'imam t • it •! t ; i u t n •i>*mm i **irm •

lf<*i*" i /-/ n -utt/ia •'t**i****tmi itt"---** *n tui'-ii* ••tin- nam : -tt ; it *m •*// //m f • -i '. < n>' *i . *m taatm i- um n+*-*r mi 1 1am it i-i- 1 1- -nimt'm-ut+m-»»ii-*.t*~infiiiitamiiti*tmt*utaauitiutm-*i m m 1111*1111 a tmi*t n* mtir*ir---*<*iiiuiU'inuiittf'titiiuutuuitf 1 t" t*i-mmi m 1 1* -in t uttai /tmm *im* «•-»•••aa im in - inm mm u it ami-am- • *m •mnm 11 >r - / •mmm-im -imi «*»ltft**t*******tai'tmn< \ • 1mmm t* 1 :mi tun 1 tm-- ill m 11 tint i*tmm —. /•*ti9*********atn uui.-'*tttiiiitimm**mtitn ttttai*- 1 tu-.m* ittt*- m*i—">*•••m a-, turn *-um tun >t t—t •mt- uti-i tt it tin t--i •///»• //-*• •• •--inaiuttfun-— **> • nit at 1m -11111mi 11m * 1 a n- \ia 1 • t-t - / " 1 1- 1 mt ?/*/**/-*»•»«.

itutt -in tit m-tti t itm 1 'a 1 1 1- 1'm < 1 **u 11 in titm t* tint t 1 it 1tm tt *'••*'••tat tt -tt -t f.*i 1 t*it*tf tt 1 t*u 1 : 1 : 1 in- ' n -'i *t t?iuimtt tt* na t-t m m-fT'i-'*niui tatf-a 'i—a auttr t- •/ / '/ t'ti '<i***tu it/ 711 tm tt n tm tr ttut mm* -*«/• /•»/,-,.,„.,,- , R , ,,. /-.nt 1 Utiu i tu tm *t •. / / / 1mnt nt it *-t t 1 111 -it 1 1 mi rat 1 1 it 11 ///• -•>*Itt . -;../-/ a r • a '-.tut niaa mannam tan ita > t-t- nut tt* 1 t-m tin mm 1 t-'fiti****'»*****\-<-*~>ititi*i.iut**tf>nntmtfti na • at-* • 1 1 •/ • un i- m mtin*- " 1tn*.*-ti-i-— <i • — at mat u»*tt*'iu- m 111* */ •iti'tit iittt-r-- in- ••* - •• tm u- t*i1 1 *••*•*-* *—*t *~i * 1m ii'* tint -—tun n" "ii-*- ••>!- if 1 •!-•!• ; ;m-umitft-*-'—nn*-**-ttni t'

ti****,**-—• ami u -i ui-uttmmit . t ''!• an ••••-' ./// mum tuna* 1 • iintf—i— miit*-* + t*>iii-t-ft*Ufit**tuitunt n • •!/•/•' ,. > //..// ..,-.; / !'"••-*mm//...,-h - a, i*u* '* i***t:*itit tun -» tivt-f tr tin tt tr -i mn i*-"tn i»im* a 1*//•*».-// 1 1 1*** 1* tut* i t-m-u 1 at i •'•*' / 1 ***•!!• •tutin • 1 1a tana •!• 1 •*•*•• it mi* tu tit* --it- >uu** i* tat* 1um t •nn.tmn tutu -nt tr r- • 1 u t <i*m- it •am a - ;• an* turn nii***-r.'*m***uumutmt*uuu 1 : 111 11 at 1

//•« • -m a 1 1- nit 1 tm 1 tt' it 1mmm 1

1

ti* ***i*-ni i-mim 1 1 u nuuutm 1 11 \-t 1 -11111 tit *t t-ut r *m it 1 •-•-* 1 1-- rtm 1tt*-»t. tun** •mint -mum* mn 11 u*-*u *u> t ~ tun i •*- ** u u ni t-~ u> mi*nm* n- -tii-*-! a -m*'t .'in ttu •uuiuutiiuu" 1

-- -u wtf it 111 -ii* tt* t- it u mimti' u-iuunu**":i' Muuiuiuui u*uut. mt 1 •"///«•/// •//- n tt tu \fi*ii' i- trim " 1 •«•*/* / u*rm 1it--*** »... /- // a 1 it t*i t **u 'i • 1 ti - -t '.

' tun 1 it *'/' 1 1 *"• 1 11 •u'"ii- - - /»» tm tr tr*tt**— > . . t uir 1 a 1 itu*mm 1 u -i*r- 11 11 1 1 1 1 ,• -1um •*u*- it 1 it tmt 1 m*t**i 1 wn u*ti"*******mu t--t laatttmt n tt t-m 1 it- fat' n um* *m ttmmur m-iimitft- 1 luuiu 1 -uii i nan at 1 11 1

— 11 •' -i • nn 1 1 1 1 u . 1 uut* t 11 1 u iim-im-im-

ft , ,. i*"- 1 itu> t , . -11111* mt 1 1 it ti it 11 nt it n -u < ft tu •t.i 1 1' -in 1 um 1 1- tit-**' 1 - 1*a : u**4 ,-.-,/..- / ., //.-«- ./*./««.. 4./ "•.•///////-!>• 1w 'it i-' 1 1*1 imii . imu ••// 1 im u nit *!*>**tt'itu n t,mum in it it. u *'ii -1 'tm ui-*um-u >i t >i mnttam *m*tm 1 1> m*u 1*tu iit*it'*i t'ti tti 1 /- «•• * '•••! tmtui 1 **i 1 am a 1 11 1am 1 1 > • 1 • immm*ir --1**1

•If •llllllll- ""•!!-* 1 un *, mi 1 4 tut* 'Iff lit •"'*•*••?•• **"f*u 'tut*- tt r* *"-***! i' ant i nuuuttr u*4 -tr— mut»-r*

FIGURE R-9 RESIDUAL SIGNAL AFTER RECURSIVE FILTERING

White Noise + Streaks + Adaptation Noise

101

.2. «.-.< t*« iii**'TP<* 7**.'f*Q i:i*«»r(t« :3*'iM! .f*,ur>*< '.'i*'im« •?•- 1 1 1 tt tuta itntrttrf**—'*<"**-—•-*—•'• »tt it mttt mi tttttf\-*

i /t a 1 t tt an I tt/ 1 1 t-/////*** u 1 1 rm u tttt **mtrf tw""-- *

:s i

*•// "

•r« «'»» • '/l":::::::: J

MM4ii* »••••• •••••• < •-! tmi* +" »««»•* .. ./;// .._...»,».. - • < it tutu »//////"-••*...*r—.-i ............ .'. /;/<.-..-.....-.;,•//,'/ ( ....,.- — -in t ttu tit tlfUUttt—**""* + •«»^» •»•"• ••- titiit*-*'**' — *ftt it- - ... •••< i • in 1 1 it ft it i ti it 1 1 1 1 r—>'- •--*•PM<«4t. ...... •»•.-** i 4 itiuttt,* *•>**! iii t tt '-•! .... « mu n. :t it utt in fn-*"-*" •*•"*»•-..-•,--,„, *• K< • ' * tilt "t t ! lilt lltl I /!• "»•"*• * \— '"' ••«..

: m; I III it till It tt 1 1 1 1 1 1 •»•*•"—-1 '•«M4MitN4^4J'l , : 4 .-. -tUH 111 I iU tlttl .'/ /- - «yii.4. v i»» i*, | ./!/ U/tl lit till till ' '- «« 4MWK- .*».-*.

»),4vi<',« » •«( tu tun "• - •«*» i- I It t tit / 1 it tit ; ttl l lilt l•<*• - "•»-•» —•< ...—.. >• tutMM *M '4 fltftfUt/ttUtt* ~ " *• - »-»<<•• '

• lUUlUUUt'lllUUttUU • • )-• — ---«»««• II III t•*ttltt at i it nut tti ut'" •«• -*»-- — 1 1 1 1 1 u 1 1n 1 1 1u it 1 1 1 1 1 1 1 1 1> .---• ••••••**•— tf rut itUlllt tilt It till U tttt t- ' -'» *•• •— •"ltllltl"»**U!llttUlltUl- 11 — "*- • 'limit Iitu 1 1 nu it tiat ttu 1 1- " • i"i i ••• • "" /////... titiutr • i • »»i..»,^ iittiia*//*//»..•.»•-• intuit- '- - ^»..» . -••?///..*..*....... ••*tttlti 1 » irii't**tun****— **•-** tit tit- .... ..

, , j , ,

,

////> • .* /»//.».*tttii*M(2/««iMtH»tM«y/i// i . .... ...» /./.-.,*<»., '/i .-/ // <-•> ** ............ ut ***** t\t lit <*** U ll ill •

..- ..... ..*..«-— ff f'i%tunt it if. .............<-'///».. i ;

tut i»** ....// *//-•-..... -~"ur>"*ttu*u» ///- -* ........... •— if?***** iutit*j'*t*»»**"*tuti 1 • ••• ...... .« •///•-•lb ii/'i; •////*: N"i'i ..... u r ***** *,

Ultt **•*••••*• "It II It »•• ••• '" • t;***-iktit*ti //// • • ••<. *—•• '///'»»•» 5titlit ***f***UlitUt • i

•••»• 4 --mil *UIU — *i- "'IT II ** »• s/tt iti •».........//////•« . ..... • - '/77/.......1 tun •*'• ' • •- -' **/*?•»*» 7t! illl u • > >i titlt tit til • »..»*.. . r- -;///..-..,. Ill It I ->>

I iti »>***4 i>l/miu -f-tn mitt lit tintuitu- *— - *." ''• it tit *» t tun* -!•' - 1 1 ttu u sin lit ittti i mutual •• ••• •••••«•« » >•— mtttt-~+--+-fiutttttt*- i -i- : i -"•!- ••"•nttmiuiLLitu uituui titif"- i— •---p.--.. •• nuuutuiim inmi—* - — -• — i

—- ttmiti 1uutiiuti/ttiitttti'"— -4 »» -•

'mu it, ut tut t mtif - •* -»- • *• u i mitt \umitiuit intuit < !•' ""' , ""'*>" mtiittiiiitttiu.it '''' i • •-•••' M- • ituniul if.u it tunu •*•»*•-*•• • .- ^ i

•--• -•• - it-**- mutual . .i-".-« rut 1 1 ttmizt a *.

,-,,-.. v'tx.^.i^tix" it u ttuunt tin-' •'**•*•*-'•*"•• ***- ' '--v ttiiui r tti 1 1 in in **- •*• •*•» *

uimmtiutiit

-*-•-«-- tint*ttttti

• leituuuuutt/u*'"- *'•"• •"•** — 4 - - - -intiituttumtittti"-*^- "«-' --.-** - .' - • t it tin ut titm , if- •

iiiuttt >"*'iimin--' nm tu •mum tf-"-ittmt 1 1 1 1 1 1- •\-

- u :m it / ui um it-""tttt 111 •»• »!•••// 1 11 1 • ' * - l lUlllt I >tll It 1 1 III- • •

ituiti -nun• '... - tutu Hi'tttm i ft t- -'tint ».».../////-» .••.••.•.--•.- //// ;//r /• • w *'Utitt •••' i

<//•»••• itat-//:/ •

r / / /«

tm ut i u u tuttttttt it n tt intitmtt

-t tmi i utmim tit mi--t it tm ui tuu intm /- »•«"*.-' -• t iiaunt tmm ttu t r- /

w*"»i luiiiiuttiitimutt .r*- • ^- - tttuinu uatimtatti tit it tat tu u unit • i'< - *•• imu tut itnm itiut tt*- .--.••*

( - um mi tt timtimi'i in um it i • \- < .».-.."- .- 'in tum intun iiim t * -.<..-.,- <•••• •mmmu <.U U U l ...-...,/,,,,', ^ /^.^..^,....»,»-«i J--. I Iff It l~..~~**lf t HI till .«4WM.I.H<1.M^-.4. '-11111111 Ifitttu**'**-**>uttutt-- i

* • 't in u*-' •-.••-/////////*•" • i- i tm itmi 1

tttt ttu •»••/// mitt t f i. -tu. —•/////...»"• *timuut"~— \—

—

/mitt it ,ita ft t>* tu ttttit *- • *""•• un t** <*<--*•*-> ut ut tu - *'\ r • • • ttrmt u ?

lilt tit

»

mutt i u --I *•• -• • 1 1 1 1 !• * iimn-" — • <.•.-- .. in mt it *u22i2 tutmu i *•• •////. t». .»•«..»•», ////// .•.-..-- . .. .. — n tut tti imm tt ***im tt tu ui-- .......... 1 1.*. ////.».»•*.. .*.. ..////// - .- "-.4 mm it ;it ttu tu »•/ lutimttt* - i ».... ' •i--«////......... ..././/////' *,.«-. .. fc it ttu a jmUluttllltUfUtUItt*'"* —i«- fc ..^i»- i i*» ./////»••«..«.».//////////.. .-v.^..-..- ^..j.//^/,,/ «_-.tauntmumut it -• •».-..*.'•

—

> •unit ——"tut iumtf ** ittmit <• ..,.,..».... ' • 1 1 1 1 1 , 1 1 1 1 1 1 1 1 1 1m it 1 1 it t it — • •— •'•"'--'* t tin ff

i <

'

..—. n tut t tttt f tut tutu nutu t • »• -—-/ntn itttf "-• •-- - * ttu i u t uu it 1 1 um t tut nm • • *** i' 'tint ;luiiuuu /" '•*- 'ttti iiium fit tituumttitf ' •-.--.-•.---••// Jmi 1 1 atum ••"-**'— •- - "••— -I ii imm tutmi u <• -• -•«.• .// 4

rj 1

mu tu 1 11 1tm 1m if"-- :"*»• 1ttuumuinumii--- -— j

t III it 1 11 1 1 itU 1 1 1 1 1 1 n " ~

•

**r t-utti tutu lift tun if-^".4—i-tiw

FIGURE R-10 Backgrounds + White noise + Streaks

Background: R(m,n) = 0.99m 0.99 n cos (0 . 143m) cos (0.143n)

Variance = 1.0

White noise: zero mean

Variance = 0.1

Streaks Intensity: 1.8

102

»»s?M 13»««?m i2?«5»m* *!)*v?m t."*«rrsi ':'«vf<* ir^*-^T8*

*«*.*/•///• 1*1111

t

// // //;

i**+i*-tttitttttt * i^***»*iltjtt*t.*itt inMM 4i> 'I - -J •#>.•---.

>

—*f*+**+*t-lU-tttlt*»</* •»/*«••//••»> tt

*ir* ///- ttf (III *t

•l"t ft-* «... w, ..J

m*litt**it*lllt I' //-../»•m*j/l *i4t*i *i i*ti tt** *-i4**t"*t*ltU**l*tt >t * .••••-•h.i//<« >(( a* ti t* <

/•-.-•••«"//•/// *' / ! -< U "••*»/**- /.....*..-.,..„—/--.—.J<!«/•".*((// (* IWi ' • •*• *••

itn /r i*i

in it : it 1

1

• •/ *'/ // r**it » *ft I

**lf*lUlt* IIIM*/ 4Mn* s.»NM*/*-* «» (.**•. 1.

-..-../- HfM ,

*t/W« «••-« '1.

till tl tilt t \

tt 1*1 I* 'IIIf tl I III 1*1 '/.

;/•/• tt *t *itii 1 1 1'in /it * -t ' -i • i

at t»ii * -t if • **

»

tt" It ill -J." £H <--

£".'*• —*;;-"*-• *•• •* ,% ti i >'ittt *

p.— .*.,,.. ,, .*}<!* Iltfl >)\ u <

»*«/»-»Hf /»•»"««/••<'-£/****- •////-//*- 2-;./•••..» „.»»•/***-.. f\**U till *

•M' • • ') .-1./- ./.-.«/ '-s. /•/-»/ /" *r» !'<*/ <- ; -v* *<"-///•*• //•»«*" •

' :£i t*i i *•ft *t --i- t**c ? •*!* ti». .. . • i* ..-- -It -It ff tu •it .*- tt ttt*itt-t tt tt -ti - •*> -* s

i*t i/ f//// * *itt n t-t• • 'tit tttt't- ittt -t*

• - »• /z *tttiii i in *-/-// , • itmttini tt ' *••W * t flt't/tt ///•»•

i / t-ttt ii*t t' t tit* >t ttttt'tt •

»

....-•.-•-*-- - t-t t + i-i **tt *~it i it •-/—/-• • •••.. /./ i * it ta tiff wit*,,F„k .../,...M.//. it l* 11 •- III •«/-•t-t >>*••-"// it-/ t*/irtr ft '//'••' .••«••

/ •/' -*i* *if + tt rr t* t /i 1 1 1 r*t * % •«•.".»/» .

.-»v/. i-ttttti* utt' t • i .•/••••• ••/ •ttiit- itttttmm it t ./-*•** ,.*- ff~*!tt "ft t- ll 'tl— "«» -.»•• ••^ f t> ft- tft * (/»»***;"'•».'»'»»• - **t- ti tt ttt- 1*** fit-"** «•••**••'

' •/- /r *//•

WHW Mtttt

rtiit till lit H » * -^ /

"**tr*ttt -tt *tt t *t utt•»'*^^/' «••(-/• .< > t'ti**i*»m ttt^tt itiiiiw«l--l -(/ U till- tt»«ititt** •'tit mil imnif t ttitt* ill til t •

»'.M'.Un..--..:,f •

"1// •/ ****f*/tttit -*

wWIii »/ >l*tll III i•

*ntt<-lili Uiti i *t \t**l-**lt .«K-i-.»-< , -.- <--«^*mi I'll nit* fill * i *

»M««/* /// tt ' lH' *t •

-it I 11 I t /••// / 1 « '

/••/'I lllt'illt -tt .-•....-.-,. i

"I",."'****'•'• ; ' *J"\ I 't'*i'l ' ?»/'""//*••• » «

* I •*•/••—•' tit l) tit' t t * > -l t /.-...-.}"."'.. J...<-.*. •"Iff 11 I 1 1* •#.* //.- '/ *•#./-/• f.-••/,•../.;./,'-//.•/ »-/•• «.«<' *-»/* +t»'*i t

-'/ •'••*/ */ tt IW ,-/.,----....,.. .-• / *../-- / Itt'/l *-•* ttft -••: -•. / l»t-ll* 1

'/**>///*

/" / *•

'-^•J /""• \*****tt*l *- •»-/ f/U* •It-It-,/.. ...,..-./. >-/»•/-//// »..//// ./•/ -*-•!-«• - ' -^ffl'f tli /-I* •<•///' -f -*.. -,^... 1. *. ftt.'tflf f*l It '/ /;.•

—

,-. . n^*t" ti-- ft it- it r • '»'t /><•««/.f •• »"•/ tit- ,it*t 'tit • /• z- •-..*-.\lllll 'ttl'f l flf 11*' •'»•-»- -in-.*

*lt til III *t $

in if i ;

it* i- if tl ?

/ /^ //// ft ttti• f -1**11% titt >itt it t it tin •

ti ttft-'ft ' /• ic • /// <ti>f .

>•*• *»r- ft '

i -ttttr f ti ti ti*it • . t'•*" t-

»•-// ti tut' /—*• ./•-/••« «* «

*t*t* it <i in if ** t i I** \i••*>—

•*»• ill <•* <lltt I *l /*»-«..-.-

-1*1- i*• •! • •»/•I till! I

I •!-

«/r /-' -.-/• // //- < //-*l-flti III 'II 1 1 -

•-

I* ft II -\t I •' I I If t

* f> t * *• /••• Ml •

tt I / • I

f-l If tt*

FIGURE R-ll RESIDUAL SIGNAL AFTER WIENER FILTERING

Figure R-10

White Noise + Streaks

103

Ul.i.?" l« -lit tit tttllttt.J S I t 1 I 1 1 l ' ! ' "If

>?--•* » tll'l//.....-/./•*/\\ vf.-'ii'"//

ttltltllttlltt

mm '/

/ i /// -

•ntti'tiinutn unit '•

III • »f ./ •..•; tin* •• i

tint li inttt .t lit W/iIII */*•'ttltti *l t »•/ f* < •

lit */•*< / **tll*-11**11 H H 1 i itl>ttntiK>l> *t~i 'I

I! ,! *> /If • i

tit-Ill''t lifttl*tt >« It' !*-'tt**t • *^

r/ "/

«

I-*/ <-i

IIHU >i i ,iinrt-""'

'I. iti'utt ,

/,'.*/ •

'< /.*--•

//<// • '.'\\ .--.

.

*i* • r ••/ '!'*"

Vittti* « 1////W F /-/• t '-

211*1 •'.mil tit til -l«- **>/_. l.v

III It/ t.<tt IIIt* *IU. [ I*- tl*<

ttt'l-tH *il> t

If'l <- VV- *.%

.til.-— i it,•t" .'-71

' ltlt it

i

:• -mil

**uf***tl •!W'/ It'

- •• -/;

• // //<f •/// //*t* n<ii * t

'

>•"•//-/

•/ * » / '«

tint-tut i it

i \itiii> i nit ittun

• * lilt t

It'll • /

win-

munniM ;<-••/ i/J**nn it • ft -t*t *

t*tmmt*i ; n- « < •;

in tni^ir ni ••

/,.:',;;.;.j.iti/ni**- i *•; *

*i*tinmt* • • ••'-'" « •;; '/'tut in % i\ ""':,'**'' ;'.. 3j.J

• HUll

It it lity.'.y.:-

t:.::.

Ill 1*1 /•"" tt nMi.« / • / / tt >i •/i «wt<« * •'•-' "////// •!

. , ..-.» «• //» / •*•/* •

1 '.'".'* / i»/.»••• •//-/ / *tn **

**ni *r****•£*' * '-i'< i) . -it*inn > k

**•' /•« ././/;/...•|**»Z»*

j -//.*/•//•*• . *** ;•• i--/j' *w i*i->//• fitti** /..,/••

./"/,** -*t t */ < /•/•/•?/-»Wh .-/./I / •/ •///

;*/•!/<i^;:.^:;^;:^!;

£* « -

1

'

•mm•uimi

£'//»; <<j/ 'i/inri • /*///•ii tr uuniii 'i it >t r ni'i•mn tt

1 u*-lf / ut u »-//// '-<mn-

1

11 til l

.

ti tiut*>ni>> '« "/• ;*/'1 1 1 1 it n ./ttiinnrirun// »./ tnnitinr-nun ftiumuin- T*"""r/ttiti :'/''/.'•-';}/-• :

/tunun i• *t*u ""' '' '.'

*u it f // •/ • fttl

•ututin •lilt,MMi' *i +t -ttt iinni/«-^/ •

n~t>- ti litu *i-t w >//nut2|ii*il tuntin ll/l'.tin-/ run • tin • tinit**i*ti*+nhnn u in

m/i tutIL-.lt mt • tn -

-Jt*li-n-*ut *mt t- ii it •tt*t i *n u \+.i i /

ttt*U • it /i •' it- •t ' f t

U+iv-i t iti i •/

ll*iit • it*t Ml \l \t ' fin t*^ t • 1

1

tut <t *uit tlltl1IAl>"'

'. i- '^ ..... l> ~t~u

111' t*/ m*ufi•/•*

/* //

•/-- / *u 'unnu

.,/ i* m- i/"-/-///• rtr-nn*rr r

••>+iim**U'»iruu* /ti:•ut •* nnm

t*It*/•/i

' uti

•Uf'IU• run***'• in m-ii

• rum*'• nt a*n- n1*1III

•-!/// i .;*» . ./ w ./ t i/ •/

/ / / • / rti /mm-/ un.ttn/t

- iiui> i u *

./// ///U 11*11r tint i

' -

tt t/i/it/ • if

FIGURE R-12 RESIDUAL SIGNAL AFTER ADAPTIVE RECURSIVE FILTERING

White Noise + Streaks + Adaptation Noise

104

VI. CONCLUSIONS

The study described herein has developed a new algorithm

for the one-dimensional signal filtering problem and extends

this to two-dimensional processing. It is an adaptive recur-

sive filtering algorithm based on the steepest descent gradient

method which employs the finite point square error for the*

gradient estimation rather than instantaneous square error.

A simplified two-dimensional version of this algorithm

is developed. It is designed to estimate the signal in real-

time operation in cases where the statistics of both signal

and corrupting noise are not available apriori. The algorithm

learns the statistics and adapts even though it is not optimal,

which means that it seeks the minimum of the error criterion.

It should be noted that Widrow's nonrecursive adaptive filtering

algorithm gives the global minimum of performance criterion due

to the fact that for the stationary process, the mean square

error is the quadratic form of weight vectors, but for the re-

cursive adaptive filter, local minima may be found instead of the

global minimum. The computer simulation shows that for the examples

considered here the algorithms presented learn the

statistics of signal and adapt. Several points can be observed

through the experimental results [see Figure R-l through R-12]

.

1) All the algorithms presented here give a satisfactory

result after the transients die out even though they are

not optimal.

105

2) The algorithm which employs the finite point average

square error for the gradient estimates gives more

rapid convergence than Feintuck's algorithm. The

possible reasons may be due to the fact that the output

information is fed back and used for the filter coef-

ficients updating process and the sensitivity information

propagates through the recursive equation as the iteration

proceeds, while Feintuck's algorithm discards the sensi-

tivity information.

3) The algorithm developed here gives the best results among

the various algorithms presented at the expense of

complex hardware. Note that the lEquired number of addi-

tional sensitivity filters (equations (3-47), (3-48))

would be the number of filter coefficients, and due to the

L point averaging process, additional storage elements are

also needed. The possible reason for the good results

may be due to the fact that the averaging process [equation

(3-4 0)] for the gradient estimate gives a smaller

error between true gradient and estimated

gradients than the gradient estimate based on instantaneous

square error does}while both give unbiased gradient

estimates

.

Due to the emerging interest in adaptive recursive

filters, further research on this subject may be worthwhile.

The following are left open for further research;

106

1) Comparison of steady state performance of the recur-

sive adaptive filter with the Kalman filtering tech-

nique. It would lead to a better understanding of the

performance of the recursive filter to express the

filter coefficients9

in terms of steady-state Kalman

filter gains.

2) Mathematical derivation of the bound in step size of

the filter coefficient updating process for convergence

and stability. It is believed that this bound may be at

tainedby setting up the constraints first such that the

value of performance criterion decreases mono-

tomically to a minimum as the iteration progresses.

3

)

Modification of the algorithms for the case that . partial

statistics of signal or noise are available apriori.

4

)

Derivation of the algorithm based on a different per-

formance criterion such as maximum likelihood ratio,

maximum signal to noise ratio, etc.

5) Derivation of the algorithm based on the different mini-

mization techniques such as Newton's method or Fletcher-

Powell methods, etc., for a given performance criterion.

107

V

LIST OF REFERENCES

Widrow, B., "Adaptive Filters," Aspects of Network andSystem Theory, R. E. Kalman and N. DeClaris, Eds., NewYork, Holt, Rinehart and Winston, 1970.

Feintuck, P. L., "An Adaptive Recursive LMS Filter,"Proceedings of IEEE . November 1976.

White, S. A., "An Adaptive Recursive Filter," Proceed-ings 9th Asilomar Conference on Circuits, Systems andComputers , November 1975.

Widrow, B., et al, "Adaptive Noise Cancelling: Principlesand Applications," Proceedings of IEEE , December 1975.

Blake, Ian F., and Thomas, John B., "The Linear RandomProcess," Proceedings of IEEE, October 1968.

Makhoul , John, "Linear Prediction: A Tutorial Review,"Proceedings of IEEE , April 1975.

7. Papoulis, A., "Probability, Random Variables and Sto-chastic Process," New York, McGraw-Hill, 1965.

8. Weiner, N. , "The Extrapolation, Interpolation andSmoothing of Stationary Time Series with EngineeringApplications," John Wiley and Sons, Inc. New York,1949.

9. Kalman, R. E., "A New Approach to Linear Filtering onPrediction Problems," ASME Trans., J. Basic Eng. ,

Vol. 83, March 1960, pp. 34-35.

108

INITIAL DISTRIBUTION LIST

No. Copies

1. Defense Documentation Center 2

Cameron StationAlexandria, Virginia 22314

2. Library, Code 0142 2

Naval Postgraduate SchoolMonterey, California 93940

3. Department Chairman, Code 62 1

Department of Electrical EngineeringNaval Postgraduate SchoolMonterey, California 93940

4. Professor Sydney R. Parker, Code 62Px(Thesis Advisor) 1

Department of Electrical EngineeringNaval Postgraduate SchoolMonterey, California 93940

5. Lieutenant Soon-Ju Ko , ROKN 2

732 -1, 1-DO 2-DONGCHEJU , CHEJUDO 590 KOREA

6. Professor Eli I. Jury 1Department of Electrical Engineering andComputer ScienceUniversity of CaliforniaBerkeley, CA 94720

7. Professor Sanjit K. Mitra 1Department of Electrical Engineeringand Computer ScienceUniversity of CaliforniaSanta Barbara, CA 93106

8. Division of Personal Education and Training 2

Republic of Korea Naval HeadquartersDAEBANG-DONG YOUNGDEUNGPO-KUSEOUL 715-18, KOREA

9. Baruch, Even-or 1SMC 2190Naval Postgraduate SchoolMonterey, CA 93940

10. Pereira, Jose 1SMC 2610Naval Postgraduate SchoolMonterey, CA 93940

109

Thesi

K716c.l

Thesis

K71625

c.l

Ko

fig«o

si\2 ???pt

1 •>

V55ve f// ter

Ve r«cor^

OCT"

i 4 785

An adaptive recur-sive filter.

thesK71625

An adaptive recursive filter.

11111111 i; ii iiII II

I

III 111 III I

llllIII llll ill nilhi i ii 1 ill hi : ill i iii II

3 2768 001 03042 2DUDLEY KNOX LIBRARY