Upload
others
View
19
Download
0
Embed Size (px)
Citation preview
26
CHAPTER – 3
ANALYSIS OF ADAPTIVE
FILTERS AND LMS
ALGORITHM
27
CHAPTER – 3
CHAPTER – 3: ANALYSIS OF ADAPTIVE FILTERS AND
LMS ALGORITHM
S.No. Name of the Sub-Title Page No.
3.1 Introduction 28
3.2 Adaptive Filtering Problem 29
3.3 Filter Structures 30
3.3.1 Finite Impulse Response Filter 31
3.3.2 Infinite Impulse Response Filter 34
3.3.3 Properties of FIR Filter 36
3.4 Task of an Adaptive Filter 37
3.5 Introduction to LMS Algorithm 38
3.6 Overview of Adaptive LMS Algorithm 40
3.7 DA based Approach for Inner Product Computation 41
28
3. ANALYSIS OF ADAPTIVE FILTERS AND LMS
ALGORITHM
3.1 INTRODUCTION
Basically, an adaptive filter is a computational device which is
used to realize a relation between the two signals in an iterative
manner in real time. Adaptive filters are often realized by running the
set of program instructions in devices such as DSP chip or as a
microprocessor or set of logic operations implemented in the field
programmable gate array (FPGA) or in a semicustom VLSI integrated
circuit or custom VLSI integrated circuit. By neglecting the errors due
to numerical precision effects in the implementation, the adaptive
filter’s primary job is characterized and physical realization is takes
place. That’s why the realization of the mathematical form of the
adaptive filter in software and hardware is opposed.
We can define the adaptive filter with the help of four aspects
which are explained below
1. The type of signals that a filter can process.
2. The architecture that defines the relationship between the
input signal and the output signal of the filter.
3. The parameters of this architecture that can be changed
iteratively to influence the filter’s input-output relationship.
4. Finally, the adaptive algorithm followed that decides how the
parameters are adjusted from one time instant to the next.
29
The parameters that can be adjusted in adaptive filters are
specified by the selection of the adaptive filter structure. The
algorithm that is used for the updating of the parameter values may
have innumerable of forms. Among this, optimization procedure is
used which minimizes the error creation that is useful for the
particular task at hand.
The general mathematical representation of the adaptive
filtering problem explains the operation of the adaptive filter. With an
overview of the advantages of adaptive filtering, they are used
successfully and widely in several applications. The simple
mathematical derivation of the least-mean- square algorithm which is
most popularly used method for the adjustment of the coefficients of
the adaptive filter is explained in this chapter.
3.2 THE ADAPTIVE FILTERING PROBLEM
The figure 3.1 shows the problem occurred in the adaptive filter
diagram. A input sample x(n) from the digital input is fed to the device,
called an adaptive filter which produces an output signal y(n) based
on the input output relationship which is nothing but the physical
architecture of the filter. Adaptive filters contains some adjustable
parameters and these values decide how y(n) is computed. An error
signal e(n) is generated by taking the difference between the desired
signal d(n) and the actual obtained output signal.
30
e(n) = d(n) - y(n) (3.1)
By using the error signal, we will produce adapts or alters that
are used to update the parameters of the filter from n time to (n+1)
time. As the time index is increased from n, it is hoping that the
output of the filter is better matched with the desired signal by the
adaption process. So that the error between the two signals is
decreased over the time.
Figure 3.1 Block Diagram of Adaptive Filter
3.3 FILTER STRUCTURES
In adaptive filters y(n) is computed from the input x(n) will be
affected by number of parameters. These parameters are defined as
coefficient vector w(n) (this is also called as the weight vector).
W(n) = [w0(n), w1(n),……., wN-1(n)] (3.2)
31
Where {wi(n)}, 0 ≤ I ≤ L-1 contains the L parameters of the
system at n time, by this definition, general I/O relation of the
adaptive filter is as follows:
y(n) = f(w(n), y(n-1), y(n-2),…….y(n-N), x(n), x(n-1),…….,x(n-M+1)) (3.3)
Where the equation 3.3 represents linear or non linear function
with M and N are positive integers. Implicitly this definition realizes a
causal filter, in which there is no necessity for the future values of x(n)
to compute the output y(n).
The general equation of the adaptive filter is given by the above
equation. In this structure main intention is to make the linear
relationship between the input and desired signal. This type of
relationship is obtained by finite impulse response (FIR) filter
structures or infinite impulse-response (IIR) filter. The direct form FIR
filters also known as transversal filter or a tapped-delay-line filter.
3.3.1 Finite Impulse Response Filter
A finite impulse response (FIR) filter is used for the
implementation of any kind of frequency response by digital means.
Usually, a series of multipliers, delays and adders are used for the
implementation of FIR filters to obtain the filter output.
The simple block diagram of an FIR filter of length N is shown in
figure.3.2., where the delay results because of the operations on prior
32
input samples. The wi values are the coefficients used for
multiplication, so that the output at time n is the summation of all the
delayed samples multiplied by the appropriate coefficients.
The general equation that defines the output of a normal FIR
filter in terms of its input vector is given by:
y[n] = w0 x[n] + w1 x[n-1] +.......+ wNx[n-N] (3.4)
● x[n] is the input signal,
● y[n] is the obtained output signal,
● wi are the filter weights and
● N is the order of the filter.
Generally, an Nth-order filter will have (N + 1) terms on the right-hand
side which are commonly referred as taps.
The above equation 3.4 can also be expressed as a convolution
result of the coefficient sequence wi with given the input vector:
y n = x n-i
L-1
i=0
wi n
= WT(n) X(n) (3.5)
33
That is, the filter output is a weighted sum of the current and a
finite number of previous values of the input.
Figure: 3.2 Structure of an FIR filter
The process of selecting the filter's length and coefficients is
called filter design. The aim is to set those parameters properly such
that certain desired stop band and pass band parameters will result
from running the filter.
The direct form configuration of the FIR filter contains a long
critical path. Why, Because of the inner product present in the filter
output path. The construction of the multiplier needs more hardware
and area occupied by this is more. In order to increase the throughput
of the filter it is necessary to reduce the critical path of the filter.
When the input sampling rate is high by making the critical path less
than the sampling period we can increase the throughput of the filter.
34
3.3.2 Infinite Impulse Response Filter
The direct form IIR filter structure is shown below. The output
of the this filter can be represented as
y n = y n-i ai
N
i=1
n + x n-j bj n
N
j=0
(3.6)
The block diagram will not be represented in this form. That can
be represented in vector form as
y(n) = WT(n) U(n)
W(n) = [ a1(n), a2(n) …… aN(n) b0(n), b1(n),…….bN(N) ]T
U(n) = [ y(n-1), y(n-2) …. y(n-N) x(n), x(n-1) …… x(n-N) ]T
To compute the output y(n) of the IIR filter involves fixed
number of adders, multipliers and delay elements unlike from the
direct form FIR structure.
This structure of the IIR filter is called as the lattice filter, which
is the third order structure of the IIR filter. This is a FIR filter
structure which contains L-1 preprocessing stages used to compute
the set of auxiliary values known as backward prediction error values
represented as {bi(n)}, 0≤ I ≤L-1. These are uncorrelated signals, which
represent the parameters of x(n) through the linear transformation.
So, the backward prediction error values are used in place of the
35
delayed inputs in a structure similar to that of FIR filter. The
convergence performance is improved because of the uncorrelated
nature of the prediction errors to the adaptive filter coefficients, when
the algorithm selection is appropriate.
Figure: 3.3 Structure of the IIR filter
The computational complexity is the sensitive issue regarding
the selection of the adaptive filter structure. In the real time the
operations of the adaptive filter typically occur. All the calculations of
the filter occur during in the one sample period tine. The structures
discussed above are useful, because the output signal y(n) can be
computed by using the simple arithmetic operations and finite
amount of memory.
When the parameter values are fixed, then we can use nonlinear
systems in addition to the linear systems discussed above for which
36
the principle of the superposition does not hold. Such systems may be
useful even when the relationship between the d(n) and x(n) is not
linear. In the field of neural networks many nonlinear models are
developed.
3.3.3 Properties of FIR filter
The reason for our interest on adaptive FIR filter structure is
because of the properties discussed below. These lines prove the
advantage of FIR filter compared to the IIR filter in the adaptive
filtering design. An FIR filter has a number of useful properties which
sometimes make it preferable to an infinite impulse response (IIR)
which are discussed below. FIR filters:
● Are stable by nature. Because, all the poles are located at the
origin and hence are located within the unit circle, which is the
reason for their inherent stability.
● Require no feedback. This is nothing but, any rounding errors
are not compounded by summed iteration and the same relative
error will occur in each calculation which makes its
implementation simpler.
● Can be designed to be linear in phase. Linearity in phase is
nothing but the phase change will be proportional to the
frequency which is usually a desired characteristic for phase-
37
sensitive applications such as crossover filters, and mastering,
where transparent filtering is adequate.
The main drawback of FIR filters is that it requires considerably
more computation power compared to a similar IIR filter. This is
especially true in the case of low frequency applications where the low
frequencies are affected by the filter.
3.4 TASK OF AN ADAPTIVE FILTER
While discussing the adaptive filters, the common doubt that
raises in most of the minds is nothing but ”If we had already known
the desired signal, then what is the need for finding out the output
signal that matches exactly with the desired signal?” The process of
equating the actual output y(n) to desired output d(n) with some
system yields the adaptive filtering task. By considering the following
issues relate to many adaptive filtering problems.
Generally, the point of interest in all the cases may not be the
same. i.e., we may not be interested in the desired signal always
and our interest may be to represent output signal y(n) in terms
of d(n) that is contains the input signal x(n). In some cases we
may be interested in the parameter values W(n) exclusively.
In some situations where d(n) is not available at all times. When
d(n) is unavailable, we will try to estimate the desired response
38
signal d(n), we will use our most-recent error based parameter
to compute y(n).
There may be some cases in which d(n) is never available. In
such cases, one can use a “hypothetical” d(n), based on its
amplitude characteristics and its predicted statistical behavior.
There is a method called blind adaptation algorithms in which
suitable estimates of d(n) are formed from the signals available
to the adaptive filter. Such algorithms even work which could
be considered as a tribute both to the developers who initiated
the algorithms and to the technological maturity that has
improved in the field of the adaptive filtering.
In general, the relationship between x(n) and d(n) will vary from
time to time. In those situations, with the help of adaptive filtering
algorithms, we will try to change or update the parameter values to
follow the changes in the relationship as “encoded” by the two
sequences of the signal. This behavior is commonly referred as
tracking.
3.5 INTRODUCTION TO LMS ALGORITHM
In high performance systems such as digital signal processors
(DSP), microprocessors, FIR filters etc. uses barrel shifters,
multipliers, multiplexers, delay elements as key components.
Performance and sequentially of the multiplier determines the
system’s performance. Multipliers are very slowest and most area
39
consuming elements in the system. Hence, optimization of speed and
area are major constrains in the design of multiplier. But these two
are conflicting constrains so improving in speed mostly increases the
area.
Adaptive digital filters are used widely in many digital signal
processing (DSP) applications like signal de-noising, sonar signal
processing, system identification, channel equalization for
communications, clutter rejection in radars, acoustic echo
cancellation and networking systems interference etc., because of
their linearity and adaptability to the changes in the signals they
process. In order to satisfy this requirement, we can make use of a
finite-impulse-response (FIR) filter because of its stable nature whose
coefficients are updated by the well known Widrow-Hoff least mean
square (LMS) algorithm, which is well known for its simplicity and
better convergence performance.
In direct form FIR filters an inner product computation present
in the forward path gives long critical path to obtain a filter output.
Therefore, it’s necessary to reduce the critical path if the input signal
has a high sampling rate by this critical path never exceed the
sampling period.
In most of the cases, we need a high performance adaptive filter
in order to reduce the system clock rate which eventually results in
low power consumption because the sampling frequencies for digital
40
processing of these signals are close to the system clock frequencies
[27].
Distributed arithmetic (DA) is one of the efficient techniques for
realization of higher order filters as it can achieve high throughput
without the help of hardware multiplier. It was first introduced by
Croisier and later developed by Peled and Liu [31].
3.6 OVERVIEW OF ADAPTIVE LMS ALGORITHM
This section gives a brief review of the LMS adaptive filter
algorithm and followed by the description of the proposing DA-based
technique for adaptive filter [7]. The least mean square (LMS)
algorithm computes an error and a filter output in each and every
cycle which is equal to the difference between the desired response
and the current filter output. The filter weights are updated in every
training cycle, for this the estimated error value is used. In the LMS
adaptive filters the weights of the nth iteration are updated by the
following equations:
w(n+1) = w(n) + µ.e(n).x(n) (3.7)
Where
e(n)=d(n)-y(n)
y(n)=wqT(n).x(n)
x(n) is the input vector and w(n) is weight vector at the nth
training iteration they are given by as follows:
41
x(n)=[x(n),x(n-1),………, x(n-N+1)]T
w(n)=[w0(n), w1(n),……., wN-1(n)]
d(n) represents the desired response, N is the filter length
y(n) represents the filter output at the nth iteration
e(n) is the error computed at the nth iteration, and this error signal
e(n) is used to update the weights, μ is the convergence factor
Generally, the pipelined architectures uses the delayed error
signal e(n-m) for updating the current weight instead of using the
recent error value. The feedback error signal e(n) may be available
after certain number of clock cycles, which is called the “adaptation
delay”, here m is considered as adaptation delay. The weight-update
equation of the pipelined LMS adaptive filter is given as follows:
w(n+1) = w(n) + µ.e(n-m).x(n-m) (3.8)
3.7 DA-BASED APPROACH FOR INNER-PRODUCT COMPUTATION
DA is basically a bit-serial computational operation that
performs an inner product computation. The advantage of DA is its
efficiency of mechanization. The LMS adaptive filter, needs to perform
an inner product computation in each cycle, the most of the critical
path is contributed by this. DA inner product generation considers the
calculation of the following sum of products:
y = wkxk
N-1
k=0
(3.9)
42
Figure: 3.4 Multiply and Accumulation Unit
Figure 3.4 shows the normal multiply and accumulation
diagram without using the distributed arithmetic concept. Where A, B,
C, D are considered as inputs and C0, C1, C2, C3 are considered as
weights.
In the equation wk denotes the weights of the filter and xk
denotes the inputs of the filter. If we Consider bit width of the weight
is L, each component in the weight vector may be represented by two’s
complementation form, such that │wk│<1, then we may express wk as
wk=-wk+ 𝑤𝑘𝑙2-l
L-1
l=1
(3.10)
Where the wkn are the bits, 0 or 1, wk0 is the sign bit, and wk,N-1
is the least significant bit (LSB). Now by combining the equations 3.9
and 3.10 the expression y in terms of the bits of wkl are shown below:
𝑦 = 𝑥𝑘 −𝑤𝑘0 + 𝑤𝑘𝑖 2−𝑙
𝐿−1
𝑙=1
𝑁−1
𝑘=0
(3.11)
43
Equation 3.11 is the conventional form of expressing the inner
product. Direct form of mechanization of this equation defines a
“lumped” arithmetic computation. Let us instead interchange the
order of the summations, which gives us:
y=- xk .
N-1
k=0
wk0 + xk
N-1
k=0
. wkl
N-1
l=1
2-l (3.12)
Figure: 3.5 MAC circuit for DA
Figure: 3.6 ROM construction in the conventional four point inner
product block
44
After the bit level rearrangement of the multiply and
accumulation the circuit design is modified as above. Equation 3.12
defines a distributed arithmetic computation. Consider the bracketed
term in the above equation:
yl = xk .
N-1
k=0
wkl (3.13)
Because each wkn may take on the values of 0 and 1 only, we
may pre-compute the values and store in a ROM. The input data can
be used to directly access the address of the memory and result can
be dropped into accumulator. After N such cycles, the memory will
contain the result. In the diagram where the ROM portion comes is
shown in figure 3.7.
Figure: 3.7 Conventional DA Based Four Point Inner Product Block