Linear Convolution Using FFT

Another useful property is that we can perform circular convolution and see how many points remain the same as those of linear convolution. When P < L and an L-point circular convolution is performed, the first (P−1) points are ‘corrupted’ by circulation. The remaining points (ie. The last L−P+1 points)

are not corrupted (i.e., they remain the same as the linear convolution result).

See below

Linear Convolution Using FFT

If 𝑥𝑥1[𝑛𝑛] is lengh-L and 𝑥𝑥2[𝑛𝑛] is lengh-P, P<L. Perform L-point circular convolution,

So, if 𝑛𝑛 ≥ 𝑃𝑃, That is, the result is the same as linear convolution when 𝑛𝑛 ≥ 𝑃𝑃.

)for 0][ (because ]mod)[(][

]mod)[(][][][][

21

1

02

1

1

02213

PmmxLmnxmx

Nmnxmxnxnxnx

P

m

L

m

>=−=

−=⊗=

∑

∑−

=

−

=

][][][][ 1

1

0221 mnxmxnxnx

L

m−=⊗ ∑

−

=

Overlapping-save method (for implementing an FIR filter)

In sum, when performing L-point circular convolution: if an L-point sequence is circularly convolved with a P-point sequence (P<L), the first (P−1) points of the result are incorrect, while the remaining points are identical to those that obtained by linear convolution.

Overlapping-save method

(for implementing an FIR filter)

Separating x[n] as overlapping segments of length L, so that each segment overlaps the preceding section by (P−1) points. The r-th segment is

Perform circular convolution for each segment, then remove the first (P-1) points obtained.

10 ],1)1([][ −≤≤+−+−+= LnPPLrnxnxr

Example of overlapping-save method

Decompose x[n] into overlapping sections of length L

Example of overlapping-save method (continue)

Result of circularly convolving each section with h[n]. The portions of each filter section to be discarded in forming the linear convolution are indicated

Discrete-time Systems

A transformation or operator that maps an input sequence with values x[n] into an output sequence with value y[n] .

y[n] = T{x[n]}

x[n] T{⋅} y[n]

Linear System

• A linear system is referred to as a system y[n]=T{x[n]} satisfying the following property for all α, β.

• A linear system employing only the current and finite previous inputs is referred to as a kind of (Finite Impulse Response) FIR system or FIR filter.

• General form:

Example: running average filters

• 3-point running average filter:

• 7-point running average filter

Time-invariant System • time-invariant system is referred to a system

y[n]=T{x[n]} satisfying

for all n0. That is, when the input is delayed (shifted) by n0, the output is delayed by the same amount.

Linear Time-invariant (LTI) System

• It can be easily verified that the FIR filter of the form

is a time-invariant system. • A system that is both linear and time-invariant is

referred to as an LTI system. – So, FIR filter is an LTI system.

IIR System or IIR Filter

• IIR filter: Linear system using both current and finite previous inputs as well as previous outputs.

• General form:

• Recursion: In IIR filter, previous outputs have been used recursively.

General form of LTI system

• It can be easily verified that IIR filter is also an LTI system.

• We have the definition of LTI system, but what is the general form of an LTI system?

• To answer this question, let us investigate what happens when we input the simplest signal, the delta function, into an LTI system.

• Recall: delta function (unit impulse) in discrete-time domain:

Impulse Response

• When taking the unit impulse δ[n] as input to an LTI system, the output h[n] is called the impulse response of this LTI system.

• Recall: Any discrete-time signal x[n] can be represented as the linear combination of delayed unit-impulse functions:

[ ] [ ] [ ]∑∞

−∞=

−=k

knkxnx δ

Impulse Response

• Then, we have the following property for the impulse response of an LTI system:

• So [ ] [ ][ ]

ky n x k h n k

∞

=−∞

= −∑

[ ] [ ] [ ]

[ ] [ ]{ }∑

∑∞

−∞=

∞

−∞=

−=

−=

k

k

knTkx

knkxTny

δ

δ

by Time-invariance

by Linearity

Impulse Response

• The operation is convolution. Hence, when we know the impulse response h[n] of a linear system, then the output signal can be completely determined by the input signal and the impulse response via the convolution.

[ ] [ ][ ]k

y n x k h n k∞

=−∞

= −∑

Knowing an LTI system

Knowing its impulse response

equivalent to

Response of LTI System

• Property • Output of an LTI system is the convolution of the input

sequence and the impulse response of the LTI system. • Convolution is commutative and associative:

[ ] [ ][ ] [ ] [ ]k

y n x n h n x k h n k∞

=−∞

= ∗ = −∑

Equivalent Systems • Hence, we have the following equivalent systems:

Equivalent Systems

h1[n] h2[n] h3[n]

x[n] y[n]

h1[n]∗h2[n]∗ h3[n]

x[n] y[n]

Convolution and Equivalent System

• Property • Convolution is also distributive over addition:

x[n] ∗ (h1[n] + h2[n]) = x[n] ∗ h1[n] + x[n] ∗ h2[n].

Ideal delay: h[n] = δ[n-nd] Moving average

Accumulator

Forward difference: h[n] = δ[n+1]−δ[n] Backward difference: h[n] = δ[n]−δ[n−1]

Impulse Responses of Some LTI Systems

[ ]

≤≤−

++=otherwise0

11

2121

MnMMMnh

[ ] ≥

=otherwise0

01 nnh

An LTI system can be realized in different ways by separating it into different subsystems.

The following systems are equivalent:

Cascading Systems

[ ] [ ] [ ]( ) [ ][ ] [ ] [ ]( )[ ] [ ]1

1111

−−=−+∗−=

−∗−+=

nnnnn

nnnnh

δδδδδδδδ

Another example of cascading systems – inverse system.

Equivalent Cascading Systems

[ ] [ ] [ ] [ ]( ) [ ] [ ] [ ]nnununnnunh δδδ =−−=−−∗= 11

Causality

• A system is causal if it does not dependent on future inputs.

• That is, if x1[n]=x2[n] when n<n0, then the output y1[n]=y2[n] when n<n0 for all n0. – or equivalently, the output y[n0] depends only the

input sequence values for n ≤ n0. • What is the property of the impulse response of

a causal system? Property: An LTI system is causal if and only if

h[n] = 0 for all n < 0.

General Form of LTI Systems

General LTI System

Causal LTI System

FIR Filter (namely, Finite Impulse Response) IIR Filter (namely, Infinite Impulse Response)

[ ] [ ][ ]k

y n h k x n k∞

=−∞

= −∑

[ ] [ ]0

[ ]k

y n h k x n k∞

=

= −∑

[ ] [ ]0

[ ]M

ky n h k x n k

=

= −∑

[ ] [ ] [ ] [ ]0 1

[ ]M N

k ky n a k x n k b k y n k

= =

= − + −∑ ∑

Difference Equation • From the above, we can find that both FIR and IIR

filters are causal LTI Systems. • They are both difference equations. • Difference equation: (variables are functions. y: unknown function; x: given)

• Both FIR and IIR filters are solutions of difference equations.

• IIR filter is the case when a0=1. Difference equation is merely a more general form.

[ ] [ ]∑∑==

−=−M

mm

N

kk mnxbknya

00

Illustration: (infinite-long) Matrix × Vector

• A General LTI system is of the form:

• Can be explained as a infinite-dimensional matrix/vector product

[ ] [ ][ ]k

y n h k x n k∞

=−∞

= −∑

[ 3] [0] [ 1] [ 2] [ 3] [ 4] [ 5][ 2] [1] [0] [ 1] [ 2] [ 3] [ 4][ 1] [2] [1] [0] [ 1] [ 2] [ 3][0] [3] [2] [1] [0] [ 1] [ 2][1] [4] [3] [2] [1] [0[2]

y h h h h h hy h h h h h hy h h h h h hy h h h h h hy h h h h hy

− − − − − −

− − − − − − − − − = − −

[ 3][ 2][ 1][0]

] [ 1] [1][5] [4] [3] [2] [1] [0] [2]

xxxx

h xh h h h h h x

−

− − −

What Happens in Frequency Domain?

• What is the influence in Frequency domain when input a discrete-time signal x[n] to an LTI system of impulse response h[n]?

• To investigate the influence, consider that an LTI system can be characterized by an infinite-dimensional matrix as illustrated above.

• Conceptually, this matrix should have “infinite-long” eigenvectors.

• Eigen function: A discrete-time signal is called eigenfunction if it satisfies the following property: When applying it as input to a system, the output is the same function multiplied by a (complex) constant, i.e., eigenvalue.

Eigenfunction of LTI System

Property: x[n] = ejwn is the eigenfunction of all LTI systems (w∈R)

Pf: Let h[n] be the impulse response of an LTI system, when ejwn is applied as the input,

Let

we can see

[ ] [ ] ( ) [ ]∑∑∞

−∞=

−∞

−∞=

− ==k

jwkjwn

k

knjw ekheekhny

( ) [ ]∑∞

−∞=

−=k

jwkjw ekheH

[ ] ( ) jwnjw eeHny =

Frequency Response

Hence, ejwn is the eigenfunction of the LTI system; The associated ‘eigenvalue’ is H(ejw). We call H(ejw) the LTI system’s frequency response

The frequency response is a complex function consisting of the real and imaginary parts, H(ejw) = HR(ejw) + jHI(ejw)

Physical meaning: When input is a signal with frequency w (i.e., ejwn)), the output is a signal of the same frequency w, but the magnitude and phase could be changed (characterized by the complex amplitude H(ejw))

Frequency Response

Another explanation of frequency response • Recall the discrete-time Fourier transform

(DTFT) pair:

• Forward DTFT

• Inverse DTFT

Frequency Response • By definition, the frequency response:

is just the DTFT of the impulse response h[n].

• This is an important reason why we need DTFT for discrete-time signal analysis, besides the use of continuous Fourier transform.

( ) [ ]∑∞

−∞=

−=k

jwkjw ekheH

Impulse Response

Frequency Response

DTFT

Inverse DTFT

LTI System and Frequency Response

• Now, let’s go back to the problem: When input a signal x[n] to an LTI system, what happens in the frequency domain?

• Remember that the output of an LTI system is y[n]=x[n]*h[n], the convolution of x[n] and h[n].

• In DTFT, we still have the convolution theorem: time domain convolution is equivalent to frequency domain multiplication.

• Hence, the output in the frequency domain is the multiplication,

Y(ejw) = X(ejw)H(ejw)

Multiplication in Frequency Domain

• In sum, the output sequence’s spectrum is the multiplication of the input spectrum and the frequency response.

Knowing an LTI system

Knowing its impulse response

equivalent to

Knowing its frequency response

equivalent to equivalent to

• Sinusoidal responses of LTI systems:

– The response of x1[n] and x2[n] are

– The total response is y[n] = y1[n] + y2[n]

Application Example of Eigenfunctions

[ ] ( )( )0 0( )2 / 2jw j w ny n H e A e ϕ− − +=

[ ] ( ) [ ] [ ]nxnxeeAeeAnwAnx njwjnjwj210

00

22+≡+=+= −− φφφcos

[ ] ( )( )0 0( )1 / 2jw j w ny n H e A e ϕ+=

Impulse response of the moving-average system is

Therefore, the frequency response is

Example: Frequency Response of the Moving-average System

[ ]

≤≤−

++=otherwise0

11

2121

MnMMMnh

[ ] ∑−=

−

++=

2

11

1

21

M

Mn

jwnjw eMM

eH

The following is the geometric series formula:

So

Geometric Series Formula

01

, 1

m m nk n k

k n kn m

m n

α α α

α αα

−

= =

+

=

−= >

−

∑ ∑

1

0

1 ,1

LLk

k

ααα

+

=

−=

−∑

Frequency Response of the Moving-average System

( )

( ) ( )( )

( ) ( )( )

2

1

21

1 2 1 22 1

1 2 1 22 1

1 2

1

1 2

1 /2 1 /21 /2

1 2

1 /2 1 /2/2

/2 /21 2

11

11 1

11 1

11

Mjw jwn

n M

jw MjwM

jw

jw M M jw M Mjw M M

jw

jw M M jw M Mjw M M

jw jw

H e eM M

e eM M e

e e eM M e

e e eM M e e

−

=−

− +

−

+ + − + +− − +

−

+ + − + +− −

−

= + +

−=

+ + −

−=

+ + −

−=

+ + −

∑

Frequency Response of the Moving-average System

( ) ( )( )

( )[ ]

( )

( ) ( )

1 2 1 22 1

2 1

1 /2 1 /2/2

/2 /21 2

1 2 /2

1 2

11

sin 1 / 211 sin / 2

expjw

jw M M jw M Mjw M Mjw

jw jw

jw M M

j H ejw

e eH e eM M e e

w M Me

M M w

H e

+ + − + +− −

−

− −

∠

− = + + −

+ + =+ +

=

(magnitude and phase)

Further evaluation

Spectrum of the Moving-average System (M1 = 0 and M2 = 4)

Recall that, in DTFT, the frequency response is repeated with period 2π. “High frequency” is close to ±π.

Amplitude response

Phase response

Example: Discrete-time Ideal Low-pass Filter

1, | |( )

0, | |cjw

lowpassc

w wH e

w w π<

= < <

• Ideal low-pass filter in DTFT domain

• Note that we usually depict the frequency response in the range [-π,π] only, and bearing in mind that the spectrum is repeated with the period 2π.

Ideal Low-pass Filter in DTFT domain

Discrete-time Ideal Low-pass Filter

• The impulse response hlowpass[n] can be found by inverse Fourier transform:

1[ ]2

1 1| ( )2 2sin

c

c

c c c

c

w jwnlowpass w

w jw n jw njwnw

c

h n e dw

e e ejn jnw nn

π

π π

π

−

−−

=

= = −

=

∫

Uniform samples of Sinc function

Approximation of Discrete-time Ideal Low-pass Filter

• In other words, the forward DTFT of the sampled sync function recovers the ideal low-pass filter:

• However, as analyzed before, the sampled sync

function cannot be realized by difference equation because it reaches infinity on both ends.

• To approximate the ideal low-pass filter , a method is to use the partial sum instead

sin( )M

jw jwncM

n M

w nH e enπ

−

=−

= ∑

sin( )jw jwnclowpass

n

w nH e enπ

∞−

=−∞

= ∑

Approximation of Discrete-time Ideal Low-pass Filter

• Examples of M=1, 3, 7, 19

The “low frequencies” are frequencies close to zero, while the “high frequencies” are those close to ±π. Since that the frequencies differing by an integer multiple

of 2π are indistinguishable, the “low frequency” are those that are close to an even multiple of π, while the “high frequencies” are those close to an odd multiple of π.

Ideal frequency-selective filters: An important class of linear-invariant systems includes

those systems for which the frequency response is unity over a certain range of frequencies and is zero at the remaining frequencies.

Ideal Frequency-selective Filters

Frequency Response of Ideal Low-pass Filter

Frequency Response of Ideal High-pass Filter

Frequency Response of Ideal Band-stop Filter

Frequency Response of Ideal Band-pass Filter