Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier...

Wavelets & Wavelet Algorithms

Vladimir Kulyukin

www.vkedco.blogspot.comwww.vkedco.blogspot.com

1D Discrete Fourier Transform&

Inverse Discrete Fourier Transform

Outline

● Review● Complex Numbers & Euler's Identity● Nth Roots of Unity● Roots of Unity & Complex Sinusoids● Discrete Fourier Transform (DFT) & Inverse DFT● DFT & Inverse DFT as Matrix Multiplication

Review

Sinusoids

(radians). phase ousinstantane theis

(radians); phase initial theis

seconds; 0.01 1/100 is sampleeach ofduration thethus

cond,samples/se 100 means 100Hz e.g. (Hz);frequency theis

(sec); timeis

(rad/sec);frequency radian theis

amplitude;peak negative-non theis

constants. are ,, variable;realt independenan is

.sin:form theoffunction a is sinusoidA

Harmonic Function Forms

tAtxtbtatx

122 tan,

where,sinsincos

Definition of Function Orthogonality

dxxgxfxgxf 0 if orthogonal are , Functions

Orthogonality of Basic Trigonometric System

.2, interval over the

0 is system tric trigonomebasic theof functionsdifferent

any two of integral that theshow formulasn integratio The

,...sin,cos,...,2sin,2cos,sin,cos,1

functions ofset infinite theis system tric trigonomebasic The

nxnxxxxx

Fourier Coefficients

,...3,2,1,sin

1ndxnxxfbn

,...3,2,1,cos

1ndxnxxfan

Fourier Series

. of seriesFourier the

called is sincos2

series tric trigonomeThe

,...3,2,1,sin1

and cos1

where,sincos2

:expansion series

tric trigonomefollowing thehas and 2 period offunction a is If

kxbkxaa

nnxxfbnxxfa

kxbkxaa

1D Fourier Analysis Algorithm

1D Fourier Analysis

● Given a 1D data array, determine the frequency range [W

lower, W

upper]

● Compute the cosine and sine coefficients for each frequency value in the frequency range

● Once the sine and cosine coefficients are computed, determine the amplitude and phase for every constituent harmonic

● Optional: If the signal function is known, recombine the reconstructed harmonics and compute how closely their sum approximates the signal function

1D Fourier Analysis: Harmonic Recovery

harmonicth - therepresents ,, tuple-3 The

;under indexed

isit that socontainer map ain ,, Save

{in every For

., e.g., range,frequency a be Let

.:001.0: e.g., interval, timea be Let

function. signal some from valuesofarray 1D a be Let

upperlower

atabaa

kaabaa

tdataabtdataaa

00000 sinsincos llllllll batbta

uuuuuuuu batbta sinsincos 22

upperlower WW ,

Frequency Reconstructed Harmonics

Example

0.001. of incrementsin dconstructe ,on of analysisFourier 1D Do

.57.30,98.7,73.1,32.12,5.0,5.4,42

,20sin573030cos987

20sin73120cos3212

10sin5010cos544

3322110

bababaa

t.t.π

Solution Steps

● Generate 1D data (this step is unnecessary if the data array is given)

● Approximate sine & cosine coefficients● Reconstruct 10th, 20th, and 30th harmonics● Combine reconstructed harmonics to reconstruct

the original sinusoid● Compute approximate error b/w reconstructed &

original sinusoids

Plotting Computed Cosine Coeffs

>> cosine_coeff_map(0)=1.5886

>> cosine_coeff_map(30) = 38.0054

Plotting Computed Sine Coeffs

>> sine_coeff_map(10)=0.5000

Complex Numbers &

Euler's Identity

Square Roots of Negative Numbers

.551515 :Example

,0number negativeany for Then .1Let

Exponents of J

,...3,2,1,0for ,

.1 of powers some through cycle usLet .1Let

22/122

Complex Numbers: Imaginary & Real Parts

.0 becausenumber complex a is number realEvery

.; Formally,

. ofpart imaginary theis and ofpart real theis and reals are

, where, as written becan number complex Every

yzimxzre

yxjyxzz

Complex Plane

Derivative of f(x) = ax & Number e

function. continuous a is .0,1

Consider

limlimlim

have welimit, theof definitionBy real. is and real positive a is where,Let

aaxfxfxf

Derivative of f(x) = ax & Number e

lims,other wordIn .1limsuch that

32number someexist must there,continuous is Since

lim3lim )2

and 112

lim2lim )1

thatlly verifycomputionacan We

What is e?

.1 lim

Therefore, .1

,0 asThen

lim shown thatjust have We

What Is e Equal To?

smaller. andsmaller makingby

2after digits decimal ofnumber any computecan We

....02874759045235367182818284.21 lim /1

Derivative of ex

. s,other wordIn

lim because ,1

limlim

any for Then .let and Let

eexfxfxf

xeaaxf

Euler's Identity

.1sincos then , If

.sincos:Identity sEuler'

.sincosThen .Let

jrrzjyxz

Rectangular & Polar Form of Complex Numbers

form.polar theis

form.r rectangula theis

.sincossincosThen

.sincos:identity sEuler'

rejrjrrz

Complex Number Multiplication

Then . and Let 212121

21212121

erreerrererzz

erzerz

Euler's Formula & Conjugate Multiplication

Then . and

. and Let

22 rerrerezz

rezrez

jyxzjyxz

Expressing Conjugates with Euler's Identity

.sincos

sincossincos

Then .Let

.sinsin i.e.,odd, is sin

while,coscos i.e.,even, is cos that Recall

jrrjrrrez

Sine & Cosine with Euler's Formula

So, .sincos

sincossincossin2

So, .sincossincoscos2

Complex Sinusoid

.sincos then ,1 and 0 If

.sincos

as defined is sinusoidcomplex then offset, the

is and seconds,in timeis (rad/sec),frequency

radian theis where, and 0Let

.sincos :identity sEuler' Recall

tjtetsA

tjAtAAets

Real & Imaginary Parts of Complex Sinusoid

. sinusoidcomplex theofpart imaginary theis sin

; sinusoidcomplex theofpart real theis cos

.sincosLet

tsteim

tstere

tjtets

Circular Motion in 3D

Let us assume that a point is moving in circles in 3D, as shown on the right. If we agree that time can be measured along the three axes (x, y, & z), then we can project the point's circular motion along the three axes (lines): x, y, and z vs. time

Projection of 3D Circular Motion on X-Axis

The X-axis projection is obtained from an observer who looks at the trace of the 3D circular motion from the x-y plane.

.cos i.e., sinusoid,complex the

ofpart real theis axis-X on the projection The

tere tj

Projection of 3D Circular Motion on Y-Axis

The Y Axis projection is obtained from an observer who looks at the trace of the 3D circular motion from the y-z plane.

.sin i.e.,

sinusoid,complex theofpart imaginary

theis axis-Y on the projection The

teim tj

Projection of 3D Circular Motion on Z

The Z Axis projection is obtained from the observer who looks at the trace of the 3D circular motion from the x-z plane.

.by defined is circle This tje

Circular Motion

increases. as

planecomplex in the circleunit thealongmotion

circular clockwise a of trace thedefines

increases. as plane

complex in the circleunit thealongmotion circular

clockwise-counter a of trace thedefines

plane.complex in the circle aon liesit

constant, is sinusoidcomplex a of modulus theSince

Positive & Negative Frequency Sinusoids

sinusoidfrequency -negative a is

sinusoidfrequency -positive a is

Positive & Negative Frequencies

s.frequencie negative and

positive of amounts equal contains signal realEvery 2)

.componentsfrequency negative and positive of

on contributi equalan of consists sinusoid realEvery 1)

:nsobservatio Two

thatshow oidentity t sEuler' used We

Modulus of Complex Sinusoid

.1sincos

sincos

:follows as is

sinusoidcomplex a of modulus the,1cossin Since

222222

tAtAtsimtsrets

Nth Roots of Unity

Finding Nth Roots

.1210for results are There

isroot th The .

Therefore, .for ,12sin2cos

., and 0 where, identity, sEuler'By

,...,N-,,kN

Zkereererez

Zkkjke

NNkjjN

Roots of Unity

.1,...,3,2,1,0, :it from

rootsother allobtain can webecause unity, ofroot th primitive a

called is 1for case special The unity. of rootsth theare These

.1,...,3,2,1,0,1

Thus, .integer every for ,1

NkjkNj

Commonly Used Formulas: Cheat Sheet

tjnTNfkjNnkj nks

A Different Notation for Roots of Unity

.1,...,3,2,1,0,

:used isnotation following the,processing signalIn

kjNNkjNTjNkN

NkjTNfjkTjkN

NkeeeW

Computing Nth Roots of Unity: Example

,,,,, So,

parts. equal 8 into circleunit thedividecan wey,Graphicall

.,,,,,,, compute usLet 8.Let

TjTjTj

TjTjTjTjTj

eWeWeW

eWeWeWeWeW

WWWWWWWWN

Computing Nth Roots of Unity

case,our In .22

that Recall 1sec. set We.,,

,,,,, So,

TeWeWeW

eWeWeWeWeW

TjTjTj

TjTjTjTjTj

.010sin0cos

cos228

2 jjeeWjj

jeeWjj

.001sincos

sincos

.01sincos4

jjeeWjj

Roots of Unity &

Complex Sinusoids

Roots of Unity & Sinusoids

unity. ofroot th -

th - theis where,1,...,2,1,0

,by defined sinusoids

complex generateunity of rootsth - The/2

nTjNknjnkN

Sinusoid at k=0

7,...,1,0,0sin0cos

.010sin0cos

nnjneW

Real Sinusoid at k = 0

Imaginary Sinusoid at k = 0

Sinusoid at k=1: Real Sinusoid

Real Sinusoid at k = 1

.7,...,1,0,4

nnjneeW

7,...,0,4

Sinusoid at k=1: Imaginary Sinusoid

.7,...,1,0,4

nnjneeW

7,...,0,4

Sinusoid at k=1: Real & Imaginary Sinusoids

.7,...,1,0,4

cos :Sinusoid 1

4cos :Root 1

nnjneeW

7,...,0,4

Real Sinusoid at k = 1 7,...,0,4

t=0:.1:8;y = cos(pi/4*t);plot(t, y);

t=0:.1:8;y = sin(pi/4*t);plot(t, y);

Sinusoid at k=3: Real & Imaginary Sinusoids

7,...,0,4

3cos :Sinusoid 3

3cos :Root 3

nnjneeW

Real Sinusoid at k = 3 7,...,0,4

t=0:.1:8;y = cos(3*pi/4*t);plot(t, y);

t=0:.1:8;y = sin(3*pi/4*t);plot(t, y);

7,...,0,4

Complex Sinusoid Basis Set

.1,...,2,1,0for

,,...,,,

,..., , ,

isset basis sinusoidcomplex Then the .1,2Let

ensensensens

NnjNnjNnj

Inner Product of 2 Sinusoids

as defined is and ofproduct inner The

.1,0for ,

and Let

NmnjNknjN

NmnjnTjm

NknjnTjk

eessss

NnWeens

Orthogonality of Complex Sinusoids

., Thus,

.1,...,0, , that Assume

.1,0, is sinusoidcomplex th - The

eeeensnsss

NnWeensk

NNmkjN

NmnjNknjmkmk

NknjnTjk

Periodicity of Complex Sinusoids

.for ,

. divide that periods with periodic all are sinusoids These

.1,...,0,

as defined is sinusoidcomplex th - The

ZmnsmNns

NnWeens

NknjnTjk

Norm of Complex Sinusoids

. Thus, .,1

/2 NsNess k

Nnkkjkk

An Orthonormal Sinusoidal Set

:condition following esatisfy th sinusoids lorthonorma The

set. lorthonormaan obtain tonormalized are sinusoidsComplex

Signal Projection

.in present is signal much the how

i.e., ,projection oft coefficien thecomputesproduct inner The

.,product inner theas computed is onto of projection The

signals. twobe ,Let

Discrete Fourier Transform

Human Ear & Sinusoids

● The human ear is a spectrum analyzer● The cochlea of the inner ear splits sound into its

sinusoidal components● A spectrum that displays the amount of each

sinusoid frequency present in sound is likely to be similar to the representation that the brain receives from the ear

● The human ear can be construed as a Fourier spectrum analyzer of sorts

DFT Definition

.1,...,1,0,,

is definition equivalent The

,2,, where

,1,...,1,0,,

:follows as defined is of (DFT) Transform

Fourier Discrete the, spans that sinusoidscomplex of basis

orthogonal thebe and signalcomplex a be Let

sskntj

NkenxenxsxX

knTtens

NknsnxsxX

Another Definition of DFT

:follows as computed is DFT the

Then signal.input thebe Let ./2 as computed

are sfrequencie that theAssume sinusoid.complex amplitude

unit phase,-zero a be 1,-..., 0,1,2,3,,Let

yenyenyeny

enyenynsnysy

Continuous Time Fourier Transform

as defined is onto of projection The .

sinusoid timecontinuous on the projected is signal The

Ydtetysy

Spectrum

. sinusoidcomplex

th - with the ofproduct inner theas defined is of

spectrum theof , sample,th - thes,other wordIn

.frequency at spectrum theof sampleth - theis

. of spectrum thecalled is ,...,,

, signalinput theand 1,...,1,0

,,...,,set basis sinusoidcomplex Given the

nsnsns

Computing Spectrum Elements

. if ,1

.,1 where,1

1 series geometric theof because

: computecan wehow is Here

. andarray signalinput thebe Let

eeeensnxX

nTjnTjkk

uuuuuuuu batbta sinsincos 22

upperlower WW ,

Frequency Reconstructed Harmonics

1D Discrete Fourier Transform: Spectral Analysis

],...,[ 10 N

Frequency Presence of complex basic sinusoid in signal x

nsnxsx

nNN nsnxsx

Projection of Signals on Sinusoids

xsk in present is

much how indicates projection oft coefficien The

Proportionality of DFT to Projection Coefficients

. onto signal of projection

theoft coefficien theis ,

t coefficien The

set. basis sinusoidcomplex theonto projection of

tscoefficien the toalproportion is DFT Thus, .1,...,1,0

Then . because ,,

Inverse DFT

.1,...,1,0,1

ly,equivalent or, ,1,...,1,0,

sprojection thefrom signal t thereconstruc nowcan We

Normalized DFT

as define is (NDFT) DFT Normalized The

sinusoids. DFT normalized theonto signal theprojectingby obtained

isIt clean.ally theoreticmore be toconsidered is DFT Normalized

Nknjkk

Normalized Inverse DFT

pure.ally mathematic more isit However,

n.computatio more require they because IDFT and DFT

as frequently as practicein usednot are NIDFT and NDFT

:follows as define is (NIDFT) DFT Inverse Normalized

Nknjkkk eX

NnsXnx

DFT as a Digital Filter

as computed isfilter thisof response

frequency theof magnitude The . isoutput whoseand

isinput whose,each for filter digital a isoperation DFT The

DFT as a Digital Filter

.4/for belowshown is

2/sin ofPlot Nk

Source: J. O. Smith III, Mathematics of the DFT with Audio Applications, 2nd Edition

DFT Example for N=2

.11cos1sin1cos1 and 10 because

,1,11,0 Thus, .1,0for ,

.1,11,0 Thus, .1,0for ,1

., compute usLet

.1,...,0, is sinusoid basis for the formula the

Recall ., :sinusoids basis twohave then we2, have weIf

1112/12

0002/02

sssneens

sssnens

DFT Example for N=2

.41216,

81216,

.components two

has spectrum The .2,6 signal theof DFT thecompute usLet

Review: Projection of Signals on Sinusoids

xsk in present is

much how indicates projection oft coefficien The

DFT Example: Projection Coefficients

2,21,1222

4,41,14 42

:sinusoids basis two theonto sprojection thecompute usLet

DFT Example:Projections on Two Sinusoids

DFT & Inverse DFT as

Matrix Multiplication

DFT Matrix & Inverse DFT Matrix

1 ... 1 0

1 1 1 1

2 2 2 2

1 1 1 1

0 0 0 0

NsNsNsNs

matrix. DFT inverse thecalled is

.matrix DFT thecalled is

eWnkSS

eWknSS

NknjknNNN

DFT as Matrix Manipulation

1 ... 1 0

......

., /2** NknjknNNN eWnkSS

DFT as Matrix Manipulation: DFT Matrix

1 ... 1 1 1

1 ... 1 0

/112/122/12

/122/8/4

/12/4/2

NNNjNNjNNj

NNjNjNj

., /2** NknjknNNN eWnkSS

Inverse DFT as Matrix Manipulation

1 ... 1 1

well.astion multiplicamatrix as DFT inverse theformulatecan We

NsXNNx

NsNsNs

Back to DFT Example for N=2

.41216,

81216,

.components twohasIt .2,6 signal theof spectrum thecompute usLet

XXxSxSx

tion.multiplicamatrix as computed DFT above The

Inverse DFT Example for N=2

41 812

.4,8 spectrum thefrom signal original erecover th usLet

References● J. O. Smith III, Mathematics of the Discrete Fourier Transform with

Audio Applications, 2nd Edition.

● G. P. Tolstov. Fourier Series.

Wavelets & Wavelet Algorithms: 1D Discrete Fourier Transform & Inverse Discrete Fourier...

Science

Discrete Fourier Transform IP Generator

Discrete-time windowing Discrete Fourier Transform ...web.cecs.pdx.edu/~ece2xx/ECE223/Slides/FFT.pdf · Fast Fourier Transform • Discrete-time windowing • Discrete Fourier Transform

Discrete Fourier Transform Dr Yvan Petillothome.eps.hw.ac.uk/~ceeyrp/www/teaching/b39se1/dsp2.pdf2 Discrete Fourier Transform 3.3 Fourier Transform • Used for spectral analysis •

Numerical Methods Discrete Fourier Transform Part: Discrete Fourier Transform

Discrete Time Fourier Transform

Discrete -Time Fourier Transform Discrete Fourier ...tania/teaching/dsp/Lectures 3-4 DTFT DFT... · Discrete -Time Fourier Transform Discrete Fourier Transform z-Transform Tania Stathaki

On Computing the Discrete Fourier Transform · On Computing the Discrete Fourier Transform By S. Winograd Abstract. A new algorithm for computing the Discrete Fourier Transform is

The Discrete Fourier Transform 1 Introductioneeweb.poly.edu/iselesni/EL713/zoom/dft.pdfThe Discrete Fourier Transform 1 Introduction The discrete Fourier transform (DFT) is a fundamental

Discrete-Time Fourier Transform Discrete Fourier Transform z-Transform

4. The Discrete Fourier Transform and Fast Fourier Transform 4 - The DFT an… · 4-1 4. The Discrete Fourier Transform and Fast Fourier Transform • Reference: Sections 8.0-8.7

Discrete Fourier Transform ‐ II.pdf

Chap3-Discrete Fourier Transform

Motivations for Discrete Fourier Transformee123/fa12/Notes/Lecture05.pdf · Motivations for Discrete Fourier Transform ... MATLAB commands: ... The discrete Fourier transform relates

Md Shiplu Hawlader Roll: SH-224. Fourier Series Theorem Fourier Transform Discrete Fourier Transform Fast Fourier Transform

Discrete Fourier TransformDiscrete Fourier Transform

discrete Time Fourier transform finite Fourier series

Discrete Fourier Transform - IMT

Chapter 5 Finite-Length Discrete Transform. §5.1 Discrete Fourier Transform (DFT) DTFT is the Fourier Transform of discrete-time sequence. It is discrete

4. The Discrete Fourier Transform and Fast Fourier Transform

M11 - Discrete Fourier Transform