FOURIER ANALYSISPART 2: Technicalities, FFT
& system analysis
Maria Elena Angoletta
AB/BDI
DISP 2003, 27 February 2003
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 2 / 21
TOPICSTOPICS
1. DFT windows
2. DFT resolution - improvement
3. Efficient DFT calculation: FFT
4. Hints on system spectral analysis
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 3 / 21
DFT – Window characteristicsDFT – Window characteristics• Finite discrete sequence spectrum convoluted with rectangular window spectrum.
• Leakage amount depends on chosen window & on how signal fits into the window.
Resolution: capability to distinguish different tones. Inversely proportional to main-lobe width. Wish: as high as possible.Wish: as high as possible.
(1)
(1)
Several windows used Several windows used ((application-dependentapplication-dependent): ): Hamming, Hanning, Blackman, Hamming, Hanning, Blackman, Kaiser ...Kaiser ...
Rectangular window
Peak-sidelobe level: maximum response outside the main lobe. Determines if small signals are hidden by nearby stronger ones.Wish: as low as possible.Wish: as low as possible.
(2)
(2)
Sidelobe roll-off: sidelobe decay per decade. Trade-off with (2).
(3)
(3)
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 4 / 21
Sampled sequence
In time it reduces end-points discontinuities.
Non windowe
d
Windowed
DFT of main windowsDFT of main windowsWindowing reduces leakage by minimising sidelobes magnitude.
Some window functions
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 5 / 21
DFT - Window choiceDFT - Window choice
Window type -3 dB Main-lobe width
[bins]
-6 dB Main-lobe width
[bins]
Max sidelobelevel[dB]
Sidelobe roll-off[dB/ decade]
Rectangular 0.89 1.21 -13.2 20
Hamming 1.3 1.81 - 41.9 20
Hanning 1.44 2 - 31.6 60
Blackman 1.68 2.35 -58 60
Common windows characteristics
NB: Strong DC component can shadow nearby small signals. Remove it!NB: Strong DC component can shadow nearby small signals. Remove it!
Far & strong interfering components high roll-off rate.
Near & strong interfering components small max sidelobe level.
Accuracy measure of single tone wide main-lobe
Observed signalObserved signal Window wish listWindow wish list
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 6 / 21
DFT - Window loss remedial DFT - Window loss remedial Smooth data-tapering windows cause information loss near Smooth data-tapering windows cause information loss near edges.edges.
• Attenuated inputs get next window’s full gain & leakage reduced.
• Usually 50% or 75% overlap (depends on main lobe width).
Drawback: increased total processing time.
Solution:
sliding (overlapping) DFTs.
2 x N samples (input signal)
DFT #1
DFT #2
DFT #3
DFT AVERAGI NG
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 7 / 21
- 1
- 0.5
0
0.5
1
0 3 6 9 12 15 18 21 24 27 30
time
0
2
4
6
8
0 3 6 9 12bin
Zero paddingZero paddingImproves DFT frequency inter-sampling spacing (“resolution”).
- 1
- 0.5
0
0.5
1
0 4 8 12 16
time
0
2
4
6
8
0 1 2 3 4 5 6bin
0
2
4
6
8
0 12 24 36 48bin- 1
- 0.5
0
0.5
1
0 10 20 30 40 50 60 70 80 90 100 110 120
time
After padding bins @ frequencies NS = original samples, L = padded.LN
fmf
S
Sm
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 8 / 21
Zero padding -2Zero padding -2
• Additional reason for zero-padding: to reach a power-of-two input samples number (see FFT).
• Zero-padding in frequency domain increases sampling rate in time domain. Note: it works only if sampling theorem satisfied!
DFT spectral resolutionDFT spectral resolution
Capability to distinguish two closely-spaced frequencies: not improvednot improved by zero-padding!.
Frequency inter-sampling spacing: increasedincreased by zero-padding (DFT “frequency span” unchanged due to same sampling frequency)
Apply zero-padding afterafter windowing (if any)! Otherwise stuffed zeros will partially distort window function.
NOTE
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 9 / 21
DFT - scalloping loss (SL)DFT - scalloping loss (SL)
Worst case when f0 falls exactly midway between 2 successive bins (|r|=½)
SL
bin, k
Note:Note: Non-rectangular windows broaden DFT main lobe SL less severeSL less severe
Correction depends on window used.Correction depends on window used.
f0
Frequency error: f = r fS/N,
relative error: R=f / f0 = r/[(kmax+r)]
R 1/(1+2 kmax)
kmax
kmaxbin, k
We’re lucky here!
May impact on data interpretation (wrong f0!)
Input frequency f0 btwn. bin centres causes magnitude loss SL = 20 Log10(|cr+kmax /ckmax|)
~~~~
|r| ½f0 = (kmax + r) fS/N
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 10 / 21
DFT - SL ExampleDFT - SL Example
0 50 100 150 200 250 300 350 400 450 5000
0.5
1
1.5
2
DC bias correction, Rectang. window, zero padding, FFT
DC bias correction, Hanning window, zero padding, FFT
• increasing N (?)
• improve windowing,
• zero-padding,
• interpolation around kmax.
SL remedialSL remedial
0 50 100 150 200 250 300 350 400 450 5000
0.25
0.5
0.75
1
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 11 / 21
DFT - parabolic interpolationDFT - parabolic interpolation
Parabolic interpolation often enough to find position of peak (i.e. frequency).
Other algorithms available depending on data.
198 199 200 201 202 2031.962
1.963
1.964
1.965
1.966
1.967
1.968
199 200 201 202 203 2040.974
0.975
0.976
0.977
Rectangular window Hanning window
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 12 / 21
DFT averagingDFT averagingIncoherent averaging
M
(k)DFT(k)DFT(k)DFT M1
INCOH
M = No. of DFT to average M = No. of DFT to average
k = bin number, k=0, 1 .. N-1k = bin number, k=0, 1 .. N-1
M
(k)DFTIm(k)DFTImj
M
(k)DFTRe(k)DFTRe(k)DFT M1M1
COH
Coherent averaging
Background noise fluctuations reduced, average noise power unchanged.
Background noise power is reduced.
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 13 / 21
Efficient DFT calculation: FFT Efficient DFT calculation: FFT
1N
0n
knN
1N
0n
Nnk2π
jk Ws[n]
N
1es[n]
N
1c~
WNn,k = twiddle factors
k = 0,1 .. N-1
DFT
Direct DFT calculation redundancy
WNkn periodic function calculated many
times.
W80,k
W84,k
W82,k
W85,k
W86,k
W83,k
W87,k
W81,k
Algorithms (= Fast Fourier Transform) developed to compute N-points DFT with ~ Nlog2N multiplications (complexity O(Nlog2N) ).
Direct DFT calculation requires ~N2 complex multiplications.
complexity O(N2) VERY BAD
VERY BAD ! !
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 14 / 21
FFT advantages FFT advantages
0
500
1000
1500
2000
0 10 20 30 40
Number of samples, N
Nu
mb
er
of
Op
era
tio
ns
DFT DFT N N22
FFT FFT N log N log22NN
NB: Usually you don’t want to write an FFT algorithm, just to “borrow” it !!! Go “shopping” onto the web!
N DFT Radix-2
4 16 4
32 1024 80
128 16384 448
1024 1048576 5120
DSPs & PLDs influenced algorithms design. ‘60s & ‘70s: multiplication counts was “quality factor”. Now: number of additions & memory access (s/w) and communication costs (h/w) also important.
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 15 / 21
FFT philosophyFFT philosophyGeneral philosophy (to be applied recursively): divide & conquerdivide & conquer.
cost(sub-problemssub-problems) + cost(mappingmapping) < cost(original problemoriginal problem)
Different algorithms balance costs differently.
Example: Decimation-in-time algorithm
time frequency
Step 3: Frequency-domain synthesis. N spectra synthesised into one.
Step 2: 1-point input spectra calculation. (Nothing to do!)
Step 1: Time-domain decomposition. N-points signal N, 1-point signals (interlace decomposition). Shuffled input data (bit-reversal). log2N stages.
(*)(*): only first decomposition shown.
(**)
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 16 / 21
FFT family treeFFT family tree
Divide & Divide & conquerconquer
N : GCD(N1,N2) <> 1
Ex: N = 2n
Cost: MAPPINGMAPPING.
• Twiddle-factors calculations.
• Easier sub-problems.
• Some algorithms:
Cooley-Tukey,
Decimation-in-time / in-frequency
Radix-2, Radix-4,
Split radix.
N : GCD(*)(N1,N2) = 1
N1, N2 co-prime. Ex: 240 = 16·3·5Cost: SUB-PROBLEMSSUB-PROBLEMS.
• No twiddle-factors calculations.
• Easier mapping (permutations).
• Some algorithms:
Good-Thomas, Kolba,
Parks, Winograd.
(*) GCD= Greatest Common Divisor
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 17 / 21
(Some) FFT concepts & notes(Some) FFT concepts & notes
Butterfly: basic FFT calculation element.
s[k+N/ 2]
s[k]
WNN/ 2
WN0
1. Decimation-in-time time data shuffling.
2. Decimation-in-frequency frequency data shuffling.
3. In-place computation: no auxiliary storage needed, allowed by most algorithms.
4. DFT pruning: only few bins needed or different from zero only they get calculated (ex: Goertzel algorithm).
5. Real-data case: Mirroring effect in DFT coeffs. only half of them calculated.
6. N power-of-two: Many common FFT algorithms work with power-of-two number of inputs. When they are not pad inputs with zeroes.
Dual approach: data to be reordered in time or in frequency!
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 18 / 21
Systems spectral analysis (hints)Systems spectral analysis (hints)
System analysis: measure input-output relationshipSystem analysis: measure input-output relationship..
DI GI TAL LTI SYSTEM
h[n]
x[n] y[n]
H(f) : LTI transfer functionH(f) : LTI transfer function
0m
h[m]m]x[nh[n]x[n]y[n]x[n] h[n]
X(f) H(f) Y(f) = X(f) · H(f)
DI GI TAL LTI
SYSTEM 0 n
[n] 1
0 n
h[n]
h[t] = impulse response
Linear Time InvariantLinear Time Invariant
y[n] predicted from { x[n], h[t] }
Transfer function can be estimated by Y(f) / X(f)
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 19 / 21
Estimating H(f) (hints)Estimating H(f) (hints)
(f)XX(f)(f)G *xx Power Spectral Density of x[t]
(FT of autocorrelation).
(f)XY(f)(f)G *yx Cross Power Spectrum of x[t] &
y[t] (FT of cross-correlation).
It is a check on H(f) validity!
(f)G(f)G
(f)G(f)C
yyxx
2yx
xy
Coherence function - values in [0,1]
- assess degree of linear relationship between x[t] & y[t].
xx
yx*
*
G
G
(f)XX(f)
(f)XY(f)
X(f)
Y(f)H(f)
Transfer Function
(ex: beam !)
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 20 / 21
References - 1References - 1
1. Tom, Dick and Mary discover the DFT, J. R. Deller Jr, IEEE Signal Processing Magazine, pg 36 - 50, April 1994.
2. On the use of windows for harmonic analysis with the Discrete Fourier Transform, F. J. Harris, IEEE Proceedings, Vol. 66, No 1, January 1978.
3. Some windows with a very good sidelobe behaviour, A. H. Nuttall, IEEE Trans. on acoustics, speech and signal processing, Vol ASSP-29, no. 1, February 1981.
4. Some novel windows and a concise tutorial comparison of windows families, N. C. Geckinli, D. Yavuz, IEEE Trans. on acoustics, speech and signal processing, Vol ASSP-26, no. 6, December 1978.
5. Study of the accuracy and computation time requirements of a FFT-based measurement of the frequency, amplitude and phase of betatron oscillations in LEP, H.J. Schmickler, LEP/BI/Note 87-10.
6. Causes et corrections des erreurs dans la mesure des caracteristiques des oscillations betatroniques obtenues a partir d’une transformation de Fourier, E. Asseo, CERN PS 85-9 (LEA).
Papers
M. E. Angoletta - DISP2003 - Fourier analysis - Part 2.2: Technicalities 21 / 21
References - 2References - 2
1. The Fourier Transform and its applications, R. N. Bracewell, McGraw-Hill, 1986.
2. A History of scientific computing, edited by S. G. Nash, ACM Press, 1990.
3. Introduction to Fourier analysis, N. Morrison, John Wiley & Sons, 1994.
4. The DFT: An owner’s manual for the Discrete Fourier Transform, W. L. Briggs, SIAM, 1995.
5. The FFT: Fundamentals and concepts, R. W. Ramirez, Prentice Hall, 1985.
Books
7. Precise measurements of the betatron tune, R. Bartolini et al., Particle Accel., 1996, vol. 55, pp 247-256.
8. How the FFT gained acceptance, J. W. Cooley, IEEE Signal Processing Magazine, January 1992.
9. A comparative analysis of FFT algorithms, A. Ganapathiraju et al., IEEE Trans.on Signal Processing, December 1997.