Upload
others
View
16
Download
0
Embed Size (px)
Citation preview
The Fourier & wavelet transforms in practiceApplication to the (non) detection of periodic signals
Frédéric Auchère [email protected] d’Astrophysique Spatiale, CNRS/ Université Paris-Sud, France
ESAC Data Analysis & Statistics Worshop – October 8, 2018, Villafranca, Spain
262
Level, requirements
o Who are you?
Master *********
PhD *********************
Phd+5 ********
PhD+10 **
PhD+++ ****
Lecture will be PhD level although there will be nuggets for the more advanced listener
o RequirementsBasic math, no prior knowledge of Fourier / wavelet analysis
Hands on sessions will be IDL but no knowledge required
362
Jean Baptiste Joseph Fourier(1768-1830) Fourier’s transform
‘‘The in-depth study of Nature isthe most fecund source of
discoveries in mathematics’’
462
Topics covered
1. Motivations2. Fourier analysis
2.1 The Fourier transform: definitions & basic properties2.2 The measurement process2.3 The continuous signal and its transform2.4 The discrete signal and its transform2.5 The effects of sampling, aliasing2.6 The discrete transform of the discrete signal2.7 How to treat the low frequencies2.8 The FFT2.9 The case of unenvenly space data
3. Wavelet analysis3.1 The wavelet transform: definitions & basic properties
4. Detection of periodic signals (Fourier & wavelets)4.1 Fourier transform of noise4.2 Side effects of detrending4.3 Local vs. Global confidence levels4.4 On the proper use of the Torrence & Compo wavelet code
Hands -on
562
Motivation
662
Some references
Many textbooks on Fourier analysis …
Bracewell, R. N. 1999, ‘‘The Fourier transform and its applications’’, 3rd Edition, McGraw Hill
Brault, J. W., & White, O. R. 1971, ‘‘The analysis and restoration of astronomical data via the FastFourier Transform’’, A&A, 13, 169
Torrence, C. & Compo, G. 1998, ‘‘A practical guide to wavelet analysis’’, BAMS, 79, 61
Scargle, J. D. 1982, ‘‘Studies in astronomical time-series analysis. II. Statistical aspects of spectral analysisof unevenly spaced data’’, ApJ, 263, 835
Gabriel, A. H., Baudin, F., Boumier, P. et al. 2002, ‘‘A search for solar g modes in the GOLF data’’, A&A, 390, 1119
Auchère, F., Froment, C., Bocchialini, K. et al. 2016, ‘‘On the Fourier and wavelet analysis of coronal time series’’, ApJ, 825, 110 + ERRATUM!
etc.
762
Fourier transform: definition & notations
o For a function 𝑠𝑠(𝑡𝑡) (𝑠𝑠 and its derivatives are supposed intergable)
o Several other common conventions (e.g. minus signs) and notations, e.g.
F 𝑠𝑠 ≡ F 𝑠𝑠 𝜈𝜈 ≡ F 𝑠𝑠 𝑡𝑡 ≡ ⋯ ≡ �̂�𝑠
Angular frequency : �̂�𝑠 𝜈𝜈 = ∫−∞+∞ 𝑠𝑠(𝑡𝑡)𝑒𝑒−𝑖𝑖𝜔𝜔𝑡𝑡𝑑𝑑𝑡𝑡
Fourier transform: �̂�𝑠 𝜈𝜈 = �−∞
+∞𝑠𝑠(𝑡𝑡)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
Inverse transform: 𝑠𝑠 𝑡𝑡 = �−∞
+∞�̂�𝑠 𝜈𝜈 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈
862
o Units & duality:
o �̂�𝑠 𝜈𝜈 is a complex function (and so is 𝑠𝑠(𝑡𝑡), see above)o 𝐴𝐴 𝜈𝜈 = �̂�𝑠 𝜈𝜈 is the amplitude spectrum
o 𝜙𝜙 𝜈𝜈 = arg �̂�𝑠 𝜈𝜈 is the phase spectrum
o 𝑃𝑃 𝜈𝜈 = �̂�𝑠 𝜈𝜈 𝑖 is the power spectrum
o �̂�𝑠 𝜈𝜈 = Re �̂�𝑠 𝜈𝜈 + Im �̂�𝑠 𝜈𝜈o �̂�𝑠 𝜈𝜈 = �̂�𝑠 𝜈𝜈 𝑒𝑒𝑖𝑖𝑖𝑖(𝑖𝑖)
Fourier transform: definition & notations
Measurement domain Fourier domain
Time Inverse time (frequency)
Length Inverse length (wave number)
unit Inverse unit
Inverse unit unit
962
Fourier transform: basic properties
�−∞
+∞𝑠𝑠 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 = �
−∞
+∞𝑠𝑠 𝑡𝑡 cos −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 + 𝑖𝑖 �
−∞
+∞𝑠𝑠 𝑡𝑡 sin −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡
If 𝑠𝑠 𝑡𝑡 ∈ ℝ (which is the case for physical signals)
𝑠𝑠 𝑡𝑡 = 𝑜𝑜 𝑡𝑡 + 𝑒𝑒 𝑡𝑡
⇒ �̂�𝑠 𝜈𝜈 = �𝑜𝑜 𝜈𝜈 + �̂�𝑒 𝜈𝜈
If 𝑠𝑠 𝑡𝑡 is even ⇒ �−∞
+∞𝑠𝑠 𝑡𝑡 sin −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 = 0 ⇒ �̂�𝑠 𝜈𝜈 ∈ ℝ and even
If 𝑠𝑠 𝑡𝑡 is odd ⇒ �−∞
+∞𝑠𝑠 𝑡𝑡 cos −2𝜋𝜋𝜈𝜈𝑡𝑡 𝑑𝑑𝑡𝑡 = 0 ⇒ �̂�𝑠 𝜈𝜈 ∈ 𝕀𝕀 and odd
∈ ℝ and even ∈ 𝕀𝕀 and odd
If 𝑠𝑠 𝑡𝑡 ∈ ℝ ⇒ �̂�𝑠 𝜈𝜈 = �𝑜𝑜 𝜈𝜈 + �̂�𝑒 𝜈𝜈 = �𝑜𝑜 −𝜈𝜈 − �̂�𝑒 −𝜈𝜈 = �̂�𝑠 −𝜈𝜈 (conjugation symmetry)
1062
Fourier transform: basic properties
o Linearity : 𝑠𝑠 𝑡𝑡 = 𝑎𝑎𝑎𝑎 𝑡𝑡 + 𝑏𝑏𝑏𝑏 𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = 𝑎𝑎𝑎𝑎 𝜈𝜈 + 𝑏𝑏 �𝑏𝑏(𝜈𝜈)
o Translation / time shifting :
o Modulation / frequency shifting :
𝑠𝑠 𝑡𝑡 = 𝑎𝑎 𝑡𝑡 − 𝑡𝑡0 ⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝑎𝑎 𝑡𝑡 − 𝑡𝑡0 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= �−∞
+∞𝑎𝑎 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖 𝑡𝑡′+𝑡𝑡0 𝑑𝑑𝑡𝑡′
= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡0𝑎𝑎 𝜈𝜈
𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡𝑎𝑎 𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝑎𝑎 𝑡𝑡 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= 𝑎𝑎 𝜈𝜈 − 𝜈𝜈0
1162
Fourier transform: basic properties
o Time / frequency scaling :
Time reversal: a = −1 ⇒ 𝑠𝑠 𝑡𝑡 = 𝑎𝑎 −𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = 𝑎𝑎(−𝜈𝜈)
o Integration :
�̂�𝑠 0 = �−∞
+∞𝑎𝑎 𝑡𝑡 𝑑𝑑𝑡𝑡
𝑠𝑠 𝑡𝑡 = 𝑎𝑎 𝑎𝑎𝑡𝑡 ⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝑎𝑎 𝑎𝑎𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= �−+∞
±∞𝑎𝑎 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖
𝑡𝑡′𝑎𝑎𝑑𝑑𝑡𝑡′
𝑎𝑎
=1𝑎𝑎 𝑎𝑎
𝜈𝜈𝑎𝑎
Time dilation ⇔ frequency contraction
Time contraction ⇔ frequency dilation
1262
Fourier transform of a derivative
�𝑠𝑠′ 𝜈𝜈 = �−∞
+∞𝑠𝑠′ 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 = �
−∞
+∞lim
𝑠𝑠 𝑡𝑡 + Δ𝑡𝑡 − 𝑠𝑠(𝑡𝑡)Δ𝑡𝑡
𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= limΔ𝑡𝑡→0
∫−∞+∞𝑠𝑠 𝑡𝑡 + Δ𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 − ∫−∞
+∞𝑠𝑠 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡Δ𝑡𝑡
= limΔ𝑡𝑡→0
𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖Δ𝑡𝑡�̂�𝑠 𝜈𝜈 − �̂�𝑠(𝜈𝜈)Δ𝑡𝑡
= �̂�𝑠 𝜈𝜈 limΔ𝑡𝑡→0
𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖Δ𝑡𝑡 − 1Δ𝑡𝑡
= 𝑖𝑖2𝜋𝜋𝜈𝜈�̂�𝑠 𝜈𝜈
⇒ �𝑠𝑠(𝑛𝑛) = 𝑖𝑖2𝜋𝜋𝜈𝜈 𝑛𝑛�̂�𝑠 𝜈𝜈
1362
Convolution theorem
�𝑎𝑎 ∗ 𝑏𝑏= �
−∞
+∞�−∞
+∞𝑎𝑎 𝜏𝜏 𝑏𝑏 𝑡𝑡 − 𝜏𝜏 𝑑𝑑𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= �−∞
+∞𝑎𝑎 𝜏𝜏 �
−∞
+∞𝑏𝑏 𝑡𝑡 − 𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡 𝑑𝑑𝜏𝜏
= �−∞
+∞𝑎𝑎 𝜏𝜏 �
−∞
+∞𝑏𝑏 𝑡𝑡′ 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖(𝑡𝑡′+𝜏𝜏)𝑑𝑑𝑡𝑡′ 𝑑𝑑𝜏𝜏
= �−∞
+∞𝑎𝑎 𝜏𝜏 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝜏𝜏 �𝑏𝑏(𝜈𝜈) 𝑑𝑑𝜏𝜏
= 𝑎𝑎 𝜈𝜈 . �𝑏𝑏(𝜈𝜈)
Likewise, �𝑎𝑎.𝑏𝑏 = 𝑎𝑎 𝜈𝜈 ∗ �𝑏𝑏(𝜈𝜈)
o The Fourier transform translates between multiplication and convolution
o In combination with the linearity and time-shifting properties, the convoutiontheorem can be used to derive the Fourier transform of many usual signals
1462
Parseval’s theorem
𝑠𝑠 𝑡𝑡 ∈ ℝ and 𝑠𝑠 𝑡𝑡 = 𝑎𝑎(𝑡𝑡). 𝑎𝑎(𝑡𝑡)
�̂�𝑠 𝜈𝜈 = �−∞
+∞𝑎𝑎𝑖 𝑡𝑡 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= 𝑎𝑎 𝜈𝜈 ∗ 𝑎𝑎(𝜈𝜈) (from the convolution theorem)
= �−∞
+∞𝑎𝑎 𝜈𝜈 𝑎𝑎(𝜈𝜈 − 𝜎𝜎)𝑑𝑑𝜎𝜎
𝜈𝜈 = 0 ⇒ �−∞
+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 = �
−∞
+∞𝑎𝑎 𝜈𝜈 𝑎𝑎(−𝜎𝜎)𝑑𝑑𝜎𝜎
⇒ �−∞
+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 = �
−∞
+∞𝑎𝑎(𝜈𝜈)
𝑖𝑑𝑑𝜈𝜈
o Physics: the average power in the signal is equal to the average power in its transform
o Signal: the variance of the signal is equal to the mean power (more on this later)
1562
Fourier transform of common functions / distributions
𝑠𝑠 𝑡𝑡 = 𝛿𝛿0(𝑡𝑡)
⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝛿𝛿0(𝑡𝑡) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= 1
�̂�𝑠 𝜈𝜈 = 𝛿𝛿0(𝜈𝜈)
⇒ 𝑠𝑠 𝑡𝑡 = 1�−∞
+∞𝛿𝛿0(𝜈𝜈) 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈
= 1
𝑠𝑠 𝑡𝑡 = 1⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿0
https://en.wikipedia.org/wiki/Fourier_transform
1662
Fourier transform of common functions / distributions
𝑠𝑠 𝑡𝑡 = 𝛿𝛿𝑇𝑇(𝑡𝑡)
⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝛿𝛿(𝑡𝑡 − 𝑇𝑇) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇
�̂�𝑠 𝜈𝜈 = 𝛿𝛿𝛮𝛮(𝜈𝜈)
⇒ 𝑠𝑠 𝑡𝑡 = �−∞
+∞𝛿𝛿(𝜈𝜈 − 𝛮𝛮) 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝜈𝜈
= 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝛮𝛮𝑡𝑡
𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖Ν𝑡𝑡
⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿(𝜈𝜈 − 𝛮𝛮)
1762
Fourier transform of common functions / distributions
𝑠𝑠 𝑡𝑡 = 𝛿𝛿𝑡𝑡0(𝑡𝑡)
⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝛿𝛿(𝑡𝑡 − 𝑡𝑡0) 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= 𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡0
𝑠𝑠 𝑡𝑡 = 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖0𝑡𝑡
⇒ �̂�𝑠(𝑡𝑡) = 𝛿𝛿(𝜈𝜈 − 𝜈𝜈0)
𝑠𝑠 𝑡𝑡 = cos(2𝜋𝜋𝜈𝜈0𝑡𝑡)
⇒ �̂�𝑠(𝑡𝑡) =12 𝛿𝛿 𝜈𝜈 + 𝜈𝜈0 + 𝛿𝛿(𝜈𝜈 − 𝜈𝜈0)
𝑠𝑠 𝑡𝑡 = sin(2𝜋𝜋𝜈𝜈0𝑡𝑡)
⇒ �̂�𝑠(𝑡𝑡) =12𝑖𝑖 𝛿𝛿 𝜈𝜈 − 𝜈𝜈0 − 𝛿𝛿(𝜈𝜈 + 𝜈𝜈0)
1862
Fourier transform of common functions / distributions
𝑠𝑠 𝑡𝑡 = �1 if 𝑥𝑥 < 𝑇𝑇/20 if 𝑥𝑥 > 𝑇𝑇/2
⇒ �̂�𝑠 𝜈𝜈 = �−𝑇𝑇/𝑖
+𝑇𝑇/𝑖𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
=𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡
𝑖𝑖2𝜋𝜋𝜈𝜈−𝑇𝑇/𝑖
𝑇𝑇/𝑖
=𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇 − 𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑇𝑇
𝑖𝑖2𝜋𝜋𝜈𝜈
= 𝑇𝑇sin(𝜋𝜋𝜈𝜈𝑇𝑇)𝜋𝜋𝜈𝜈𝑇𝑇
= 𝑇𝑇sinc(𝜋𝜋𝜈𝜈𝑇𝑇)
1962
Fourier transform of common functions / distributions
𝑠𝑠 𝑡𝑡 = 𝑒𝑒−𝑎𝑎𝑡𝑡2
⇒ �̂�𝑠 𝜈𝜈 = �−∞
+∞𝑒𝑒−𝑎𝑎𝑡𝑡2𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡𝑑𝑑𝑡𝑡
= �−∞
+∞𝑒𝑒− 𝑎𝑎𝑡𝑡2+𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡 𝑑𝑑𝑡𝑡
= 𝑒𝑒−𝑖𝑖2𝑖𝑖2𝑎𝑎 �
−∞
+∞𝑒𝑒− 𝑎𝑎𝑡𝑡2+𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑡𝑡−𝑖𝑖
2𝑖𝑖2𝑎𝑎 𝑑𝑑𝑡𝑡
= 𝑒𝑒−𝑖𝑖2𝑖𝑖2𝑎𝑎 �
−∞
+∞𝑒𝑒− 𝑎𝑎𝑡𝑡+𝑖𝑖𝑖𝑖𝑖𝑖𝑎𝑎
2
𝑑𝑑𝑡𝑡
=𝜋𝜋𝑎𝑎 𝑒𝑒
−𝑖𝑖2𝑖𝑖2𝑎𝑎
2062
Use the basic properties to understand your signal and its transform!
𝑎𝑎 𝑡𝑡 = �𝑚𝑚=0
𝑀𝑀−1
𝑎𝑎𝑚𝑚𝑝𝑝(𝑡𝑡 − 𝑚𝑚𝑇𝑇)
𝜓𝜓 𝜈𝜈 = 𝑃𝑃(𝜈𝜈) 𝑖 𝑀𝑀𝜎𝜎𝑖 + 𝜇𝜇𝑖sin 𝜋𝜋𝜈𝜈𝑇𝑇𝑀𝑀sin 𝜋𝜋𝜈𝜈𝑇𝑇
𝑖
𝑀𝑀 pulses equally spaced by 𝑇𝑇
Random amplitudes
Individual pulse
(variance 𝜎𝜎𝑖, mean 𝜇𝜇)
× {Continuum + discrete harmonics}Individual
pulse power
Expected Fourier spectra for Gaussianand double exponential pulses
o Analytic expressions of Fourier transforms are often available even for complex signalso Example: random pulse train
o Well know result in telecommunications! (see also Auchère et al. 2018, ApJ, 827, 152)
2162
The measurement process
o Physical measurements are in general a sample, discrete and finite in space and time, of an underlying signal that is more extended in time and space.o Light curve of a star between 𝑡𝑡1 and 𝑡𝑡𝑖 every Δ𝑡𝑡o Spectrum of a star between 𝜆𝜆1 and 𝜆𝜆𝑖 every Δ𝜆𝜆o Image of an object between (𝜃𝜃1, 𝜃𝜃𝑖, 𝜙𝜙1, 𝜙𝜙𝑖) at the angular resolution Δ𝜃𝜃, Δ𝜙𝜙o Etc.
o The signal has intrinsic cutoff frequencies in time and space: SDo Characteristic times of the physical processes (e.g. cooling time)o Natural width of spectral lineso Etc.
o The measurement apparatus introduces noise and thus frequencies potentially higher than the intrinsic frequencies
o The measurement apparatus itself has intrinsic cutoff frequencies: SC
o Finite response timeo Finite integration timeo Spatial or spectral resolution (e.g. Airy disk)o Etc.
2262
The continuous signal and its transform
o Let’s assume a continuous signal, e.g. the spectrum of a staro The transform appears ‘noisy’: beats dues to the various components required to
reproduce the signalo The transform is symmetrical (more on this later)o The transform is zero above a given cutoff frequency SD.o In a well-designed experiment, one has SC > SD
o The noise expands the range of frequencies over which the transform is non zero
2362
The discrete signal and its transform
o We now sample the signal regularlyo One value every Δ𝑡𝑡 or one value every Δ𝜃𝜃, Δ𝜙𝜙, etc.o The sampled signal is 𝑠𝑠𝑠𝑠 = 𝑠𝑠𝑐𝑐ШΔ𝑥𝑥 (Ш = «Dirac comb», “sha” Cyrillic letter)
o The continuous transform of this signal is thus:
o The sampling induces a replication of the transform every Δ𝜈𝜈 = 1/Δ𝑥𝑥
o We have �Ш𝑎𝑎 = Ш1/𝑎𝑎 and �𝑎𝑎 .𝑏𝑏 = �𝑎𝑎 ∗ �𝑏𝑏
o Thus �𝑠𝑠𝑠𝑠 = �𝑠𝑠𝑐𝑐 ∗�ШΔ𝑥𝑥 = �𝑠𝑠𝑐𝑐 ∗ Ш1/Δ𝑥𝑥
o The continuous transform of a discrete signal is thus periodic!o One replica (alias) centered on each peak of the Dirac comb.
o Other way: the above expression is that of a Fourier series ! And these represent periodic functions.
𝑠𝑠(𝑥𝑥𝑗𝑗): signalN: number of samples𝑥𝑥𝑗𝑗 = 𝑗𝑗Δ𝑥𝑥
�𝑠𝑠𝑠𝑠 𝜈𝜈 = �𝑗𝑗=0
𝑁𝑁−1
𝑠𝑠𝑐𝑐(𝑥𝑥)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑥𝑥𝑗𝑗
2462
The sampling theorem
o A bad sampling choice can lead to the superimposition of the replicas (aliases). The transform is then « aliased »
o The frequency in between two replicas is called the Nyquist frequency: SNy = 0.5/Δ𝑥𝑥o The sampling theorem
o 1/Δ𝑥𝑥 must be greater or equal to twice the maximum frequency Sc contained in the signal
o If this condition is verified, the discrete samples contain all the information contained in the continuous signal
2562
Aliasing is back!
o Beware of noise!o The measurement noise is broad-bando If possible a low-pass filter must be included in the instrumento Hard to escape to photon noise
o The measurements are finite in space and timeo Represented by the multiplication of the signal by a rectangular function Π(x)o Produces high frequencies due to the discontinuity between the beginning and end of
the measuremento The transform can be aliased even if the sampling is sufficient wrt sampling theorem!
2662
Aliasing still!
o Due to the finite nature of the measurement, the transform is convolved by a sinc functiono 𝑠𝑠 = 𝑠𝑠0ΠΔ𝑥𝑥 → �̂�𝑠 = �𝑠𝑠0 ∗ �ΠΔ𝑥𝑥 → �̂�𝑠 = �𝑠𝑠0 ∗ sinc1/𝑁𝑁Δ𝑥𝑥
o The practical transform can’t be identical to the transform of the underlying signal even in the limit of a continuous signalo We will see how to deal with that
o The longer the observing time / the FOV extended / etc., the higher the frequency resolution: 𝛿𝛿𝜈𝜈 = 1/𝑁𝑁Δ𝑥𝑥
2762
Aliasing (the end)
o Aliasing: the transform at frequency 𝜈𝜈 is contaminated by frequencies > Nyquisto The Fourier representation of undersampled data is ambiguous:
o The 5 right-hand side components can’t be distinguished from the lefto The problem can’t be solved by calculating the transform at frequencies > Nyquisto There is no additional information beyond Nyquisto Did I say that there is no additional information beyond Nyquist?o One last time: there is no additional information beyond Nyquist
2862
DFT (Discrete Fourier Transform)
o Given the length 𝑁𝑁Δ𝑥𝑥 of the measurement, the frequency resolution is 1/𝑁𝑁Δ𝑥𝑥o We thus evaluate the transform at 𝑁𝑁 discrete frequencies separated by Δ𝜈𝜈 = 1/𝑁𝑁Δ𝑥𝑥
�̂�𝑠 𝜈𝜈𝑘𝑘 = �𝑗𝑗=0
𝑁𝑁−1
𝑠𝑠(𝑥𝑥𝑗𝑗)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗
o The inverse transform is
𝑠𝑠 𝑥𝑥𝑗𝑗 = �𝑘𝑘=0
𝑁𝑁−1
�̂�𝑠(𝜈𝜈𝑘𝑘)𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗
o The transform of a discrete signal is periodic ↔ the underlying signal itself is periodic !
2962
Symmetry properties
o The N measured values are realo The transform is composed of 𝑁𝑁 complex valueso Symmetry with respect to the origin and Nyquist
o �̂�𝑠 𝜈𝜈 = �̂�𝑠 −𝜈𝜈 ∗
o �̂�𝑠 𝜈𝜈𝑁𝑁𝑁𝑁 + Δ𝜈𝜈 = �̂�𝑠 𝜈𝜈𝑁𝑁𝑁𝑁 − Δ𝜈𝜈 ∗
o The imaginary part of the coefficients at the origin and at Nyquist is zero
o There are 𝑁𝑁 independent Fourier coefficientso 𝑁𝑁/2 + 1 realo 𝑁𝑁/2 − 1 imaginary
3062
Dealing with low frequencies (if necessary!)
o Subtract the mean of the signal first!
o The mean is simply the value of the transform at the origin
o Using zero-mean signal simplifies the treatment of the edges
o Low frequency trends produce a peak near the origin. Its intensity is redistributed by the window function (convolution by sinc for a square window)
o Possible to de-trend the data prior to Fourier analysiso But beware of nasty side effects !o More on this later …
3162
Treating the edges
o The discontinuities at the edged create aliasingo The is minimized by forcing a smooth transition at the edgeso Apodization of the signal with a window functiono Equivalent to replace the square window (finite length of the measurements) by a window
transitioning smoothly to zero: 𝑠𝑠 𝑡𝑡 = 𝑠𝑠0 𝑡𝑡 ∗ 𝑊𝑊(𝑡𝑡) → �̂�𝑠 𝜈𝜈 = �̂�𝑠0 𝜈𝜈 ∗ �𝑊𝑊 𝜈𝜈o There is a large number of possible windows, each with advantages and drawbacks
o Generally, a wide window will have a good frequency resolution but high lobes,o and vice versa
3262
Treating the edges
o Hann
o Tukey
3362
Summary
o Transform �̂�𝑠 𝜈𝜈𝑘𝑘 = ∑𝑗𝑗=0𝑁𝑁−1 𝑠𝑠(𝑥𝑥)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗 and its inverse 𝑠𝑠 𝑥𝑥𝑗𝑗 = ∑𝑘𝑘=0𝑁𝑁−1 �̂�𝑠(𝜈𝜈𝑘𝑘)𝑒𝑒𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗
o The Fourier transform is periodic and symmetric with respect to the origin and Nyquist
o The measurement is one period of an infinite, periodic signal
o If possible the instrument must have a low pass filter and the signal must be sampled at high enough frequency to avoid aliasing: 1/Δx > 2𝑆𝑆𝑐𝑐
o The mean should be subtracted. Low frequency trends can be subtracted (if the application requires it, see later)
o If necessary, a window can be applied to minimize the impact of discontinuities at the edges
3462
What about unenvenly sampled data?
o The definition of the discrete Fourier transform still holds for unevenly spaced data
�̂�𝑠 𝜈𝜈𝑘𝑘 = �𝑗𝑗=0
𝑁𝑁−1
𝑠𝑠(𝑥𝑥𝑗𝑗)𝑒𝑒−𝑖𝑖𝑖𝑖𝑖𝑖𝑖𝑘𝑘𝑥𝑥𝑗𝑗
o The periodogram is the power spectrum of this evevenly spaced data (Scargle, 1981, 1982)
o Please READ the papers before using ito The periodogram is not a magical tool that will let you violate information theoryo You can’t beat Shannon (no information beyond Nyquist)o The smoother peaks contain no additional information (merely interpolations)o Choice of optimal frequencies will depend on the observation window
o The Scargle periodogram has the same statistical properties as for evenly spaced data
o If the sampling is not pathological, then the detection statistics presented below is valid
3562
The Fast Fourier Transform
o FFT = Fast Fourier Transform
o The FFT computes exactly and efficiently the DFTo The acronym FFT is often used (wrongly) as synonymous of FTo Directly computing the DFT requires O(N2) operationso The FFT does the same in O(N log N) operationso The gain is of the order of N / log(N): N=10 gain 10, N=106 gain >105, etc.o The FFT is also more accurate than the direct evaluation of the DFT
o Several FFT algorithms existo Cooley-Tukey, Prime-factor, Bruun, Rader, etc.o Dedicated algorithmes for real data, and / or symmetric
o Cooley-Tukeyo Published by J. W. Cooley and J. W. Tukey in 1965o In fact already known by Gauss in 1805 …o Maximum error O(ε log N) instead of O(εN3/2) for straight DFTo RMS error in O(ε √log N) for Cooley–Tukey and O(ε √N) for the DFT
3662
Applications: filtering, convolutio, deconvolution, compression, etc. etc. etc. etc.
o Frequency filtering is very easy to implement in the Fourier space:𝑠𝑠 𝑡𝑡 = 𝑠𝑠0 𝑡𝑡 𝐹𝐹 → �̂�𝑠 𝜈𝜈 = �𝑠𝑠0 𝜈𝜈 ∗ �𝐹𝐹 𝜈𝜈See convolution in the Hands on session
o Visualizing of the power spectrum is a good way to choose the filter
3762
Wavelet analysis
o Will use the code by Torrence, C. & Compo, G. P. 1998, BAMS, 79(1), 61-78o Reasons for success (2602 citations, >0.5 cite/day)
o Practical guide to wavelet analysis (very good paper, READ it!)o Provides quantitative confidence levels for wavelet spectrao Ready to use IDL code (can be used as a black box: dangerous!)o Produces compelling graphics
o But use it properly!o Global confidence levelso Proper background noise model
3862
Wavelet transform
o If the signal is non stationary, how to have information on both the frequency and the location?o Crude approach: run Fourier transforms over a
running segment and vary the segment length →Windowed Fourier Transform
o If resolution in time no resolution in frequency
o Need for a scale independent method → wavelets:
o Ψ0 𝜂𝜂 = 𝜋𝜋−1/4𝑒𝑒𝑖𝑖𝜔𝜔0𝜂𝜂𝑒𝑒−𝜂𝜂2/𝑖 (Morlet)𝜂𝜂 = 𝑡𝑡/𝑠𝑠 (dimensionless ‘‘time’’)
o Ψ0 𝜂𝜂 must have zero mean and be bounded in time and frequency
o Wavelet can be real or complex functions
3962
Wavelet transform
o The continuous wavelet transform of a discrete signal (TC98 notations), defined as
𝑊𝑊𝑛𝑛 𝑠𝑠 = �𝑛𝑛′=0
𝑁𝑁−1
𝑥𝑥𝑛𝑛′ Ψ∗ 𝑛𝑛′ − 𝑛𝑛 𝛿𝛿𝑡𝑡𝑠𝑠
is the convolution of 𝑥𝑥𝑛𝑛 with a scaled and translated (and normalized) version of Ψ0 𝜂𝜂
o Convolution theorem → the wavelet transform can be easily computed in Fourier space
𝑊𝑊𝑛𝑛 𝑠𝑠 = �𝑘𝑘=0
𝑁𝑁−1
�𝑥𝑥𝑘𝑘 �Ψ∗ 𝑠𝑠𝜔𝜔𝑘𝑘 𝑒𝑒𝑖𝑖𝜔𝜔𝑘𝑘𝑛𝑛𝑛𝑛𝑡𝑡
𝜔𝜔𝑘𝑘 = ± 𝑖𝑖𝑖𝑘𝑘𝑁𝑁𝑛𝑛𝑡𝑡
𝑘𝑘 ≤ 𝑁𝑁𝑖
, 𝑘𝑘 > 𝑁𝑁𝑖
o Wavelet power spectrumo The wavelet trasnform is imaginary and by analogy with the Fourier transform one
defines real part, imaginary part, amplitude, phase, and power
𝑃𝑃𝑛𝑛 𝑠𝑠 = 𝑊𝑊𝑛𝑛(𝑠𝑠) 𝑖
4062
Which one to chose? Depends on the application!
4162
Power spectrum normalization (Fourier or wavelets)
o It is useful to be able to compare different power spectrao Remember Parseval in Fourier
�−∞
+∞𝑎𝑎𝑖 𝑡𝑡 𝑑𝑑𝑡𝑡 =�
−∞
+∞𝑎𝑎(𝜈𝜈)
𝑖𝑑𝑑𝜈𝜈
o Discrete equivalent for a zero-mean signal of variance 𝜎𝜎0𝑖:
1𝑁𝑁�
𝑖𝑖=0
𝑁𝑁−1
𝑠𝑠 𝑥𝑥𝑖𝑖 𝑖 = 𝜎𝜎0𝑖 =1𝑁𝑁�
𝑖𝑖=0
𝑁𝑁−1
�̂�𝑠(𝜈𝜈𝑖𝑖) 𝑖
o We have equivalent expressions for wavelets (the total energy is conserved under wavelet transform)
o Since the expectation value of white noise is 𝜎𝜎0𝑖at all frequencies (and at all scales for wavelets), we normalize all power spectra to 𝜎𝜎0𝑖 (i.e. multiply the IDL Fourier spectrum by 𝑁𝑁/𝜎𝜎0𝑖, but check the definition of YOUR Fourier transform). This gives a measure of power relative to white noise. This is the convention used in TC98.
4262
o A Fourier or wavelet spectrum shows a bunch of peakso Which ones are significant?
o How to define confidence levels?
o Robust quantitative detection is importanto Many oscillatory phenomena have been reported in the astrophysics literature
o In the solar corona: periods from sub-second to tens of hours, in all types of structures
o Several authors have questioned the soundness of some of these detections
o Cenko et al. 2010, Gruber et al. 2011, Inglis et al. 2015, Ireland et al. 2015, Auchère et al. 2016, Threlfall et al. 2017
o The methods and / or assumptions used in several papers are flawed/wrong
o Beware of your brain / eyes: ‘‘I can see the pattern’’ is not a statistical proof!
o Preconceptions (“there must be periods in my data”)
o Unconscious biases
o Optical illusions
o It’s OK not to find periods !o A negative result often tells more than a positive one
Search for periodic components in a noisy signal
4362
cf. Stephen Jay Gould (1991) on the glow-worms in the Waitomo cave of New Zealand
To find a periodic signal is to find a pattern in randomness. Which one is random?
4462
Fourier transform of noise (stationary process, constant variance)
o The Fourier transform of Gaussian noise is white (same expected value at all frequencies)o The PSD of noise is distributed as 𝜒𝜒𝑖𝑖 at each frequency:
o Both real and imaginary parts are Gaussian distributedo If 𝑍𝑍1,𝑍𝑍𝑖 … ,𝑍𝑍𝑘𝑘 are 𝑘𝑘 independent, standard, normal random variables, then:
𝑄𝑄 = ∑𝑖𝑖=1𝑘𝑘 𝑍𝑍𝑖𝑖𝑖 ~𝜒𝜒𝑘𝑘𝑖
o Taking into account normalization factors: 𝑃𝑃 𝑝𝑝 = 𝑒𝑒− 𝑝𝑝𝜎𝜎02
o Most probable value is 0 while the mean is 𝜎𝜎0𝑖
4562
Confidence levels: Local vs. Global (e.g. Scargle 1982, Gabriel et al. 2002)
Red noiseWhite noise
N =
256
N =
819
2
Local confidence levels𝑃𝑃(𝑚𝑚) = 𝑒𝑒−𝑚𝑚𝑚𝑚 = − ln 𝑃𝑃95% confidence level
𝑃𝑃 = 0.05𝑚𝑚 = 3.0
Global confidence levels𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚) ⁄𝑁𝑁 𝑖
𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑒𝑒−𝑚𝑚 ⁄𝑁𝑁 𝑖
𝑚𝑚 = − ln 1 − 1 − 𝑃𝑃𝑔𝑔⁄𝑖 𝑁𝑁
95% confidence level:𝑃𝑃 = 0.05𝑁𝑁 𝑚𝑚
256 7.8
8192 11.3
4662
How to find periods in any time series in 3 easy steps with the TC98 code
True El Nino time series Fake El Nino time series
1. Take a signal that has a power law Fourier spectrum (basically any Solar time series)2. De-trend using a running boxcar filter3. Apply blindly the (very good) Torrence & Compo wavelet package
or more generally do not estimate properly the background poweror trust your eyes/brain
What happens?
4762
Important observation
Irel
and,
J. e
t al.
2015
, ApJ
, 798
, 1
Power laws (or similar) are everywhere: −1.3 < α < −4.
4862 1st problem: combined with a power law, detrending produces a colored signal
𝑠𝑠𝑑𝑑 =𝑆𝑆0(𝑡𝑡)
𝑆𝑆0(𝑡𝑡) ∗ 𝑎𝑎(𝑡𝑡) − 1 ≈𝛿𝛿𝑆𝑆0 𝑡𝑡 − 𝛿𝛿𝑆𝑆0 𝑡𝑡 ∗ 𝑎𝑎(𝑡𝑡)
𝑆𝑆0(𝑡𝑡)
Effects of detrending
Foul
lon
et a
l. 20
009,
ApJ
, 700
, 165
8
4962
Detrending of a time series whose PSD is a power-law
1∆𝑇𝑇
4 + 𝑠𝑠2Maximum ~
Ψ 𝜐𝜐 = 𝑎𝑎𝜐𝜐𝑠𝑠 1 −sin 𝜋𝜋𝜐𝜐Δ𝑡𝑡𝜋𝜋𝜐𝜐Δ𝑡𝑡
𝑖
𝑠𝑠𝑑𝑑 =𝑆𝑆0(𝑡𝑡)
𝑆𝑆0(𝑡𝑡) ∗ 𝑎𝑎(𝑡𝑡)− 1 ≈
𝛿𝛿𝑆𝑆0 𝑡𝑡 − 𝛿𝛿𝑆𝑆0 𝑡𝑡 ∗ 𝑎𝑎(𝑡𝑡)𝑆𝑆0(𝑡𝑡)
5062
On simulated data, α=-2
Obvious pulsations, right?
5162
The detrended signal IS narrowband (while the original IS NOT!)
Obvious pulsations, right ?
2nd problem: white noise is NOT a pertinent background model
5262
Periods tend to be found close to the cutoff of the detrending filter…Object Analysis Noise model Data Detrending Filter width Period
EUV filament Wavelets (TC) red EIT195 running boxcar (M) 20h 10-20h
Full disk EUV Wavelets (TC) red SEM304 running boxcar (M) 14h 12-15h
EUV filament Fourier white EIT195 running boxcar (M) 14h 12h
Full disk EUV Fourier white SEM304 running boxcar (M) 14h 12h
EUV filament Fourier white EIT195 running boxcar (M) 50h 28-40h
Full disk EUV Fourier white SEM304 running boxcar (M) 50h 28-40h
EUV filament Wavelets (TC) red EIT195 running boxcar (M) 14h 8-16h
EUV filament Fourier white EIT195 running boxcar (M) 14h 12h
QPP Fourier white NoRH17 running boxcar (S) 20s 16s
QPP Fourier white SXT running boxcar (S) 20s 16s
none Fourier white Red noise simulations running boxcar (S) 50s 33-100s
QPP Visual running boxcar (S) 20s 1-30s
Coronal bright points Wavelets (TC) white ? TRACE1216 & EIT195 smoothed running difference 3.5m 8-64m
Coronal bright points Wavelets (TC) white? SUMER SVI933 running boxcar (S) 1500s 491s
Coronal hole Wavelets (TC) white EIS HeII Fe XI Fe XII runnning boxcar ? 600s 394-860s
Magnetic flux asymmetry Wavelets (TC), Fourier white NOAA magnetograms running boxcar ?
None ? 1yr? 1.5-3.6yr
Loops, QS BP Wavelets (TC) white? EIS Fe XIII I, v running boxcar (S) 10 min 4.5-13.8 min
Loops Wavelets (TC) white EIS Fe XII v running boxcar (S?) 10 min 4-6 min
XBP Wavelets (TC) ?
QPPs Fourier white XRT ? 28-60 min
3th problem: AR(1) noise model of TC98 is not a good model either
5362 4th problem: used confidence levels are usually local
AR(1)𝑥𝑥𝑛𝑛= 𝛼𝛼𝑥𝑥𝑛𝑛−1 + 𝑧𝑧𝑛𝑛
PSD1 − 𝛼𝛼𝑖
1 + 𝛼𝛼𝑖 − 2𝛼𝛼cos 2𝜋𝜋𝜐𝜐𝛿𝛿𝑡𝑡
𝛼𝛼 determined from the data
If 𝛼𝛼 ≈ 1 and 𝜐𝜐 ≪ 1
⇒ 𝑃𝑃𝑆𝑆𝑃𝑃 ≈1 − 𝛼𝛼
2 𝜋𝜋𝜐𝜐𝛿𝛿𝑡𝑡 𝑖
But TC98 cade CAN be used with any background noise model! (see hands-on)
TC98 red noise model on simulated data, α=-3
5462
Local confidence levels (as done in TC98)
o In the wavelet spectrumo The power at each point is 𝜒𝜒𝑖𝑖 distributedo Monte Carlo simulations of white & AR(1) red noise
5562
Local confidence levels (as done in TC98)
o In the time-averaged wavelet spectrumo The power at each point is not 𝜒𝜒𝑖𝑖 distributedo Sum of a number of correlated 𝜒𝜒𝑖𝑖 distributed variableso The number and correlation depends on the scaleo Number of degrees of freedom determined from Monte-Carlo simulations
𝜈𝜈 = 2 1 + 𝑛𝑛𝑎𝑎𝑛𝑛𝑡𝑡𝛾𝛾𝑠𝑠
𝑖
5662
How significant are extended peaks of wavelet power?
o For an event or probability 𝑝𝑝, the probability to get 𝑘𝑘 successes in 𝑙𝑙 trials is
𝑃𝑃𝑏𝑏 𝑘𝑘; 𝑙𝑙, 𝑝𝑝 =𝑙𝑙!
𝑘𝑘! 𝑙𝑙 − 𝑘𝑘 !𝑝𝑝𝑘𝑘 1 − 𝑝𝑝 𝑙𝑙−𝑘𝑘
o Example with 95% confidence levels (𝑝𝑝 = 0.05)o TC98 give the degree of freedom at each scale: 𝑙𝑙 = 4.5o Length of the structure 𝑘𝑘 = 1.8o → 𝑃𝑃𝑏𝑏~2 × 10−𝑖(upper boundary due to binomial distribution approximation)o Local probability in the time-averaged spectrum 10−3
5762
Global confidence levels: wavelet spectrum
o Generalization of the Fourier global confidence levels
o The bins of the wavelet spectrum are notindependento Intuitively, 𝑛𝑛 should be the number of degrees
of freedom in the spectrum, outside the COI, normalized by the scale resolution 𝛿𝛿𝑗𝑗
o 𝑎𝑎 should be close to 1o 𝑎𝑎 and 𝑛𝑛 are empirically derived
o Monte-Carlo simulationso 100 000 white noise time-serieso For each couple 𝑁𝑁, 𝛿𝛿𝑗𝑗:
o For each local probability 𝑃𝑃 𝑚𝑚 ,compute the global probability 𝑃𝑃𝑔𝑔(𝑚𝑚)
o Fit the resulting data (Morlet)𝑎𝑎 = 0.801 𝑁𝑁𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 0.01
𝑛𝑛 = 0.491 𝑁𝑁𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 0.9𝑖6
𝑃𝑃𝑔𝑔 𝑚𝑚 = 1 − 1 − 𝑃𝑃 𝑚𝑚𝑁𝑁𝑖
→ 𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚)𝑎𝑎 𝑛𝑛
5862
Global confidence levels: time-averaged wavelet spectrum
o Same principle for the time-averaged wavelet spectrumo Number of degrees of freedom a function of scaleo But probability is relative to the expected power at each scale
𝑃𝑃𝑔𝑔(𝑚𝑚) = 1 − 1 − 𝑃𝑃(𝑚𝑚)𝑎𝑎 𝑛𝑛
o Monte-Carlo simulationso Compute 𝑃𝑃𝑔𝑔(𝑚𝑚) for varying 𝑃𝑃(𝑚𝑚)o Fit the resulting data (Morlet)
𝑎𝑎 = 0.805 + 0.45 × 2−𝑆𝑆𝑜𝑜𝑜𝑜𝑜𝑜𝑛𝑛𝑗𝑗𝑛𝑛 = 1.136 𝑆𝑆𝑜𝑜𝑜𝑜𝑡𝑡𝛿𝛿𝑗𝑗 1.𝑖
5962
All this in practice (no detrending + custom noise model + global confidence levels)
𝜎𝜎 𝜈𝜈 = 𝑎𝑎𝜐𝜐𝑠𝑠 + Κ𝜌𝜌 𝜐𝜐 + 𝐶𝐶
Demonstration code: https://idoc.ias.usud.fr/MEDOC/wavelets_tc98
+ photon noise+ pulseBackground activity
The period seems evident in that case, but much can be learned from the understanding of the spectral shape → train of random pulses, not a wave!
6062
Are these confidence levels too harsh?
o The short answer: no (even if they can be disappointing)
o Bias of the TC98 code (higher peak if the signal is low frequency, see their website)o Automatically taken care of by the Monte-Carlo derived confidence levels.
o Peaks are blobs, not single pixels, and correlation higher at low frequency, thus more likely to have many bins above a threshold at low frequency; thus global wavelet levels should be scale dependento No, a “global detection” is defined as a whole spectrum with at least one bin above a
threshold. Does not matter if a blob or a bin.o Automatically taken care of by Monte-Carlo
6162
Analysis of an extended field of view
o Compare only normalized power values! (i.e. whitened, or divided by the expected power)o With non-normalized spectra, if the power in one pixel is higher it may simply be because the
variance of the signal is higher there (the power is higher at all frequencies, more activity !)
o The principle behind 𝑃𝑃𝑔𝑔 𝑚𝑚 = 1 − 1 − 𝑃𝑃 𝑚𝑚⁄𝑁𝑁 𝑖
can in principle be generalized to 3Do Nearby pixels are not independento Coherence length function of Fourier frequency (and wavelength)
6262
Advices
o Don’t be afraid not to find anything in your data
o Look at your data and at its spectrum (log-log plots!)
o Estimate what to expect in the power spectrum o Analytic solutions to many signals existo Decompose the signal into simpler components and use the basic theorems
(shift, convolution, etc.) to compute the Fourier transform of the signal you think may be in there
o If the signal that you expect is periodic, use Fourier. If there is nothing in Fourier, wavelets won’t help for the detection (it is not “more powerful” because “fancier”).
o Do not use the global wavelet levels for detection of periodic signals. Use the time-averaged spectrum (if you must) or Fourier.
o Wavelets will however give you the extra information on when (or where) there is something significant if the signal is intermittent.