Representation of Digital Signalsjowa/notes/digisig_pres.pdf · Representation of Digital Signals Representation of Digital Signals I) Discrete Signals 1) Discretization and Sampling

Representation of Digital Signals


Why a special lecture?

from NMSOP- Bormann (2002)


Almost every analysis in Geophysics (and meanwhile also in Geology) uses digi-tal data, successive spectral analysis and filtering

Scherbaum, “Of poles and Zeroes” (1996)


Richter et al. (2004)


Representation of Digital SignalsI) Discrete Signals

1) Discretization and Sampling Theorem2) Fourier Transform3) From Analog to Digital

II) Statistical Description of Signals1) Definition deterministic vs. stochastic (stationary)2) Some important statistical Moments3) The uncertainty principle

III) Spectral Estimation1) Non-Parametric Methods2) Parametric Methods3) Time-Frequency-Representation

IV) Digital Filter1) LTI Systems2) The concept of response and transfer functions3) The seismometer as filter4) Problems of FIR and IIR

I) Discrete Signals Discretization - Sampling

!

I) Discrete Signals

1.Discretization - SamplingA continuous Signal Function: taken at specific time steps results in:

;

= sampling interval; = sampling rate or sampling frequency

Note! The amplitude values are still !!!!

xa t( ) Ts

x n[ ] xa nTs( )=

Ts fs1Ts-----=

xa t( ) R∈


Different mathematical notation using the 1-Impulse:

;

and using a series of such “1-Impulses” describes the sampling:

This results in:

δ t( ) 1 if t=0 0 sonst

=

δTst( ) δ t nTs–( )

n ∞–=

∞

∑=

x n[ ] xa t( )δTs t( )=


!
The Sampling TheoremIn order to describe a continuous signal or function complete and unique using amplitude values taken at discrete times , the sampled signal MUST NOT
HAVE energy above a certain frequency . This frequency is also called

Nyquist-Frequency.

The corresponding continuous signal could be reconstructed using a linear

combination of the discrete function weighted by a function :

Tsfs2--- 1

2Ts--------=

xa t( )

sinc t( ) t( )sint

--------------=

xa t( ) x n[ ]sinc πfs t nTs–( )( )n ∞–=

∞

∑=

I) Discrete Signals Sampling - Fourier Transform

2.Sampling - Fourier Transform

Definition:

;

Strictly only valid if: the function is absolute integrable:

Latter point is not always the case in Geophysics!

Xa jω( ) xa t( )e jωt– td∞–

∞

∫= xa t( )⇔ 12π------ Xa jω( )ejωt ωd

∞–

∞

∫=

xa t( ) td∫ c ∞<≤


Important Properties of the Fourier Transform

Time domain Frequency domainx(t) real X(-jω) = [X(jω)]*x(t) imaginary X(-jω) = -[X(jω)]*x(t) = x(-t) (x(t) even) X(-jω) = X(jω) (X(jω) even)x(t) = - x(-t) (x(t) odd X(-jω) = -X(jω) (X(jω) odd)x(t) real and even X(jω) real and evenx(t) real and odd X(jω) imaginary and oddx(t) imaginary and even X(jω) imaginary and evenx(t) imaginary and odd X(jω) real and odd

* complex conjugate


multiplication: convolution:

convolution: multiplication:

differentiation: multiplication:

integration: multiplication:

for

Time domain Frequency domain

* complex conjugate

xa t( )ha t( ) 12π------Xa jω( ) Ha jω( )•

12π------ Xa jΩ( )Ha jω jΩ–( ) Ωd∫=

xa t( ) ha t( )• Xa jω( )Ha jω( )

dx t( )dt

------------ jω X jω( )⋅

xa t( ) td∫ 1jω------Xa jω( )

xa t a–( ) Xa jω( )e jωa– a 0>


Parsevals-Theorem:

;

Back to sampling process:

using

this results in

xa t( ) 2 td∞–

∞

∫1

2π------ Xa jω( ) 2 ωd

∞–

∞

∫=

FT δ t nTs–( )n ∞–=

∞

∑⎝ ⎠⎜ ⎟⎛ ⎞ 2π

Ts------ δ ω k2π

Ts------–⎝ ⎠

⎛ ⎞

k ∞–=

∞

∑ ∆T jω( )= =

ωs 2πfs2πTs------= = ∆T jω( ) 2π

Ts------ δ ω kωs–( )k ∞–=

∞

∑=


!

FT of a sampled signal can be represented by a convolution of and

What the heck does that mean????

The sampled signal will be periodic in frequency (sampling frequency). It follows that the continuous signal can be reconstructed using only one

period. Only valid if the sampling theorem is not violated and no energy above

is present in the signal :;

FT ∆T

FT Xa jω( )

X jω( ) 1Ts----- Xa jω kjωs–( )k ∞–=

∞

∑=

x n[ ] ωsx t( )

ωs2

------

x t( )Xa jω( ) TsX jω( )=


!
The sampling theorem MUST be applied BEFORE the sampling process. There-fore an analog lowpass filter must be applied before sampling - regardless which sampling frequency is used. The corner frequency (!) of that filter should satisfy:
.fc 0.4 fs⋅=


a) FT of analog signalb) FT of discrete signal (sampling theorem complied)c) FT of discrete signal (sampling theorem violated)

ω

ω

ω

ωs 2⁄ω– s 2⁄

ωs 2⁄ 3ωs 2⁄ω– s 2⁄3ωs 2⁄–

Ts X jω( )⋅

Ts X jω( )⋅

Xa jω( )

a

b

c


Consequence of violation - ALIASING:

Problem 1:Assume we are sampling with 125 Hz without an analog lowpass. Estimate the alias frequencies of noise signals at 70 Hz, 120 Hz and 300 Hz!

0ωs2

------

ωs2

------ωs

ωs 3ωs2

---------


Sampling Theorem complied


Sampling Theorem violated


Consequence: error in reconstruction

I) Discrete Signals A/D-Conversion

3.A/D-Conversion

Decimal system:

;

Example:

LSB MSB

Binary system:

Example: represents “Little Endian”

LSB MSB 000000001

x 10( ) di 10( )10i

i∑=

1024 10( ) 4 100⋅ 2 101⋅ 0 102⋅ 1 103⋅+ + +=

x 2( ) di 2( )2i∑=

512 10( ) 0 20⋅ … 0 28⋅ 1 29⋅+ + +=


!

A 16 bit A/D-converter could represent in principle output states in its maxi-mum (values between 0 -( -1) are possible).

The LSB (least significant bit) or smallest step width of the A/D-converter (resolu-tion) is defined by:

.

As the resolution is directly dependent on the number of bits, a n-bit A/D-converter has “n-bit” resolution. Unfortunately, there is no rule, which would specify a “criti-cal” number of “must have” bits. It is simply like that: if we have more bits we will decrease the noise added to the signal

216

2n

LSB Maximale Voltage2n

-------------------------------------------- Q= =



An equivalent important parameter of A/D conversion is the so called dynamic range:

and therefore

Note: this definition intrinsically assumes proportionality to power (20*log) of the signal - NOT energy (10*log)!

16 bit A/D-converter: 90dB;24 bit A/D-converter: 138 dB;

Be aware of the sign!

D 20log10AmaxAmin------------⎝ ⎠⎛ ⎞=

D 20log10 2n 1–( ) nlog10 2( )≈ n 6⋅= =

III) Statistic Description Definition: deterministic vs. stochas-tic

III) Statistic Description

1.Definition: deterministic vs. stochastic

stochastic

deterministic

III) Statistic Description Definition: deterministic vs. stochas-tic

!
Deterministic signals are, at least in principle, absolute reproducible. If the bound-ary conditions are the same an event could be completely reproducible. Two earthquakes exactly at location and source mechanism will produce exactly iden-tical seismograms (if the propagation properties did not change!). The propaga-tion medium is modelled as a so called Linear-Time-Invariant Filter (LTI).
On the other hand seismic noise measurements used in microzonation analysis, volcanic tremor and microseisms defy themselves of this simple description. These signals could not be forecasted as well as exactly described.

To describe stochastic signals, we have to learn some theorems of probability

and the probability density function: . Pr x( ) Pr x a≤( ) f x( ) xd∞–

a

∫=

III) Statistic Description Some Statistical Moments

The complete description of a statistical parameter requires the estimation of probability of every event in the parameter space. However, this is not always pos-sible and, fortunately, not always needed. Sometimes it is sufficient to describe the mean behavior of the variable.

2.Some Statistical Moments

a) Sample Mean

Example: Dice: random variable; the dice was thrown N-times; the number occurred

times. The sample mean is defined through:

x k nk

x⟨ ⟩N1n1 2n2 3n3 4n4 5n5 6n6+ + + + +

N-------------------------------------------------------------------------------------=


!

Approximation for : and therefore:

The expectation value of the variable is defined by:

bzw.

discrete continuous

Problem 2:Can we expect an exact estimation of the expectation value using digital signals?

N 1»nkN----- Pr x=k( )≈

x⟨ ⟩N kPr x=k( )k 1=

6

∑≈

E x αkPr x=αk( )k∑= E x αfx α( ) αd

∞–

∞

∫=


!

!

b) Variance

The variance describes the deviation of the actual value from the expectation value of a variable. It is defined by:

c) Covariance and Correlation

Often we want to estimate the statistical relationship of two or more variables. A frequently asked questions is whether two variables are statistically independent or not:

Correlation: Covariance:

complex conjugate and ,

Var x σ2 E x E x –( )2 E x2 E2 x –= = =

rxy E xy* = cxy Cov x y, E xy* mxmy*–= =

* mx E x = my E y =


!

!

Sometimes the correlation coefficient - the normalized covariance is used instead:

with ;

Definition:Two random variables are statistical independent if their joint probability density function is separable:

A somewhat weaker but more frequently used formulation uses the separability of covariance or correlation:

If or than the signals are called uncorrelated.

ρxyE xy* mxmy*–

σxσy---------------------------------------= ρxy 1≤

fx y, α β,( ) fx α( )fy β( )=

E xy* E x E y* = rxy mxmy*=


An additional property of uncorrelated signals is:

If , than the signals are called orthogonal. Consequence: two signals with zero mean, which are uncorrelated are always orthogonal.

d) Bias and Consistency

Problem: only part of data are available (often the case in seismology/geophys-ics). Can we estimate the already defined statistical moments? No!

The deviation of the sample moments (e.g., mean and variance) is called bias.

Assume: the statistical moment should be estimated from sample measure-ments of the random variable with .

Var x y+ Var x Var y +=rxy 0=

θxn n 1 …N,=


Assume: the mean of equals the true value .

Than the bias of the estimate is defined by:.

If the estimate is unbiased

The estimate is called asymptotic unbiased if:.

An estimate is consistent if it converge in some way towards its true value

θ θ

B θ θ E θ –=

B θ 0=

E θ N ∞→lim θ=


Example: “Sample-Mean”

.

The expectation value is:

.

Therefore the sample mean is a unbiased estimate.

The variance on the other hand:

mxˆ 1

N---- xnn 1=

N

∑=

E mx 1N---- E xn n 1=

N

∑ mx= =

Var mx 1N2------ Var x n 1=

N

∑σx

2

N------= =


!

is a consistent estimate as its value converges towards zero if N approaches infin-itye) Covariance- Autocorrelation and Cross correlation function

In order to describe a stochastic time series (also for deterministic signals), the definition of autocovariance, cross correlation and cross correlation function is often used. We will introduce these important analysis tools using 1D time series. The extension to n dimensions is easy, however:

Autocovariance function: ;Cross covariance function: ;

Cross correlation function:

What does this mean? How can we use these function in our daily analysis?

cxx l( ) E x t( ) mx–( ) x t l+( ) mx–( ) =

cxy l( ) E x t( ) mx–( ) y t l+( ) my–( )– =

ρxy l( )cxy l( )cxy 0( )---------------=


To put it straight first: the covariance and correlation of deterministic and stochas-tic signals are formally different. However, this difference does not matter in our daily work!

Assume a periodic function: with ;

the autocovariance is then:

It is easy to see that the resulting signal is again periodic with the same fre-quency. However, all phase information is lost!

This is different for a stochastic (random) signal. There will be only one maxi-mum at time lag zero!

for .

x t( ) A0 ω0t φ–( )sin= T02πω0------=

cxx l( )1T0----- A0

2 ω0t φ–( )sin ω0 t l+( ) φ–( )sin tdo

T0

∫A0

2

2------ ωol( )cos= =

cxx l( ) cxx 0( )< l 0>


The behavior of the autocovariance function for gives also some informa-tion about the possibility to predict the corresponding signal!

l ∞→



!
Definition of Stationary m-th Order ProcessesA signal or process is called m-th order stationary if the statistical moments up to order m are existing and identical for all times and all time lags .
Example: 2. order stationary process:

1)

2) 3) only dependent on time lag

X t1 l+( ) … X tn l+( ), ,

t1…tnl

E x t( ) mx const= =

Var x t( ) σ2 const·= =cxx l( ) E x t( ) mx–( ) x t l+( ) mx–( ) = l


Properties of autocovariance and cross covariance function:• the autocovariance and cross covariance function of real signals are real• the autocovariance and cross covariance function of stationary signals is only depending on time lag . Different time segments will result in identical autocova-riance.

• the autocovariance function contain information about the internal coherence (?) of different time segments within the signal .

• the autocovariance has its maximum at time lag 0. On the other hand, the cross covariance will have its maximum at arbitrary (dependent on the signals) time lag . The cross covariance at time lag 0 corresponds to the mean of the product of

both signals.• the autocovariance function is even.

l

x t( )

l


The cross covariance and its normalized version the cross correlation functions are of great importance when analyzing the similarity of two signals or in estimat-ing the time shift between two traces.

Example:An earthquake signal , recorded at two stations, is buried in noise at one station and, as the stations are at different locations, shifted by the travel time dif-ference:

; ;

The cross correlation function of those signal results in:. If: than:

x t( ) n t( )

x1 t( ) s t( )= x2 t( ) s t t0–( ) n t( )+=

ρx1x2l( ) ρss l t0+( ) ρsn l( )+= ρns l( ) ρss 0( )« ρx1x2

l( ) ρss l t0+( )≈



!

Nice but, we can do a couple of things wrong! There are several pitfalls when applying the covariance functions. Therefore, we will hold a moment and look for it!

All given definitions assume infinite sequences or continuous signals. In reality measured signals are always finite. It is only possible to estimate the statistical properties (-> Bias)!

Problem 3: Give the formulas for mean, variance, covariance function and cross covariance function for finite, discrete time signals.


Pitfall 1: What would you expect cross correlating these two traces?


Cure and Pitfall 2: sliding window analysis


Cure and Pitfall 2: sliding window analysis


Pittfall 3: one-sided signals - Why that?

III) Statistic Description The uncertainty principle

3.The uncertainty principleWe already know it from physics: it is not possible to resolve the impulse of a par-ticle as well as its location in arbitrary precision at the same time. Same is true for digital signal processing: It is not possible to increase time and frequency resolution at the same time. If we increase the frequency resolution, we MUST increase the corresponding time win-dow (i.e., decrease the time resolution) and vice versa.

In its best time-frequency analysis will choose an “optimum” of Time/Frequency resolution.


2 spikes (0.6s separated)

(Bandpass: 1.0 - 15 Hz)

(Bandpass: 1.0 - 1.5 Hz)


!
The uncertainty principle in digital signal processing connects time - and fre-quency resolution in following relation:
or

with frequency resolution,and time resolution

∆f∆t 14π------≥ ∆ω∆t 1

2---≥

∆ω ∆f,∆t

Spectral Estimation Non-Parametric Methods

Spectral Estimation

1.Non-Parametric Methods

We have already used the properties of Fourier Transform when discussing the effects of sampling a continuous signal. Mapping time domain signals into the cor-responding frequency space is one of the most fundamental analysis tools in sig-nal processing. To take a short cut, we will immediately start with discrete Fourier Transform based analysis.

First: What can we gain out of that FT-analysis? What does FT means mathemat-ically?

Problem 4: What happens, when we estimate the cross covariance (NOT the function!) of a sinus and a cosine with the same frequency?


!

But wait: remembering definition II.2.c) a zero value means that sinus and cosine are orthogonal functions!

The FT is therefore nothing else but a convolution of the signal with a (infinitive) number of sinuses and cosines - also called basis functions. In other words, the signal is represented by a linear combination of sinuses and cosines. The func-tional space spanned by this “vectors” is called Hilbert space.


a) Discrete Fourier Transform (DFT)

Assume: aperiodic, discrete signal. The discrete Fourier Transform is defined as:

Valid under the condition that the signal is square integrable:

;

The back transform is defined as:

;

X jω( ) x nTs[ ]e jnTsω–

n ∞–=

∞

∑=

x nTs[ ] 2

n ∞–=

∞

∑ c≤ ∞<

x nTs[ ] 12π------ X jω( )ejnTsω ωd

π–

π

∫=


!

NOTE:1) sign convention in the argument of the exponential function;2) FT of infinite length of a discrete signal will be continuous.3) all properties of the FT are also valid in this case (Parseval and Convolu-

tion)

Be aware of fact 2)

The FT (DFT) of a discrete and finite signal is defined as (IEEE):

and its back transform:

X k∆f[ ] Ts x nTs[ ]ej2πkn–N

------------------

n 0=

N 1–

∑=


!

;

The continuous frequency maps on or . Here, represents the

window (length), we have transformed. It is immediately clear that the best fre-quency resolution is defined by the observation length of our signal:

.

As shown before, a discrete signal will have a periodic spectrum. In this case the periodicity is in N.

x nTs[ ] 1NTs--------- X k∆f[ ]e

j2πk nN----

k 0=

N 1–

∑=

ω kTsN---------

kfsN------ 1

NTs--------- 1

T---=

∆f

∆f 1T---=


Amplitude and Phase of DFT:

Applying the DFT we map our signal into the complex plane:.

An often used representation of the resulting complex series is in polar form:

;

The absolute value of the DFT is called amplitude-spectrum, while the quantity stands for the phase-spectrum. The amplitude spectrum gives the amplitude

present in the signal at a certain frequency. However, the phase spectrum of an aperiodic discrete time signal could be interpreted as:

x nTs[ ]

X k∆f[ ] C∈

X k∆f[ ] X k∆f[ ] e jΦ k∆f[ ]–=

Φ k[ ]


; with time shift of the signal (regarding start of analysis window).

Φ k∆f[ ] 2πk∆fτ=τ



!

In other words, the time shift of the signal can be interpret as slope of the corre-sponding phase spectrum.

This estimate is NOT depending on the sampling rate!!!

Note: If the time shift of the signal is large, the phase exists as a multiple of (Phase wrapping). This could be easily seen by:

. is known to be periodic in .

Known Problem in: TDOA-estimation, InSAR, seismic Source decomposition, RADAR, GPS etc.

2π

Φ k[ ] Im X k∆f[ ][ ]Re X k∆f[ ][ ]-----------------------------⎝ ⎠⎛ ⎞atan= atan 2π


In order to avoid this wrapping, so called phase unwrapping is applied. There are several techniques for doing so, but none of them are without drawbacks or doubts.

2π


Windowing - Leakage

The finiteness of the signal bears an additional complexity. Selecting just a portion of a (physical) infinite signal is equivalent to a multiplication of the data with a box-car window:


The resulting signal can be written as: .

The DFT of a box car window is simply:

;

w n[ ]1 if t T

2---≤⎝ ⎠

⎛ ⎞

0 if t T2--->⎝ ⎠

⎛ ⎞=

xT n[ ] x n[ ]w n[ ]=

W jω( ) w nTs[ ]e jωnTs–

n ∞–=

∞

∑ e jωnTs–

n N2----–=

N2----

∑ 2

ωT2

-------⎝ ⎠⎛ ⎞sin

ω---------------------= = =


Remember the rule that multiplication in time domain will result in convolution in frequency domain!The convolution of the FT of a box car with a discrete periodic (cosine) function will result in:

;

If time is an even multiple n of the signals period than:

;

XTaper j2πf[ ] S jg[ ]W jf jg–[ ] gd∞–

∞

∫

T2---

π f f0–( )T( )sinπ f fo–( )T

------------------------------------π f f0+( )T( )sin

π f f0+( )T-------------------------------------+⎝ ⎠

⎛ ⎞

= =

=

T 1f0----

XTaper j2πf[ ]f 2πfn 1 f0⁄( )( )sin

π f2 f02–( )-------------------------------------------=



We are of course not restricted to the boxcar window. We should chose a window, which has more “convenient” properties. Best would be a window function, which has a sharp main lobe and strongly suppressed side-lobes. Unfortunately, there is a trade off between sharpness and side-lobe suppression.


Boxcar

Barlett

Henning/Cosine

Hamming

Welch




The width of the main lobe has severe consequences. Let’s assume a signal,where two spectral lines at and exist. In order to detect both lines, the halfwidth of the window must be smaller than the distance between the two spectrallines.The half width of the window function is defined as width of the main lobe taken at50% of its maximum. For the box car that is:

.

and results in:

; ; ; ;

f f ∆f+

T π∆fTaperT( )sinπfT

------------------------------------------ 12---T=

TBoxcar1 2,∆f

---------> TBarlett1 8,∆f

---------> TTukey2∆f-----> TParzen

2 5,∆f

--------->


!
Note: The boxcar window has the best frequency resolution! The actual frequency
resolution will always be smaller than the formal (DFT).

Note: usually, today algorithms for computing the discrete FT use the so calledFastFourierTransform (FFT), which is optimized (in some way) for computers.However, the data length (in points) must be given in power of 2 (e.g., Press et al.,1988, Numerical Recipes in Fortran (C; C++; Pascal)).

∆f 1T---=


b) Cross spectral estimates, Convolution and Coherence

We have already learned how the convolution of two time traces is transformed in the frequency domain. The discrete version is defined as:

for ;

or using the properties of the DFT:

;

The phase properties can be estimated by:

g mTs[ ] y nTs[ ]x m n–( )Ts[ ]n 0=

N 1–

∑=

m 0 1 … N 1–, , ,=

g mTs[ ] DFT 1– G k∆f[ ] DFT 1– Y k∆f[ ]X k∆f[ ] = =

G k∆f[ ] Y k∆f[ ] X k∆f[ ] ejΦy k∆f( )ejΦx k∆f( )=


!
Note a pitfall:We already recognized that the FT of a finite, discrete signal will produce a peri-odic spectrum. This is also valid for a infinite, discrete signal, but the spectrum is only computed up to the Nyquist frequency as all frequencies above contain no new information. However, if we multiply two DFTs with length and where
and transform the resulting spectrum back into time using the length we will suffer from a so called “wrap around effect”. It appears that part of the convolution trace is wrapped from the “beginning” into the convolution trace.

M1 M2

M1 M2< M1

M1>



To be correct the resulting trace should have a spike at 11s (Why?). As the origi-nal trace is restricted to 10.24 s, the convolution results in a spike at position 0.76 s.

Problem 5: How many samples long should the zero padding be in the latter example in order to avoid the wrap around effect?

The cross spectrum of two discrete signals is defined as:

A not very precise but instructive interpretation of the cross spectrum is, that it estimates how much energy is present in both signals at a certain frequency.

A more important property will be obvious, if we rewrite the definition as:

.

C k∆f[ ] Y k∆f[ ]X* k∆f[ ]=

C k∆f[ ] Y k∆f[ ] X k∆f[ ] ejΦy k∆f( )e jΦx k∆f( )–=


!

Note: The cross phase spectrum is estimated by the difference of the individual phase spectra.

Remembering the relation it is easy to see that the phase of the cross spectrum contains the time difference between both signals

Φ k∆f[ ] 2πk∆fτ=



!

Using the properties of the DFT and the so called Wiener-Khintchine Theorem (we do not care about that at present) enables us to estimate the autocovariance and cross correlation function in a very effective way:

cxy lTs[ ] 12N 1+---------------- x n l+( )Ts[ ]y nTs[ ]

n N– l+=

N l–

∑

DFT 1– X jk∆f[ ]Y * jk∆f[ ]

=

=

ρxy lTs[ ] 12N 1+---------------- x n l+( )Ts[ ]y nTs[ ]

k N– l+=

N l–

∑

DFT 1– X jk∆f[ ]Y* jk∆f[ ]Cxx1 2/ Cyy1 2/

------------------------------------------⎩ ⎭⎨ ⎬⎧ ⎫

=

=


!

!

This leads us directly to a new quantity the so called Coherency:

;

More commonly used is the so called Coherence - the absolute value of the coherency:

;

The coherence is real - but what does it mean?

CohXY jk∆f[ ] X jk∆f[ ]Y* jk∆f[ ]CXX1 2/ CYY1 2/

-----------------------------------------Re CXY[ ] iIm CXY[ ]+

CXX1 2/ CYY1 2/----------------------------------------------------= =

CCohXY jk∆f[ ]Re CXY[ ]( )2 Im CXY[ ]( )2+

CXXCYY-----------------------------------------------------------------=


Let us compute the coherence of two signals:


non-averaged spectrum


!

We observe, that the coherence is equal to one everywhere (actually that is notreally surprising)! Let’s start from scratch. The coherence should be one if two traces are identical(up to a certain phase shift) within a frequency band or conterminously one fre-quency and its phase are stable within both signals within a certain time window.

Consequently, we must either average our cross- and auto spectra within certainfrequency bands (Daniell’s method) or we average the corresponding spectra ofseveral time windows.Using the Daniell approach (average within 2m+1 frequency steps) results in:

.

The same average procedure must be also applied in case of the auto spectra:

Cl jk∆f[ ] 12m 1+----------------- C j k l+( )∆f[ ]

l m–=

m

∑=



The coherence is an important assessment tool for evaluating the similarity of twotraces in frequency domain. This is especially important in array seismology whenthe coherent signal is buried in also coherent noise, acting in a different frequencyrange.


!

d) The Power Spectral Density (PSD)

The DFT (or more general FT) is only defined if the signal is square integrable.The overwhelming numbers of processes are violating this definition. However,these signals have infinite energy (density) but finite power (density). On theother hand, applying this statement to our data we neglect the finiteness of ourdata!There is another reason for using power density, which is more important in dailylife work: if we select a certain time window out of a random, stationary processand estimate its spectral content via a DFT, we suffer from a 100% variance in ourestimate! I.e., applying the DFT for another time window results in a complete dif-ferent spectrum:

Formal definition of power spectral density:

;Pxt1

2T------

T ∞→lim x2 t( ) td

T–

T

∫ P f( ) fd∞–

∞

∫= =


The PSD estimates the power/per frequency bin present in the signal. If we havea random, stationary and zero mean process we can adhere:

;Pxt σ2=

DFT

PSD


!

There are several ways for estimating the PSD, however, we will restrict ourself tothe most simple one (regarding theory). Just for completeness we will introducethe

Theorem of Wiener and Khintchine

and vice versa .

It follows that starting from the PSD, we can only reconstruct the autocovarianceof the corresponding signal but NOT the signal itself.

cxx DFT 1– P f( ) = P f( ) DFT cxx =


!

PSD via Periodogram Estimate

In the most simple approach we use directly the DFT:

However, as already noted, this estimate would have a variance of 100%. In orderto reduce this variance, several modifications are on the market (Taper, averagingin frequency or time, etc.). We will focus on two of the various possibilities: TheDaniell- and the Welch approach.

The Periodogram estimate intrinsically assumes a periodic repetition of theselected data outside the observed interval. Again we multiply the data to be ana-lyzed with a box car window. Applying arbitrary window function we will deal withan expectation value of:

;

PPˆ f( )

TsN----- xne

i2πfnTs–

n 0=

N 1–

∑⎝ ⎠⎜ ⎟⎛ ⎞

2 TsN----- xne

i2πk nN----–

n 0=

N 1–

∑⎝ ⎠⎜ ⎟⎛ ⎞

2

P k∆f[ ]= = =

E Ppˆ k∆f[ ] P f( ) W k∆f[ ] 2

N------------------------×=


!

and Bias:

Using a boxcar the window term will be (FT: ):

.

The estimate of PSD using Periodogram is asymptotically bias free.But as the variance of a periodogram from a random process is:

The variance will be 100% if N approaches infinity. The Periodogram is not a con-sistent estimate

b Ppˆ k∆f[ ] P f( ) W k∆f[ ] 2

N------------------------× P f( )–=

x( )sinx

---------------

WN ∞→lim k∆f[ ] 1→

Var Ppˆ E Pp

ˆ ( )2≥


!
There exists a trade of between bias and variance reduction when analyzingfinite data sequences using the periodogram approach.
Daniell-Estimate:No modified taper is applied but the spectral estimates are averaged within fre-quency bins. This will reduce the variance significantly:

Pbˆ k∆f( ) 1

2M 1+----------------- Pp

ˆ m∆f[ ]m M–=

M

∑=


Welch-EstimateOne of the best direct approaches. Here, segmentation in time, tapering, 50%overlapping data segments and averaging the resulting spectra are applied:

segments of length L and tapering

with the taper and finally averaging over S segments:

, The variance reduction using a boxcar (not the best one! - leackage) would be:

Psw k∆f[ ]TsLU-------- x

l s 1–( )L2---+ule

j2πk lL---–

l 0=

L 1–

∑2

=

U 1L--- ul2

l 0=

L 1–

∑=

Pwˆ k∆f[ ] 1

S--- Pws k∆f[ ]s∑=

Var Pw k∆f[ ] 32---1S---P f( )=


Some words about confidence limits. They are used in order to quantify the qualityof a spectral estimate.Confidence intervals specify the upper and lower limit of the interval, in whichregarding a certain probability, the true value of the spectral estimates will befound.

Estimation of confidence limits:

We must specify according the probability that the true P(f) is smaller

than and that P(f) is smaller than .

The bandwidth or spectral resolution using a periodogram and a box car taper

results in: or segmenting in S segments of length L: .

Prob x1 P k∆f[ ] x2≤ ≤[ ] β=

β 1 α–= α2---

x1 1 α2---– x2

bp1NTs---------= bb

1LTs---------=


!

The variable follows a distribution with degrees of freedom

A single segment gives: ;

Averaging S non-overlapping segments will result in degrees of freedom.Finally we can estimate the confidence interval of P(f) with probability as:

Which can be found in mathematical tablesThe Welch approach with its modified taper and overlapping segments results inslightly less degrees of freedom:

and therefore .

νPp f( )P f( )

---------------- χ2 ν

ν2E2 Pp f( )[ ]

Var Pp f( )[ ]---------------------------- 2≈=

ν 2S=1 α–

νuν

2 1 α2---–

--------------------Pp f( )ν

uν1 α

2---

-------------Pp f( ),

ν2NTs------------- bw= ν

2E2 Pw f( )

Var Pw f( ) ------------------------------ 1 6S,= = bw

1 16,L

------------=


Finally some PSD values of common random processes:

white noise: with autocovariance function ;

red noise: and ;

P f( ) P0 f∀= cxx l[ ] P0δ=

P f( ) σ2 πα

--------------eπ2f2

α2-----------–

= cxx l[ ] σ2e α2l–=

Spectral Estimation Parametric Methods

2.Parametric Methods

We have already notices that applying classical, non-parametric spectral estima-tor will result in a trade off between bias and variance reduction.A different class of spectral estimators is based on signal models (physical pro-cesses), which are generating the signal. As a by-product when estimating theparameters of these models we can also estimate the spectral content of the sig-nal. We will focus only on one of the various existing methods to demonstrate thecapabilities but also the drawbacks of that class of estimators.

AR - Estimate (similar to MEM)An estimate of the PSD or the extrapolation of an autocovariance function will bemore and more successful, if we have a deeper knowledge about the signal or thesignal generating process. Using the theorem of Wold (1938) - all stationary sig-nals can be separated in a deterministic (forecast) and a non-deterministic part - awhole class of signals “recursive processes” come to life.The most fundamental (stationary) process is called Autoregresive-Moving-Aver-age (ARMA):


;

(difference equation of ARMA-process), with the gaussian, random input signal.

There are two special cases existing:

a) Moving Average (MA):

;

b) Autoregressive Model (AR):

y i[ ] α a[ ]y i k–[ ]a 1=

A

∑+ β m[ ]x i l–[ ]m 1=

M

∑ x i[ ]+=

x i[ ]

y i[ ] β m[ ]x i m–[ ]m 1=

M

∑ x i[ ]+=

y i[ ] α a[ ]y i a–[ ]a 1=

A

∑– x i[ ]+=


Here, represents the non-deterministic part of the signal. The process iscalled (A,M)-th order ARMA Process and AR-2 or MA-2 process, if .

Problem: Which process should be modelled?

A frequently used technique for estimating the signals parameter is based on theextrapolation of an AR autocovariance function. Let’s assume an A-th order AR-process. If we multiply the equation with it results in:

;(ACF)

We can re-write these equations for different lags a as “normal-equation” in matrixform:

x i[ ]A M, 2=

y i a–[ ]cyy a[ ] α1cyy a 1–[ ] … αAcyy a A–[ ] E y i a–[ ]x j[ ] + + +=


If the ACF for are known, we can estimate the coefficients.

After this parameter estimation, we can FT the equation and result in:

or

and therefore the PSD is given by:

cyy 0[ ] cyy 1[ ] … cyy A[ ]

cyy 1[ ] cyy 0[ ] … cyy A[ ]

… …cyy A[ ] cyy A 1–[ ] … cyy 0[ ]

1α 1[ ]–…α A[ ]–

σx2

0…0

=

cyy 0[ ] … cyy A[ ], , A

α 1[ ] … α A[ ], ,

Y jω[ ] Y jω[ ] α 1[ ]e jωt– α 2[ ]e 2jωt– … α A[ ]e Ajωt–+ + +( )– X jω[ ]=

Y jω( ) 1AA jω[ ]------------------X jω[ ]=


;

The nominator represents the variance of the driving white noise (gaussianrandom process). This value must also be estimated while computing the coeffi-cients.

Py jω[ ]Tsσx

2

1 α a[ ]e 2πfaTs–

a 1=

A

∑–2

---------------------------------------------------------=

σx2


Let’s have a look on two different AR-2 processes:


One of the problems when applying model based spectral estimators is hidden inthe estimation of the process length (A & M). In the next example we will see, what happens if our process length is to small orto large.



There exist several different ways for fixing the length of the process in advance(data-driven). However, non is working really reliable. Of course this is different ifwe have some a priori knowledge of the physical process itself.

In order to estimate the filter (!) length, driven completely by the data, we use thedefinition of Burg (1967):

;

Here represents the coefficients estimated using a length .If is chosen correctly represents the variance of a gaussian process.

PA 1+1

2 N A–( )--------------------- y i A+[ ] αA a[ ]y i A a–+[ ]

a 1=

A

∑–⎝ ⎠⎜ ⎟⎛ ⎞

2

y i[ ] αA a[ ]y i a+[ ]a 1=

A

∑–⎝ ⎠⎜ ⎟⎛ ⎞

2

+⎝

⎠

⎜

⎟

⎛

⎞i 1=

N A–

∑=

αA A

A PA 1+


Akaike’s Final Prediction Error (FPE):

;

That will be used, at which the function has its (first!) minimum.

Properties:• the resolution is, in principle, infinite!• spectral estimates using a too coarse , (free to chose) will be badly resolved;• the position of spectral peaks will be shifted to an unknown amount, if the filter

length is wrong or variable noise conditions within the signal. When applying toshort data segments, the result depend strongly on the phase of the signal.

• the most severe point is embedded in the estimate of the process length. If thesignal is noise and/or includes also harmonic components, the computationof filter length is difficult and non-unique.

FPE A( ) N A 1+ +N A– 1–-----------------------PA 1+=

A FPE A( )

∆f

A

Spectral Estimation Time-Frequency-Representation

3. Time-Frequency-Representation

Very often, not only the spectral content of a signal is the desired output but alsoits time-dependence. An adhoc method consists of a running segmentation of thedata and successive FT of the corresponding segments. This is generally (withtapering and overlap) the procedure to estimate so called ShortTermFourierTrans-from (STFT):


STFT.


Whatsoever, in this chapter all is about finding the optimum “time-frequency-atoms”. We will have a brief look onto the problem of optimum tapers as well asoptimum overlap.A general answer to the question about best taper was given by Gabor (1949).According to the uncertainty principle the best time-frequency-localization will beachieved using a gaussian function:

;

with . The Gabor Transform is given by:

;

The FT of the signal is centered at . The width of the taper is defined bya “damping” constant . This damping results in a time resolution of:

;

gα t( )1

2 πα---------------e

t24α-------–

=

α 0> const=

Gbα x t( ) x t( )gα t b–( )( )e jωt– td∞–

∞

∫=

x t( ) t b=α

∆t α=


A different view on the use of the gaussian taper opens a complete new field insignal processing.

Rewriting the equation to: the TF is defined as:

;

and therefore the GT consists in the inner product of two functions.The FT of this modified gaussian can be written as:

The gaussian is therefore also a gaussian in the frequency range!

Gαb ω, t( ) gα t b–( )ejωt=

Gbα x t( ) x t( )Gαb ω, * t( ) td

∞–

∞

∫ x t( ) Gαb ω, t( ),⟨ ⟩= =

FT Gαb ωk, t( ) α

π---e α ω ωk–( )2–=


Again some re-writing and choosing special parameters (octave bands!) results inan optimum localization in time and frequency!Gabor Transform


Gabor Transform - constantQ


GT


constant Q


As we have not averaged any spectrum, we can also display the phase proper-ties.This latter definition of the Gabor Transform and carefully chosen parameters (socalled constant Q) end up in the new field of “Continuous Wavelet Transform”(Morlet, 1987).

Continuous Wavelet Transform (CWT)The CWT is defined:

;

here and , with .

In other words, the Wavelet Transform is the inner product (convolution) of thesignal with an analysis function. The analysis function, when orthogonal, formsagain a Hilbert space!

Wψa b, x t( ) 1

a---------- x t( )ψ* t b–

a----------⎝ ⎠⎛ ⎞ td

∞–

∞

∫=

a b, R∈ a 0≠ f L2 R( )∈


The only difference to the constant Q filter introduced while focusing on the GT isits more general definition. Today, a complete library of different analyzing func-tions (wavelets) are existing with different properties and different advantages.Whatsoever, this flexibility is at the same time also its drawback. In order to inter-pret the result of a CWT, the corresponding wavelet must be given. Different CWTwith different wavelets can often not be compared directly.

It is also possible to define a coherence measure based on wavelet theory. Theadvantage is its scaling law, resulting in always constant averaging properties atdifferent frequency bands.


Wavelet Coherence:

Documents

Representation of Digital Signalsjowa/notes/digisig_pres.pdf · Representation of Digital Signals Representation of Digital Signals I) Discrete Signals 1) Discretization and Sampling