5/26/2006 1

Beyond the Fourier Transform :Coping with Nonlinear, Nonstationary Time Series

Norden E. HuangGoddard Institute for Data AnalysisNASA Goddard Space Flight Center

5/26/2006 2

Seminar Announcement, Johns Hopkins University, 1998

5/26/2006 3

Jean-Baptiste-Joseph Fourier

1807 “On the Propagation of Heat in Solid Bodies”

1812 Grand Prize of Paris Institute

“Théorie analytique de la chaleur”

‘... the manner in which the author arrives at these equations is not exempt of difficulties and that his analysis to integrate them still leaves something to be desired on the score of generality and even rigor.’

1817 Elected to Académie des Sciences

1822 Appointed as Secretary of Math Section

paper published

Fourier’s work is a great mathematical poem.Lord Kelvin

5/26/2006 4

Fourier Integral

( ) ( ) ;

1( ) ( )

2

i t

i t

F f t e dt

f t F e d

ω

ω

ω

ω ωπ

∞

−∞

∞−

−∞

=

=

∫

∫

5/26/2006 5

Fourier Spectrum

5/26/2006 6

Fourier Series Expansion:

n Any function f(t) can be expanded in terms of discrete sine or cosine functions as

( )01

1( ) cos sin .

2 n n n nn

f t a a t b tω ω∞

=

= + +∑

5/26/2006 7

Random and Delta Functions

5/26/2006 8

Fourier Components : Random Function

5/26/2006 9

Fourier Components : Delta Function

5/26/2006 10

Fourier Sums : Delta Function

5/26/2006 11

Problems with Fourier Expansion

n Linear and Stationary assumptions.n Trigonometric function with constant

frequency and amplitude over the whole time span

n Superposition holds true limited to linear systems.

n Phase information not fully used.n No difference between delta and random

functions in frequency spectral representation.

5/26/2006 12

Data Analysis is equivalent to Information Extraction

n Data is the only connection between us and the realty.

n All our information is contained in the data.

n Data analysis is the means to extract information form the data.

n Unless we have clear understanding of the underlying processes, data analysis should not be based on a priori basis methods.

n Adaptive basis is the best approach to extract the maximum amount information.

n Hilbert-Huang Transform (HHT) is based on an adaptive approach.

n Data analysis is mechanical; result interpretation is the key to yield information.

5/26/2006 13

The Main Data Analysis Tasks

n Distribution: global properties limited to homogeneous population only; HHT can help extract component with homogeneous scale.

n Filtering: mostly Fourier based in frequency space; HHT is a nonlinear time scale based filter.

n Regression: fit data to an a priori functional; HHT fits adaptively with spline.

n Correlation: need to detrend; HHT offers adaptive detrend.

n Spectral Analysis: time-frequency representation; HHT for data from nonlinear and nonstationary processes.

n Prediction: stationary processes; HHT could help here too by provide band-limited components fro easier prediction.

5/26/2006 14

Motivations for a New Method

n Physical processes are mostly nonstationary

n Physical Processes are mostly nonlinear

n Data from observations are invariably too short

n Physical processes are mostly non-repeatable.

∪ Ensemble mean impossible, and temporal mean might not be meaningful for lack of ergodicity. Traditional methods inadequate.

5/26/2006 15

Available Data Analysis Methodsfor Nonstationary (but Linear) time series

n Various probability distributionsn Spectral analysis and Spectrogramn Wavelet Analysisn Wigner-Ville Distributionsn Empirical Orthogonal Functions aka Singular

Spectral Analysisn Moving meansn Successive differentiations

5/26/2006 16

Available Data Analysis Methodsfor Nonlinear (but Stationary and Deterministic) time series

n Phase space method• Delay reconstruction and embedding• Poincaré surface of section• Self-similarity, attractor geometry &

fractals

n Nonlinear Prediction

n Lyapunov Exponents for stability

5/26/2006 17

The Need for Instantaneous Frequency in Nonstationary and Nonlinear Processes

( )

32

2

2

22

d xx cos t

dt

d xx cos t

dt

Spring with positiondependent cons tan t ,int ra wave frequency m o d ulation;therefore,we need ins tan

x

1

tan eous frequenc

x

y .

γε ω

ε γ ω

+ + =

⇒ + =

⇒

+

−

5/26/2006 18

Duffing Pendulum

2

22( co .) s1

d xx tx

d tε γ ω=++

x

5/26/2006 19

( )

p

2 2 1 / 2 1

i ( t )

For any x( t ) L ,

1 x( )y( t ) d ,

t

then, x ( t )and y( t )arecomplexconjugate :

z( t ) x ( t ) i y ( t ) ,

wherey ( t )

a ( t ) x y and ( t

a(

)

t ) e

tan .x ( t )

θ

τ

ττ

π τ

θ −

∈

= ℘−

= + =

= + =

∫

Hilbert Transform : Definition

5/26/2006 20

Hilbert Transform Fit

5/26/2006 21

The Traditional View of the Hilbert Transform for Data Analysis

5/26/2006 22

Traditional Viewa la Hahn (1995) : Data LOD

5/26/2006 23

Traditional Viewa la Hahn (1995) : Hilbert

5/26/2006 24

Traditional Viewa la Hahn (1995) : Phase Angle

5/26/2006 25

Traditional Viewa la Hahn (1995) : Phase Angle Details

5/26/2006 26

Traditional Viewa la Hahn (1995) : Frequency

5/26/2006 27

Why the traditional view does not work?

5/26/2006 28

Hilbert Transform a cos q + b : Data

5/26/2006 29

Hilbert Transform a cos q + b : Phase Diagram

5/26/2006 30

Hilbert Transform a cos q + b : Phase Angle Details

5/26/2006 31

Hilbert Transform a cos q + b : Frequency

5/26/2006 32

The Empirical Mode Decomposition Method and Hilbert Spectral Analysis

Sifting

5/26/2006 33

Empirical Mode Decomposition: Methodology : Test Data

5/26/2006 34

Empirical Mode Decomposition: Methodology : data and m1

5/26/2006 35

Empirical Mode Decomposition: Methodology : data & h1

5/26/2006 36

Empirical Mode Decomposition: Methodology : h1 & m2

5/26/2006 37

Empirical Mode Decomposition: Methodology : h3 & m4

5/26/2006 38

Empirical Mode Decomposition: Methodology : h4 & m5

5/26/2006 39

Empirical Mode DecompositionSifting : to get one IMF component

1 1

1 2 2

k 1 k k

k 1

x ( t ) m h ,

h m h ,

. . . . .

. . . . .h m h

.h c

.−

− =

− =

− =

=⇒

5/26/2006 40

Two Stoppage Criteria : S and SD

A. The S number : S is defined as the consecutive number of siftings, in which the numbers of zero-crossing and extrema are the same for these S siftings.

B. SD is small than a pre-set value, where

2Tk 1 k

2t 0 k 1

h ( t ) h ( t )SD .

h ( t )−

= −

−= ∑

5/26/2006 41

Empirical Mode Decomposition: Methodology : IMF c1

5/26/2006 42

Definition of the Intrinsic Mode Function (IMF)

Any function having the same numbers ofzero cros sin gs and extrema,and also havingsymmetric envelopesdefined by local m a x i m aand min ima respectively isdefined asanIntrinsic M odeFunction( IMF ).

All IMF enjoys good HilbertTransfo

−

i ( t )

rm :

c( t ) a( t ) e θ⇒ ⇒ =

5/26/2006 43

Empirical Mode DecompositionSifting : to get all the IMF components

1 1

1 2 2

n 1 n n

n

j nj 1

x( t ) c r ,

r c r ,

x( t ) c r

. . .r c r .

.

−

=

− =

− =

−⇒ =

− =

∑

5/26/2006 44

Empirical Mode Decomposition: Methodology : data & r1

5/26/2006 45

Empirical Mode Decomposition: Methodology : IMFs

5/26/2006 46

Definition of Instantaneous Frequency

i ( t )

t

The FourierTransform of the Instrinsic M odeFunnction, c( t ), gives

W ( ) a( t ) e dt

By Stationary phaseapproximation we have

d ( t ),

dt

This isdefined as the Ins tan tan eous Frequency .

θ ωω

θω

−=

=

∫

5/26/2006 47

Comparison between FFT and HHT

j

jt

i tj

j

i ( ) d

jj

1 . F F T :

x ( t ) a e .

2 . H H T :

x ( t ) a ( t ) e .

ω

ω τ τ

= ℜ

∫= ℜ

∑

∑

5/26/2006 48

Comparisons: Fourier, Hilbert & Wavelet

5/26/2006 49

Speech AnalysisHello : Data

5/26/2006 50

Four comparsions D

5/26/2006 51

An Example of Sifting

5/26/2006 52

Length Of Day Data

5/26/2006 53

LOD : IMF

5/26/2006 54

Orthogonality Check

n Pair-wise %

n 0.0003n 0.0001n 0.0215n 0.0117n 0.0022n 0.0031n 0.0026n 0.0083n 0.0042n 0.0369n 0.0400

n Overall %

n 0.0452

5/26/2006 55

LOD : Data & c12

5/26/2006 56

LOD : Data & Sum c11-12

5/26/2006 57

LOD : Data & sum c10-12

5/26/2006 58

LOD : Data & c9 - 12

5/26/2006 59

LOD : Data & c8 - 12

5/26/2006 60

LOD : Detailed Data and Sum c8-c12

5/26/2006 61

LOD : Data & c7 - 12

5/26/2006 62

LOD : Detail Data and Sum IMF c7-c12

5/26/2006 63

LOD : Difference Data – sum all IMFs

5/26/2006 64

Traditional Viewa la Hahn (1995) : Hilbert

5/26/2006 65

Mean Annual Cycle & Envelope: 9 CEI Cases

5/26/2006 66

Hilbert’s View on Nonlinear Data

5/26/2006 67

Duffing Type WaveData: x = cos(wt+0.3 sin2wt)

5/26/2006 68

Duffing Type WavePerturbation Expansion

( )( ) ( )

For 1 , we can have

x( t ) cos t sin2 t

cos t cos sin2 t sin t sin sin2 t

cos t sin t sin2 t ....

1 cos t cos 3 t ....2 2

This isvery similar tothe solutionof Duffingequation .

ε

ω ε ω

ω ε ω ω ε ω

ω ε ω ω

ε εω ω

= +

= −

= − +

= − + +

=

5/26/2006 69

Duffing Type WaveWavelet Spectrum

5/26/2006 70

Duffing Type WaveHilbert Spectrum

5/26/2006 71

Duffing Type WaveMarginal Spectra

5/26/2006 72

Duffing Equation

23

2 .

Solved with for t 0 to 200 with1

0.1

od

0.04 H z

Initial condition :[ x ( o ) ,

d xx x c

x ' ( 0 ) ] [ 1

os t

, 1 ]

3

t

e 2

d

tbε

ε γ ω

γω

== −==

=

+ + =

5/26/2006 73

Duffing Equation : Data

5/26/2006 74

Duffing Equation : IMFs

5/26/2006 75

Duffing Equation : IMFs

5/26/2006 76

Duffing Equation : Hilbert Spectrum

5/26/2006 77

Duffing Equation : Detailed Hilbert Spectrum

5/26/2006 78

Duffing Equation : Wavelet Spectrum

5/26/2006 79

Duffing Equation : Hilbert & Wavelet Spectra

5/26/2006 80

What This Means

n Instantaneous Frequency offers a total different view for nonlinear data: instantaneous frequency with no need for harmonics and unlimited by uncertainty.

n Adaptive basis is indispensable for nonstationary and nonlinear data analysis

n HHT establishes a new paradigm of data analysis

5/26/2006 81

Comparisons

noyesyesHarmonics

noyesyesUncertainty

yesyesnoNon-stationary

yesnonoNonlinear

Energy-time-frequency

Energy-time-frequency

Energy-frequency

Presentation

Differentiation:Local

Convolution: Regional

Convolution: Global

Frequency

Adaptivea prioria prioriBasis

HilbertWaveletFourier

5/26/2006 82

Current Applications

n Non-destructive Evaluation for Structural Health Monitoring n (DOT, NSWC, and DFRC/NASA, KSC/NASA Shuttle)

n Vibration, speech, and acoustic signal analysesn (FBI, MIT, and DARPA)

n Earthquake Engineeringn (DOT)

n Bio-medical applicationsn (Harvard, UCSD, Johns Hopkins, and Southampton, UK)

n Global Primary Productivity Evolution map from LandSat data n (NASA Goddard, NOAA)

n Cosmological Gravity Wave and Planets huntingn (NASA Goddard, and Nicholas Copernicus University, Poland)

n Financial market data analysisn (NASA and HKUST)