Lecture4:

TimeSeriesAnalyses

ShaneElipotTheRosenstielSchoolofMarineandAtmosphericScience,

UniversityofMiami

Createdwith{Remark.js}using{Markdown}+{MathJax}

Loading[MathJax]/jax/output/HTML-CSS/jax.js

ForewordThislectureisheavilybasedonalongercourse(TheOsloLectures)givenbyJonathanM.LillyinOsloattheinvitationoftheNorwegianResearchSchoolinClimateDynamics(ResClim),duringtheweekMay23–27,2016.JML'sOsloLecturesarefreelyavailableforonlineviewingordownloadat{www.jmlilly.net/talks/oslo/index.html}

References

[1]Bendat,J.S.,&Piersol,A.G.(2011).Randomdata:analysisandmeasurementprocedures(Vol.729).JohnWiley&Sons.

[2]Percival,D.B.andWalden,A.T.(1993).SpectralAnalysisforPhysicalApplications.CambridgeUniversityPress

[3]Jenkins,G.M.andWatts,D.G.(1968).SpectralAnalysisanditsApplications.HoldenDays

Outline1. Thetimedomain2. Stationarity,non-stationarity,andtrends3. FourierSpectralanalysis4. Bivariatetimeseries5. Filteringandothertopics6. Multitaperrevisited

1.Thetimedomain

TheSampleIntervalWehaveasequenceof observations

whichcoincidewithtimes

Thesequence iscalledadiscretetimeseries.

Itisassumedthatthesampleinterval, ,isconstant

withthetimeat definedtobe .Thedurationis .

Ifthesampleintervalinyourdataisnotuniform,thefirstprocessingstepistointerpolateittobeso.

N

, n = 0, 1, 2,…N − 1xn

, n = 0, 1, 2,…N − 1.tn

xn

Δt

= ntn Δt

n = 0 0 T = NΔt

TheUnderlyingProcessAcriticalassumptionisthatthereexistssome“process” thatourdatasequence isasampleof:

Unlike , isbelievedtoexistforalltimes.

(i)Theprocess existsincontinuoustime,while onlyexistsatdiscretetimes.

(ii)Theprocess existsforallpastandfuturetimes,while isonlyavailableoveracertaintimeinterval.

Itisthepropertiesof thatwearetryingtoestimate,basedontheavailablesample .

x(t)xn

= x(n ), n = 0, 1, 2,…N − 1.xn Δt

xn x(t)

x(t) xn

x(t) xn

x(t)xn

MeasurementNoiseInreality,themeasurementdeviceand/ordataprocessingprobablyintroducessomeartificalvariability,termednoise.

Itismorerealistictoconsiderthattheobservations containsamplesoftheprocessofinterest, ,plussomenoise :

Thisisanexampleoftheunobservedcomponentsmodel.Thismeansthatwebelievethatthedataiscomposedofdifferentcomponents,butwecannotobservethesecomponentsindividually.

Theprocess ispotentiallyobscuredordegradedbythelimitationsofdatacollectioninthreeways:(i)finitesampleinterval,(ii)finiteduration,(iii)noise.

Becauseofthis,thetimeseriesisanimperfectrepresentationofthereal-worldprocesseswearetryingtostudy.

xny(t) ϵn

= y(n ) + , n = 0, 1, 2,…N − 1.xn Δt ϵn

y(t)

TimeversusFrequencyTherearetwocomplementarypointsofviewregardingthetimeseries .

Thefirstregards asbeingbuiltupasasequenceofdiscretevalues .

Thisisthedomainofstatistics:themean,variance,histogram,etc.

Whenwelookatdatastatistics,generally,theorderinwhichthevaluesareobserveddoesn'tmatter.

Thesecondpointofviewregards asbeingbuiltupofsinusoids:purelyperiodicfunctionsspanningthewholedurationofthedata.

ThisisthedomainofFourierspectralanalysis.

Inbetweenthesetwoextremesiswaveletanalysiswhichisnotcoveredhere,seetheOslolectures.

xn

xn, ,…x0 x2 xN−1

xn

Time-DomainStatisticsTimedomainstatisticsconsistoftheparameterswehaveconsideredearlierduringtheweek:samplemean,samplevariance,skewness,kurtosis,andhighermoments.

Thetermsampleisbeingusedtodistinguishthesequantitiescalculatedfromthedatasamplefromthepopulation,ortrue,statisticsoftheassumedunderlyingprocess .That'swhyweusex(t)(⋅)

2.Stationarityvsnon-stationarity,trends

FirstExample

FirstExample

FirstExample

ObservableFeatures1. Thedataconsistsoftwotimeseriesthataresimilarincharacter.2. Bothtimeseriespresentasuperpositionofscalesandahigh

degreeofroughness.3. Thedataseemstoconsistofdifferenttimeperiodswithdistinct

statisticalcharacteristics—thedataisnonstationary.4. Zoomingintooneparticularperiodshowregularoscillationsof

roughlyuniformamplitudeandfrequency.5. Thephasingoftheseshowacircularpolarizationorbitedina

counterclockwisedirection.6. Thezoomed-inplotshowsafairamountofwhatappearstobe

measurementnoisesuperimposedontheoscillatorysignal.

ObservableFeatures1. Thedataconsistsoftwotimeseriesthataresimilarincharacter.2. Bothtimeseriespresentasuperpositionofscalesandahigh

degreeofroughness.3. Thedataseemstoconsistofdifferenttimeperiodswithdistinct

statisticalcharacteristics—thedataisnonstationary.4. Zoomingintooneparticularperiodshowregularoscillationsof

roughlyuniformamplitudeandfrequency.5. Thephasingoftheseshowacircularpolarizationorbitedina

counterclockwisedirection.6. Thezoomed-inplotshowsafairamountofwhatappearstobe

measurementnoisesuperimposedontheoscillatorysignal.

Thisisarecordofvelocitiesofasinglesurfacedrifterat6-hourintervals.AllSurfacedrifterdataarefreelyavailablefromtheDataAssemblyCenteroftheGlobalDrifterProgram{www.aoml.noaa.gov/phod/dac/}.

SecondExampleWehavealreadyencounteredthistimeseries...

SecondExample

ObservableFeatures1. Thedataexhibitaverystrongpositivetrend,roughlylinearwith

time.Thus,thistimeseriesdoesnotpresentameanstatisticsthatrepresentsa"typical"value.

2. Ontopofthetrendthereseemstobeasinusoid-likeoscillationthatdoesnotappeartochangewithtime.

3. Thezoomed-inplotshowsnoisesuperimposedonthesinusoidalandtrendprocesses.

ObservableFeatures1. Thedataexhibitaverystrongpositivetrend,roughlylinearwith

time.Thus,thistimeseriesdoesnotpresentameanstatisticsthatrepresentsa"typical"value.

2. Ontopofthetrendthereseemstobeasinusoid-likeoscillationthatdoesnotappeartochangewithtime.

3. Thezoomed-inplotshowsnoisesuperimposedonthesinusoidalandtrendprocesses.

ThisisarecordofdailyatmosphericCO measuredatMaunaLoainHawaiiatanaltitudeof3400m.DatafromDr.PieterTans,NOAA/ESRL({www.esrl.noaa.gov/gmd/ccgg/trends/})andDr.RalphKeeling,ScrippsInstitutionofOceanography({scrippsco2.ucsd.edu}).

Wewillinvestigateagainthistimeseriesduringthepracticalsessionthisafternoon,thistimeusingaspectralanalysisapproach.

2

Non-stationarityThesamplestatisticsmaybechangingwithtimebecausetheunderlyingprocess(thatisitsstatistics)ischangingwithtime.Theprocessissaidtobe“non-stationary”.

Sometimesweneedtore-thinkourmodelfortheunderlyingprocess.AswesayinLecture3,wecanhypothesizethattheprocess

isthesumofanunknownprocess ,plusalineartrend ,plusnoise:x(t) y(t) a

x(t) = y(t) + at+ ϵ(t),

Non-stationarityThesamplestatisticsmaybechangingwithtimebecausetheunderlyingprocess(thatisitsstatistics)ischangingwithtime.Theprocessissaidtobe“non-stationary”.

Sometimesweneedtore-thinkourmodelfortheunderlyingprocess.AswesayinLecture3,wecanhypothesizethattheprocess

isthesumofanunknownprocess ,plusalineartrend ,plusnoise:

ormaybethetrendisbetterdescribedasbeingquadraticwithtimebecauseofanacceleration:

x(t) y(t) a

x(t) = y(t) + at+ ϵ(t),

x(t) = y(t) + b + at+ ϵ(t).t2

Non-stationarityThesamplestatisticsmaybechangingwithtimebecausetheunderlyingprocess(thatisitsstatistics)ischangingwithtime.Theprocessissaidtobe“non-stationary”.

Sometimesweneedtore-thinkourmodelfortheunderlyingprocess.AswesayinLecture3,wecanhypothesizethattheprocess

isthesumofanunknownprocess ,plusalineartrend ,plusnoise:

ormaybethetrendisbetterdescribedasbeingquadraticwithtimebecauseofanacceleration:

Thegoalisthentoestimatetheunknowns ,whichconsistsofmethodsofanalysesgenerallycalled“parametric”.Itisabitlikeanalyzingthedataintermsofitsstatistics(withnopriorexpectations)orassumingaformforthedata.

x(t) y(t) a

x(t) = y(t) + at+ ϵ(t),

x(t) = y(t) + b + at+ ϵ(t).t2

a, b

3.FourierSpectralAnalysis

ComplexFourierSeriesItispossibletorepresentadiscretetimeseries asasumof

complexexponentials,acomplexFourierseries:

Weleaveoutfornowhowtoobtainthecomplexcoefficients ...

xn

= , n = 0, 1,…N − 1xn1

NΔt∑m=0

N−1

Xmei2πmn/N

Xm

AboutFrequencyYouwilltypicallyfindtwofrequencynotations:

iscalledthecyclicfrequency.Itsunitsarecycles/time.Example:Hz=cycles/sec.

iscalledtheradianorangularfrequency.Itsunitsarerad/time.Theassociatedperiodofoscillationis .

As goesfrom to , goesfrom to andgoesfrom to .

AverycommonerrorinFourieranalysisismixingupcyclicandradianfrequencies!

Note:neither“cycles”nor“radians”actuallyhaveanyunits,thusboth and haveunitsof1/time.However,specifyingforexample'cyclesperday'or'radiansperday'helpstoavoidconfusion.

cos(2πft) or cos(ωt)

f

ω = 2πfP = 1/f = 2π/ω

t 0 1/f = 2π/ω = P 2πft 0 2πωt 0 2π

f ω

Review:SinusoidsCosinefunction(blue)andsinefunction(orange)

ComplexExponentials,2DNowconsideraplot vs. .

That'sthesameas .

cos(t) sin(t)

cos(t) + i sin(t) = eit

ComplexExponentials,3DThisisbetterseenin3Dasaspiralastimeincreases.

cos(t) + i sin(t) = eit

ThecomplexFourierseriesThediscretetimeseries canwrittenasasumofcomplexexponentials:

xn

= = , n = 0, 1,…xn1

NΔt∑m=0

N−1

Xmei2πmn/N 1

NΔt∑m=0

N−1

Xmei2πn ⋅(m/N )Δt Δt

ThecomplexFourierseriesThediscretetimeseries canwrittenasasumofcomplexexponentials:

The thtermbehavesas,where

.

Notethatintheliterature, isoftensettoone,andthusomitted,leadingtoalotofconfusion(includingforme!).

Thequantity iscalledthe thFourierfrequency.Theperiodassociatedwith is .Thus tellsusthenumberofoscillationscontainedinthelength timeseries.

xn

= = , n = 0, 1,…xn1

NΔt∑m=0

N−1

Xmei2πmn/N 1

NΔt∑m=0

N−1

Xmei2πn ⋅(m/N )Δt Δt

m= cos(2π n ) + i sin(2π n )ei2π nfm Δt fm Δt fm Δt

≡ m/Nfm Δt

Δt

≡ m/Nfm Δt mfm 1/ = N/mfm Δt m

NΔt

ContinuousTimeand

cos(2π t)fm sin(2π t)fm

= 0,fm 1/100, 2/100, 3/100 t = [0…100]

DiscreteTimeand

cos(2π n )fm Δt sin(2π n )fm Δt

= 0,fm 1/100, 2/100, 3/100 n = 0, 1, 2,…99 = 1Δt

TheNyquistFrequencyThesinglemostimportantfrequencyisthehighestresolvable

frequency,theNyquistfrequency.

Thehighestresolvablefrequencyishalfthesamplingrateoronecyclepertwosamplingintervals.

Notethatthereisno“sine”componentatNyquistintheFourierseries!

≡ = ⋅ ≡ ⋅ =fN 12Δt

12

1Δt

ωN 12

2πΔt

πΔt

= = = (−1 = 1,−1, 1,−1,…ei2π nfN Δt ei2π⋅1/(2 )⋅nΔt Δt eiπn )n

TheRayleighFrequencyThesecondmostimportantfrequencyisthelowestresolvable

frequency,theRayleighfrequency.

Thelowestresolvablefrequencyisonecycleovertheentirerecord.Herethesampleinterval andthenumberofpointsis

.

≡ ≡fR 1NΔt

ωR 2πNΔt

= 1Δt

N = 10

ImportanceofRayleighTheRayleighfrequency isimportantbecauseitgivesthespacingbetweentheFourierfrequencies:

, ,

Thus,itcontrolsthefrequency-domainresolution.Ifyouwanttodistiguishbetweentwocloselyspacedpeaks,youneedthedatasetdurationtobesufficientlylargesothattheRayleighfrequencyissufficientlysmall.

fR

= 0f0 =f11

NΔt= ,…f2

2NΔt

= n , =fn fR fR 1NΔt

ImportanceofRayleighTheRayleighfrequency isimportantbecauseitgivesthespacingbetweentheFourierfrequencies:

, ,

Thus,itcontrolsthefrequency-domainresolution.Ifyouwanttodistiguishbetweentwocloselyspacedpeaks,youneedthedatasetdurationtobesufficientlylargesothattheRayleighfrequencyissufficientlysmall.

Asanexample,thetwoprincipalsemi-diurnaltidal“species”haveperiodof12h(M )and12.4206012h(S ).Theminimumrecordlengthtodistinguishthetwofrequenciesisthus

fR

= 0f0 =f11

NΔt= ,…f2

2NΔt

= n , =fn fR fR 1NΔt

2 2

N = = = = 354.36hours.Δt1fR

1−f S2 fM2

1

−112

112.4206012

RayleighandNyquistfrequenciesTheratiooftheRayleightoNyquistfrequenciestellsyouhowmanydifferentfrequenciesyoucanresolve.

Sowhydowehave frequenciesinthesumforthecomplexFourierseries?

= =fN

fR

NΔt2Δt

N

2

N

=xn1

NΔt∑m=0

N−1

Xmei2πmn/N

TheFourierFrequenciesThefirstfewFourierfrequencies are:

whilethelastfeware

ButnoticethatthelastFourierexponentialtermis

because forallintegers !FrequencieshigherthantheNyquistcannotappearduetooursamplerate.Therefore,thesetermsinsteadspecifytermsthathaveafrequencylessthantheNyquistbutthatrotateinthenegativedirection.

= m/(N )fm Δt

= , = , = ,…f00

NΔtf1

1NΔt

f22

NΔt

…, = = − , = = − .fN−2N − 2NΔt

1Δt

2NΔt

fN−1N − 1NΔt

1Δt

1NΔt

= = = =ei2π nfN−1 Δt ei2π(N−1)n/N ei2πne−i2πn/N e−i2πn/N e−i2π nf1 Δt

= 1ei2πn n

TheFourierFrequenciesInthevicinityof ,foreven ,wehave

butactuallythefirstfrequencyhigherthantheNyquististhehighestnegativefrequency:

ThusthepositivefrequenciesandnegativefrequenciesarebothincreasingtowardthemiddleoftheFouriertransformarray.

ForthisreasonMatlabprovidesfftshift,toshiftsthezerofrequency,nottheNyquist,tobeinthemiddleofthearray.

m = N/2 N

= − , = , = + ,…fN/2−112Δt

1NΔt

fN/212Δt

fN/2+112Δt

1NΔt

= − , = ,…fN/2−112Δt

1NΔt

fN/212Δt

= − = −( − ) ,… .fN/2+1 fN/2−112Δt

1NΔt

One-Sidedvs.Two-SidedThereexiststwostrictlyequivalentrepresentations,two-sidedandone-sided,ofthediscreteFouriertransform:

where and areanamplitudeandphase,with.

Thetwo-sidedrepresentationismorecompactmathematically.

Forreal-valued ,theone-sidedrepresentationismoreintuitiveasitexpresses asasumofphase-shiftedcosinusoids.

xn

xn

=

=

,1

NΔt

∑m=0

N−1

Xmei2πmn/N

+ cos(2πmn/N + ) + (−1 ,1

NΔtX0

2NΔt

∑m=1

N/2−1

Am Φm XN/2 )n

Am Φm=Xm Ame

iΦm

xnxn

One-Sidedvs.Two-SidedThereexiststwostrictlyequivalentrepresentations,two-sidedandone-sided,ofthediscreteFouriertransform:

where and areanamplitudeandphase,with.

Thetwo-sidedrepresentationismorecompactmathematically.

Forreal-valued ,theone-sidedrepresentationismoreintuitiveasitexpresses asasumofphase-shiftedcosinusoids.Apriceoftheone-sidedformisthatevenandodd aresomewhatdifferent!Theexpressionaboveisforeven-valued .

xn

xn

=

=

,1

NΔt

∑m=0

N−1

Xmei2πmn/N

+ cos(2πmn/N + ) + (−1 ,1

NΔtX0

2NΔt

∑m=1

N/2−1

Am Φm XN/2 )n

Am Φm=Xm Ame

iΦm

xnxn

NN

TheForwardDFTSo?HowdoweknowthevaluesoftheFouriercoefficients ?Itcanbeshownthat:

ThisiscalledthediscreteFouriertransformof .

TheDFTtransforms fromthetimedomaintothefrequencydomain.TheDFTdefinesasequenceof complex-valuednumbers, ,for ,whicharetermedtheFouriercoefficients.

InMatlab,thediscreteFouriertransformdefinedaboveiscomputedbyfft(x) .

Xm

=Xm Δt ∑n=0

N−1

xne−i2πmn/N

xn

xnN

Xm m = 0, 1, 2,…N − 1

×Δt

TheInverseDFTInfact,

iscalledtheinversediscreteFouriertransform.ItexpresseshowmaybeconstructedusingtheFouriercoefficientsmultiplying

complexexponentials—or,aswesawearlier,phase-shiftedcosinusoids.

≡xn1

NΔt∑m=0

N−1

Xmei2πmn/N

xn

TheSpectrumOneofseveraldefinitionsofthespectrum,orspectraldensityfunction,atfrequency ,is:

iscalledtheexpectation,itisaconceptual“average”overastatisticalensemble,anditcannotobtainedinpractice.

fm

S( ) ≡ E{ } .fm limN→∞

|Xm|2

N

E{⋅}

TheSpectrumOneofseveraldefinitionsofthespectrum,orspectraldensityfunction,atfrequency ,is:

iscalledtheexpectation,itisaconceptual“average”overastatisticalensemble,anditcannotobtainedinpractice.

Formally,thefunction isdefinedforallfrequencies ,notonly,butas ,theRayleighfrequency becomes

infinitesimallysmall,andallfrequenciesareobtained.

fm

S( ) ≡ E{ } .fm limN→∞

|Xm|2

N

E{⋅}

S ffm N →∞ 1/NΔt

TheSpectrumOneofseveraldefinitionsofthespectrum,orspectraldensityfunction,atfrequency ,is:

iscalledtheexpectation,itisaconceptual“average”overastatisticalensemble,anditcannotobtainedinpractice.

Formally,thefunction isdefinedforallfrequencies ,notonly,butas ,theRayleighfrequency becomes

infinitesimallysmall,andallfrequenciesareobtained.

However, isnotachievable...Therefore,oneaspectofspectralanalysisistofindanacceptableestimateofthetrue,unknown,spectrum oftheprocess .

fm

S( ) ≡ E{ } .fm limN→∞

|Xm|2

N

E{⋅}

S ffm N →∞ 1/NΔt

N →∞

S(f) x(t)

TheParsevalTheoremAveryimportanttheoremisParseval'stheoremwhichtakesthefollowingformforthediscretecase:

When ,thistheoremshowsthatthetotalvarianceof isrecoverablefromthesumofabsoluteFouriercoefficientssquared.

Whichcanbeinterpretedassayingthatthespectrumgivesyouthedistributionofthevarianceasafunctionoffrequency.

| = | .Δt ∑n=0

N−1

xn |2 1

NΔt∑m=0

N−1

Xm|2

= 0μx xn

SpectralEstimatesThesimplestwaytoestimatethespectrum functionoffrequency istosimplytakethemodulussquaredoftheFouriertransform,

Thisquantityisknownastheperiodogram.

NotethattheMatlabfft(x)commandassumes soyouneedtoplotabs(fft(x)) .

Asweshallsee,theperiodogramisnotthespectrum!Itisanestimateofthespectrum—andgenerallyspeaking,averypoorone.

Itisalsosaidtobethenaivespectralestimate,meaningitisthespectralestimatethatyougetifyoudon'tknowthatthereissomethingbetter.Pleasedonotusetheperiodograminyourpublications.

S(f)f

( ) = ≡ , m = 0, 1, 2,… , (N − 1).S fm Sm1N| |Xm

2

= 1Δt

× /N2 Δt

TheMultitaperMethodAnalternatespectralestimatecalledthemultitapermethod.Hereisaquicksketchofthismethod.

Weformasetof differentsequencesthesamelengthasthedata,thatis,having pointsintime.Thesesequencesarechosenfromaspecialfamilyoffunctionsthatiscloselyrelatedtofamiliarorthogonalfunctions,e.g.theHermitefunctions.

These differentsequencesaredenotedas for.Foreachofthesesequence,weformaspectral

estimateas

whichinvolvesmultiplyingthedatabythesequence beforetakingtheFouriertransform.

KN

K ψ{k}n

k = 1, 2,…K

≡ , n = 0, 1, 2,… , (N − 1).S{k}m

∣

∣∣∣Δt ∑

n=0

N−1

ψnxn e−i2πmn/N

∣

∣∣∣

2

ψ{k}n

TheMultitaperMethodTheactionofmultiplyingthedatabysomesequencebeforeFouriertransformingit,asin

iscalledtapering.Thegoalistoreducethebias(systematicerror)ofthespectralestimate.These differentindividualestimates(akaeigenspectra),arecombinedintooneaveragespectralestimate,inordertoreducethevariance(randomerror)oftheestimate

Themultitapermethodthereforeinvolvestwosteps:(i)taperingthedata,and(ii)averagingovermultipleindividualspectralestimates.

≡ , n = 0, 1, 2,… , (N − 1)S{k}m

∣

∣∣∣Δt ∑

n=0

N−1

ψnxn e−i2πmn/N

∣

∣∣∣

2

K

≡ .Sψ

m

1K

∑k=1

K

S{k}m

TheTaperFunctions

Here Slepiantapersareshown.Theseareorthogonalfunctionsthatbecomemoreoscillatoryforincreasing .

K = 5K

TheMultitaperMethodThemultitapermethodcontrolsthedegreesofspectralsmoothingandaveragingthroughchangingthepropertiesofthetapers.

Themultitapermethodisgenerallythefavoritespectralanalysismethodamongthoseresearcherswhohavethoughtthemostaboutspectralanalysis.

Itisrecommendedbecause(i)itavoidsthedeficienciesoftheperiodogram,(ii)ithas,inacertainsense,provableoptimalproperties,(iii)itisveryeasytoimplementandadjust,(iv)itallowsanestimateofthespectrumfortheperiodequaltothelengthofyourtimeseries(noneedtodivideupyourtimeseriesasfortheWelch'smethod!).

SeeThomson(1982),Parketal.(1987),andPercivalandWalden,SpectralAnalysisforPhysicalApplications.

ExampleAgulhascurrentboundarytransportfromBeal,L.M.andS.Elipot,BroadeningnotstrengtheningoftheAgulhasCurrentsincetheearly1990s,Nature,540,570573,doi:10.1038/nature19853

Example:periodogramLinearplot

Example:periodogramLog-logplot

Example:multitaperEffectoffirsttaper

Example:multitaperSecondtaper

Example:multitaperThirdtaper

Example:multitaperFourthtaper

Example:multitaperFifthtaper

Example:multitaperFifthtaper

Example:multitaperEigenspectra(colors)andmultitaperestimate(black)

Example:period.vsmtPeriodogram(gray)vsmultitaper(black)

Uncertaintyofthespectrum

Itcanbeshown(nothere)thattheestimateofthespectrumwithtapers

K

(ω) ∼ S(ω)Sχ22K2K

Uncertaintyofthespectrum

Itcanbeshown(nothere)thattheestimateofthespectrumwithtapers

Assuch,a CIis

K

(ω) ∼ S(ω)Sχ22K2K

(1 − α)100%

[ < S(ω) < ]2K (ω)S

χ22K;α/2

2KSχ22K;1−α/2

Uncertaintyofthespectrum

Itcanbeshown(nothere)thattheestimateofthespectrumwithtapers

Assuch,a CIis

Thismeansthatyoumultiply by togetthelowerboundandsimilarlyfortheupperbound.

K

(ω) ∼ S(ω)Sχ22K2K

(1 − α)100%

[ < S(ω) < ]2K (ω)S

χ22K;α/2

2KSχ22K;1−α/2

S(ω) 2K/χ22K;α/2

Uncertaintyofthespectrum

Itcanbeshown(nothere)thattheestimateofthespectrumwithtapers

Assuch,a CIis

Thismeansthatyoumultiply by togetthelowerboundandsimilarlyfortheupperbound.Ifyouplotyourestimatesonalogarithmicscale,youobtain

K

(ω) ∼ S(ω)Sχ22K2K

(1 − α)100%

[ < S(ω) < ]2K (ω)S

χ22K;α/2

2KSχ22K;1−α/2

S(ω) 2K/χ22K;α/2

[log( )+ log < logS < log( )+ log ]2Kχ22K;α/2

S2K

χ22K;1−α/2S

Uncertaintyofthespectrum

Periodogram(left)andmultitaper(right)estimatewithCIsonlinear-linearscales

Uncertaintyofthespectrum

MultitaperestimatewithCIsonlog-logscalesPeriodogram(left)andmultitaper(right)estimatewithCIsonlog-logscales

4.Bivariatetimeseries

VectorandcomplexnotationsWhatifyourprocessofinterestiscomposedoftwotimeseries,let'ssay and ?Asinthevectorcomponentsofoceancurrentsoratmosphericwinds:

Often,abivariatetimeseriesisconvenientlywrittenasacomplex-valuedtimeseries:

where and isthecomplexargument(orpolarangle)of intheinterval .

x(t) y(t)

z(t) = [ ]x(t)y(t)

z(t) = x(t) + iy(t) = |z(t)| ,ei arg (z)

i = −1−−−√ arg (z)z [−π,+π]

TheMeanofBivariateDataThesamplemeanofthevectortimeseries isalsoavector,

thatconsistsofthesamplemeansofthe and componentsof.

zn

≡ = [ ]μz1N

∑n=0

N−1

znμx

μy

xn ynzn

VarianceofBivariateDataThevarianceofthevector-valuedtimesseries isnotascalaroravector,itisa matrix

where“ ”isthematrixtranspose, , .

Carryingoutthematrixmultiplicationleadsto

Thediagonalelementsof arethesamplevariances and ,whiletheoff-diagonalgivesthecovariancebetween and .Notethatthetwooff-diagonalelementsareidentical.

zn2 × 2

Σ ≡ ( − )1N

∑n=0

N−1

zn μz ( − )zn μzT

T = [ ]znxn

yn= [ ]zTn xn yn

Σ = [ ]1N

∑n=0

N−1 ( − )xn μx2

( − ) ( − )xn μx yn μy

( − ) ( − )xn μx yn μy

( − )yn μy2

Σ σ2x σ2yxn yn

FouriertransformTheFouriertheorypresentedearlierforscalartimeseriesiscompletelyapplicabletocomplex-valuedtimeseries,indiscreteform( even):

Thefirstsumcorrespondstopositivefrequencies,andthesecondsumtotheassociatednegativefrequencies(exceptthezeroandNyquistfrequenciesfor ).

N

zn =

=

1NΔt

∑m=0

N/2

Zmei2πmn/N

1NΔt

∑m=0

N/2

Z+mei2πmn/N

+1

NΔt∑

m=N/2+1

N−1

Zmei2πmn/N

+ .1

NΔt∑m=1

N/2−1

Z−me−i2πmn/N

m = 0,N/2

RotarySpectra

Thisintroducestheconceptofrotaryspectrum:

Thisisveryusefulingeophysicalfluidmechanicsbecausecounterclockwisemotionsarecyclonicinthenorthernhemisphereandclockwisemotionsareanticyclonic,andvice-versainthesouthernhemisphere.

= + .zn1

NΔt∑m=0

N/2

Z+mei2πmn/N 1

NΔt∑m=1

N/2−1

Z−me−i2πmn/N

S( > 0) ≡ ( )fm S+ fm

S( < 0) ≡ (− )fm S− fm

≡

≡

E{ } counterclockwisespectrum,limN→∞

|Z+m |2

N

E{ } clockwisespectrum.limN→∞

|Z−m |2

N

= , m = 0,…,N/2fmm

NΔt

RotaryvarianceImagineyouhaveonlytwooppositecomponentspresentinyourtimeseriesatfrequency :

where

fk

zn =

=

=

+Z+kei2πkn/N Z−

ke−i2πkn/N

{ } + { }A+eiϕ+ei2πkn/N A−eiϕ

−e−i2πkn/N

{A cos(2πkn/N + ϕ) + iB sin(2πkn/N + ϕ)}eiθ

θ

ϕ

A

B

====

( − )/2ϕ+ ϕ−

( + )/2ϕ+ ϕ−

+A+ A−

− .A+ A−

Ellipticvariance

Thisistheequationforanellipseorientedatanangle fromtheaxis,withsemi-majorandsemi-minoraxes and ,respectively,rotatingatfrequency ,inthedirectiongivenbythesignof .

SeemoredetailsaboutellipticvarianceinJML'sOslolectures.

= {A cos(2πkn/N + ϕ) + iB sin(2πkn/N + ϕ)}zn eiθ

θ xA B

= k/(N )fk Δt

B

CartesianSpectraRotaryandCartesianspectraaretwoalternaterepresentationofthevarianceofthecomplextimeseries:

= + .zn1

NΔt∑m=0

N/2

Z+mei2πmn/N 1

NΔt∑m=1

N/2−1

Z−me−i2πmn/N

≡ , ≡ RotaryspectraestimatesS+m

1N∣∣Z+m ∣∣

2S−m

1N∣∣Z−m ∣∣

2

= + i = + i{ } .zn xn yn1

NΔt∑m=0

N−1

Xmei2πmn/N 1

NΔt∑m=0

N−1

Ymei2πmn/N

≡ , ≡ CartesianspectraestimatesSx

m

1N| |Xm

2 Sy

m

1N| |Ym

2

ParsevaltheoremForbivariatedata,thediscreteformoftheParsevaltheoremtakestheform:

| = |Δt ∑n=0

N−1

zn|2 1

NΔt∑m=0

N−1

Zm|2 =

=

| + |1

NΔt

∑m=0

N−1

Xm|2 1

NΔt∑m=0

N−1

Ym |2

| + |1

NΔt∑m=0

N/2

Z+m |2 1

NΔt∑m=1

N/2−1

Z−m |2

ParsevaltheoremForbivariatedata,thediscreteformoftheParsevaltheoremtakestheform:

ThisshowsthatthetotalvarianceofthebivariateprocessisrecoveredcompletelybytheCartesian,orrotaryFourierrepresentation.

| = |Δt ∑n=0

N−1

zn|2 1

NΔt∑m=0

N−1

Zm|2 =

=

| + |1

NΔt

∑m=0

N−1

Xm|2 1

NΔt∑m=0

N−1

Ym |2

| + |1

NΔt∑m=0

N/2

Z+m |2 1

NΔt∑m=1

N/2−1

Z−m |2

ExampleHourlycurrentmeterrecordat110mdepthfromtheBravomooringintheLabradorSea,LillyandRhines(2002)

Example1Hourlycurrentmeterrecordat110mdepthfromtheBravomooringintheLabradorSea,LillyandRhines(2002)

Example1Hourlycurrentmeterrecordat110mdepthfromtheBravomooringintheLabradorSea,LillyandRhines(2002)

ObservableFeatures1. Thedataconsistsoftwotimeseriesthataresimilarincharacter.2. Bothtimeseriespresentasuperpositionofscales.3. Atthesmallestscale,thereisanapparentlyoscillatory

roughnesswhichchangesitsamplitudeintime.4. Alargerscalepresentsitselfeitheraslocalizedfeatures,oras

wavelikeinnature.5. Severalsuddentransitionsareassociatedwithisolatedevents.6. Zoomingin,weseethesmall-scaleoscillatorybehavioris

sometimes degreesoutofphase,andsometimes .7. Theamplitudeofthisoscillatoryvariabilitychangeswithtime.

Thefactthattheoscillatorybehaviorisnotconsistently outofphaseremovesthepossibilityofthesefeaturesbeingpurelyinertialoscillations.Theamplitudemodulationsuggeststidalbeating.

90∘ 180∘

90∘

ObservableFeatures1. Thedataconsistsoftwotimeseriesthataresimilarincharacter.2. Bothtimeseriespresentasuperpositionofscales.3. Atthesmallestscale,thereisanapparentlyoscillatory

roughnesswhichchangesitsamplitudeintime.4. Alargerscalepresentsitselfeitheraslocalizedfeatures,oras

wavelikeinnature.5. Severalsuddentransitionsareassociatedwithisolatedevents.6. Zoomingin,weseethesmall-scaleoscillatorybehavioris

sometimes degreesoutofphase,andsometimes .7. Theamplitudeofthisoscillatoryvariabilitychangeswithtime.

Thefactthattheoscillatorybehaviorisnotconsistently outofphaseremovesthepossibilityofthesefeaturesbeingpurelyinertialoscillations.Theamplitudemodulationsuggeststidalbeating.

Theisolatedeventsareeddies,whichcausethecurrentstosuddenlyrotateastheypassby.Theoscillationsareduetotidesandinternalwaves.

90∘ 180∘

90∘

CartesianvsRotarySpectra

CartesianvsRotarySpectra

CartesianvsRotarySpectra

CartesianvsRotarySpectra

ExampleGlobalzonally-averagedrotaryspectrafromhourlydriftervelocities,seeElipotetal.2016.

5.Filteringandothertopics

ContinuousFourierWehaveconsideredtheFTforadiscretetimeseries :xn

= , =xn1

NΔt∑m=0

N−1

Xme2πmn/N Xm Δt ∑

n=0

N−1

xne−i2πmn/N

ContinuousFourierWehaveconsideredtheFTforadiscretetimeseries :

butitextendstocontinuoustimeseries :

Notethathereweareusingradianfrequency .

xn

= , =xn1

NΔt∑m=0

N−1

Xme2πmn/N Xm Δt ∑

n=0

N−1

xne−i2πmn/N

x(t)

x(t) = X(ω) dω, X(ω) ≡ x(t) dt.12π

∫ ∞

−∞eiωt ∫ ∞

−∞e−iωt

ω

ContinuousFourierWehaveconsideredtheFTforadiscretetimeseries :

butitextendstocontinuoustimeseries :

Notethathereweareusingradianfrequency .

Thecontinuousnotationiseasiertounderstandthemechanicsoffilteringatimeseries,aswellasspectralblurring.

xn

= , =xn1

NΔt∑m=0

N−1

Xme2πmn/N Xm Δt ∑

n=0

N−1

xne−i2πmn/N

x(t)

x(t) = X(ω) dω, X(ω) ≡ x(t) dt.12π

∫ ∞

−∞eiωt ∫ ∞

−∞e−iωt

ω

TheSpectrum(revisited)

Analternativedefinitionofthespectrum isthatitistheFouriertransformoftheautocorrelationfunction :

S(ω)R(ω)

S(ω) ≡ R(τ)dτ, R(τ) = S(ω) dω∫ ∞

−∞e−iωτ

12π

∫ ∞

−∞eiωτ

TheSpectrum(revisited)

Analternativedefinitionofthespectrum isthatitistheFouriertransformoftheautocorrelationfunction :

ThisiscalledtheWiener–Khintchinetheorem.ThespectrumandtheautocorrelationfunctionareFouriertransformpairs.Whilebothareessentiallyequivalentinthattheycapturethesamesecond-orderstatisticalinformationindifferentforms,thespectrumturnsouttogenerallybefarmoreilluminating,aswellaseasiertoworkwithinpractice.

S(ω)R(ω)

S(ω) ≡ R(τ)dτ, R(τ) = S(ω) dω∫ ∞

−∞e−iωτ

12π

∫ ∞

−∞eiωτ

TheSpectrum(revisited)

Analternativedefinitionofthespectrum isthatitistheFouriertransformoftheautocorrelationfunction :

ThisiscalledtheWiener–Khintchinetheorem.ThespectrumandtheautocorrelationfunctionareFouriertransformpairs.Whilebothareessentiallyequivalentinthattheycapturethesamesecond-orderstatisticalinformationindifferentforms,thespectrumturnsouttogenerallybefarmoreilluminating,aswellaseasiertoworkwithinpractice.

Butthetrueautocorrelationfunctionisnotobservableunlesswehave(i)infinitetimeand(ii)accesstoanabstractsetofotheruniverseswherethingsmighthavehappeneddifferently!

S(ω)R(ω)

S(ω) ≡ R(τ)dτ, R(τ) = S(ω) dω∫ ∞

−∞e−iωτ

12π

∫ ∞

−∞eiωτ

TheConvolutionIntegralTheconvolution ofafunction and isdefinedas:

Notethatinconvolution,theorderdoesnotmatterandwecanshowthat

Thismathematicaloperationisactuallywhatisbeingdonewhen“smoothing”data.(Itislikeslidingtheirononthetablecloth,orpullingthetableclothunderastaticiron).

h(t) f(t) g(t)

h(t) ≡ f(τ)g(t− τ)dτ.∫ ∞

−∞

h(t) ≡ g(τ)f(t− τ)dτ∫ ∞

−∞

ConvolutionTheoremTheconvolutiontheoremstatesconvolving and inthetimedomainisthesameasamultiplicationinthefrequencydomain.

Let and betheFouriertransformsof and ,respectively.Itcanbeshownthatif

thenthefouriertransformof is

f(t) g(t)

F (ω) G(ω) f(t) g(t)

h(t) = f(τ)g(t− τ)dτ∫ ∞

−∞

h(t)

H(ω) = F (ω)G(ω).

ConvolutionTheoremTheconvolutiontheoremstatesconvolving and inthetimedomainisthesameasamultiplicationinthefrequencydomain.

Let and betheFouriertransformsof and ,respectively.Itcanbeshownthatif

thenthefouriertransformof is

ThisresultiskeytounderstandwhathappensintheFourierdomainwhenyouperformatime-domainsmoothing.

f(t) g(t)

F (ω) G(ω) f(t) g(t)

h(t) = f(τ)g(t− τ)dτ∫ ∞

−∞

h(t)

H(ω) = F (ω)G(ω).

SmoothingNowweconsiderwhathappenswhenwesmooththetimeseries

bythefilter toobtainasmoothedversion ofyourtimeseries:

bytheconvolutiontheorem.

Whenweperformsimplesmoothing,wearealsoreshapingtheFouriertransformofthesignalbymultiplyingitsFouriertransformbythatofthesmoothingwindow.

x(t) g(t) (t)x

(t) = x(t− τ)g(τ)dτx ∫ ∞

−∞≡

=

(ω) dω.12π

∫ ∞

−∞X eiωt

X(ω)G(ω) dω,12π

∫ ∞

−∞eiωt

ThreeWindowexamples

ThreeTaperingWindows

ThreeTaperingWindows

Lowpass&HighpassFiltersFromtheconvolutiontheorem,weunderstandthatfilteringwillkeepthefrequenciesnearzerobutrejecthigherfrequencies.Forthisreasontheyarecalledlow-passfilters.

Thereversetypeoffiltration,rejectingthelowfrequenciesbutkeepingthehighfrequencies,iscalledhigh-passfiltering.

Theresidual isanexampleofahigh-passfilteredtimeseries.

Inpractice,tofindthefrequencyformofyourfilter,youpaditwithzerossothatitbecomesthesamelengthasyourtimeseries,andthenyoutakeitsdiscreteFouriertransform.

(t) ≡ x(t) − (t)x x

ConvolutionTheoremIIThistheoremisreciprocal:isyourmultiplyinthetimedomain,youconvolveintheFourierdomain.

Itcanbeshownthatif

thenthefouriertransformof is

ThisresultiskeytounderstandwhathappensintheFourierdomainwhenyoutrytoestimatespectra,i.e.spectralbluring,ortodesignband-passfilters.

h(t) = f(t)g(t)

h(t)

H(ω) = F (ν)G(ω− ν)dω.∫ ∞

−∞

BandpassfilteringWecanusetheconvolutiontheoremtobuildaband-passfilter.

Wewanttomodifythelowpassfilter sothatitsFouriertransformislocalizednotaboutzero,butaboutsomenon-zerofrequency .Todothis,wemultiply byacomplexexponential

.

Itcanbeshown(seeOslolectures)thattheFouriertransformofis ,whichislocalizedaround .

Thus,aconvolutionwith willbandpassthedatainthevicinityof .

Infact,alowpassfilterisaparticulartypeofbandpassinwhichthecenterofthepassbandhasbeenchosenaszerofrequency.

g(t)

ωo g(t)g(t)ei tωo

g(t)ei tωo G(ω− )ωo ωo

g(t)ei tωo

ωo

EffectofTruncationNowimagineinsteadthatwehaveacontinuouslysampledtimeseriesoflength ,thatis,wehave butonlybetweentimes

and .Thisislikemultiplying byafunctionwhichisequaltoonebetween and and0otherwise:

Wewilldenotethistrunctedversionof by .

Howdoesthespectrumcompareof comparewiththatof ?

T z(t)−T/2 T/2 z(t) g(t)

−T/2 T/2

(t) = g(t) × z(t)zT

z(t) (t)zT

(t)zT z(t)

EffectofTruncationNowimagineinsteadthatwehaveacontinuouslysampledtimeseriesoflength ,thatis,wehave butonlybetweentimes

and .Thisislikemultiplying byafunctionwhichisequaltoonebetween and and0otherwise:

Wewilldenotethistrunctedversionof by .

Howdoesthespectrumcompareof comparewiththatof ?

Q.Usingoneofthewindowsencounteredtoday,howcanweexpresstherelationshipbetween and ?

T z(t)−T/2 T/2 z(t) g(t)

−T/2 T/2

(t) = g(t) × z(t)zT

z(t) (t)zT

(t)zT z(t)

z(t) (t)zT

EffectofTruncationNowimagineinsteadthatwehaveacontinuouslysampledtimeseriesoflength ,thatis,wehave butonlybetweentimes

and .Thisislikemultiplying byafunctionwhichisequaltoonebetween and and0otherwise:

Wewilldenotethistrunctedversionof by .

Howdoesthespectrumcompareof comparewiththatof ?

Q.Usingoneofthewindowsencounteredtoday,howcanweexpresstherelationshipbetween and ?

Q.Therefore,usinganothertheoremlearnedtoday,whatisthedifferencebetweentheirspectra?

T z(t)−T/2 T/2 z(t) g(t)

−T/2 T/2

(t) = g(t) × z(t)zT

z(t) (t)zT

(t)zT z(t)

z(t) (t)zT

SpectralBlurringThespectrumofthetruncatedtimeseriesisblurredthroughsmoothingwithafunction thatisthesquareoftheFouriertransformofaboxcar:

Thesmoothingfunction,whichisknownastheFejérkernel

isessentiallyasquaredversionofthe“sinc”or function.

However, isnotaverysmoothfunctionatall!

(ω)FT

(ω) ≡ S(ν) (ν − ω) dν.S12π

∫ ∞

−∞FT

(ω) ≡ (1 − ) dτ =FT ∫ T

−T

|τ|T

eiωτ1T

(ωT/2)sin2

(ω/2)2

sin(x)/x

sin(x)/x

MultitaperingRevisitedWecannowunderstandthepurposeofmultitapering.

Doingnothinginyourspectralestimateisequivalenttotruncatingyourdata,thusimplicitlysmoothingthetruespectrumbyanextremelyundesirablefunction!

TheFejérkernelhasamajorprobleminthatitisnotwellconcentrated.Its“sidelobes”arelarge,leadingtoakindoferrorcalledbroadbandbias.

Thisisthesourceoftheerrorshowninthemotivatingexample.

NextwetakealookatthreedifferenttaperingfunctionsandtheirsquaredFouriertransforms.Thebroadbandbiasismostclearifweuselogarithmicscalingforthe -axis.y

ThreeTaperingWindows

ThreeTaperingWindows

ThreeTaperingWindows

EpilogueDuringthepracticalsessionthisafternoonwewillcoverthematerialpresentedthismorning,aswellcoversomeofthetopicoffiltering.

Thankyou!

ShaneElipot

email:selipot@rsmas.miami.edu