DART Tutorial Sec’on 1: Filtering For a One Variable System · DART Tutorial Sec’on 1: Slide 2...

TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.

©UCAR

DARTTutorialSec'on1:FilteringForaOneVariableSystem

DARTTutorialSec'on1:Slide2

Introduc'on

Thisseriesoftutorialpresenta'onsisdesignedtointroducebothbasicEnsembleKalmanfiltertheoryandtheDataAssimila'onResearchTestbedCommunityFacilityforEnsembleDataAssimila'on.ThereissignificantoverlapwiththeDART_LABtutorialthatisalsopartoftheDARTsubversioncheckout.IfyouhavealreadystudiedDART_LAB,feelfreetoskipthroughtheredundanttheoryslides.However,doingtheexercisesinallsec'onsofthistutorialisrecommendedinordertolearnthebestwaystousetheDARTsystem.

−6 −4 −2 0 2 4 60

0.2 Prior PDF

abilit

A :PriorEs'matebasedonallpreviousinforma'on,C.B :Anaddi'onalobserva'on.p(A|BC) :Posterior(updatedes'mate)basedonCandB.

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

= p(B | AC)p(A |C)p(B | x)p(x |C)dx∫

Bayes’Rule

DARTTutorialSec'on1:Slide3DARTTutorialSec'on1:Slide3

−6 −4 −2 0 2 4 60

0.2 Prior PDF

abilit

y Obs. Likelihood

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Bayes’Rule

−6 −4 −2 0 2 4 60

0.2 Prior PDF

abilit

y Obs. Likelihood

Product (Numerator)

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Bayes’Rule

−6 −4 −2 0 2 4 60

0.2 Prior PDF

abilit

y Obs. Likelihood

Normalization (Denom.)

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Bayes’Rule

−6 −4 −2 0 2 4 60

0.2 Prior PDF

abilit

y Obs. Likelihood

Normalization (Denom.)

Posterior

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Bayes’Rule

Green==Prior

Red==Observa'on

Blue==Posterior

ThesamecolorschemeisusedthroughoutALLTutorialmaterials.

ColorScheme

ThisproductisclosedforGaussiandistribu'ons.

−4 −2 0 2 40

abilit

Prior PDF

Obs. Likelihood

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

ProductofTwoGaussians

−4 −2 0 2 40

abilit

Prior PDF

Obs. Likelihood

Posterior PDF

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

N (µ1,∑1)N (µ2 ,∑2 ) = cN (µ,∑)

Productofd-dimensionalnormalswithmeansandandcovariancematricesand

isnormal.

µ1 µ2∑1 ∑2

isnormal.

Covariance:

N (µ1,∑1)N (µ2 ,∑2 ) = cN (µ,∑)

∑ = (∑1−1+∑2

−1)−1

µ = (∑1−1+∑2

−1)−1(∑1−1µ1 +∑2

−1µ2 )

µ1 µ2∑1 ∑2

isnormal.

Covariance:

Weight:

N (µ1,∑1)N (µ2 ,∑2 ) = cN (µ,∑)

∑ = (∑1−1+∑2

−1)−1

µ = (∑1−1+∑2

−1)−1(∑1−1µ1 +∑2

−1µ2 )

µ1 µ2∑1 ∑2

We’llignoretheweightunlessnotedsinceweimmediatelynormalizeproductstobePDFs.

c = 1(2∏)d /2 ∑1 +∑2

1/2 exp − 12

µ2 − µ1( )T (∑1 +∑2 )−1 µ2 − µ1( )⎡

⎣⎤⎦

⎧⎨⎩

⎫⎬⎭

isnormal.

Covariance:

Weight:

N (µ1,∑1)N (µ2 ,∑2 ) = cN (µ,∑)

∑ = (∑1−1+∑2

−1)−1

µ = (∑1−1+∑2

−1)−1(∑1−1µ1 +∑2

−1µ2 )

µ1 µ2∑1 ∑2

Easytoderivefor1-DGaussians;justdoproductsofexponen'als.

c = 1(2∏)d /2 ∑1 +∑2

1/2 exp − 12

µ2 − µ1( )T (∑1 +∑2 )−1 µ2 − µ1( )⎡

⎣⎤⎦

⎧⎨⎩

⎫⎬⎭

−4 −2 0 2 40

abilit

y Prior PDF

Thereareotherfamiliesoffunc'onsforwhichitisclosed…But,forgeneraldistribu'ons,there’snoanaly'calproduct.

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

−4 −2 0 2 40

abilit

y Prior PDF

Obs. Likelihood

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

−4 −2 0 2 40

abilit

y Prior PDF

Obs. Likelihood

Posterior PDF

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Don’tknowmuchaboutproper'esofthissample.Maynaivelyassumeitisrandomdrawfrom‘truth’.

Ensemblefilters:Priorisavailableasfinitesample.

−4 −2 0 2 40

abilit

Prior Ensemble

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Howcanwetakeproductofsamplewithcon'nuouslikelihood?

Fitacon'nuous(Gaussianfornow)distribu'ontosample.−4 −2 0 2 40

abilit

Prior PDF

Prior Ensemble

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Observa'onlikelihoodusuallycon'nuous(nearlyalwaysGaussian).

IfObs.Likelihoodisn’tGaussian,cangeneralizemethodsbelow.Forinstance,canfitsetofGaussiankernelstoobs.likelihood.

−4 −2 0 2 40

abilit

Prior PDF

Obs. Likelihood

Prior Ensemble

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

ProductofpriorGaussianfitandObs.likelihoodisGaussian.

Compu'ngcon'nuousposteriorissimple.BUT,needtohaveaSAMPLEofthisPDF.

−4 −2 0 2 40

abilit

Prior PDF

Obs. Likelihood

Posterior PDF

Prior Ensemble

p A | BC( ) = p(B | AC)p(A |C)p(B |C)

Therearemanywaystodothis.

Exactproper'esofdifferentmethodsmaybeunclear.Trialanderrors'llbestwaytoseehowtheyperform.Willinteractwithproper'esofpredic'onmodels,etc.

−2 −1 0 1 2 30

0.6Posterior PDF

abilit

SamplingPosteriorPDF

EnsembleAdjustment(Kalman)Filter

−4 −2 0 2 40

Prior Ensemble

abilit

Again,fitaGaussiantosample.

−4 −2 0 2 40

Prior Ensemble

abilit

Prior PDF

ComputeposteriorPDF(sameaspreviousalgorithms).

−4 −2 0 2 40

Prior Ensemble

abilit

Prior PDFObs. Likelihood

Posterior PDF

Usedeterminis'calgorithmto‘adjust’ensemble.1.  ‘Shil’ensembletohaveexactmeanofposterior.2.  Uselinearcontrac'ontohaveexactvarianceofposterior.

−4 −2 0 2 40

abilit

Posterior PDF

−4 −2 0 2 40

abilit

Posterior PDF

Mean Shifted

−4 −2 0 2 40

abilit

Posterior PDF

Mean ShiftedVariance Adjusted

p isprior,u isupdate(posterior),

isstandarddevia'on,overbarisensemblemean.

−4 −2 0 2 40

abilit

Posterior PDF

xiu = xi

p − x p( ) i σ u /σ p( ) + x uσi=1,...,ensemblesize.

−4 −2 0 2 40

abilit

Posterior PDF

Bimodalitymaintained,butnotappropriatelyposi'onedorweighted.Noproblemwithrandomoutliers.

−4 −2 0 2 40

abilit

Posterior PDF

Thereareavarietyofotherwaystodeterminis'callyadjustensemble.Classofalgorithmssome'mescalleddeterminis'csquarerootfilters.

1stlookatDARTDiagnos'cs

cd models/lorenz_63/work inyourDARTsandbox.csh workshop_setup.csh Doesstuffyou’lllearntodolater.matlab -nodesktop

OutputfromaDARTassimila'onin3-variablemodel.20memberensemble.Observa'onsofeachvariableonceevery‘6hours’;errorvariance8.Observa'onONLYimpactsitsownstatevariable.Forassimila'on,lookslike3independentsinglevariableproblems.Modeladvancebetweenassimila'onsisn’tindependent.

Ini'alensemblemembersarerandomselec'onfromlongmodelrun.Ini'alerrorshouldbeanupperbound(randomguess).

1stlookatDARTDiagnos'cs

TrythefollowingMatlabcommands.Eachwillaskyouto:Inputnameofensembletrajectoryfile:<cr>forpreassim.nc

JustselectthedefaultfilebyhipngcarriagereturnforallMatlabexercisesfornow.

'meseriesofdistancebetweenpriorensemblemeanandtruthinblue;spread,averagepriordistancebetweenensemblemembersandmean,inred.(Recordtotalvaluesoftotalerrorandspreadforlater.)'meseriesoftruthinblue;ensemblemeanpriorinred.Figure1isseparatepanelforeachstatevariable.Figure2isthree-dimensionalplot.alsoincludespriorensemblemembersingreenforFigure1.

plot_total_err

plot_ens_mean_-me_series

plot_ens_-me_series

SimpleExample:Lorenz633-variablechao'cmodel

20 −20

2010203040

Observa'oninred.Priorensembleingreen.Observingallthreestatevariables.Obs.Errorvariance=4.0.Four20-memberensembles.

20 −20

2010203040

Observa'oninred.Priorensembleingreen.

20 −20

2010203040

20 −20

2010203040

20 −20

2010203040

Observa'oninred.Priorensembleingreen.Ensembleispassingthroughanunpredictableregion.

20 −20

2010203040

Observa'oninred.Priorensembleingreen.Partoftheensembleheadsforonelobe,therestfortheother..

SimpleExample:Lorenz633-variablemodel

20 −20

2010203040

UsingDARTDiagnos'cs

UsingDARTdiagnos'csfromthesimpleLorenz63assimila'on:Canyouseeevidenceofenhanceduncertainty?Wheredoesthisoccur?Doestheensembleappeartobeconsistentwiththetruth?

(Isthetruthnormallyinsidetheensemblerange?)

1.   FilteringForaOneVariableSystem2.   TheDARTDirectoryTree3.   DARTRun>meControlandDocumenta>on4.   Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?

Mul>variateassimila>on.5.   ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6.   OtherUpdatesforAnObservedVariable7.   SomeAddi>onalLow-OrderModels8.   DealingwithSamplingError9.   MoreonDealingwithError;Infla>on10.   RegressionandNonlinearEffects11.   Crea>ngDARTExecutables12.   Adap>veInfla>on13.   HierarchicalGroupFiltersandLocaliza>on14.   QualityControl15.   DARTExperiments:ControlandDesign16.   Diagnos>cOutput17.   Crea>ngObserva>onSequences18.   LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19.   DART-CompliantModelsandMakingModelsCompliant20.   ModelParameterEs>ma>on21.   Observa>onTypesandObservingSystemDesign22.   ParallelAlgorithmImplementa>on23.  Loca'onmoduledesign(notavailable)24.  Fixedlagsmoother(notavailable)25.   Asimple1Dadvec>onmodel:TracerDataAssimila>on

DARTTutorialIndextoSec'ons

DART Tutorial Sec’on 1: Filtering For a One Variable System · DART Tutorial Sec’on 1: Slide 2...

Documents

DART Tutorial Sec’on 7: Some Addi’onal Low-Order Models · DART Tutorial Sec’on 7: Slide 9 preassim.nc preassim.nc preassim.nc true_state.nc models/lorenz_96/work/ Lorenz 96

DART Tutorial Sec’on 20: Model Parameter Es’maon

DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try

Testing Equipment for Construction Materials H-4114C.3F … · 18/02/2011 · 4" Dart— H-4114.4 6" Dart— H-4114.6 8" Dart— H-4114.8 10" Dart— H-4114.10 12" Dart— H-4114.12

waist dart

DART Tutorial Sec’on 19: Making DART-Compliant Models · DART uses iden’cal assimilaon code for menagerie of models. Same namelists you’ve been using in low-order models s’ll

Loewen Turnier Dart 1990 - Dart News | Dart Technik

Data Assimilation Research Testbed Tutorial...DART-LAB Tutorial -- June 09 pg 2 Single observed variable, single unobserved variable So far, we have a known observation likelihood

Dart intro

Dart Workshop

DART Tutorial Sec’on 25: A simple 1D advec’on model ...€¦ · source_random_amp_frac = 0.000002 &perfect_model_obs_nml input_state_files = ‘perfect_input_source_noise.nc’

- UNIGRAZ · in Accounting, Reporting, DART. and Taxation Doctoral Program in Accounting, Reporting, DART. and Taxation DART is funded by . About DART DART is a doctoral program in

DART Tutorial Sec’on 2: The DART Directory Tree · DART Tutorial Sec’on 2: The DART Directory Tree The DART Code Tree Much of DART is implemented as Fortran-90 modules and programs

DART 2013 - oficina.exagono.netoficina.exagono.net/DC2/2012/Dodge/pdf/fichas-tecnicas/dart-2013.pdf · ESPECIFICACIONES TÉCNICAS Dodge Dart SXT Dodge Dart Limited Motor Potencia

Intrinsic DART

Dart manupulation

DART Tutorial Section 18:Lost in Phase Space: The Challenge of … · 2020-06-04 · DART Tutorial Section 18:Lost in Phase Space: The Challenge of Not Knowing the Truth. Reality

Introduction Dart

Start dart

DART Tutorial Sec’on 6: Other Updates for an Observed Variable · Obs. Likelihood Random Draws from Obs. Proceed to associate a random draw from obs. with each prior sample. Earliest