Ensemble Dynamics and Bred Vectors · Ensemble Dynamics and Bred Vectors Nusret Balci Anna L. Mazzucato1 George R. Sell 2 Juan M. Restrepo 3 1Penn State University 2University of

Ensemble Dynamics and Bred Vectors

Nusret Balci Anna L. Mazzucato1 George R. Sell 2 Juan M.Restrepo 3

1Penn State University

2University of Minnesota

3Oregon State University

Workshop “Dynamics and Differential Equations”,IMA, June 22, 2016

In memory of George R. Sell

Work partially supported by the US National Science Foundation.

Thanks to IMA and Gulf of Mexico Research Institute.

Introduction The model BV and EBV Lorenz63 Finite-time LV Continuum limit CY92 Lyapunov Theory

Motivation

Data sensitivity in weather forecasting models.

These models come from discretization of 3D coupled, nonlinear PDEsplus boundary conditions.

Wide range of scales involved:∼ 10 – ∼ 104kmSubscale-grid processes aretypically parametrized. Initialand boundary conditionsmeasured on a (sparse) grid.Uncertainty in both parametersand initial/boundary data⇒ “thecurrent state of the atmosphere”.

(Courtesy NASA c©)

Ensemble Bred Vectors 2 / 48


Motivation




(Courtesy NASA c©)



Motivation




(Courtesy NASA c©)



Motivation

Want a reduced, faithful representation of variability in the currentstate:

a reduced representation of the model output covariance matrix;a representation of the sensitivity matrix = the derivative of thetangent linear equations (TLE) wrt data and parameters.

Many approaches developed both linear and nonlinear:

singular vectors and empirical orthogonal functions;ensemble Kalman filters;bred vectors;Lyapunov vectors for the TLE.



Motivation

Want a reduced, faithful representation of variability in the currentstate:

a reduced representation of the model output covariance matrix;a representation of the sensitivity matrix = the derivative of thetangent linear equations (TLE) wrt data and parameters.

Many approaches developed both linear and nonlinear:

singular vectors and empirical orthogonal functions;ensemble Kalman filters;bred vectors;Lyapunov vectors for the TLE.



Data assimilation

Error estimation of the current state of the system.

Based on an analysis cycle followed by the forecast cycle. Theanalysis cycle uses a (discrete) forward model to reduce uncertainty.

In the analysis step, observations of the current state are fed into theforward model to output an “analysis”, best estimate of the currentstate of the system.

Analysis step tries to balance error in the data and forecast.

Model is next advanced in time and result used for next analysis cycle.



Data assimilation








Data assimilation








Motivation cont.

The Bred Vector (BV) algorithm is popular algorithm to capture mostunstable modes in weather forecasting models (Toth & Kalnay 1993,1997).

In use until mid 2000’s at National Centers for EnvironmentalPrediction (NCEP). Now replaced by ensemble Kalman filtering withBV at core (Bishop et al 2001, Wang-Bishop 2003, Wei et al2006-2008).

We propose a variant of the BV algorithm, the Ensemble Bred Vector(EBV) algorithm. Our goals are:

provide a mathematical foundation for the BV algorithm and itsrelation to Lyapunov vectors (LV);

obtain a more stable and accurate algorithm for data sensitivity.



Motivation cont.








Motivation cont.








Case Studies: Lorenz63

Consider two “toy” models for weather and climate predictions.

• The Lorenz 3D system (E. Lorenz 1963), chaotic dynamical systemwith three degrees of freedom:

∂tX = A X + N(X ),

with X = (X1,X2,X3) and

A =

−σ σ 0r −1 00 0 −b

, N(X ) =

0−X1X3X1X2

,σ, r , b parameters to be chosen.






∂tX = A X + N(X ),


A =

−σ σ 0r −1 00 0 −b

, N(X ) =

0−X1X3X1X2




Case studies: the Cessi-Young (CY92) model

• A forced-dissipative system given by a Cahn-Hilliard type partialdifferential equation (PDE) in 1D.

Model for the thermohaline circulation in the oceans (W. Young and P.Cessi 1992):

∂S∂t = α ∂2

∂x2 [f (x) + µS(S − sin(x))2 + S − γ ∂2S∂x2 ] t > 0,

S(x ,0) = S0(x).

S(x , t) ocean salinity, zonally-averaged, as a function of latitudex ∈ [−π, π] and time t ≥ 0.

Impose periodic boundary conditions, instead of more physical no-flux,no-stress b.c. Take forcing f (x) asymmetric (Eyink 2005).Refer to the Cahn-Hilliard model as the CY92 model.






∂S∂t = α ∂2


S(x ,0) = S0(x).








∂S∂t = α ∂2


S(x ,0) = S0(x).





Thermohaline Circulation

(Courtesy NOAA)



The General Nonlinear Problem

Weather models are typically finite-dimensional:

by mode selection or

by spatial discretization on a grid.

⇒ reduce to systems of ODEs

Consider an initial-value problem (IVP) for a nonlinear system of ODEson RK , K possibly large:

dydt

= G(y), y(0) = y0, t ≥ 0, y , y0 ∈ RK .

with G(y) sufficiently smooth.

y(t) denotes the solution of the IVP.



The General Nonlinear Problem

Weather models are typically finite-dimensional:

by mode selection or

by spatial discretization on a grid.

⇒ reduce to systems of ODEs

Consider an initial-value problem (IVP) for a nonlinear system of ODEson RK , K possibly large:

dydt

= G(y), y(0) = y0, t ≥ 0, y , y0 ∈ RK .

with G(y) sufficiently smooth.

y(t) denotes the solution of the IVP.



The Tangent Linear Equations

Model the long-time dynamics of the system ⇒ consider trajectorieson or near the attractor A for non-linear system.

Goal: Study the error growth due to small changes in the initialconditions on A.

Derive Tangent Linear Equations (TLE) to the flow on the attractor:

∂t x = A(θ · t) x

x(0) = x0 ∈ RK ,

where A(y) = DG(y) is the Jacobian matrix of G (non-autonomoussystem).

θ denotes point in A and θ · t = y(t) denotes the unique solution ofnon-linear system s.t. y(0) = θ.







∂t x = A(θ · t) x

x(0) = x0 ∈ RK ,









∂t x = A(θ · t) x

x(0) = x0 ∈ RK ,









∂t x = A(θ · t) x

x(0) = x0 ∈ RK ,





Skew Product Semiflow

The TLE generate a skew-product semiflow Π = Π(t):

Π(t)(θ, x0) := (θ · t , U(θ, t)x0),

where U(θ, t) denote the solution operator for TLE.

1 Π(0) = I and

Π(τ + t) = Π(t) Π(τ), τ, t ∈ R,

from the cocycle identity U(θ, τ + t) = U(θ · τ, t) U(θ, τ).

2 A× RK is invariant under Π(t).



Skew Product Semiflow

The TLE generate a skew-product semiflow Π = Π(t):

Π(t)(θ, x0) := (θ · t , U(θ, t)x0),

where U(θ, t) denote the solution operator for TLE.

1 Π(0) = I and

Π(τ + t) = Π(t) Π(τ), τ, t ∈ R,

from the cocycle identity U(θ, τ + t) = U(θ · τ, t) U(θ, τ).

2 A× RK is invariant under Π(t).



Time Discretization.

Recall continuous time solution y = y(t) for the non-linear model.

Approximate solution Yn = Y (tn); on uniform time grid tn ⇒tn+1 = tn + δt , δt positive and small, t0 = 0.

Let M(Y , δt) be a discrete-time solution operator on RK for thenon-linear system. Set:

Yn+1 = M(Yn, δt), Y0 = Y (0),

Assume explicit time discretization.

In numerical test, use 4th-order Runga-Kutta scheme.

Examine connection between the continuous-time and discrete-timetheories.







Yn+1 = M(Yn, δt), Y0 = Y (0),










Yn+1 = M(Yn, δt), Y0 = Y (0),










Yn+1 = M(Yn, δt), Y0 = Y (0),






Families of Perturbations

To assess sensitivity to initial conditions, run discrete forward model forensemble perturbations of the current state.

Assume initial data Y0 is on, or near the attractor A.

Next consider a (finite) family of tangent vectors at t0 = 0,{δY0(ι)}ι∈I, denoted the cloud.

The collection {(Y0, δY0(ι))}ι∈I is the Family of InitialPerturbations to Y0.

Choose cloud such that, for given ε positive, small,

‖δY0(ι)‖ = ε, for all ι ∈ I,

Evolve the cloud and the family of perturbation in time.











































The Bred Vector (BV) algorithm (Toth & Kalnay, 1993)

Given an initial condition Y0 and a family of perturbations, advance thefamily in discrete time:

(Y0, δY0(ι))→ {(Yn, δYn(ι))}, for n ≥ 0 and ι ∈ I.

The (n + 1)th step, defined recursively by:

δYn+1(ι) = M(Yn + δYn(ι), δt)−M(Yn, δt),

δYn+1(ι) = Rn+1(ι) δYn+1(ι),

where

Rn+1 = Rn+1(ι) :=‖δYn(ι)‖‖δYn+1(ι)‖

=‖δY0(ι)‖‖δYn+1(ι)‖

.








δYn+1(ι) = Rn+1(ι) δYn+1(ι),

where

Rn+1 = Rn+1(ι) :=‖δYn(ι)‖‖δYn+1(ι)‖

=‖δY0(ι)‖‖δYn+1(ι)‖

.








δYn+1(ι) = Rn+1(ι) δYn+1(ι),

where

Rn+1 = Rn+1(ι) :=‖δYn(ι)‖‖δYn+1(ι)‖

=‖δY0(ι)‖‖δYn+1(ι)‖

.



The BV algorithm cont.

The outcome of the BV algorithm is a Bred Vector ⇒ “breed"perturbations through forward, nonlinear (time-discrete) model M.

Rescaling via factor Rn+1 avoids spurious growth due to nonlinearamplification.

No derivation of the TLE needed (handle legacy codes).

Rescaling time step δτ = tn+km − tn+(k−1)m, m ∈ N, possiblydifferent than discretization time step δt .

Choice of size ε is ad hoc.

Rn+1 = ‖δY0(ι)‖ / ‖δYn+1(ι)‖ depends on Y0 and ‖δYn(ι)‖ = ε,for all n ≥ 0 and ι ∈ I.

⇒ members of the ensemble rescaled independently to samesize.























BV variants

Suppression of non-linear growth leads to eventual alignment of allBV’s with leading Lyapunov vector.

Uniform rescaling may suppress strong, transient growth, which maybe important for local dynamics.

Orthogonalize output periodically (Annan 2004, Keller et al. 2010),span phase space;

Optimize choice of norm (Primo et al. 2008, Pazo et al. 2011,Hallerberg et al. 2010). Logarithmic BV: use the geometricpseudo-norm (discrete “0 norm”):

‖Y‖0 =n∏ι=1

|Y (ι)|1/n.

non-linear optimal singular vectors (Mu-Jiang 2008, Rivière et al.2008), expensive.



BV variants

Suppression of non-linear growth leads to eventual alignment of allBV’s with leading Lyapunov vector.

Uniform rescaling may suppress strong, transient growth, which maybe important for local dynamics.

Orthogonalize output periodically (Annan 2004, Keller et al. 2010),span phase space;

Optimize choice of norm (Primo et al. 2008, Pazo et al. 2011,Hallerberg et al. 2010). Logarithmic BV: use the geometricpseudo-norm (discrete “0 norm”):

‖Y‖0 =n∏ι=1

|Y (ι)|1/n.

non-linear optimal singular vectors (Mu-Jiang 2008, Rivière et al.2008), expensive.



The Ensemble BV Algorithm (EBV)

Only difference is in rescaling rule:

Use scaling rule Rminn+1 adapted to the ensemble dynamics of the

cloud ⇒ uniform rescaling, the same for all ι ∈ I.

The EBV algorithm then reads:


δYn+1(ι) = Rminn+1 δYn+1(ι),

with

Rminn+1 = ε [max

ι∈I(‖δYn+1(ι)‖)]−1.

Note that ‖δYn(ι)‖ ≤ ε, for all n ≥ 0.Ensemble Bred Vectors 17 / 48









with

Rminn+1 = ε [max

ι∈I(‖δYn+1(ι)‖)]−1.










with

Rminn+1 = ε [max

ι∈I(‖δYn+1(ι)‖)]−1.










with

Rminn+1 = ε [max

ι∈I(‖δYn+1(ι)‖)]−1.



The EBV algorithm cont.

The output of the EBV algorithm called an Ensemble Bred Vector(EBV).

Simple change in rescaling has far reaching consequences.

Denote:V (tn−1): an EBV member at t = tn−1,

V̄ (tn): the vector at t = tn before rescaling, and

V (tn): V̄ (tn) rescaled.

Analyze the growth or decay of the EBV size ⇒

‖V (tn)‖‖V (tn−1)‖

= Rminn‖V̄ (tn)‖‖V (tn−1)‖

,










‖V (tn)‖‖V (tn−1)‖


,










‖V (tn)‖‖V (tn−1)‖


,










‖V (tn)‖‖V (tn−1)‖


,



The new EBV algorithm cont.

Consequences of the new rescaling rule:

Automatic size ordering among all EBV members.

Contrary to EBV, in the BV algorithm dynamics of each ensemblemember is decoupled.

Recurring patterns (nontrivial attractor in action): ratio ‖V (tn)‖‖V (tn−1)‖

exhibits complex behavior.

Separation of scales: different growth rate of different-size vectors,not suppressed by rescaling to initial size.

Can better monitor transient growth due to nonlinear effect.

Perturbation size: the choice of ε in the EBV more robust becauseof size ordering.


















































∂tX = A X + N(X ),


A =

−σ σ 0r −1 00 0 −b

, N(X ) =

0−X1X3X1X2







∂tX = A X + N(X ),


A =

−σ σ 0r −1 00 0 −b

, N(X ) =

0−X1X3X1X2




Lorenz63 Tangent Linear Equations and Attractor

Choose σ = 10, r = 28, and b = 8/3 ⇒The Lorenz system has a (compact) chaotic attractor.

The Tangent Linear Equations (TLE) are given by

∂tX = A X + N(X )

∂tW = (A + DN(X )) W ,

where DN(X ) = ∂X N(X ).

The EBV gives insight into the fractal nature of the attractor, withoutderiving TLE.



Lorenz63 Tangent Linear Equations and Attractor

Choose σ = 10, r = 28, and b = 8/3 ⇒The Lorenz system has a (compact) chaotic attractor.

The Tangent Linear Equations (TLE) are given by

∂tX = A X + N(X )

∂tW = (A + DN(X )) W ,

where DN(X ) = ∂X N(X ).

The EBV gives insight into the fractal nature of the attractor, withoutderiving TLE.



Numerical test for Lorenz 63

Use 2-norm for rescaling.

Initial data for both BV and TLE:

X = 0.5688,Y = 0.4694,Z = 0.0119, δY0 = [1,1,0].

Initial point near the attractor.

Initial perturbations: projection onto sphere in R3 of:

{−1,−0.75,−0.5,−0.25,0,0.25,0.5,0.75,1}3

ε = 0.1

Cloud of 584 distinct points spanning tangent space fairly uniformly.

Time stepping:

δt = 0.0001, δτ = 0.001, 0.004.

Final time T = 2.Ensemble Bred Vectors 22 / 48





X = 0.5688,Y = 0.4694,Z = 0.0119, δY0 = [1,1,0].



{−1,−0.75,−0.5,−0.25,0,0.25,0.5,0.75,1}3

ε = 0.1


Time stepping:

δt = 0.0001, δτ = 0.001, 0.004.






X = 0.5688,Y = 0.4694,Z = 0.0119, δY0 = [1,1,0].



{−1,−0.75,−0.5,−0.25,0,0.25,0.5,0.75,1}3

ε = 0.1


Time stepping:

δt = 0.0001, δτ = 0.001, 0.004.



Views of the Lorenz attractor

(a)−0.100.1 −0.100.1

−0.1

−0.05

0

0.05

0.1

YX

Z

(b) −0.10

0.1

−0.1−0.0500.050.1−0.1

−0.05

0

0.05

0.1

XY

Z

(c)−0.1

00.1−0.1 0 0.1

−0.1

−0.05

0

0.05

0.1

XY

Z

Figure: Three different angle views. Rotation about Z = X3 axis. "Flower"structure obtained by plotting the output of EBV for subsequent time steps.



Fractal behavior of the Lorenz attractor

(a)−0.0100.01 −0.0100.01

−0.01

−0.005

0

0.005

0.01

YX

Z

(b)−202

x 10−3 −202

x 10−3

−2

0

2

x 10−3

YX

Z

(c)−101

x 10−3 −101

x 10−3

−1

0

1

x 10−3

YX

Z

Figure: Figure 23 a): zoomed in 8 times (a); 32 times (b); 60 times (c).Observe the self-similar structure of the attractor.



Sample EBV output - 3D Lorenz, δt = 0.005, T = 40

0 5 10 15 20 25 30 35 400

0.2

0.4

0.6

0.8

1

time

2-norm for EBV output of 98 initial perturbations of the Lorenz63model.Note size ordering after transient (t ≤ 13), yet strong local growth isretained for t > 17.



Lyapunov Exponents and Vectors:

Extension of spectral theory for autonomous linear systems tonon-autonomous system (Lyapunov 1892.)

For θ on the attractor, x0 ∈ RK , x0 6= 0, consider the 4 Lyapunovrelations for exponential growth:

lim supt→±∞

1t

log (‖U(θ, t)x0‖) , lim inft→±∞

1t

log (‖U(θ, t)x0‖) ,

U(θ, t) solution operator of the TLE along trajectory θ · t .

When 4 limits the same, set λ = λ(θ, x0) to satisfy

λ := limt→∞

1t

log (‖U(θ, t)x0‖) = limt→−∞

1t

log (‖U(θ, t)x0‖) (1)

λ is a strong Lyapunov exponent, x0 a related Lyapunov vector.






lim supt→±∞

1t


1t

log (‖U(θ, t)x0‖) ,



λ := limt→∞

1t


1t

log (‖U(θ, t)x0‖) (1)







lim supt→±∞

1t


1t

log (‖U(θ, t)x0‖) ,



λ := limt→∞

1t


1t

log (‖U(θ, t)x0‖) (1)




Finite-time Lyapunov Vectors

In practice, can only integrate system for a finite time interval [0,T ].

Define a finite-time version of Lyapunov vectors using piece-wiseautonomous approximation of the TLE:

∂tY (t) = An Y (t), for tn < t ≤ tn+1,

Y (0) = Y0,

tn = n δt , n = 0,1,2, · · · ,N − 1.

where An = A(y(tn)) := ∂G∂y |y=(y(tn)), y(t) solution of non-linear

problem.



Finite-time Lyapunov Vectors cont.

Define leading finite-time LV as direction of steepest ascent of

Z (T ) : = eδt AN−1eδt AN−2 · · · eδt A1eδt A0

∼ (I + δt AN−1) · · · (I + δt A1) · · · (I + δtA0),

That is the singular vector associated to largest singular value of Z (T ).

In our tests, compute finite-time LV by running discrete approx to TLEand periodically rescaling (to size 1).

Compare BV and LV.



Finite-time Lyapunov Vectors cont.

Define leading finite-time LV as direction of steepest ascent of

Z (T ) : = eδt AN−1eδt AN−2 · · · eδt A1eδt A0

∼ (I + δt AN−1) · · · (I + δt A1) · · · (I + δtA0),

That is the singular vector associated to largest singular value of Z (T ).

In our tests, compute finite-time LV by running discrete approx to TLEand periodically rescaling (to size 1).

Compare BV and LV.



Lorenz 63: BV versus LV

0 5 10 15 20 25 30 35 40−2

0

2

δ X

0 5 10 15 20 25 30 35 40−2

0

2

δ Y

0 5 10 15 20 25 30 35 40−2

0

2

δ Z

Time

The three components of BV and of the finite-time Lyapunov vectors,as a function of time, for the Lorenz63 system. The perturbation size is1, in all of the components. Transient regular behavior disappears att = 13, approximately.



Projective Semiflow and Projective Metric

Solution operator U(θ, t) of the TLE maps lines onto lines in RK

⇒ induced semiflow Σ(θ, t) on the projective space PK−1, with [u]the line in RK through u.

Σ(θ, t) [u] = [U(θ, t) u], for [u] ∈ PK−1.

Induced projective semiflow [Π] on A× PK−1:

[Π(t)](θ, [x0]) = (θ · t ,Σ(θ, t)[x0]), for (θ, [x0]) ∈ A× PK−1.

d(U,V ) denotes the projective metric on PK−1:

d(u1,u2) = mins1,s2‖s1u1 ± s2u2‖,

where numbers s1, s2 satisfy ‖s1u1‖ = ‖s2u2‖ = 1.



Projective Semiflow and Projective Metric

Solution operator U(θ, t) of the TLE maps lines onto lines in RK

⇒ induced semiflow Σ(θ, t) on the projective space PK−1, with [u]the line in RK through u.

Σ(θ, t) [u] = [U(θ, t) u], for [u] ∈ PK−1.

Induced projective semiflow [Π] on A× PK−1:

[Π(t)](θ, [x0]) = (θ · t ,Σ(θ, t)[x0]), for (θ, [x0]) ∈ A× PK−1.

d(U,V ) denotes the projective metric on PK−1:

d(u1,u2) = mins1,s2‖s1u1 ± s2u2‖,

where numbers s1, s2 satisfy ‖s1u1‖ = ‖s2u2‖ = 1.



Continuum Limits

Return to the ODE dydt = G(y).

Let U(t) = U(θ, t)x0 the unique solution of the IVP for the TLE.

Set BV (t) and EBV (t) to be the respective algorithms with sameclouds and initial data (θ, x0).

Use the projective metric d to measure distance between outputs.

TheoremUniformly on 0 ≤ t ≤ T ,

limδt→0,ε→0

d(BV (t),U(t)) = limδt→0,ε→0

d(EBV (t),U(t)) = 0.



Continuum Limits






limδt→0,ε→0


d(EBV (t),U(t)) = 0.



Continuum Limits






limδt→0,ε→0


d(EBV (t),U(t)) = 0.



Continuum Limits: Numerical Validation

δt max BV min BV max EBV min EBV0.004 1.40× 10−1 5.26× 10−4 1.01× 10−1 1.10× 10−3

0.001 7.77× 10−2 1.03× 10−4 3.48× 10−2 4.16× 10−5

I denote the index set over the 584 scaled initial pertubations inR3.

Evaluate U(t), BV (t), and EBV (t) over the interval 0 ≤ t ≤ T = 2,using the same initial conditions.

Table gives the minimum and maximum values (over I), ford(BV (2),U(2)) and d(EBV (2),U(2)), with step sizes δt = 0.004and δt = 0.001, ε = 0.1.

Note faster drop in the distance for EVB. Direction of fastestdecay approx leading finite-time LV.





0.001 7.77× 10−2 1.03× 10−4 3.48× 10−2 4.16× 10−5









0.001 7.77× 10−2 1.03× 10−4 3.48× 10−2 4.16× 10−5










∂S∂t = α ∂2


S(x ,0) = S0(x).








∂S∂t = α ∂2


S(x ,0) = S0(x).








∂S∂t = α ∂2


S(x ,0) = S0(x).





The CY92 model cont.

Set S = ∂xu and f = ∂xF ⇒ the CY92 can be related to theCahn-Hilliard Equation (CHE) with forcing:

∂tu + ν42 u = 4(g(u)) + F ,

with g is a degree-3 polynomial g(u) =∑3

j=1 aj uj , with a3 > 0.

Cahn-Hilliard has a Lyapunov functional for F = 0.

Cahn-Hilliard has an inertial manifold (Foias et al. 1988, Sell-You2002) ⇒ long-time dynamics contained in the compact attractor of afinite-dimensional system .

Determining modes are obtained by projecting onto the first Neigenfunctions of the bi-Laplacian, N large.



The CY92 model cont.

Set S = ∂xu and f = ∂xF ⇒ the CY92 can be related to theCahn-Hilliard Equation (CHE) with forcing:

∂tu + ν42 u = 4(g(u)) + F ,

with g is a degree-3 polynomial g(u) =∑3

j=1 aj uj , with a3 > 0.

Cahn-Hilliard has a Lyapunov functional for F = 0.

Cahn-Hilliard has an inertial manifold (Foias et al. 1988, Sell-You2002) ⇒ long-time dynamics contained in the compact attractor of afinite-dimensional system .

Determining modes are obtained by projecting onto the first Neigenfunctions of the bi-Laplacian, N large.



BV and EBV for CY92

Numerical experiments indicate:

EBV converges quickly and is more robust than BV.

EBV is less sensitive to detail of the initial perturbation (e.g.frequency and size) than BV.

Information on multiple scales retained.

Contrast with BV algorithm (all vectors become O(1).)

BVs eventually and then largest member of the EBV approachleading Lyapunov vector.



BV and EBV for CY92









BV and EBV for CY92









Numerics for the CY92 model

Model:∂S∂t

= α∂2

∂x2 [f (x) + µS(S − sin(x))2 + S − γ ∂2S∂x2 ], with

∂2x f (x) =

227(36− 39µ2 cos2 x − 81 cos2 x + 8µ2 + 25µ2 cos4 x) sin x ,

for x ∈ [−π,0],

−29(3µ2 cos2 x − 3− µ2) sin x ,

for x ∈ (0, π].

Parameters α = 3.5× 10−3 to α = 0.01, γ = 0.001, µ =√

10.

Initial condition S0(x) = cos(x), final time T = 70.

Initial perturbations of size ε = 0.25 to ε = 1.2:

δY0 = ε sin[jx +13

(j − 1) exp(1)], j = 1, . . . ,6.



Numerics for the CY92 model

Model:∂S∂t

= α∂2

∂x2 [f (x) + µS(S − sin(x))2 + S − γ ∂2S∂x2 ], with

∂2x f (x) =

227(36− 39µ2 cos2 x − 81 cos2 x + 8µ2 + 25µ2 cos4 x) sin x ,

for x ∈ [−π,0],

−29(3µ2 cos2 x − 3− µ2) sin x ,

for x ∈ (0, π].

Parameters α = 3.5× 10−3 to α = 0.01, γ = 0.001, µ =√

10.

Initial condition S0(x) = cos(x), final time T = 70.

Initial perturbations of size ε = 0.25 to ε = 1.2:

δY0 = ε sin[jx +13

(j − 1) exp(1)], j = 1, . . . ,6.



CY92: forcing and base solution

Time stepping: δt = 0.01 to δt = 0.000015,

Space discretization: second-order centered differences, 121 to 320grid points, −π = x0, x1, ..., x120 = π.

!3 !2 !1 0 1 2 3!8

!6

!4

!2

0

2

X time

X

0 10 20 30 40 50 60!3

!2

!1

0

1

2

3

!1

!0.5

0

0.5

(a) (b)

(a) Forcing function fxx ; b) the numerical solution.



CY92 numerics: LV and BV sensitivity

!3 !2 !1 0 1 2 3

!0.5

0

0.5

1

X

LVBV

!3 !2 !1 0 1 2 3

!1

!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

X

LVBV

!3 !2 !1 0 1 2 3

!0.5

0

0.5

1

X

LVBV

!3 !2 !1 0 1 2 3

!0.5

0

0.5

1

X

LVBV

!3 !2 !1 0 1 2 3

!1

!0.5

0

0.5

X

LVBV

!3 !2 !1 0 1 2 3

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

1

X

LVBV

(a) (b)

(c) (d)

(e) (f)

Figure: Wavenumber dependence of BVs and finite-time Lyapunov vectors,εj = 0.25, (a) j = 1, (b) j = 2, ..., (f) j = 6.



CY92 numerics: BV

(a) −3 −2 −1 0 1 2 3

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

X (b) −3 −2 −1 0 1 2 3

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0.8

X (c) −3 −2 −1 0 1 2 3

−1

−0.5

0

0.5

1

X

Figure: BVs at t = 19.05, corresponding to εj = ε, for all j . (a) ε = 0.6, (b)ε = 0.8, (c) ε = 1.2. Initial perturbation δY j

0 = ε sin[jx + 13 (j − 1) exp(1)]. The

outcomes are sensitive to nonlinearity. There is a resulting ambiguity instructure of the perturbation field.



CY92 numerics: EBV

!3 !2 !1 0 1 2 3

!0.4

!0.2

0

0.2

0.4

X!3 !2 !1 0 1 2 3

!0.4

!0.2

0

0.2

0.4

0.6

X

!3 !2 !1 0 1 2 3!0.8

!0.6

!0.4

!0.2

0

0.2

0.4

0.6

0.8

X

(a) (b)

(c)

Figure: EBV with (a) ε = 0.6, (b) ε = 0.8, (c) ε = 1.2. The vectors are shownin their original scales.



(a) −3 −2 −1 0 1 2 3

−1

−0.5

0

0.5

1

X (b) −3 −2 −1 0 1 2 3

−1

−0.5

0

0.5

1

X (c) −3 −2 −1 0 1 2 3

−1

−0.5

0

0.5

1

X

Figure: EBV outcomes, with each vector rescaled to an L2-norm of 1 to aid invisual comparison. Perturbation amplitudes (a) ε = 0.6, (b) ε = 0.8, and (c)ε = 1.2. Note that as the perturbation amplitude increases, the resemblancebetween the BV and the EBV outcomes is lost, except for one of the vectors.






lim supt→±∞

1t


1t

log (‖U(θ, t)x0‖) ,



λ := limt→∞

1t


1t

log (‖U(θ, t)x0‖) (1)







lim supt→±∞

1t


1t

log (‖U(θ, t)x0‖) ,



λ := limt→∞

1t


1t

log (‖U(θ, t)x0‖) (1)







lim supt→±∞

1t


1t

log (‖U(θ, t)x0‖) ,



λ := limt→∞

1t


1t

log (‖U(θ, t)x0‖) (1)




The Red Vector

The Red Vector is defined as a Lyapunov vector x1 such that for anyother Lyapunov vector x2, λ(θ, x2) ≤ λ(θ, x1)

Show x1 is the direction of maximal growth of the error due to changesin the initial conditions → x1 is a good choice as initial condition for theTLE.

Lyapunov vectors are infinite-time objects. To compute, exploitdynamics on the attractor and use BV, EBV to approximate TLE.

Main tools (R. A. Johnson, K. J. Palmer, and G. R. Sell, 1997):Foliations for the linear solution operator U(θ, t):

Exponential Dichotomies and the Sacker-Sell Spectrum.

The Multiplicative Ergodic Theorem (MET) and ergodic measures.



The Red Vector









The Red Vector









The Red Vector









Robust Foliations and Exponential Dichotomies:

LetM denote a compact, invariant set in the attractor A.

Consider the family of shifted semiflows

Uλ(θ, t) = e−λtU(θ, t), for λ ∈ R,

and the associated skew-product flows

Πλ(t)(θ, x0) = (θ · t , Uλ(θ, t)x0).

The Sacker-Sell (SS) Spectrum SS Σ: all the λ ∈ R, for whichΠλ(t) has no exponential dichotomy overM.

Πλ has an exponential dichotomy overM if there exist projectorsPλ and Qλ and constants K ≥ 1 and α > 0, such thatPλ + Qλ = I, and for all θ ∈ M and all u ∈ RK ,

‖Uλ(θ, t)Qλ(θ) u‖ ≤ K e(−α t)‖u‖, t ≥ 0,‖Uλ(θ, t)Pλ(θ) u‖ ≤ K e(α t)‖u‖, t ≤ 0.








Πλ(t)(θ, x0) = (θ · t , Uλ(θ, t)x0).











Πλ(t)(θ, x0) = (θ · t , Uλ(θ, t)x0).











Πλ(t)(θ, x0) = (θ · t , Uλ(θ, t)x0).






The Sacker-Sell continuous foliation

There is an integer `, with 1 ≤ ` ≤ K , such thatSS Σ = ∪i=`

i=1[ai ,bi ], ai ≤ bi and bi < ai+1.

If λ ∈ SS Σ, there is (θ, x0) in A× RK such that

supt∈R‖Uλ(θ, t)x0‖ <∞.

There is a continuous foliation

RK =⊕̀i=1

Vi(θ), for θ ∈M,

with∑i=`

i=1 dim Vi(θ) = K .

The foliation is robust to small perturbations.









RK =⊕̀i=1


with∑i=`











RK =⊕̀i=1


with∑i=`





Measurable foliations

For each i , the vector bundle {(θ, x0) ∈M× Vi(θ)} is invariant underthe skew-product flow Π.

To compute LV, strong Lyapunov exponents are needed.⇒ Employ refinement of the foliation measurable with respect toergodic measures onM.

Denote by PRB = PRB(M) the collection of invariant probabilitymeasures onM.µ ∈ PRB ⇒ µ is non-negative and µ(M) = 1.

Denote by ERG = ERG(M) all the measures µ ∈ PRB that areergodic: there are no invariant subsets Q inM with the property that0 < µ(Q) < 1.































The Multiplicative Ergodic Theorem (MET)

Theorem (the MET - Oseledets)For every ergodic measure µ onM, there is an invariant setMµ inM,with µ(Mµ) = 1 and

There is a measurable foliation:

RK =k⊕

j=1

Wj(θ), for θ ∈Mµ,

with dim Wj(θ) = mj ≥ 1, and m1 + · · ·+ mk = K .

There are real numbers λj , for 1 ≤ j ≤ k, and strong Lyapunovexponents λ(θ, x0) = λj , for all (θ, x0) ∈Mµ ×Wj(θ), withx0 6= 0, and λk < · · · < λ1.






RK =k⊕

j=1









RK =k⊕

j=1






Measurable Foliation: Properties

For each j , the measurable vector bundle

{(θ, x) : θ ∈Mµ, x ∈Wj(θ)}

invariant under projective flow Π(t).

The measurable projective bundle Mµ × Pj is also invariantunder the projective flow, where

Pj = Pj(θ) = {[u] ∈ PK−1 : u ∈Wj(θ) }.The projective fiberMµ × P1 is an attractor for the projective flow,due to exponential separation.

The Lyapunov vector corresponding to the largest Lyapunovexponent defines the direction of maximal growth of the error dueto changes in the initial conditions.

If dimV1>dimW1 = 1, the red vector is unique, but is onlymeasurable and may not be robust to perturbations.





{(θ, x) : θ ∈Mµ, x ∈Wj(θ)}










{(θ, x) : θ ∈Mµ, x ∈Wj(θ)}










{(θ, x) : θ ∈Mµ, x ∈Wj(θ)}







Documents

Ensemble Dynamics and Bred Vectors · Ensemble Dynamics and Bred Vectors Nusret Balci Anna L. Mazzucato1 George R. Sell 2 Juan M. Restrepo 3 1Penn State University 2University of