Approaches to climate change study and neural network modelling

Summer School on Climate Change

UniKore, 6-10 September 2008 1

Approaches to climate change study and neural network

modelling

Antonello PasiniCNR - Institute of Atmospheric Pollution

Rome, Italy



Outline

• Climate science and dynamical modelling;

• neural network model;• assessment on the past;• about predictability in past

and future scenarios;• NN downscaling;• conclusions and prospects.



Climate scientists as grown-up babies

If you give a child a toy, he will eventually open it up.

Let’s open it up!



Climate scientists as grown-up babies

Let’s understand how it works...

… and let’s reassemble it!



Decomposing the system



Theoretical knowledgeWe possess theoretical knowledge of single sub-systems from experiments in “real laboratories” (e.g., laws from fluid-dynamics and thermodynamics of oceans and atmosphere).

In order to recompose the complexity of the system, we need a “virtual laboratory”...



Recomposing in a model



From dynamical modelling...

• Physical characterization and forecasting in the climate system is a very difficult task, if we adopt an approach with complete dynamics.

• Global Climate Models (GCMs) are the standard tools for grasping this complexity.

• They permit to recognize the role of some cause-effect relationships.







• However, the results of GCMs could crucially depend on the delicate balance (fine tuning) among the relative strength of feedbacks and the various parameter-ization routines doubtful results .

• Furthermore, they show limits in reconstruction and forecasting at regional and local scales.

• So, an independent (more “holystic”) analysis could be interesting.



… to a different strategy

• The simplest idea: application of a multivariate linear model to the analysis of influence/causality:

forcings (which influence temperature)vs.

temperature itself

• Bad results: the linear model is too simple neural networks!



A biological “inspiration”




Natural inputs

Anthropogenic inputs

Climatic behaviour




hl

j

jj

n

hhg

exp1

1

0

0.2

0.4

0.6

0.8

1

-10 -8 -6 -4 -2 0 2 4 6 8 10

weighted sum

sigm

oid

outp

ut n=3

n=8

n=20

n=50

A neural network model



i

ii OTE2

2

1

j kkjkjijii IwgWgO




1=

11

tWtWmVOThgtW

tWtWmtW

EtWtW

ijijjiiiiij

ijijij

ijij




1

11

twtwmIOThgWhg +tw

twtwmtw

Etwtw

jkjkki

iiiiijjjjk

jkjkjk

jkjk





• Tool for short historical time series of data (“all-frame” or “leave-one-out” procedure);

• early stopping method.



Assessment on the past

Global case study;

regional case study.



Global case studyInput data:• solar irradiance and stratospheric optical

thickness as indices of natural forcings coming from Sun and volcanoes;

• CO2 concentration and sulfate emissions as anthropogenic forcings;

• SOI index (ENSO) as a circulation pattern in ocean and atmosphere which can be important for better catching the inter-annual temperature variability.



Global case study4 case studies:a) natural forcings only;b) anthropogenic forcings only;c) natural + anthropogenic forcings;d) natural + anthropogenic forcings + ENSO.

In cases when anthropogenic forcings are considered, a strong improvement in the reconstruction performance is achieved by neural modelling (vs. linear modelling).



Global case studyNatural forcings

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

1860 1880 1900 1920 1940 1960 1980 2000

Years

Te

mp

era

ture

an

om

ali

es

[°C

]

Anthropogenic forcings

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

1860 1880 1900 1920 1940 1960 1980 2000

Years

Te

mp

era

ture

an

om

ali

es

[°C

]



Global case studyNatural + anthropogenic forcings

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

1860 1880 1900 1920 1940 1960 1980 2000

Years

Te

mp

era

ture

an

om

ali

es

[°C

]

Natural + anthropogenic forcings + ENSO

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

1860 1880 1900 1920 1940 1960 1980 2000

Years

Te

mp

era

ture

an

om

ali

es

[°C

]



Remarks• Anthropogenic forcings appear as a main

probable cause of the changes in T;• the input related to ENSO acts as a 2nd-

order corrector to the estimation obtained by anthropogenic and natural forcings (nevertheless, in a nonlinear system we cannot separate the single contributions to the final result);

• the amount of variance not explained by our final model is low



Remarks

Is this low amount due to the natural variability of climate system or to some hidden dynamics coming from one or more neglected dynamical causes?

Look at the residuals!Three tests:•Fourier spectrum;•autocorrelation function;•MonteCarlo Singular Spectrum Analysis (MCSSA).



The residualsNo particular peak and periodicity;

the spectrum trend is almost flat…

… but, decrease in the amplitude above 3 cycles per 10 years;

we cannot exclude red or pink noise.



The residualsThe auto-correlation function is almost completely confined inside the white noise limits;

some oscillations are visible but more uncoupled than in previous results.



The residuals

The plots show results obtained applying MCSSA: due to some points exceeding the confidence limits provided by an AR(1) process, the presence of components different from red noise is suggested.



The residuals

No undoubted conclusion can be reached by our analysis of the residuals (besides, it is well known how is difficult to distinguish between noise and chaotic dynamical signals in short time series).

Anyway, we can be confident that the major causes of temperature change have been considered and only 2nd-order dynamics has been neglected in our study.



Regional case studyWe want to analyze the fundamental

elements that drive the temperature behaviour at a regional scale, with the same strategy adopted in the previous global case study.

It is well known that the North Atlantic Oscillation (NAO) correlates quite well with temperatures in a period called “extended winter” (December to March).

We want to assess the relative influences of global forcings and NAO on temperature in Central England.



Regional case study NAO - NAO +



Regional case study (CET)

3 case studies and input data:a) global (natural + anthropogenic) forcings;b) NAO only;c) global forcings + NAO.

CET series in extended winters

1

2

3

4

5

6

7

1860 1880 1900 1920 1940 1960 1980 2000

Years

Te

mp

era

ture

[°C

]



Regional case study (CET)

Case Bias [°C] MAE [°C](a) -0.002 0.995(b) 0.117 0.601(c) -0.037 0.651

(a)

-4

-3

-2

-1

0

1

2

3

4

5

1860 1880 1900 1920 1940 1960 1980 2000

Years

Re

sid

ua

ls [

°C]

(b)

-4

-3

-2

-1

0

1

2

3

4

5

1860 1880 1900 1920 1940 1960 1980 2000

Years

Re

sid

ua

ls [

°C]

(c)

-4

-3

-2

-1

0

1

2

3

4

5

1860 1880 1900 1920 1940 1960 1980 2000

Years

Re

sid

ua

ls [

°C]



Regional case study (CET)Global forcings have a very little influence on

the behaviour of temperatures in the Central England during extended winter.

NAO - driving force: when NAO is considered the values of linear correlation coefficients (estimated T vs. observed T) are about 0.72 0.75 in the two cases. These values are lower than in the analogous situations of the previous global case study (about 0.88).

This is probably due to the enhanced inter-annual variability of climate at regional scale.



Discussion

A non-dynamical approach allows us to obtain simple assessments in a complex system.

At a global scale we are able to reconstruct the global temperature behaviour only if we take the anthropogenic forcings into account.

Furthermore, we are able to recognize the influence of ENSO in better catching the inter-annual variability of our global time series of temperature.



Discussion

At a regional scale, the recognition of the major influence of NAO on the CET time series appears very important (a further discussion in the afternoon exercise session).

In general, our results can be used in order to identify the fundamental elements for obtaining both:

• successful dynamical regional models• and reliable statistical downscaling of GCMs

in the past and for future scenarios.



DiscussionWe possess a phenomenological tool for

obtaining preliminary assessments on the past in the climate system.

In particular it is worthwhile:• to consider an extension to inputs related

to other kinds of forcings, circulation patterns and oscillations;

• to apply our method to other regions of the world;

• to extend our treatment to the reconstruction of precipitation regimes.



Impact studies (animals)

How rainfall, snow cover and temperature affect them?



From Pasini et al. (submitted)

Bivariate linear and nonlinear analyses (meteo-climatic forcings vs. rodent density)





Neural reconstruction of rodent density in the Apennines starting from data of meteo-climatic forcings





Neural “backcast” of rodent density in the Apennines starting from data of meteo-climatic forcings




Predictability

• Paper by Lorenz (1963) and the discovery of “deterministic chaos” in meteo-climatic systems;

• predictability problem and change of perspective in the forecasting activity at medium- and long-range;

• ensemble integrations for estimating the predictability horizon in different meteorological situations.



Preliminary considerations

Deterministic

Ensemble

0

5

10

15

20

25

TIME (12 - hours interval)

10m

WIN

D S

PE

ED

(K

TS

)



Preliminary considerationsIs the Lorenz-63 model important only

for historical reasons?In the present situation, we deal with

very complex meteo-climatic models (107 degrees of freedom);

inside these models, their physical behaviour can be obscured and also the ensemble strategy cannot be fully followed up (because of the large amount of computer time needed).



Preliminary considerations

In this framework, the Lorenz-63 model represents a toy model which mimics some features of both the atmosphere and the climate system:

for instance, their chaotic behaviour …… and the presence of preferred

states or “regimes”.Furthermore, the local predictability on

the Lorenz attractor resembles the predictability of single real states.



The Lorenz system

dx/dt = (y-x)dy/dt = rx - y - xz

dz/dt = xy - bz

Our choice of the parameters: = 10, b = 8/3, r = 28

chaotic solutions.



The forced Lorenz system

dx/dt = (y-x) + f0 cos dy/dt = rx - y - xz + f0 sin

dz/dt = xy - bz

Toy simulation of an increase of anthropogenic forcings in the climate system

Our choice of the parameters:f0 = 2.5 5, = /2 still chaotic solutions.



The Lorenz system



Unforced vs. forced



Predictability (dynamics)



Predictability (dynamics)The concept of bred vector:

• Bred vectors are simply the difference v between two model runs after a certain number (n) of time steps, if the second run is originated from slightly perturbed initial conditions v0.

• We define the bred-growth rate as:g = 1/n ln(v/v0).

• g can be used to identify regions of distinct predictability on the attractor.




n = 8

Blue: g < 0

Green: 0 g < 0.04

Yellow: 0.04 g < 0.064

Red: g 0.064




Unforced Forced






Predictability studies by NNs

The idea to forecast future states of the Lorenz system by NN is not new…

… but previous works considered the prediction of the time series for a single variable (usually the x variable) in order to reconstruct the complete dynamics under the conditions of the Takens theorem;

this permits to mimic the reconstruction of an unknown dynamics by observational data in a complex system.



Predictability studies by NNsHere, we consider the full 3D dynamics of the

Lorenz system and try to estimate the predictability on its attractor in several regions (related to bred-growth classes), by considering changes in NN forecasting performance:

• network topology: 3 - 15 - 3;• single-step forecast from t0 to t0+n (n=8);• total set of Lorenz simulated data (20,000

input-target patterns) divided into a training set (80%) and a validation/test set (20%);




• the 3D-Euclidean distance between output and target points as a measure of our forecast performance.

• The NN forecast performance “feels” increased predictability in forced situations.




Distributions of distance errors for each class

Blue class

0

2

4

6

8

10

12

14

16

18

20

1 5 9 13 17 21 25 29 33 37 41 45

NN forecast error (distance)

Fre

qu

ency

(%

)

Yellow class

0

2

4

6

8

10

12

14

16

18

20

1 5 9 13 17 21 25 29 33 37 41 45

NN forecast error (distance)

Fre

qu

ency

(%

)




In short, the average forecast error decreases in the forecasting activity on the forced system.

This can be obviously due to a more frequent permanence of the system’s state in regions of high predictability (blue points).

Can this be due to a change in local predictability of single points in the Euclidean 3D-space, too?

Is this due to both of these factors?



Predictability studies by NNsSome points:• Of course, the Lorenz system is only a toy

model of the atmosphere and the climate system;

• operationally, we would like to obtain an estimate of predictability for future times, when observations are not still available, while here the recognition of distinct predictability regions are obtained by NN just in comparison with the “observed” states in Lorenz models (obtained after dynamical integration).



Thus, we obtain just an a posteriori recognition of the predictability over the Lorenz attractors is it possible to obtain an operational estimation of predictability?

Yes, by forecasting (via NNs) the bred-growth rates directly (1 output).

Main result: NNs are able to forecast g and a statistical significant increase of performance is shown when the external forcing is applied.




Thus, not only the presence of an external forcing permits to better forecast the future states over the attractors (as shown previously), but also the NN estimation of the predictability itself is improved in these forced situations.


Related to the forecast of g



Provisional conclusions• Neural modelling is able to distinguish

regions of distinct predictability over Lorenz attractors.

• Increased predictability has been found in the forced case (confirmed by dynamical quantities) and operational estimation of g has been obtained (here, it is an exercise, but it could become important as emulation of dynamical computations for predictability assessments in real dynamical models, where ensemble runs are very time-consuming).



Prospects

Our NN performance is not very good

improvements can be envisaged by:• obtaining extended data sets by

prolonged Runge-Kutta integrations;• consideration of different input sets (e.g.,

truncated time series of delayed data);• application of other NN architectures and

learning paradigms.



NN downscaling

• Up to now we have considered NNs as a strategy which is alternative to dynamical modelling.

• In doing so we have obtained both results comparable with those coming from GCMs (in the case of influence analysis on the past) and new results (e.g., in the predictability case study on unforced and forced Lorenz systems).



NN downscaling

• As a matter of fact, these strategies are based on two distinct view-points for the analysis of a system: a dynamical decomposition-recomposition approach vs. an analysis of the system as a whole by learning directly on data.

• The challenge of complexity is extremely hard and different view-points (and the associated strategies) are welcome!



NN downscaling

• Probably, these strategies can be seen more appropriately as complementary than as alternative.

• A concrete example of “synergies” between them is represented by the case of GCMs downscaling via NNs.

• In what follows we will discuss this complementary approach and the work in progress about it.



The rationale



The rationale

GCMs are not able to determine climate at regional/local scale.

So, there is a need for downscaling.It can be achieved either dynamically or

statistically, so that we have two cases:• dynamical downscaling (regional climate

models - RCMs);• statistical downscaling (regression models,

weather classification, weather generators).



The rationale

Here we do not discuss about weather classification and weather generators: see Wilby et al. (2004) in the references.

In short, statistical downscaling is based on the view-point that the regional/local climate is conditioned by two factors:

• the large scale climatic state;• the regional/local physiographic features

(e.g., topography, land/sea distribution, land use).



The rationale

So the process for a statistical downscaling is as follows:

• to establish a statistical model which is able to link large-scale climate variables (predictors) with regional/local variables (predictands);

• to feed the large-scale output of a GCM to the statistical model;

• to estimate the corresponding regional/local climate characteristics.



Pros and Cons

Advantages of a statistical downscaling:• the techniques used for building and

applying the statistical model are usually quite inexpensive from the computer-time point of view;

• they can be used to provide site-specific information, which can be critical for many climate change impact studies.



Pros and ConsA major theoretical weakness:• we are not able to verify the basic

assumption that underlies these models;• that is to say, we cannot be sure that the

statistical relationships developed for the present-day climate also hold under the different forcing conditions of possible future climates (“stationarity” assumption);

• however, this is a limitation that affects also physical parameterizations of GCMs.



Be aware...

• Predictors relevant to a regional/local predictand should be adequately reproduced by the GCM to be downloaded (e.g., remind NAO as an important element to determine European climate);

• therefore, predictors have to be chosen on the balance of their relevance to the target predictand and their accurate representation by climate models.



NNs in downscaling

• Among the regression models, NNs appear particular for their characteristic feature of achieving nonlinear relationships between predictors and predictands.

• This feature is obviously important in the nonlinear climate system and it becomes increasingly crucial when dealing with regional/local variables (predictands) which are heterogeneous and discontinuous in space and time, such as daily precipitation.



A simple example

• I would like to present just a simple example of application (by Trigo & Palutikof, 1999).

• Reconstruction on the past and future scenarios for minimum and maximum daily temperatures in Coimbra (Portugal).

• Feed-forward networks with one hidden layer and backpropagation training.

• Training and validation on the past; test on future scenarios.



A simple example

Predictor 500hPa SLP24 h mean (nearest grid point) *24 h north-south gradient * *24 h east-west gradient * *24 h geostrophic vorticity *

6 variables + values of the same variables for the

previous day + sin and cos (Julian day)



A simple exampleValidation



A simple exampleFuture scenarios



Recent advances

• Recently, a particular attention has been devoted to the combination of dynamical and moisture variables as predictors;

• furthermore, some researchers stressed the importance of a cross-validation of the downscaling model from observational data for periods that represent independent or different climate regimes (thus somewhat validating the “stationarity” assumption).



Recent advances

• Recently, NNs used for downscaling were extended to SOM (Kohonen networks);

• inter-comparisons of NNs and other methods for a statistical downscaling show that neural network modelling is one of the best methods to do so (see more in Pasini (2008)).

• At present, in general the scores of methods of statistical downscaling are comparable with those coming from dynamical downscaling (RCMs).



Conclusions and prospects

• Modelling the dynamics of the climate system is a difficult task.

• In this framework, neural network modelling begins to help in grasping this complexity, both as an alternative strategy to dynamical modelling, and as a complementary technique that may be used together with GCMs.

• Climate change studies represent a field in which NNs (and, more generally, AI techniques) can be applied successfully.



Essential references

• Climate modelling: A. Pasini (2005), From Observations to Simulations: a conceptual introduction to weather and climate modelling, World Scientific, www.worldscibooks.com/environsci/5930.html




• Assessment on the past: A. Pasini, M. Lorè, F. Ameli (2006), Ecological Modelling 191, 58-67.

• Predictability: A. Pasini (2007), Predictability in past and future climate conditions: a preliminary analysis by neural networks using unforced and forced Lorenz systems as toy models, in Proceedings of the 87th AMS annual meeting (5th AI Conference), San Antonio, AMS, CD-ROM.




• NN downscaling: R.L. Wilby et al. (2004), Guidelines for use of climate scenarios developed from statistical downscaling methods. IPCC Task Group TGICA, http://ipcc-ddc.cru.uea.ac.uk/guidelines/StatDown_Guide.pdf (and references therein).

• R.M. Trigo & J.P. Palutikof (1999), Climate Research 13, 45-59.



Essential references• All these topics are now

reviewed in A. Pasini (2008), Neural network modeling in climate change studies, in Artificial Intelligence Methods in the Environmental Sciences (S.E. Haupt, A. Pasini and C. Marzban eds.), Springer (in press).



For Italian readers...

[email protected]

Documents

Approaches to climate change study and neural network modelling