Upload
khushivansh
View
212
Download
0
Embed Size (px)
Citation preview
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 1/6
An Introduction to Probability andStochastic Processes for Ocean,
Atmosphere, and Climate Dynamics
1: Basic Probability
Adam Monahan
School of Earth and Ocean Sciences, University of Victoria
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 1/23
Introduction
“Climate is what you expect, weather is what you get.”
- Robert A. Heinlein
⇒ “expectation” lies at heart of notion of climate
⇒ this is a fundamentally probabilistic perspective
Probability is a natural way of looking at atmosphere, ocean, and
climate dynamics, both in terms of
statistics (probability and data)
stochastics (the dynamics of probability)
These lectures will provide an overview of probability theory &
stochastic processes in the context of atmosphere, ocean, and climate
dynamics
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 2/23
Probability: Basic concepts
Probabilities are used to characterise processes with indeterminate
outcomes - that is, that are random
Any given random process X is associated with a set of possible
basic outcomes
Ω = x1, x2,...,xn
(which doesn’t have to be discrete set)
Can define an “event” A as any subset of these basic outcomes, and
assign to that a probability P (A) of its occurrence, with the rules:
(i) 0 ≤ P (A) ≤ 1
(ii) P (Ω) = 1 (something’s got to happen)
(iii) P (A or B) = P (A) + P (B) − P (A and B)
(additivity of mutually exclusive events)An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 3/23
Probability densities
Our main focus will be case of basic outcome space of continuous
variables Ω = x = ω ∈ R
Define “differential” probability of x falling in small volume dx
dP (x0) = P (x0 ≤ x ≤ x0 + dx)
If P (x) smooth, can define probability density function (pdf)
P (x0 ≤ x ≤ x0 + dx) = px(x) dx
for which px(x) dx = 1
Can also define cumulative distribution function
F (x0) = P (x ≤ x0) =
x0
−∞
px(x) dx
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 4/23
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 2/6
Joint and marginal pdfs
Probability of A and B together is joint probability
P (A andB) = P (A,B)
For x ∈ RN , px(x) = px(x1, x2,...,xN ) is the joint pdf of multiple
scalar random variables
Can find pdf over subspace of x (the marginal pdf ) by integrating
out other variables
E.g. in R2:
px1(x1) =
px(x1, x2) dx2
px2(x2) =
px(x1, x2) dx1
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 5/23
Conditional probabilities
The conditional probability P (A|B) measures the probability of A
given that B is known to be true
Conditional probabilities are related to joint probabilities:
P (A,B) = P (A|B)P (B) = P (B|A)P (A)
P (A|B) is a measure of the dependence of A and B
Independence: B irrelevant to A, so P (A|B) = P (A) and
P (A,B
) = P
(A
)P
(B
)
For x, y independent, joint pdf factors as product of marginals:
pxy(x, y) = px(x) py(y) (independence)
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 6/23
Bayes’ Theorem
From properties of conditional distributions
p(x|y) =
p(y|x) p(x)
p(y) =
p(y|x) p(x) p(x, y)dx =
p(y|x) p(x) p(y|x) p(x)dx
⇒
p(x|y) = p(y|x) p(x) p(y|x) p(x)dx
Provides formal model of “knowledge acquisition” of system
variable x through “experimental outcome” y:
p(x) initial uncertain knowledge of x (the prior )
p(y|x) predicted outcome of experiment
p(x|y) improved knowledge of x given outcome of experiment y
(the posterior )
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 7/23
Expectation and moments
The pdf px(x) defines a measure of relative probabilities, leading to
the definition of the expectation (average) of a function f (x):
Ef =
f (x) px(x) dx
Can interpret marginal pdf as “expectation” of conditional:
px1(x1) =
px(x1, x2) dx2 =
p(x1|x2) px2(x2) dx2
Expectation used to define moments, which are useful
characterisations of a pdf
For scalar random variables:
mean(x) = Ex pdf centre
var(x) = std2(x) = E(x − Ex)2 pdf spread
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 8/23
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 3/6
Expectation and moments
Two common higher-order (reduced) moments:
skewness
skew(x) = E(x− Ex)3
std3(x)
measures asymmetry of pdf
0 1 2 3 4 50
0.2
0.4
0.6
0.8
1
1.2
1.4
x
p x
( x )
skew(x) > 0skew(x) = 0
skew(x) < 0
kurtosis
kurt(x) = E(x− Ex)4
std4(x) −3
measures flatness of pdf
−4 −2 0 2 40
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
x
p x
( x )
kurt(x) > 0
kurt(x) = 0kurt(x) < 0
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 9/23
Moments of sea surface winds
mean(U) (m/s)
0 50 100 150 200 250 300 350
−60
−40
−20
0
20
40
60
−10
−5
0
5
10
std(U) (m/s)
0 50 100 150 200 250 300 350
−60
−40
−20
0
20
40
60
0
2
4
6
8
skew(U)
0 50 100 150 200 250 300 350
−60
−40
−20
0
20
40
60
−2
−1
0
1
2
Zonal Wind
mean(w) (ms−1
)
0 50 100 150 200 250 300 350
−50
0
50
2
4
6
8
10
12
std(w) (ms−1
)
0 50 100 150 200 250 300 350
−50
0
50
1
2
3
4
5
skew(w)
0 50 100 150 200 250 300 350
−50
0
50
−1
−0.5
0
0.5
1
Wind speed
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 10/23
Covariance and correlation
Given two variables x and y with joint pdf pxy(x,y) define
covariance
cov(x, y) = E(x − Ex)(y − Ey)
and correlation
corr(x, y) = cov(x,y)
std(x)std(y)
These are measures of dependence between x and y
If x and y are independent, covariance and correlation vanish - but
not vice versa! (in general)
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 11/23
Covariance and correlation
x 2
(a)
−4 −2 0 2 4−4
−2
0
2
4
(b)
−4 −2 0 2 4−4
−2
0
2
4
x1
x 2
(c)
−4 −2 0 2 4−4
−2
0
2
4
x1
(d)
−4 −2 0 2 4−4
−2
0
2
4
(a) indep, uncorr
(b) dep, corr
(c) dep, uncorr
(d) dep, corr
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 12/23
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 4/6
Covariance matrix & EOFs
For x ∈ RN define covariance matrix
Σxx = E
(x − Ex)(x − Ex)T
so var(x) = diag(Σxx)
Efficiently described in terms of eigenvectors ek known as empirical
orthogonal functions (EOFs):
Σxxek = λkek
EOFs ⇒ orthogonal basis in RN which efficiently partitions variance
In EOF basis new (uncorrelated) variables given by projections
ak = x · ek with diagonal covariance matrix
(Σaa)ij = λ jδ ij
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 13/23
Families of pdfs
There are many families of pdfs characterised by one or more
parameters whose properties are well-studied
The most important of these are the two-parameter Gaussian (or
normal) distributions
For a Gaussian scalar x
px(x) = 1√
2πσ2exp
−
(x− µ)2
2σ2
such that
mean(x) = µ std(x) = σ
skew(x) = 0 kurt(x) = 0
−4 −2 0 2 40
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
(x−µ)/ σ
p x
( x )
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 14/23
Multivariate Gaussian
Random vector X ∈ RN is multivariate Gaussian when pdf is
p(x
) =
1 (2π)N detΣxx exp
−
1
2 (x− µ)
T
Σ−1
xx (x− µ)
where µ = mean(x)
Σxx = cov(x,x)
For a multivariate Gaussian
corr(xi, x j) = 0
⇒ x1, x2 independent
all marginals are Gaussian
x1
x 2
−5 0 5−4
−3
−2
−1
0
1
2
3
4p(x
1,x2)
EOF1
EOF2
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 15/23
Central limit theorem and law of large numbers
One reason that Gaussian distributions are important is the central
limit theorem, which states (loosely) that if xn is a sequence of
independent mean-zero random variables with finite variance, then
Z n = 1√ n
n
i=1
xi
becomes Gaussian with finite variance as n →∞
Another useful asymptotic result is the law of large numbers: for xn
a sequence of independent and identically distributed random
variableslimn→∞
1
n
n
i=1
xn → mean(x)
That is: as n becomes very large, fluctuations in sum vanish
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 16/23
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 5/6
Maximum entropy pdfs
Get moments from pdf; but how can we go the other way?
The entropy
H = −
px(x) ln px(x)dx
measures “information content” of a pdf
Given first N moments, can find pdf which maximises H subject to
the constraint it must have specified moments; takes form
px(x) = 1
Z
exp Lixi
where Li are Lagrange multipliers of constrained optimisation
Provides “least biased” pdf with given information
Only mean and variance given, maximum entropy pdf Gaussian
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 17/23
Change of variables
Suppose y = f (x) where f is invertible
Variable y
is just another way of talking about x
; intuitively expect
P (y1 < y < y2) = P (x1 < x < x2)
if y1 = f (x1) and y2 = f (x2) ⇒ “conservation of probability”
In terms of densities:
px(x) dx = py(y) dy = py(y) df
dx dx
so
py(y) = px(f −1(y))
df/dx
For x, y ∈ RN
py(y) = px(f −1(y))
det(∂f i/∂x j)An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 18/23
Change of variables: example
Know the joint pdf of zonal and meridional wind components:
puv(u, v), but want marginal pdf of wind speed pw(w)
1. move from (u, v) to polar coordinates (w, θ):
(u, v) = (w cos θ, w sin θ)
2. ∂ wu ∂ θu
∂ wv ∂ θv
=
cos θ −w sin θ
sin θ w cos θ
= w
⇒ pwθ(w, θ) = wpuv(w cos θ, w sin θ)
3. Integrate over θ to get marginal:
pw(w) = w
π
−π
puv(w cos θ, w sin θ) dθ
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 19/23
Case study: Tropical Pacific SST
EOF1
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16o
N
0 0.2 0.4 0.6 0.8
EOF2
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16o
N
−0.4 −0.2 0 0.2 0.4
Antisymmetric Component a(1)
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16o
N
−0.2 0 0.2 0.4 0.6 0.8
Symmetric Component a(2)
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16o
N
−0.15 −0.1 −0.05 0 0.05 0.1
Standard Deviation
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16
o
N
0.2 0.4 0.6 0.8 1
Skewness
150oE 180
oW 150
oW 120
oW 90
oW
16oS
8oS
0o
8oN
16
o
N
−0.5 0 0.5 1
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 20/23
8/20/2019 monahan_1
http://slidepdf.com/reader/full/monahan1 6/6
Case study: Tropical Pacific SST
−3 −2 −1 0 1 2 3 4−3
−2
−1
0
1
2
3
PC1
P C
2
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 21/23
Case Study: 20 hPa geopotential height
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 22/23
References
Gardiner, C.W. Handbook of Stochastic Methods, Springer
Monahan, A.H. & A. Dai, 2004. “The spatial and temporal structure
of ENSO nonlinearity”. J. Clim, 17, 3026-3036.
Monahan, A.H., J.C. Fyfe, & L. Pandolfo, 2003. “The vertical
structure of wintertime climate regimes of the Northern Hemisphere
extratropical atmosphere”. J. Clim, 16, 2005-2021.
von Storch, H. & F.W. Zwiers, Statistical Analysis in Climate
Research, Cambridge University Press
Wilks, D.S. Statistical Methods in the Atmospheric Sciences,Academic Press
An Introduction to Probability and Stochastic Processes for Ocean, Atmosphere, and Climate Dynamics1: Basic Probability – p. 23/23