Embed Size (px)
Citation preview
Stochastic processes
Lecture 5
Some random processes
Random process
1st order Distribution & density function
First-order distribution
First-order density function
2end order Distribution & density function
2end order distribution
2end order density function
EXPECTATIONS
• Expected value
• The autocorrelation
Some random processes
• Single pulse
• Multiple pulses
• Periodic Random Processes
• The Gaussian Process
• The Poisson Process
• Bernoulli and Binomial Processes
• The Random Walk Wiener Processes
• The Markov Process
Single pulse
Single pulse with random amplitude and arrival time: Deterministic pulse: S(t): Deterministic function. Random variables: A: gain a random variable Θ: arrival time. A and Θ are statistically independent
X (t) = A S(t −Θ)
0 0.5 1 1.5 2 2.5 3-2
0
2
t (ms)
Am
plit
ude
Nerve spike
0 0.5 1 1.5 2 2.5 3-2
0
2
t (ms)A
mplit
ude
0 0.5 1 1.5 2 2.5 3-2
0
2
Am
plit
ude
t (ms)
Single pulse expected value
X (t) = A S(t −Θ)
Expected value
E[X(t)]= E[A S(t −Θ)]
Since A and Θ are statistically independent, we have
𝐸 𝑋(𝑡) = 𝐸 𝐴 𝐸 S(t −Θ) = 𝐸 𝐴 S(t −θ)∞
−∞𝑓Θ(θ)dθ
E[X (t)] = E[A] s(t) ∗ fΘ (θ)
Convolution sum
𝑥 (𝑡) = 𝐸 𝑋 𝑡 = x
∞
−∞
𝑓𝑥 x,t 𝑑𝑥
Single pulse autocorrelation
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝑋 𝑡1 𝑋 𝑡2
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴 𝑆(𝑡1 − 𝛩)𝐴 𝑆(𝑡2− 𝛩)
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴2 𝐸 𝑆(𝑡1− 𝛩)𝑆(𝑡2− 𝛩)
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴2 S(t1 −θ)S(t2 −θ)𝑓Θ(θ)dθ
∞
−∞
Single pulse uniform arrival time
• The likelihood of a pulse is uniform for all times (from t=0 to t=T)
• Expected value
• Autocorrelation
𝑓Θ =1
𝑇
𝐸 𝑋(𝑡) = 𝐸 𝐴 S(t −θ)∞
−∞𝑓Θ(θ)dθ=
𝐸 𝐴
𝑇 S(t −θ)∞
−∞dθ
𝑅𝑥𝑥 𝑡1, 𝑡2 =𝐸 𝐴2
𝑇 S(t1 −θ)S(t2 −θ) dθ
∞
−∞
Multiple pulses
Single pulse with random amplitude and arrival time: Deterministic pulse: S(t): Deterministic function. Random variables: Ak: gain a random variable Θk: arrival time. n: number of pulses Ak and Θk are statistically independent
x 𝑡 = 𝐴𝑘𝑆(𝑡 − 𝛩𝑘)𝑛𝑘=1
0 0.5 1 1.5 2 2.5 3-2
0
2
Multiple Nerve spikes
0 0.5 1 1.5 2 2.5 3-2
0
2
0 0.5 1 1.5 2 2.5 3-2
0
2
Multiple pulses expected value
Expected value
𝐸 𝑋(𝑡) = 𝑛 𝐸 𝐴 S(t −θ)∞
−∞𝑓Θ(θ)dθ
E[X (t)] = n E[A] s(t) ∗ fΘ (θ)
n times 𝐸 𝐴𝑘 𝐸 S(t −Θ𝑘) like n times a single pulse
𝑥 (𝑡) = 𝐸 𝑋 𝑡 = x
∞
−∞
𝑓𝑥 x,t 𝑑𝑥
x 𝑡 = 𝐴𝑘𝑆(𝑡 − 𝛩𝑘)𝑛𝑘=1
𝐸 𝑋(𝑡) =E 𝐴𝑘𝑆(𝑡 − 𝛩𝑘)𝑛𝑘=1 = 𝐸 𝐴𝑘 𝐸 S(t −Θ𝑘)
𝑛𝑘=1
𝐸 𝑋(𝑡) =n 𝐸 𝐴𝑘 𝐸 S(t −Θ𝑘)
Multiple pulses autocorrelation
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝑋 𝑡1 𝑋 𝑡2
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴𝑘𝑆(𝑡1− 𝛩𝑘)
𝑛
𝑘=1
𝐴𝑗𝑆(𝑡2− 𝛩𝑗)
𝑛
𝑗=1
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴𝑘𝑆(𝑡1− 𝛩𝑘)𝐴𝑗𝑆(𝑡2− 𝛩𝑗)
𝑛
𝑗=1
𝑛
𝑘=1
𝑅𝑥𝑥 𝑡1, 𝑡2 = 𝐸 𝐴𝑘𝐴𝑗
𝑛
𝑗=1
𝑛
𝑘=1
𝐸 𝑆(𝑡1− 𝛩𝑘)𝑆(𝑡2− 𝛩𝑗)
𝑅𝑥𝑥 𝑡1, 𝑡2
j=k
j≠k
Periodic Random Processes
• A process which is periodic with T
x 𝑡 = 𝑥 𝑡 + 𝑛𝑇 n is an integrer
x 𝑡 = 𝑠𝑖𝑛2𝜋𝑡
50+ Θ + 𝑠𝑖𝑛
2𝜋𝑡
100+ Θ
0 100 200 300 400 500 600 700 800 900 1000-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
t
X(t
)
Signal
T=100
Periodity and the autocorrelation
• Theorem. If the random process is stationary in the wide-sense and period with T
• Then
• This is also true the other way around
x 𝑡 = 𝑥(𝑡 + 𝑛𝑇)
Rxx(τ) = Rxx(τ+nT)
Example
-500 -400 -300 -200 -100 0 100 200 300 400 500-1
-0.5
0
0.5
1
1.5
lag
Rxx(lag)
Autocorrelation
T=100
0 100 200 300 400 500 600 700 800 900 1000-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
t
X(t
)
Signal
T=100
Proof
• We agree that
• And that
• Therefore
𝑥 𝑡 + 𝜏 + 𝑛𝑇 = 𝑥(𝑡 + 𝜏)
𝑅𝑥𝑥 𝜏 = 𝐸[𝑥 𝑡 + 𝜏 𝑥(𝑡)]
𝑅𝑥𝑥 𝜏 = 𝐸[𝑥 𝑡 + 𝜏 𝑥(𝑡)]=𝐸[𝑥 𝑡 + 𝜏 + 𝑛𝑇 𝑥(𝑡)] = 𝑅𝑥𝑥 𝜏 + 𝑛𝑇
Expected value
• The mean of one period
Real life example
• Aim: Identify heart rate from a heart sound recording
• Assumption: the heart beats are periodic
0 500 1000 1500 2000 2500 3000 3500-0.02
-0.01
0
0.01
0.02Signal
Am
plitu
de
0 1000 2000 3000
0
0.5
1
1.5Envelope
Envelo
pe
Segmentation result
Time (ms)
Sta
te
0 500 1000 1500 2000 2500 3000 3500
siSyssiDia S2S1
S1 S1S2 S2 S2S1
Calculate the signal envelope
• env(t)=filter(abs(x(t)))
0 500 1000 1500 2000 2500 3000 3500-0.02
-0.01
0
0.01
0.02Signal
Am
plitu
de
0 1000 2000 3000
0
0.5
1
1.5Envelope
Envelo
pe
Segmentation result
Time (ms)
Sta
te
0 500 1000 1500 2000 2500 3000 3500
siSyssiDia S2S1
S1 S1S2 S2 S2S1
Autocorrelation of the envelope
The Gaussian Process
• X(t1),X(t2),X(t3),….X(tn) are jointly Gaussian fro all t and n values
• Example: randn() in Matlab
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-4
-3
-2
-1
0
1
2
3
4
5Gaussian process
-4 -3 -2 -1 0 1 2 3 4 50
100
200
300
400
500
600
700Histogram of Gaussian process
The Gaussian Process and a linear time invariant systems
• Out put = convolution between input and impulse response
Gaussian input Gaussian output
Example
• x(t):
• h(t): Low pass filter
• y(t):
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-4
-3
-2
-1
0
1
2
3
4
5Gaussian process
-4 -3 -2 -1 0 1 2 3 4 50
100
200
300
400
500
600
700Histogram of Gaussian process
-1.5 -1 -0.5 0 0.5 1 1.50
100
200
300
400
500
600Histogram of y(t)
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000-1.5
-1
-0.5
0
0.5
1
1.5
The Poisson Process
• Typically used for modeling of cumulative number of events over time.
– Example: counting the number of phone call from a phone
𝑃 𝑋 𝑡 = 𝑘 =𝜆 𝑡 𝑘
𝑘!𝑒−𝜆(𝑡)
Definition of the Poisson Process
1. X(t) is a nondecreasing step function, with unit jumps at each time tk, and k is a finite and countable number.
2. For any time span from t1 to t2, the number of jumps that occur in the follow a Poisson distribution.
3. The number of events that occur in any interval of time t is independent of the number of events that occur in any other nonoverlapping interval
Condition 1
X(t) is a nondecreasing step function, with unit jumps at each time tk, and k is a finite and countable number.
Condition 2
For any time span from t1 to t2, the number of jumps that occur in the follow a Poisson distribution.
• 𝜆 is the arrival time
𝑃 𝑋 𝑡2 − 𝑋 𝑡1 = 𝑘 =𝜆 𝑡2− 𝑡1
𝑘
𝑘!𝑒−𝜆(𝑡2−𝑡1)
Poisson distribution
Condition 3
The number of events that occur in any interval of time t is independent of the number of events that occur in any other nonoverlapping interval
Thereby:
Alternative definition Poisson points
• The number of events in an interval
N(t1,t2)
𝑃 𝑁 0, 𝑡2 = 𝑘 = 𝑃 𝑋 𝑡 = 𝑘 =𝜆𝑡 𝑘
𝑘!𝑒−𝜆𝑡
𝑃 𝑁 𝑡1, 𝑡2 = 𝑘 = 𝑃 𝑋 𝑡2 − 𝑋 𝑡1 = 𝑘 =𝜆 𝑡2− 𝑡1
𝑘
𝑘!𝑒−𝜆(𝑡2−𝑡1)
First order distribution (Fx)
• The probability that x1 events has occurred at time t1
𝐹𝑋1 𝑥1; 𝑡1 = 𝑃[𝑋(𝑡1) ≤ 𝑥1] = 𝜆 𝑡1
𝑘
𝑘!𝑒−𝜆(𝑡1)
𝑥1
𝑘=0
Expected value
Bernoulli Processes
• A process of zeros and ones
X=[0 0 1 1 0 1 0 0 1 1 1 0]
Each sample must be independent and identically distributed Bernoulli variables.
– The likelihood of 1 is defined by p
– The likelihood of 0 is defined by q=1-p
The density function of Bernoulli Processes
1st order
2st order
δ() is the impuls respons δ(0)=1 and δ(≠0)=0
(X[n1] = 0, X[n2] = 1),
(X[n1] = 0, X[n2] = 0)
( X[n1 ] = 1, X[n2] = 0),
(X[n1] = 1, X[n2] = 1).
Binomial process
Summed Bernoulli Processes
Where X[n] is a Bernoulli Processes
Binomial distribution
• S[n]=k mean that out of n cases 1 occurs k time
Bernoulli Processes: X=[1 0 0 1 0 1]
Binomial process S=[1 1 1 2 2 3]
• S[6]=3;
• There by P(S[n]=k) follows a binomial distribution
• first-order density function
Remember
Random walk
• For every T seconds take a step (size Δ) to the left or right after tossing a fair coin
0 5 10 15 20 25 30 35 40 45 50
-8
-6
-4
-2
0
2
4
6
n
x[n
]
Random walks
Random walk
• After n steps (time nT) x(nT) will be
• Where k is the number of time we walk to positive direction k varies from 0 to n X(nT) varies from –nΔ to nΔ The change of getting k out n positive steps is Bernoulli random variable with p=0.5 and q=0.5 The density function is then
𝑋 𝑛𝑇 = 𝑘∆ − 𝑛 − 𝑘 ∆= (2𝑘 − 𝑛)∆
Random walk autocorrelation (1/2)
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛1 𝑋 𝑛2
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛1 (𝑋 𝑛2 + 𝑋 𝑛1 − 𝑋 𝑛1 )
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛12+ 𝑋 𝑛1 𝑋 𝑛2 − 𝑋 𝑛1
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛12 + 𝐸 𝑋 𝑛1 𝑋 𝑛2 − 𝑋 𝑛1
Independent if n2>n1
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛12 + 𝐸 𝑋 𝑛1 𝐸 𝑋 𝑛2 − 𝑋 𝑛1
Random walk autocorrelation (2/2)
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛12 + 𝐸 𝑋 𝑛1 𝐸 𝑋 𝑛2 − 𝑋 𝑛1
𝐸 𝑋 𝑛1 = 1
2∆ +1
2(−∆
𝑛1
𝑘=1
) = 0
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝐸 𝑋 𝑛12
𝐸 𝑋 𝑛12 =
1
2∆2+1
2(−∆
𝑛1
𝑘=1
2) = 𝑛1∆2
𝑅𝑥𝑥 𝑛1, 𝑛2 = 𝑛1∆2
The Wiener process
• A random walk where
𝑇 → 0 𝑎𝑛𝑑 𝑛 → ∞
The Markov Process
• 1st order Markov process
– The current sample is only depended on the previous sample
Density function
Expected value
Transition probability
• The probability of transition from on time to the next time instant
Continuous / Discrete
Continuous time (t) Discrete time (n)
Continuous X(t) Continuous random process
Continuous random sequence
Discrete X(t) Discrete random process
Discrete random sequence (Markov chain)
State diagram of Markov chain
• Two state system
Transition probabilities
The Markov Process
• The probability of a given sequence
Homogeneous and stationery process
• Homogeneous process
Is independent on n
But might dependent on n
• Stationary
is also independent on n
Predictions next sample
• Pj(n)=P(x(n)=j) State probability
• Pij(n-1,n)=P(x(n)=j | x(n-1)=i) Transition probability
• Inhomogeneous case
𝑃𝑗 𝑛 = 𝑃𝑖 𝑛 − 1 Pij(n−1,n)
𝑎𝑙𝑙 𝑖
• Homogeneous case
𝑃𝑗 𝑛 = 𝑃𝑖 𝑛 − 1 Pij𝑎𝑙𝑙 𝑖
Example
• Weather forecast (two state homogeneous model)
– The probability for rain at day one is Pr(0)=0.3
– The probability for sun at day one is Ps(0)=0.7
– The transition probabilities are
• Rain to sun Prs=0.4
• Sun to Rain Psr=0.2
Example
• What is the chance of rain tomorrow (n=1)
What is the chance of sum tomorrow (n=1)
𝑃𝑗 𝑛 = 𝑃𝑖 𝑛 − 1 Pij𝑎𝑙𝑙 𝑖
𝑃𝑟 1 = Pr 0 𝑃𝑟𝑟 + P𝑠 0 𝑃𝑠𝑟 𝑃𝑟 1 = 0.3*0.6+0.7*0.2=0.32
𝑃𝑠 1 = 0.3*0.4+0.7*0.8=0.68
51
Segmentation of heart sounds without reference signal
• Task – Identification of first and
second heart sounds (S1 and S2)
• Problems – Stethoscope recordings are
contaminated with recording noise
– Recordings from clinical settings are contaminated with background noise
500 1000 1500 2000 2500 3000 3500 4000-0.1
0
0.1
Arb
itrar
y am
plitu
de
Time (ms)
Heart sound recording: subject 1
500 1000 1500 2000 2500 3000 3500 4000
-0.01
0
0.01
0.02
Time (ms)
Arb
itrar
y am
plitu
de
Heart sound recording: subject 10
S1 S2 S1 S2 S1 S2 S1 S2 S1 S2
S1S2S1S2S1S2S1S2S1
Data and preprocessing
• Heart sound recorded with a 3M/Littmann electronic stethoscope from 100 patients referred for coronary arterial angiography
• Band pass filtered
– 4th order Butterworth 25-400 Hz
• Homomorphic envelogram
• Down sampled to 50 samples pr second
52
0 500 1000 1500 2000 2500 3000 3500-0.02
-0.01
0
0.01
0.02Signal
Am
plitu
de
0 1000 2000 3000
0
0.5
1
1.5Envelope
Envelo
pe
Segmentation result
Time (ms)
Sta
te
0 500 1000 1500 2000 2500 3000 3500
siSyssiDia S2S1
S1 S1S2 S2 S2S1
Hidden Markov models • A double stochastic process
– consisting of an underlying hidden Markov process
– which generates an observable stochastic output
• Weakness
– The transition probabilities are independent of time spent in the current state
53
O
Heart cycle states
S2 siSys S1 siDia Q
The observable output,
the heart sounds
The hidden part
as1 s1 asiSys siSys
asiDia siDia
aS2 S2
aS2 siDia asiSys S2 aS1 siSys
asiDia S1
Estimation of the most likely state sequence (Q)
• Find the state sequence which is the most likely to have generated the observed output given the current model
– Maximize P(Q|O,λ) according to Q
54
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
State sequences
Output: Heart sound
Q is the state sequence, O the observations, λ is the model parameters
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
S1
siSys
S2
siDia
Results
55
Models Sensitivity Positive
predictive value
DHMM 99.3% (CI: 98.4-99.7%)
99.1% (98.1-99.5%)
HMM 59.5% (CI: 56-63%)
55% (CI: 51.6-58.4%)
0 500 1000 1500 2000 2500 3000 3500-0.02
-0.01
0
0.01
0.02Signal
Am
plitu
de
0 1000 2000 3000
0
0.5
1
1.5Envelope
Envelo
pe
Segmentation result
Time (ms)
Sta
te
0 500 1000 1500 2000 2500 3000 3500
siSyssiDia S2S1
S1 S1S2 S2 S2S1
Table 1.
Trained at 40 recordings Test at 60 recordings including 744 S1 and S2 sounds.
