View
220
Download
0
Category
Preview:
Citation preview
8/10/2019 Topic 5_Intro to Random Processes
1/64
Topic 5: Stochastic/Random Processes Introduction(Read Chapter 9 and some sections in Chapter 10] in Papoulis) -2 weeks
A general random, or stochastic, process can be described as: Collection of time functions (signals) corresponding to various outcomes
of random experiments (events).
Collection of random variables observed at different times.
Rather than consider fixed random variablesX, Y, etc. or even
sequences of i.i.d random variables, we consider sequencesX0
,X1
,
X2, . WhereXtrepresent some random quantity at time t.
In general, the valueXtmight depend on the quantityXt-1at time t-1, oreven the valueXsfor other timess < t.
Examples of random processes in communications:
Channel noise,
Information generated by a source,
Interference.
Examples in other fields: Daily stream flow, hourly rainfall of storm
events , stock index,
Note: we will use the terms stochastic and random process interchangeably 1
8/10/2019 Topic 5_Intro to Random Processes
2/64
A RANDOM VARIABLEX, is a rule for assigning to everyoutcome, !, of an experiment a numberX(!).
Note:Xdenotes a random variable andX(!)denotes a particular value.
A RANDOM PROCESSX(t) is a rule for assigning to every !,afunctionX(t,!).
Note: for notational simplicity we often omit the dependence on !.
Random Processes
An example of a stochastic processX(t) is shown below
time
a sample path
a random variable for each fixed t
t
X(t)
2
8/10/2019 Topic 5_Intro to Random Processes
3/64
Random
Event
0
Random Process
t
t
1+n(t)
0+n(t)
1
Random
Process
0=!
1=!
( )tX ,!
Ensemble average = 0.5
Time average = 1
Time average = 0
A random processes can be either discrete-time or continuous-time.
A random processes can be either discrete valued or continuous valued.
3
8/10/2019 Topic 5_Intro to Random Processes
4/64
Classification of Stochastic Processes
Four classes of stochastic processes:
Discrete-state process!chain
Discrete-time process!stochastic sequence {Xn| n!T} (e.g.,probing a system every 10 ms.)
Continuous time and discrete state!quantized signal
Continuous time and continuous state!white noise
8/10/2019 Topic 5_Intro to Random Processes
5/64
Types of Random Processes
5
8/10/2019 Topic 5_Intro to Random Processes
6/64
Random Process for a Continuous Sample Space
6
8/10/2019 Topic 5_Intro to Random Processes
7/64
Stochastic/Random Processes Are Important
Most of the real life signals (observations) are contaminated by random
noise.
Noise contamination may lead to unpredictablechanges in the parameters
(e.g., amplitude and phase) of the signal.
Many information-bearing signals are random (the data bits are random).
Signals (processes)
Deterministicsignal.
One possible value for any time
instance. Therefore, we canpredict the exact value of the
signal for a desired time instance.
Random (stochastic):
Many (infinitely) possible values for
any time instance. Therefore, we canpredict only the expected value of
the signal for a desired time
instance.
Stock market, speech, medical
data, communication signals,
8/10/2019 Topic 5_Intro to Random Processes
8/64
Chapter 4aStochastic Processes(a bit more formally)
A deterministic model predicts a single outcome from a giveset of circumstances;
A stochastic model predicts a set of possible outcomes
weighted by their likelihood or probabilities.
A stochastic process is a special stochastic model that dealwith a class of random variables.
A stochastic process{X(t), t"T} is a collection of random
variablesX(t) indexed by t. That is, for each t"T, X(t) is a
random variable, with toften interpreted as time and the values
ofX(t) are referred to as statesof the process (especially if thevalues ofX(t) are discrete).
e.g., X(t) = the received signal voltage
X(t) = number of customers in a supermarket at time t;
X(t) = the quantity of a commodity in inventory at time t.8
8/10/2019 Topic 5_Intro to Random Processes
9/64
Chapter 4a
Stochastic Processes-2
The set Tis called the index set of the process. When Tis
countable, the stochastic process is said to be a discrete-timeprocess; if Tis an interval of the real line, it is said to be a
continuous-time process.
The set of possible values that the random variableX(t) can assume
is called the state space of the stochastic process.
If the state space is countable,X(t) is a discrete-state process.
Otherwise it is a continuous-state process.
Thus, a stochastic process is a family of random variables that
describes the evolution through time of some (physical) process.
Stochastic processes are classified by the type of state space, by the
index set T, and the dependence relations among the random variables
X(t) and their distributions --- giving the names such as Gaussian,
Markov,and Poisson Process.9
8/10/2019 Topic 5_Intro to Random Processes
10/64
Random Processes Recall that a random variable,X, is
a rule for assigning to every
outcome, !, of an experiment anumberX(!).
Note:Xdenotes a randomvariable andX(!)denotes a
particular value.
A random processX(t)is a rule for
assigning to every !,a functionX(!,t).
Note: for notational simplicitywe often omit the dependenceon !.
Analytical descriptionX(t) =X(t,!)
where !is an outcome of a randomevent.
Statistical description: for any
integer n and any choice of (t1,
t2, . . ., tn) the joint PDF of (X(t1), X
(t2
), . . ., X(tn
) ) is known. 10
8/10/2019 Topic 5_Intro to Random Processes
11/64
Example: Analytical Description ---Random Phase Sine Wave
Let X(t) =A cos(2"f0t +")where "is a random variable uniformlydistributed on [0,2#).Ais a fixed amplitude.
Complete statistical description ofX(to)is:
Let Y= 2"f0t +" [which is a uniform RV]
Then, we need to transform fromy tox:
pX(x)dx = pY(y1)dy + pY(y2)dy
We need bothy1andy2because for a givenxthe equationx=A
cos yhas two solutions in [0,2#).
Since
SincepYis uniform in [2#f0t, 2#f0t + 2#], we get
2 2sin
dxA y A x
dy= = !
( ) 2 21
0 elsewhere
X
A x Ap x A x!
"# <
8/10/2019 Topic 5_Intro to Random Processes
12/64
Example: Statistical Description---Gaussian Random Process
Suppose a random processX(t)has the property that forany nand (t0,t1, . . .,tn) the joint density function of {x(ti)}is a jointly distributed Gaussian vector with zero meanand covariance
This gives complete statistical description of the randomprocessX(t).
The Gaussian Random Process is unique in that the firstand second-order statistics completely describe theprocess.
( )2 min ,ij i jt t! !=
12
(5-2)
8/10/2019 Topic 5_Intro to Random Processes
13/64
Random Processes Basic Concepts
The univariate probability density function describes the general distribution of
the magnitude of the random process, but it gives no information on the time or
frequency content of the process.
We will use the multi-variate PDF/PMF, as well as other parameters such ascorrelation function and power spectral density to describe the Random Process.
fX(x)
time, t
x(t)
13
8/10/2019 Topic 5_Intro to Random Processes
14/64
Probability Distribution of a Random Process
For any stochastic process with index setI, its probability
distribution function is uniquely determined by its finite dimensional
distributions.
The kdimensional distribution function of a process is defined by
for any and any real numbersx1, ,xk.
The distribution function tells us everything we need to know about
the process {Xt}.
However, in many cases the joint distribution function is not
available and so other metrics are used to describe a random
process.
( ) kttkXX
xXxXPxxFkktt
!!= ,...,,...,11,..., 11
Ittk!,...,1
14
(5-3)
8/10/2019 Topic 5_Intro to Random Processes
15/64
Moments of Stochastic Process
We can describe a stochastic process via its moments, i.e.,
We often use the first two moments.
The mean function of the process is
The variance function of the process is
The covariance function betweenXt,Xsis
The correlation function betweenXt,Xsis
These moments are often functions of time, but not always
(!stationarity)
( ) ( ) etc.,, 2sttt
XXEXEXE !
( ) .tt
XE =
( ) .2ttXVar !=
( ) ( )( )( )ssttst
XXEXX !!=,Cov
( ) ( )
22
,Cov,
st
st
st
XXXX
!!
" =
15
(5-4c)
(5-4d)
(5-4b)
(5-4a)
8/10/2019 Topic 5_Intro to Random Processes
16/64
Example: Consider the complex RV
Then the second moment ofZis given by
Z = X(t)dt!T
T
" .
E[| Z |2]=E[ZZ*]= E{X(t1)X
*(t2 )}dt1 dt2
!T
T
"!T
T
"
= RXX(t1, t2 )dt1 dt2
!T
T
"!T
T
"
16
(5-5a)
(5-5b)
8/10/2019 Topic 5_Intro to Random Processes
17/64
Stationary Random Processes
Means of collecting statistics:
Sample records which are individual representations of the underlyingprocess.
Ensemble averaging: properties of the process are obtained by
averaging over a collection or ensembleof sample records using
values at corresponding times
Time averaging: properties are obtained by averaging over a singlerecord in time.
Stationary Processes ----ensemble averages do not vary with time
Ergodic process: stationary process in which averages from a single
record are the same as those obtained from averaging over the
ensemble.
Strictly stationarity: the joint PDF is time invariant
(Second-order)Wide-sense stationary (WSS): the first and second
moments are time invariant.
17
8/10/2019 Topic 5_Intro to Random Processes
18/64
Ergodic Random Process
(ergodic stationary)
Definition: A random process is ergodicif all time averages of any samplefunction are equal to the corresponding ensemble averages
Example, for ergodic processes, can use ensemble statistics to compute:
DC values
RMS values
Ergodic processes are always stationary; Stationary processes are not necessarilyergodic.
Example: X(t) =A sin(2"f0t +")
Aand !0are constants; "0is a uniformly distributed RV from [-#,#);tis time.
Mean (Ensemble statistics)
( ) ( ) ( )01
sin 02
xm x x f d A t d !
"!
" " " # " " !
$
%$ %= = = + =& &
( )2
2 2 2
0
1sin
2 2x
AA t d
!
!
" # $ $ !%
= + =&
18
(5-6a)
(5-6b)
8/10/2019 Topic 5_Intro to Random Processes
19/64
Example: Ergodic Process
Mean (Time Average) T is large
Variance
The ensemble and time averages are thesame, at least for the the momentsconsidered, so the process is ergodic
( ) ( )00
1sin 0lim
T
T
x t A t dtT
! "
#$
= + =%
( ) ( )2
2 2 2
00
1sin
2lim
T
T
Ax t A t dt
T! "
#$
= + =%
19
(5-7a)
(5-7b)
8/10/2019 Topic 5_Intro to Random Processes
20/64
Strictly Stationary Processes
A process is said to be strictly stationary if has the samejoint distribution as . That is, if
If {Xt} is a strictly stationary process and then, the mean
function is a constant and the variance function is also a constant.
Moreover, for a strictly stationary process with first two momentsfinite, the covariance function, and the correlation function dependonly on the time differences.
This means that the process behaves similarly (follows the same
PDF) regardless of when you measure it. Is the random process from the coin tossing experiment stationary?
ktt XX ,...,1
!+!+ ktt XX ,...,
1
( ) ( )kXXkXX
xxFxxFkttktt
,...,,...,1,...,1,...,
11 !+!+=
!
Recommended