Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
COMM 1004: Detection & Estimation
Prof. Ahmed El-MahdyDean of the faculty of IET
The German University in Cairo
Text Books
• H.L. Van Trees, Detection, Estimation, and Linear Modulation Theory, vol. I. John Wiley& sons, New York, 2001.
• Don. H. Johnson, Statistical Signal Processing: Detection Theory, Houston, TX, 2013.
• S. Kay, Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice Hall, 1993.
• S. Kay, Fundamentals of Statistical Signal Processing: Detection Theory, Prentice Hall, 1993.
Grading
• Quizzes (2Quizzes) 15%
(No Compensation Quizzes)
• Assignments 15%
• Project 30%
• Final Exam 40%
Course Contents1-Estimation Theory:
2-Detection: Simple binary hypothesis testing, likelihood ratio, Bayes criterion, Neyman-Pearson Criterion, Min-Max Performance
Parameter Estimation
random
Applications: Communication channel estimation, Range Estimation,
Sinusoidal Parameter Estimation, communication receivers, Noise Canceller
COMM 1004: Detection & Estimation
Lecture 1
- Introduction- Estimation Theory
Introduction to Detection & Estimation
Goal: Extract useful information from noisy signals
Detection: Decision between two (or a small number of) possible hypothesis to choose the best of the two hypothesis.
Parameter Estimation: Given a set of observations and given an assumed probabilistic model, we get the best estimate of the parameters of the model.
What is the detection and estimation??
Detection: example 1: digital Communications
Detection example 3: In a speaker classification problem we know the speaker is German, British, or American. There are three possible hypotheses Ho, H1, H2.
Decision: After observing the outcome in the observation space, we guess which hypothesis is true.
Examples for Estimation
Estimation of the phase of the signal:
Estimation of a DC level of a signal:
Useful in coherent modulation:
• Estimation of fading Channel:
• Parameter estimation of a signal:
Estimate h[m]???
Difference between Detection & Estimation?
Detection:
Estimation:
Try to extract a parameter from them
Estimation theory
Definitions
Parameter Estimation
random
Performance of Estimators1- Unbiased Estimators:
- For an estimator to be unbiased we mean that on the averagethe estimator will yield the true value of the unknown parameter.
- Since the parameter value may in general be anywhere in the interval , unbiasedness asserts that no matter what the true value of θ, our estimator will yield it on the average.
𝐸[ 𝜃]=𝜃
Otherwise, the estimate is said to be biased: 𝐸[ 𝜃]≠ 𝜃
a b
The bias 𝑏[𝜃] is usually considered to be additive, so that:
𝐸[ 𝜃]=𝜃 + 𝑏[𝜃].
When we have a biased estimate, the bias usually depends on the number of observations N. An estimate is said to be asymptotically unbiased if the bias tends to zero for large N: lim
𝑁→∞𝑏=0
Variance of Estimator: The variance of an estimator 𝜃 is defined as:
𝑣𝑎𝑟( 𝜃)=𝐸[( 𝜃 − 𝐸[ 𝜃])2]
Expectations are taken over x (meaning 𝜃 is random but not 𝜃).
An estimate’s variance equals the mean-squared estimation error only if the estimate is unbiased.
Performance of Estimators
Example:
Unbiased Estimators
• An estimator is unbiased does not necessarily mean that it is a good estimator. We need to Check some other performance measure.
• It only guarantees that on the average it will attain the true value.
• A continuous bias will always result in a poor estimator.
21
2-Efficiency:An unbiased estimator is said to be efficient if it has lower variance than all other estimators. Example: If we compare two unbiased estimators .
Cramer-Rao bound is a lower bound of the variance of any unbiased estimators. Then:
An estimator is said to be efficient if:
-It is unbiased-It satisfies Cramer-Rao bound.
If an efficient estimate exists, it is optimum in the mean-squared sense: No other estimate has a smaller mean-squared error.Efficiency states that the estimator is “best”
21ˆandˆ
)ˆ()ˆ(ˆthanefficientmoreisˆ2121 VarVarif
3- Consistency:
• An unbiased estimator is consistent if its variance decreases as sample size increases.
• In consistent unbiased estimator, the distribution of the estimator converges to the true value as the sample size increases.
0)ˆ(lim 1
Varn
• Consistency is a relatively weak property in contrast to optimal properties such as efficiency. Unbiased and
Consistent Estimator
Thus, a consistent estimate must be at least asymptotically unbiased.
Appendix A :Revision of Matrices
Revision of Matrices
)/1(0
0)/1()()(
:then,0
0matrixdiagonoalFor)8(
and
matrixunitarycalledisthen,if)7(
)6(
)5(
)()4(
constantfor(3)
matrixsymmetricisthen,if(2)
)1(
:C and B, A, matrices For the
2
11
2
1
1
1
b
bBbBBa
b
bB
AA
AAA
ABCCBA
ABBA
BABA
AA
AAA
AA
T
T
T
TTTT
TTT
TTT
TT
T
TT
Determinant of matrices
Inverse of matrices
There exist an inverse of the matrix A when det (A) does not equal to zero.
For the matrix A:
Eigen values and Eigen vectors of a matrix :
. of each valuefor 0)-( :solve rs,eigenvecto thedetermine To-
. of for values 0)-det(
:equation sticcharacteri thesolve s,eigenvalue thedetermine To
.eigenvaluethecalledis
.= such that , any vector isr eigenvectoan ,matrix square aGiven
vIA
IA
vAvvA
Example to find the Eigen values and vectors of a matrix :
)(7/3
7/1
3
13Repeat
5/1
5/2
512:vectoroflengthbydivideunitbetovectortheFor
.1
2then,2Assume.
5.02
1063
02
0
0
63
21:getwe,04Solving
63
21
40
04
23
234:4For
:vectorseigenthefindTo
3,4:arevalueseigentheThen
0340120-2-3
2-3
0)-det(:isequation sticcharacteriThe
23
23:matrixtheofvectorseigeningcorrespondtheandvalueseigentheFind
222
1
22
1
111
11
11
111121211
1211
12
11
1
11
21
2
vectorunitfor
vv
vvvvv
vv
v
v
VV
VV
VV
VIA
IAIA
IA
A
Appendix B :Revision of Random Variables
Revision of Random Variables
Mean of a Random Variable
Covariance of a Random Variable
Independence and Uncorrelation
)()......()(
)(),.....,,(:variablesrandomtindependenFor
21
1
21
N
N
i
iN
xpxpxp
xpxxxpN
Remember: Two Statistically Independent Random Variables
)()()( YEXEXYE
)()()( YVarXVarYXVar
If X and Y are statistically independent, then
LMMSE