EEL 5544 - Noise in LinearSystems
(Should be called ldquoProbability andRandom Processes for Electrical
Engineersrdquo)
Dr John M Shea
Fall 2008
EEL5544
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Pre-requisite Very strong mathematical skillsSolid understanding of systems theory includingconvolution Fourier transforms and impulsefunctions Knowledge of basic linear algebraincluding matrix properties and eigen-decomposition
Computer requirement Some problems will requireMATLAB Students may want to purchase thestudent version of MATLAB as departmentalcomputer resources are limited Not being able toget on a computer is not a valid excuse for late workWeb access with the ability to run Java programs isalso required
EEL5544 S-1
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFriday
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufledu
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Meeting Time 935ndash1025 MondayWednesdayFridayMeeting Room NEB 201Final Exam The third exam will be given during thefinal exam time slot determined by the Universitywhich is December 18th at 1000 AM ndash 1200 noon
E-mail jsheaeceufleduInstant messaging On AIM eel5544
Class Web page httpwirelesseceufledueel5544Personal Web page httpwirelesseceufledujshea
EEL5544 S-2
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Office 439 New Engineering Building
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Office 439 New Engineering BuildingPhone (352)846-3042
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Office 439 New Engineering BuildingPhone (352)846-3042Office hours 1040ndash1130 Mondays and 155ndash350Wednesdays
Textbook Henry Stark and John W WoodsProbability and Random Processes with Applicationsto Signal Processing Prentice Hall 3rd ed 2002(ISBN 0-13-020071-9)
EEL5544 S-3
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processes
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Suggested References
bull If you feel like you are having a hard time with basicprobability I suggestD P Bertsekas and J N Tsitsiklis Introduction toProbability 2nd ed 2008 (ISBN 978-1-886529-23-6)
Sheldon Ross A First Course in ProbabilityPrentice Hall 6th ed 2002 (ISBN 0-13-033851-6)
bull For more depth on filtering of random processesMichael B Pursley Random Processes inLinear Systems Prentice Hall 2002 (ISBN0-13-067391-9)
EEL5544 S-4
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grader Byonghyok Choi (aru0080ufledu)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grader Byonghyok Choi (aru0080ufledu)
Course Topics (as time allows)
Goals and Objectives
EEL5544 S-5
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10)
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Grading Grading for on-campus students will bebased on three exams (30 each) and classparticipation and quizzes (10)Grading for EDGEstudents will be based on three exams (25each) homework (15) and class participation andquizzes (10) The participation score will takeinto account in-class participation e-mail or instantmessaging exchanges discussions outside of classetc I will also give unannounced quizzes that willcount toward the participation grade A grade ofgt 90 is guaranteed an A gt 80 is guaranteeda B etc
EEL5544 S-6
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Because of limited grading support most homeworksets will not be graded for on-campus students
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Because of limited grading support most homeworksets will not be graded for on-campus studentsHomework sets for off-campus students will begraded on a a spot-check basis if I give tenproblems I may only ask the grader to check four orfive of them Homework will be accepted late up totwo times with an automatic 25 reduction in grade
EEL5544 S-7
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
No formal project is required but as mention abovestudents will be required to use MATLAB in solvingsome homework problems
When students request that a submission (testor homework) be regraded I reserve the right toregrade the entire submission rather than just asingle problem
EEL5544 S-8
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Attendance Attendance is not mandatory Howeverstudents are expected to know all material coveredin class even if it is not in the book Furthermore theinstructor reserves the right to give unannouncedldquopoprdquo quizzes with no make-up option Studentswho miss such quizzes will receive zeros for thatgrade If an exam must be missed the studentmust see the instructor and make arrangementsin advance unless an emergency makes thisimpossible Approval for make-up exams is muchmore likely if the student is willing to take the examearly
EEL5544 S-9
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Academic Honesty
All students admitted to the University of Floridahave signed a statement of academic honestycommitting themselves to be honest in all academicwork and understanding that failure to comply with thiscommitment will result in disciplinary action
This statement is a reminder to uphold yourobligation as a student at the University of Florida andto be honest in all work submitted and exams taken inthis class and all others
EEL5544 S-10
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Honor statements on tests must be signed in orderto receive any credit for that test
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
Honor statements on tests must be signed in orderto receive any credit for that test
Collaboration on homework is permitted unlessexplicitly prohibited provided that
1 Collaboration is restricted to students currently inthis course
2 Collaboration must be a shared effort
3 Each student must write up hisher homeworkindependently
EEL5544 S-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
4 On problems involving MATLAB programs eachstudent should write their own program
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
4 On problems involving MATLAB programs eachstudent should write their own program Studentsmay discuss the implementations of the programbut students should not work as a group in writingthe programs
EEL5544 S-12
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
I have a zero-tolerance policy for cheating in thisclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass
I have a zero-tolerance policy for cheating in thisclass If you talk to anyone other than me during anexam I will give you a zero If you plagiarize (copysomeone elsersquos words) or otherwise copy someoneelsersquos work I will give you a failing grade for theclass Furthermore I will be forced to bring academicdishonesty charges against anyone who violates theHonor Code
EEL5544 S-13
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
EEL 5544 Noise in Linear Systems Lecture 1
RANDOM EXPERIMENTS
What is a random experiment
QWhat do we mean by random
Output is unpredictable in some sense
EEL5544 L1-1
DEFN A random experiment
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure
DEFN A random experiment is an experimentin which the outcome varies in anunpredictable fashion when the experimentis repeated under the same conditions
A random experiment is specified by
1 an experimental procedure2 a set of outcomes and measurement restrictions
EEL5544 L1-2
DEFN An outcome
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace
DEFN An outcome of a random experiment isa result that cannot be decomposed intoother results
DEFN The set of all possible outcomes for arandom experiment is called the samplespace and is denoted by Ω
EEL5544 L1-3
EX
Tossing a coin
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
EX
Tossing a coinA coin (heads on one side tails on theother) is tossed one time and the side thatis face up is recorded
bull Q What are the outcomes
Ω =heads tails
EEL5544 L1-4
EX
Rolling a 6-sided die
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
EX
Rolling a 6-sided dieA 6-sided die is rolled and the number onthe top face is observed
bull QWhat are the outcomes
Ω =123456
EXA 6-sided die is rolled and whether the topface is even or odd is observed
bull QWhat are the outcomes
Ω =even odd
EEL5544 L1-5
If the outcome of an experiment is random how canwe apply quantitative techniques to it
If the outcome of an experiment is random how canwe apply quantitative techniques to it
This is the theory of probability
EEL5544 L1-6
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
People have tried to develop probability through avariety of approaches which are discussed in thebook in Section 12
1 Probability as Intuition
2 Probability as the Ratio of Favorable to UnfavorableOutcomes (Classical Theory)
3 Probability as a Measure of Frequency of Occurrence
4 Probability Based on Axiomatic Theory
EEL5544 L1-7
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
Probability as the Ratio of Favorable toUnfavorable Outcomes (Classical Theory)
bull Some experiments are said to be ldquofairrdquo
ndash For a fair coin or a fair die toss the outcomes areequally likely
ndash Equally likely outcomes are common in manysituations
bull Problems involving a finite number of equally likelyoutcomes can be solved through the mathematicsof counting which is called combinatorial analysis
bull Given an experiment with equally likely outcomeswe can determine the probabilities easily
EEL5544 L1-8
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
bull First we need a little more knowledge aboutprobabilities of an experiment with a finite numberof outcomes
ndash An outcome or set of outcomes with probability 1(one) is certain to occur
ndash An outcome or set of outcomes with probability 0(zero) is certain not to occur
ndash If you find a probability for a question on one of onmy tests and your answer is lt 0 or gt 1 then youare certain to lose a lot of points
EEL5544 L1-9
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
bull Using the first two rules we can find the exactprobabilities of individual outcomes for experimentswith a finite number of equally likely outcomes
EX Tossing a fair coin
Let pH = Prob heads pT = Prob tails
Then pH + pT = 1(the probability that something occurs is 1)
Since pH = pT pH = pT = 12
EEL5544 L1-10
EX Rolling a fair 6-sided die
EX Rolling a fair 6-sided die
Let pi = Prob top face is i
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
EX Rolling a fair 6-sided die
Let pi = Prob top face is iThen
6sumi=1
pi = 1
rArr 6p1 = 1
rArr p1 =16
So pi = 16 for i = 1 2 3 4 5 6
EEL5544 L1-11
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
bull We can use the probabilities of the outcomes to findprobabilities that involve multiple outcomes
EX Rolling a fair 6-sided die
What is Prob 1 or 2
Prob 1 or 2 = Prob 1+ Prob 2
=16
+16
=13
EEL5544 L1-12
The basic principle of counting
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1
The basic principle of countingIf two experiments are to be performedwhere experiment 1 has m outcomes andexperiment 2 has n outcomes for each outcomeof experiment 1 then the combined experimenthas mn outcomes
EEL5544 L1-13
We can use equally-likely outcomes on repeatedtrials but we have to be careful
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
We can use equally-likely outcomes on repeatedtrials but we have to be careful
EX Rolling a fair 6-sided dice twice
Suppose we roll the die two times What is the probthat we observe a 1 or 2 on either roll of the die
EEL5544 L1-14
EEL5544 L1-15
What are some problems with defining probability inthis way
What are some problems with defining probability inthis way
1 Requires equally likely outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomes
What are some problems with defining probability inthis way
1 Requires equally likely outcomes ndash thus it onlyapplies to a small set of random phenomena
2 Can only deal with a finite number of outcomesndash thisagain limits its usefulness
EEL5544 L1-16
Probability as a Measure of Frequency ofOccurrence
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
Probability as a Measure of Frequency ofOccurrence
bull Consider a random experiment that has K possibleoutcomes K lt infin
bull Let Nk(n) = the number of times the outcome is k
bull Then we can tabulate Nk(n) for various values of k
and n
bull We can reduce the dependence on n by dividingN(k) by n to find out ldquohow often did k occurrdquo
EEL5544 L1-17
DEFN The relative frequency
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
DEFN The relative frequency of outcome k of arandom experiment is
fk(n) =Nk(n)
n
bull Observation In our previous experiments as n getslarge fk(n) converges to some constant value
EEL5544 L1-18
DEFN An experiment possesses statisticalregularity
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN An experiment possesses statisticalregularity if
limnrarrinfin
fk(n) = pk (a constant)forallk
DEFN For experiments with statistical regularityas defined above pk is called the probabilityof outcome k
EEL5544 L1-19
PROPERTIES OF RELATIVE FREQUENCY
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
PROPERTIES OF RELATIVE FREQUENCY
bull Note that
0 le Nk(n) le nforallk
because Nk(n) is just the of times outcome k
occurs in n trials
Dividing by n yields
0 le Nk(n)n
= fk(n) le 1 forallk = 1 K (1)
EEL5544 L1-20
bull If 1 2 K are all of the possible outcomes
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
bull If 1 2 K are all of the possible outcomes then
Ksumk=1
Nk(n) = n
Again dividing by n yields
Ksumk=1
fk(n) = 1 (2)
EEL5544 L1-21
Consider the event E that an even number occurs
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
Consider the event E that an even number occurs
What can we say about the number of times E isobserved in n trials
NE(n) = N2(n) + N4(n) + N6(n)
What have we assumed in developing this equation
EEL5544 L1-22
Then dividing by n
fE(n) =NE(n)
n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General property
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs
Then dividing by n
fE(n) =NE(n)
n=
N2(n) + N4(n) + N6(n)n
= f2(n) + f4(n) + f6(n)
bull General propertyIf A and B are 2 events thatcannot occur simultaneously and C is the event thateither A or B occurs then
fC(n) = fA(n) + fB(n) (3)
EEL5544 L1-23
bull In some sense we can define the probability of anevent as its long-term relative frequency
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
bull In some sense we can define the probability of anevent as its long-term relative frequency
What are some problems with this definition
1 It is not clear when and in what sense the limitexists
2 It is not possible to perform an experiment aninfinite number of times so the probabilities cannever be known exactly
3 We cannot use this definition if the experimentcannot be repeated
EEL5544 L1-24
We need a mathematical model ofprobability that is not based on a particularapplication or interpretation
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition
We need a mathematical model ofprobability that is not based on a particularapplication or interpretationHowever any such model should
1 be useful for solving real problems
2 agree with our interpretation of probabilityas relative frequency
3 agree with our intuition (whereappropriate)
EEL5544 L2-1
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elements
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the set
EEL 5544 Lecture 2PROBABILITY SPACES
We define a probability spaceas a mathematical constructioncontaining three elementsWe saythat a probability space is a triple(ΩF P )
bull The elements of a probability space for a randomexperiment are
1 DEFN The sample space denoted by Ω is the setof all outcomes
EEL5544 L2-2
The sample space is also known as the universal setor reference set
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite
The sample space is also known as the universal setor reference set
Sample spaces come in two basic varieties discreteor continuous
DEFN A discrete set is either finite or countablyinfinite
DEFN A set is countably infinite if it can be putinto one-to-one correspondence with theintegers
EEL5544 L2-3
EX
Examplesbull The integers Z
EX
Examplesbull The integers Zbull Positive integers Z+
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
EX
Examplesbull The integers Zbull Positive integers Z+
bull The rationals Q
DEFN A continuous set is not countable
DEFN If a and b gt a are in an interval I then ifa le x le b x isin I
EEL5544 L2-4
bull Intervals can be either open closed or half-open
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can be either open closed or half-openndash A closed interval [a b] contains the endpoints a
and bndash An open interval (a b) does not contain the
endpoints a and bndash An interval can be half-open such as (a b] which
does not contain a or [a b) which does notcontain b
bull Intervals can also be either finite infinite orpartially infinite
EEL5544 L2-5
EX
Example of continuous sets
EX
Example of continuous setsbull the real line R
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbers
EX
Example of continuous setsbull the real line Rbull the transcendental numbers (ie the
numbers in R that are not rationals)bull complex numbersbull Any interval finite or infinite
EEL5544 L2-6
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is even
bull We wish to ask questions about the probabilities ofnot only the outcomes but also combinations ofthe outcomes For instance if we roll a six-sideddie and record the number on the top face we maystill ask questions like
ndash What is the probability that the result is evenndash What is the probability that the result is le 2
EEL5544 L2-7
DEFNEvents are sets of outcomes to which weassign probability
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top face
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
DEFNEvents are sets of outcomes to which weassign probability
Examples based on previous examples ofsample spaces
(a) EX
Roll a fair 6-sided die and note the numberon the top faceLet L4 = the event that the result is lessthan or equal to 4Express L as a set of outcomes
L = 1 2 3 4
EEL5544 L2-8
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor odd
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcome
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
(b) EX
Roll a fair 6-sided die and determinewhether the number on the top face is evenor oddLet E = even outcome O = odd outcomeList all events
E = 2 4 6 and O = 1 3 5
EEL5544 L2-9
(c) EX
Toss a coin 3 times and note the sequenceof outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tails
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttoss
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
(c) EX
Toss a coin 3 times and note the sequenceof outcomesLet H=heads T=tailsLet A1= event that heads occurs on firsttossExpress A1 as a set of outcomes
A1 = HHHHTH HHT HTT
EEL5544 L2-10
(d) EX
Toss a coin 3 times and note the number ofheads
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occurs
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
(d) EX
Toss a coin 3 times and note the number ofheadsLet O = odd number of heads occursExpress O as a set of outcomes
O = 1 3
EEL5544 L2-11